Hello,
I try to exploit the utility shared_stack.py to maintain regular weekly based evolution of the stack in the Unix machines at CCIN2P3
I did manage to make it work some weeks ago but now it fails:
[lsstprod@ccwige0003 shared-stack]$ ./shared_stack.py
$* is no longer supported at /usr/bin/latex2html line 10588.
texexpand V2008 (Revision 1.12)
*** initialising colors ***
Warning: No implementation found for package: hyperref
Warning: No implementation found for package: fullpageThis is dvips(k) 5.96.1 Copyright 2007 Radical Eye Software (www.radicaleye.com)
' TeX output 2016.07.11:1025' -> /tmp/l2h100210/image
(-> /tmp/l2h100210/image001) </usr/share/texmf/dvips/base/tex.pro>
</usr/share/texmf/dvips/config/alt-rule.pro>
</usr/share/texmf/dvips/base/texps.pro>
</usr/share/texmf/dvips/base/special.pro>
</usr/share/texmf/dvips/base/color.pro>
</usr/share/texmf/fonts/type1/bluesky/latex-fonts/line10.pfb>
</usr/share/texmf/fonts/type1/bluesky/cm/cmr10.pfb>[1]
(-> /tmp/l2h100210/image002) </usr/share/texmf/dvips/base/tex.pro>
</usr/share/texmf/dvips/config/alt-rule.pro>
</usr/share/texmf/dvips/base/texps.pro>
</usr/share/texmf/dvips/base/special.pro>
</usr/share/texmf/dvips/base/color.pro>
</usr/share/texmf/fonts/type1/bluesky/latex-fonts/line10.pfb>
</usr/share/texmf/fonts/type1/bluesky/cm/cmr10.pfb>[2]
(-> /tmp/l2h100210/image003) </usr/share/texmf/dvips/base/tex.pro>
</usr/share/texmf/dvips/config/alt-rule.pro>
</usr/share/texmf/dvips/base/texps.pro>
</usr/share/texmf/dvips/base/special.pro>
</usr/share/texmf/dvips/base/color.pro>
</usr/share/texmf/fonts/type1/bluesky/latex-fonts/line10.pfb>
</usr/share/texmf/fonts/type1/bluesky/cm/cmr10.pfb>[3]
(-> /tmp/l2h100210/image004) </usr/share/texmf/dvips/base/tex.pro>
</usr/share/texmf/dvips/config/alt-rule.pro>
</usr/share/texmf/dvips/base/texps.pro>
</usr/share/texmf/dvips/base/special.pro>
</usr/share/texmf/dvips/base/color.pro>
</usr/share/texmf/fonts/type1/bluesky/cm/cmsy10.pfb>[4]
Creating user data directory: /tmp/tmpPt3vpY
eups list: No products found
[ 1/1 ] miniconda2 3.19.0.lsst4 ... done.
Error: no packages found to remove from environment: /sps/lsst/Library/weeklies/Linux64/miniconda2/3.19.0.lsst4
Failed process output:
Fetching package metadata .......
Solving package specifications: ..........
Traceback (most recent call last):
File "./shared_stack.py", line 590, in <module>
main()
File "./shared_stack.py", line 554, in main
sm = StackManager.create_stack(ROOT, userdata=userdata)
File "./shared_stack.py", line 498, in create_stack
sm.conda("remove", package)
File "./shared_stack.py", line 391, in conda
universal_newlines=True)
File "./shared_stack.py", line 542, in _check_output
raise subprocess.CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['conda', 'remove', '--yes', 'mkl-service']' returned non-zero exit status 1
The tool is unchanged from the master branch.
Can someone help me ?
Regards
swinbank
July 11, 2016, 1:52pm
#2
I see the same error. I’ll try to chase it down.
swinbank
July 11, 2016, 3:49pm
#4
I believe this is now fixed (or, at least, worked-around—the problem seems to be a change in Anaconda behaviour which I don’t understand) in the current master version of shared_stack.py Please let me know if that works for you.
Much better!!
Christian
Hi
Unfortunately there is now another failure (in getting the log package ) :
[lsstprod@cca004 shared-stack]$ ./shared_stack.py
Creating user data directory: /scratch/tmp_fusRR
Considering lsst_distrib
Installing lsst_distrib tagged w_2016_28
Using /sps/lsst/Library/weeklies/Linux64/miniconda2/3.19.0.lsst4/bin/python (Python 2.7.11 :: Anaconda custom (64-bit)).
Using externally provided astropy v1.2.1.
Using externally provided numpy v1.11.1.
Using externally provided matplotlib v1.5.1.
Using externally provided scipy v0.17.1.
***** error: from /sps/lsst/Library/weeklies/EupsBuildDir/Linux64/log-12.0+2/build.log:
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests/logTest.py", line 66, in tearDown
shutil.rmtree(self.tempDir)
File "/sps/lsst/Library/weeklies/Linux64/miniconda2/3.19.0.lsst4/lib/python2.7/shutil.py", line 256, in rmtree
onerror(os.rmdir, path, sys.exc_info())
File "/sps/lsst/Library/weeklies/Linux64/miniconda2/3.19.0.lsst4/lib/python2.7/shutil.py", line 254, in rmtree
os.rmdir(path)
OSError: [Errno 39] Directory not empty: '/scratch/tmptZiOOc'
----------------------------------------------------------------------
Ran 9 tests in 0.143s
FAILED (errors=1)
The following tests failed:
/sps/lsst/Library/weeklies/EupsBuildDir/Linux64/log-12.0+2/log-12.0+2/tests/.tests/logTest.py.failed
1 tests failed
scons: *** [checkTestStatus] Error 1
scons: building terminated because of errors.
+ exit -4
eups distrib: Failed to build log-12.0+2.eupspkg: Command:
source /sps/lsst/Library/weeklies/eups/bin/setups.sh; export EUPS_PATH=/sps/lsst/Library/weeklies; (/sps/lsst/Library/weeklies/EupsBuildDir/Linux64/log-12.0+2/build.sh) >> /sps/lsst/Library/weeklies/EupsBuildDir/Linux64/log-12.0+2/build.log 2>&1 4>/sps/lsst/Library/weeklies/EupsBuildDir/Linux64/log-12.0+2/build.msg
exited with code 252
Failed process output:
[ 1/91 ] apr 1.5.2 ... done.
[ 2/91 ] cfitsio 3360.lsst4 ... done.
[ 3/91 ] doxygen 1.8.5.lsst1 ... done.
[ 4/91 ] eigen 3.2.5.lsst2 ... done.
[ 5/91 ] fftw 3.3.4.lsst2 ... done.
[ 6/91 ] gsl 1.16.lsst3 ... done.
[ 7/91 ] mariadb 10.1.11.lsst2 ... done.
[ 8/91 ] mariadbclient 10.1.11.lsst3 ... done.
[ 9/91 ] minuit2 5.34.14 ... done.
[ 10/91 ] mpich 3.2 ... done.
[ 11/91 ] python 0.0.4 ... done.
[ 12/91 ] python_d2to1 0.2.12.lsst1 ... done.
[ 13/91 ] swig 3.0.2.lsst1 ... done.
[ 14/91 ] xpa 2.1.15.lsst3 ... done.
[ 15/91 ] activemqcpp 3.9.0.lsst2+3 ... done.
[ 16/91 ] apr_util 1.5.4 ... done.
[ 17/91 ] astropy 0.0.1.lsst2 ... done.
[ 18/91 ] boost 1.60+1 ... done.
[ 19/91 ] mpi 0.0.1+1 ... done.
[ 20/91 ] mysqlpython 1.2.3.lsst2+2 ... done.
[ 21/91 ] numpy 0.0.2+1 ... done.
[ 22/91 ] python_psutil 4.1.0+1 ... done.
[ 23/91 ] pyyaml 3.11.lsst1+1 ... done.
[ 24/91 ] scons 2.5.0.lsst1 ... done.
[ 25/91 ] sqlalchemy 1.0.8.lsst3+1 ... done.
[ 26/91 ] stsci_distutils 0.3.7.lsst1 ... done.
[ 27/91 ] wcslib 5.13.lsst1 ... done.
[ 28/91 ] astrometry_net 0.50.lsst3+1 ... done.
[ 29/91 ] esutil 0.5.3+1 ... done.
[ 30/91 ] log4cxx 0.10.0.lsst6+1 ... done.
[ 31/91 ] matplotlib 0.0.2-1-g49c793a ... done.
[ 32/91 ] mpi4py 1.3.1.lsst2+1 ... done.
[ 33/91 ] pyfits 3.4.0+4 ... done.
[ 34/91 ] scipy 0.0.1.lsst1+1 ... done.
[ 35/91 ] scisql 0.3.5+14 ... done.
[ 36/91 ] sconsUtils 12.0-1-g08b90fd ... done.
[ 37/91 ] tmv 0.73+2 ... done.
[ 38/91 ] astrometry_net_data 10.0+78 ... done.
[ 39/91 ] base 12.0-4-g0d328d3 ... done.
[ 40/91 ] galsim 1.3.2.lsst2+2 ... done.
[ 41/91 ] geom 10.0+62 ... done.
[ 42/91 ] lmfit 0.9.3+2 ... done.
[ 43/91 ] lsst 12.0.rc1-2-g4137827+3 ... done.
[ 44/91 ] psfex 12.0+1 ... done.
[ 45/91 ] log 12.0+2 ...
Traceback (most recent call last):
File "./shared_stack.py", line 594, in <module>
main()
File "./shared_stack.py", line 572, in main
sm.distrib_install(product, tag=tag)
File "./shared_stack.py", line 416, in distrib_install
print(self._run_cmd("distrib", *args))
File "./shared_stack.py", line 368, in _run_cmd
universal_newlines=True)
File "./shared_stack.py", line 546, in _check_output
raise subprocess.CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['eups', '--nolocks', 'distrib', 'install', '--no-server-tags', 'lsst_distrib', '-t', 'w_2016_28']' returned non-zero exit status 9
Christian
…
Other failure in meas_extensions_shapeHSM-12.0+11
***** error: from /sps/lsst/Library/weeklies/EupsBuildDir/Linux64/meas_extensions_shapeHSM-12.0+11/build.log:
warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.')
ERROR: ImportError: /usr/local/grid/emi-3/WN/SL6_64/3.10.0-1.2/usr/lib64/libgomp.so.1: version `GOMP_4.0' not found (required by /sps/lsst/Library/weeklies/Linux64/tmv/0.73+2/lib/libtmv.so.0) [lsstimport]
Traceback (most recent call last):
File "tests/testHsm.py", line 43, in <module>
import lsst.meas.extensions.shapeHSM
File "/sps/lsst/Library/weeklies/EupsBuildDir/Linux64/meas_extensions_shapeHSM-12.0+11/meas_extensions_shapeHSM-12.0+11/python/lsst/meas/extensions/shapeHSM/__init__.py", line 25, in <module>
from .hsmLib import *
File "/sps/lsst/Library/weeklies/EupsBuildDir/Linux64/meas_extensions_shapeHSM-12.0+11/meas_extensions_shapeHSM-12.0+11/python/lsst/meas/extensions/shapeHSM/hsmLib.py", line 34, in <module>
_hsmLib = swig_import_helper()
File "/sps/lsst/Library/weeklies/EupsBuildDir/Linux64/meas_extensions_shapeHSM-12.0+11/meas_extensions_shapeHSM-12.0+11/python/lsst/meas/extensions/shapeHSM/hsmLib.py", line 30, in swig_import_helper
_mod = imp.load_module('_hsmLib', fp, pathname, description)
File "/sps/lsst/Library/weeklies/Linux64/base/12.0-4-g0d328d3/python/lsstimport.py", line 102, in imp_load_module
module = orig_imp_load_module(name, *args)
ImportError: /usr/local/grid/emi-3/WN/SL6_64/3.10.0-1.2/usr/lib64/libgomp.so.1: version `GOMP_4.0' not found (required by /sps/lsst/Library/weeklies/Linux64/tmv/0.73+2/lib/libtmv.so.0)
The following tests failed:
/sps/lsst/Library/weeklies/EupsBuildDir/Linux64/meas_extensions_shapeHSM-12.0+11/meas_extensions_shapeHSM-12.0+11/tests/.tests/testHsm.py.failed
1 tests failed
Christian
swinbank
July 12, 2016, 1:01pm
#8
shapeHSM problem moved here: Problem in trying to install meas_extensions_shapeHSM-12.0+11
Am I correct in assuming you solved the log problem, since the shapeHSM issue occurs after that in the build?