V14_0 source and binary installations on RHEL6

Hi,

I’ve tried out both the binary and source installations on RHEL6-64, with gcc 4.9.2. The source installation fails during one of the obs_base unit tests (testCameraMapper.py), so I fell back to the binary install. Does the binary installation run the unit tests? While the binary installation completed, I went back and tried to run testCameraMapper.py which still fails in a similar fashion, though the output is more verbose. This leads me to believe that the unit tests are not executed during the binary installation. As a sanity check for a binary installation is there a way to systematically run the unit tests for all the packages - short of running them all by hand? I did a successful source installation at NERSC on cori with gcc 4.9.3. I also ran the obs_base testCameraMapper.py unit test by hand and it completed successfully.
Anyway, here is the error on RHEL6-64:

bash-4.1$ stack/current/Linux64/obs_base/14.0/tests/testCameraMapper.py 
CameraMapper INFO: Loading exposure registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/registry.sqlite3
/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/miniconda3-4.2.12-7c8e67/Linux64/obs_base/14.0/python/lsst/obs/base/cameraMapper.py:247: ResourceWarning: unclosed file <_io.TextIOWrapper name='/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/registry.sqlite3' mode='r' encoding='ISO-8859-1'>
  posixIfNoSql=(not parentRegistry))
CameraMapper INFO: Loading exposure registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/registry.sqlite3
.CameraMapper INFO: Loading exposure registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/registry.sqlite3
.CameraMapper INFO: Loading exposure registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/registry.sqlite3
.CameraMapper INFO: Loading exposure registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/registry.sqlite3
.CameraMapper INFO: Loading exposure registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/registry.sqlite3
.CameraMapper INFO: Loading exposure registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/cfhtls.sqlite3
/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/miniconda3-4.2.12-7c8e67/Linux64/obs_base/14.0/python/lsst/obs/base/cameraMapper.py:247: ResourceWarning: unclosed file <_io.TextIOWrapper name='/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/cfhtls.sqlite3' mode='r' encoding='ISO-8859-1'>
  posixIfNoSql=(not parentRegistry))
CameraMapper WARN: Unable to locate registry at policy path (also looked in root): tests/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/calibRegistry.sqlite3
/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/miniconda3-4.2.12-7c8e67/Linux64/obs_base/14.0/python/lsst/obs/base/cameraMapper.py:254: ResourceWarning: unclosed file <_io.TextIOWrapper name='/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/calibRegistry.sqlite3' mode='r' encoding='ISO-8859-1'>
  "calibRegistryPath", calibStorage)
daf.persistence.butler WARN: Passing an instantiated mapper into Butler.__init__ will prevent Butler from passing parentRegistry or repositoryCfg information to the mapper, which is done only at init time. It is better to pass a importable string or class object.
daf.persistence.butler WARN: Passing an instantiated mapper into Butler.__init__ will prevent Butler from passing parentRegistry or repositoryCfg information to the mapper, which is done only at init time. It is better to pass a importable string or class object.
.CameraMapper INFO: Loading exposure registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/cfhtls.sqlite3
CameraMapper WARN: Unable to locate registry at policy path (also looked in root): tests/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/calibRegistry.sqlite3
.CameraMapper INFO: Loading exposure registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/cfhtls.sqlite3
CameraMapper WARN: Unable to locate registry at policy path (also looked in root): tests/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/calibRegistry.sqlite3
.CameraMapper INFO: Loading exposure registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/cfhtls.sqlite3
CameraMapper WARN: Unable to locate registry at policy path (also looked in root): tests/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/calibRegistry.sqlite3
daf.persistence.butler WARN: Passing an instantiated mapper into Butler.__init__ will prevent Butler from passing parentRegistry or repositoryCfg information to the mapper, which is done only at init time. It is better to pass a importable string or class object.
daf.persistence.butler WARN: Passing an instantiated mapper into Butler.__init__ will prevent Butler from passing parentRegistry or repositoryCfg information to the mapper, which is done only at init time. It is better to pass a importable string or class object.
.CameraMapper INFO: Loading exposure registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/cfhtls.sqlite3
CameraMapper WARN: Unable to locate registry at policy path (also looked in root): tests/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/calibRegistry.sqlite3
daf.persistence.butler WARN: Passing an instantiated mapper into Butler.__init__ will prevent Butler from passing parentRegistry or repositoryCfg information to the mapper, which is done only at init time. It is better to pass a importable string or class object.
daf.persistence.butler WARN: Passing an instantiated mapper into Butler.__init__ will prevent Butler from passing parentRegistry or repositoryCfg information to the mapper, which is done only at init time. It is better to pass a importable string or class object.
.CameraMapper INFO: Loading exposure registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/cfhtls.sqlite3
CameraMapper WARN: Unable to locate registry at policy path (also looked in root): tests/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/calibRegistry.sqlite3
CameraMapper INFO: Loading exposure registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/registry.sqlite3
.CameraMapper INFO: Loading exposure registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/cfhtls.sqlite3
CameraMapper WARN: Unable to locate registry at policy path (also looked in root): tests/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/calibRegistry.sqlite3
daf.persistence.butler WARN: Passing an instantiated mapper into Butler.__init__ will prevent Butler from passing parentRegistry or repositoryCfg information to the mapper, which is done only at init time. It is better to pass a importable string or class object.
daf.persistence.butler WARN: Passing an instantiated mapper into Butler.__init__ will prevent Butler from passing parentRegistry or repositoryCfg information to the mapper, which is done only at init time. It is better to pass a importable string or class object.
.CameraMapper INFO: Loading exposure registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/cfhtls.sqlite3
CameraMapper WARN: Unable to locate registry at policy path (also looked in root): tests/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/calibRegistry.sqlite3
daf.persistence.butler WARN: Passing an instantiated mapper into Butler.__init__ will prevent Butler from passing parentRegistry or repositoryCfg information to the mapper, which is done only at init time. It is better to pass a importable string or class object.
daf.persistence.butler WARN: Passing an instantiated mapper into Butler.__init__ will prevent Butler from passing parentRegistry or repositoryCfg information to the mapper, which is done only at init time. It is better to pass a importable string or class object.
.CameraMapper INFO: Loading exposure registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/cfhtls.sqlite3
CameraMapper WARN: Unable to locate registry at policy path (also looked in root): tests/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/calibRegistry.sqlite3
..CameraMapper INFO: Loading exposure registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/cfhtls.sqlite3
CameraMapper WARN: Unable to locate registry at policy path (also looked in root): tests/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/calibRegistry.sqlite3
xCameraMapper INFO: Loading exposure registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/cfhtls.sqlite3
CameraMapper WARN: Unable to locate registry at policy path (also looked in root): tests/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/calibRegistry.sqlite3
.CameraMapper INFO: Loading exposure registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/cfhtls.sqlite3
CameraMapper WARN: Unable to locate registry at policy path (also looked in root): tests/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/calibRegistry.sqlite3
daf.persistence.butler WARN: Passing an instantiated mapper into Butler.__init__ will prevent Butler from passing parentRegistry or repositoryCfg information to the mapper, which is done only at init time. It is better to pass a importable string or class object.
daf.persistence.butler WARN: Passing an instantiated mapper into Butler.__init__ will prevent Butler from passing parentRegistry or repositoryCfg information to the mapper, which is done only at init time. It is better to pass a importable string or class object.
.CameraMapper INFO: Loading exposure registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/cfhtls.sqlite3
CameraMapper WARN: Unable to locate registry at policy path (also looked in root): tests/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/calibRegistry.sqlite3
.CameraMapper INFO: Loading exposure registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/cfhtls.sqlite3
CameraMapper WARN: Unable to locate registry at policy path (also looked in root): tests/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/calibRegistry.sqlite3
.CameraMapper INFO: Loading exposure registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/registry.sqlite3
.CameraMapper INFO: Loading Posix exposure registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/ParentRegistryTestCase-fca2pq80/a
CameraMapper INFO: Loading exposure registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/ParentRegistryTestCase-fca2pq80/a/registry.sqlite3
/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/miniconda3-4.2.12-7c8e67/Linux64/obs_base/14.0/python/lsst/obs/base/cameraMapper.py:247: ResourceWarning: unclosed file <_io.TextIOWrapper name='/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/ParentRegistryTestCase-fca2pq80/a/registry.sqlite3' mode='r' encoding='ISO-8859-1'>
  posixIfNoSql=(not parentRegistry))
CameraMapper INFO: Loading exposure registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/ParentRegistryTestCase-fca2pq80/a/registry.sqlite3
CameraMapper INFO: Loading exposure registry from /nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/ParentRegistryTestCase-fca2pq80/b/registry.sqlite3
/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/miniconda3-4.2.12-7c8e67/Linux64/obs_base/14.0/python/lsst/obs/base/cameraMapper.py:247: ResourceWarning: unclosed file <_io.TextIOWrapper name='/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/stack/current/Linux64/obs_base/14.0/tests/ParentRegistryTestCase-fca2pq80/b/registry.sqlite3' mode='r' encoding='ISO-8859-1'>
  posixIfNoSql=(not parentRegistry))
E..
======================================================================
ERROR: test (__main__.ParentRegistryTestCase)
Verify that when the child repo does not have a registry it is assigned the registry from the
----------------------------------------------------------------------
Traceback (most recent call last):
  File "stack/current/Linux64/obs_base/14.0/tests/testCameraMapper.py", line 403, in tearDown
    shutil.rmtree(self.ROOT)
  File "/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/python/miniconda3-4.2.12/lib/python3.5/shutil.py", line 474, in rmtree
    _rmtree_safe_fd(fd, path, onerror)
  File "/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/python/miniconda3-4.2.12/lib/python3.5/shutil.py", line 416, in _rmtree_safe_fd
    onerror(os.rmdir, fullname, sys.exc_info())
  File "/nfs/farm/g/lsst/u1/software/redhat6-x86_64-64bit-gcc44/DMstack/v14_0/python/miniconda3-4.2.12/lib/python3.5/shutil.py", line 414, in _rmtree_safe_fd
    os.rmdir(name, dir_fd=topfd)
OSError: [Errno 39] Directory not empty: 'b'

----------------------------------------------------------------------
Ran 24 tests in 8.351s

FAILED (errors=1, expected failures=1)

There are some bugs on NFS for the tests in the 14.0 release. In this case it is a problem with the test and not a problem with the core code. I believe the current master works fine on NFS now. Binary installs don’t run tests and there is no easy way to run them all post installation in 14.0. In the current master we are pytest compatible and you can do things like pytest $AFW_DIR to run all the afw tests.

1 Like