Hello
I have some trouble to interpret the error message here after my ingestion seemed to be fine:
processCcd.py decamData --id visit=511404 ccdnum=10 --rerun processCcdOutputs4 --calib decamData/CALIB --config calibrate.doPhotoCal=False calibrate.doAstrometry=False
root INFO: Loading config overrride file ‘/cvmfs/sw.lsst.eu/linux-x86_64/lsst_distrib/v16.0/stack/miniconda3-4.3.21-10a4fa6/Linux64/obs_decam/16.0+1/config/processCcd.py’
CameraMapper INFO: Loading exposure registry from /home/fraison/lsst/try5/decamData/registry.sqlite3
CameraMapper INFO: Loading calib registry from /home/fraison/lsst/try5/decamData/CALIB/calibRegistry.sqlite3
CameraMapper INFO: Loading calib registry from /home/fraison/lsst/try5/decamData/CALIB/calibRegistry.sqlite3
root INFO: Running: /cvmfs/sw.lsst.eu/linux-x86_64/lsst_distrib/v16.0/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_tasks/16.0+1/bin/processCcd.py decamData --id visit=511404 ccdnum=10 --rerun processCcdOutputs4 --calib decamData/CALIB --config calibrate.doPhotoCal=False calibrate.doAstrometry=False
WARNING: You are using OpenBLAS with multiple threads (2), but have not
specified the number of threads using one of the OpenBLAS environment variables:
OPENBLAS_NUM_THREADS, GOTO_NUM_THREADS, OMP_NUM_THREADS.
This may indicate that you are unintentionally using multiple threads, which may
cause problems. WE HAVE THEREFORE DISABLED OpenBLAS THREADING. If you know
what you are doing and want threads enabled implicitly, set the environment
variable LSST_ALLOW_IMPLICIT_THREADS.
processCcd INFO: Processing {‘visit’: 511404, ‘ccdnum’: 10, ‘date’: ‘2016-01-15’, ‘filter’: ‘r’, ‘hdu’: 28, ‘object’: ‘DES survey hex 666-247 tiling 5’}
processCcd.isr INFO: Performing ISR on sensor {‘visit’: 511404, ‘ccdnum’: 10, ‘date’: ‘2016-01-15’, ‘filter’: ‘r’, ‘hdu’: 28, ‘object’: ‘DES survey hex 666-247 tiling 5’}
CameraMapper WARN: I can only set the VisitInfo if you provide a mapper
processCcd FATAL: Failed on dataId={‘visit’: 511404, ‘ccdnum’: 10, ‘date’: ‘2016-01-15’, ‘filter’: ‘r’, ‘hdu’: 28, ‘object’: ‘DES survey hex 666-247 tiling 5’}: RuntimeError: Unable to retrieve bias for {‘visit’: 511404, ‘ccdnum’: 10, ‘date’: ‘2016-01-15’, ‘filter’: ‘r’, ‘hdu’: 28, ‘object’: ‘DES survey hex 666-247 tiling 5’}: No locations for get: datasetType:bias dataId:DataId(initialdata={‘visit’: 511404, ‘ccdnum’: 10, ‘date’: ‘2016-01-15’, ‘filter’: ‘r’, ‘hdu’: 28, ‘object’: ‘DES survey hex 666-247 tiling 5’}, tag=set())
Traceback (most recent call last):
File “/cvmfs/sw.lsst.eu/linux-x86_64/lsst_distrib/v16.0/stack/miniconda3-4.3.21-10a4fa6/Linux64/ip_isr/16.0+1/python/lsst/ip/isr/isrTask.py”, line 776, in getIsrExposure
exp = dataRef.get(datasetType, immediate=immediate)
File “/cvmfs/sw.lsst.eu/linux-x86_64/lsst_distrib/v16.0/stack/miniconda3-4.3.21-10a4fa6/Linux64/daf_persistence/16.0/python/lsst/daf/persistence/butlerSubset.py”, line 198, in get
return self.butlerSubset.butler.get(datasetType, self.dataId, **rest)
File “/cvmfs/sw.lsst.eu/linux-x86_64/lsst_distrib/v16.0/stack/miniconda3-4.3.21-10a4fa6/Linux64/daf_persistence/16.0/python/lsst/daf/persistence/butler.py”, line 1412, in get
raise NoResults(“No locations for get:”, datasetType, dataId)
lsst.daf.persistence.butlerExceptions.NoResults: No locations for get: datasetType:bias dataId:DataId(initialdata={‘visit’: 511404, ‘ccdnum’: 10, ‘date’: ‘2016-01-15’, ‘filter’: ‘r’, ‘hdu’: 28, ‘object’: ‘DES survey hex 666-247 tiling 5’}, tag=set())
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “/cvmfs/sw.lsst.eu/linux-x86_64/lsst_distrib/v16.0/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_base/16.0+1/python/lsst/pipe/base/cmdLineTask.py”, line 392, in call
result = task.run(dataRef, **kwargs)
File “/cvmfs/sw.lsst.eu/linux-x86_64/lsst_distrib/v16.0/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_base/16.0+1/python/lsst/pipe/base/timer.py”, line 150, in wrapper
res = func(self, *args, **keyArgs)
File “/cvmfs/sw.lsst.eu/linux-x86_64/lsst_distrib/v16.0/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_tasks/16.0+1/python/lsst/pipe/tasks/processCcd.py”, line 183, in run
exposure = self.isr.runDataRef(sensorRef).exposure
File “/cvmfs/sw.lsst.eu/linux-x86_64/lsst_distrib/v16.0/stack/miniconda3-4.3.21-10a4fa6/Linux64/pipe_base/16.0+1/python/lsst/pipe/base/timer.py”, line 150, in wrapper
res = func(self, *args, **keyArgs)
File “/cvmfs/sw.lsst.eu/linux-x86_64/lsst_distrib/v16.0/stack/miniconda3-4.3.21-10a4fa6/Linux64/ip_isr/16.0+1/python/lsst/ip/isr/isrTask.py”, line 647, in runDataRef
isrData = self.readIsrData(sensorRef, ccdExposure)
File “/cvmfs/sw.lsst.eu/linux-x86_64/lsst_distrib/v16.0/stack/miniconda3-4.3.21-10a4fa6/Linux64/ip_isr/16.0+1/python/lsst/ip/isr/isrTask.py”, line 413, in readIsrData
if self.config.doBias else None
File “/cvmfs/sw.lsst.eu/linux-x86_64/lsst_distrib/v16.0/stack/miniconda3-4.3.21-10a4fa6/Linux64/ip_isr/16.0+1/python/lsst/ip/isr/isrTask.py”, line 779, in getIsrExposure
raise RuntimeError(“Unable to retrieve %s for %s: %s” % (datasetType, dataRef.dataId, exc1))
RuntimeError: Unable to retrieve bias for {‘visit’: 511404, ‘ccdnum’: 10, ‘date’: ‘2016-01-15’, ‘filter’: ‘r’, ‘hdu’: 28, ‘object’: ‘DES survey hex 666-247 tiling 5’}: No locations for get: datasetType:bias dataId:DataId(initialdata={‘visit’: 511404, ‘ccdnum’: 10, ‘date’: ‘2016-01-15’, ‘filter’: ‘r’, ‘hdu’: 28, ‘object’: ‘DES survey hex 666-247 tiling 5’}, tag=set())
Here is my butler repository tree (minus most of the reference catalog files for brevity):
.
├── 2016-01-15
│ ├── g
│ │ └── decam0511406.fits.fz -> /data/euclid/u/fraison/DES_lepus3/20160114/src/EUC_EXT_DPDEXTDESRAWFRAME_DECAM-00511406_20180516T162540.8Z_00.00.fits.fz
│ ├── i
│ │ └── decam0511405.fits.fz -> /data/euclid/u/fraison/DES_lepus3/20160114/src/EUC_EXT_DPDEXTDESRAWFRAME_DECAM-00511405_20180516T162513.8Z_00.00.fits.fz
│ └── r
│ └── decam0511404.fits.fz -> /data/euclid/u/fraison/DES_lepus3/20160114/src/EUC_EXT_DPDEXTDESRAWFRAME_DECAM-00511404_20180516T162510.7Z_00.00.fits.fz
├── CALIB
│ ├── calibRegistry.sqlite3
│ ├── cpBIAS
│ │ └── 2016-01-15
│ │ └── BIAS-2016-01-15.fits -> /home/fraison/lsst/try5/calib/EUC_EXT_DPDEXTDESBIASFRAME_DECAM-00511575_20180516T162509.1Z_00.00.fits
│ └── cpFLAT
│ └── 2016-01-15
│ ├── g
│ │ └── FLAT-2016-01-15.fits -> /home/fraison/lsst/try5/calib/EUC_EXT_DPDEXTDESFLATFRAME_DECAM-00511666_20180516T162539.3Z_00.00.fits.fz
│ ├── i
│ │ └── FLAT-2016-01-15.fits -> /home/fraison/lsst/try5/calib/EUC_EXT_DPDEXTDESFLATFRAME_DECAM-00511641_20180516T162517.2Z_00.00.fits.fz
│ ├── r
│ │ └── FLAT-2016-01-15.fits -> /home/fraison/lsst/try5/calib/EUC_EXT_DPDEXTDESFLATFRAME_DECAM-00511663_20180516T162456.3Z_00.00.fits.fz
│ └── z
│ └── FLAT-2016-01-15.fits -> /home/fraison/lsst/try5/calib/EUC_EXT_DPDEXTDESFLATFRAME_DECAM-00511628_20180516T162553.4Z_00.00.fits
├── Decam.list
├── _mapper
├── ref_cats
│ └── ps1_pv3_3pi_20170110
│ ├── 141826.fits
│ ├── 141832.fits
…
│ ├── 142399.fits
│ ├── config.py
│ └── master_schema.fits
├── registry.sqlite3
└── rerun
└── processCcdOutputs4
├── 0511404
│ └── metadata
│ └── metadata-0511404_10.boost
├── config
│ ├── packages.pickle
│ └── processCcd.py
├── repositoryCfg.yaml
└── schema
├── icSrc.fits
└── src.fits
Any help is welcomed. Thanks!