Hi Folks,
I’m working on a small project right now with DECam data which requires moving back to single-band deblending. I was able to run it alright by changing the deblend task in DRP-Merian to DeblendCoaddSourcesSingleTask, but I’m running into issues on the measure step. Right now I’ve followed the guidance outlined in comments on github, and have added the following to my configuration file:
config.inputCatalog='deblendedFlux'
config.doAddFootprints=False
But the qgraph generation fails with the following error:
FutureWarning: Dataset type(s) ['deepCoadd_deblendedCatalog'] are not registered; this will be an error after v26.
with registry.queryDataIds(**queryArgs).materialize() as commonDataIds:
lsst.ctrl.bps.pre_transform INFO: CRITICAL 2024-10-18T22:51:31.525-04:00 lsst.pipe.base.graphBuilder ()(graphBuilder.py:1128) - Initial data ID query returned no rows, so QuantumGraph will be empty.
lsst.ctrl.bps.pre_transform INFO: CRITICAL 2024-10-18T22:51:31.526-04:00 lsst.pipe.base.graphBuilder ()(graphBuilder.py:1131) - Dataset type deepCoadd_deblendedCatalog is not registered.
lsst.ctrl.bps.pre_transform INFO: ERROR 2024-10-18T22:51:31.529-04:00 lsst.daf.butler.cli.utils ()(utils.py:1127) - Caught an exception, details are in traceback:
Traceback (most recent call last):
File "/users/idellant/data/Clusters/gen3_processing/lsst_stack_v26_0_0/stack/miniconda3-py38_4.9.2-7.0.1/Linux64/ctrl_mpexec/g218a3a8f53+ca4789321c/python/lsst/ctrl/mpexec/cli/cmd/commands.py", line 178, in qgraph
if script.qgraph(pipelineObj=pipeline, **kwargs, show=show) is None:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/users/idellant/data/Clusters/gen3_processing/lsst_stack_v26_0_0/stack/miniconda3-py38_4.9.2-7.0.1/Linux64/ctrl_mpexec/g218a3a8f53+ca4789321c/python/lsst/ctrl/mpexec/cli/script/qgraph.py", line 210, in qgraph
qgraph = f.makeGraph(pipelineObj, args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/users/idellant/data/Clusters/gen3_processing/lsst_stack_v26_0_0/stack/miniconda3-py38_4.9.2-7.0.1/Linux64/ctrl_mpexec/g218a3a8f53+ca4789321c/python/lsst/ctrl/mpexec/cmdLineFwk.py", line 622, in makeGraph
qgraph = graphBuilder.makeGraph(
^^^^^^^^^^^^^^^^^^^^^^^
File "/users/idellant/data/Clusters/gen3_processing/lsst_stack_v26_0_0/stack/miniconda3-py38_4.9.2-7.0.1/Linux64/pipe_base/g8798d61f7d+6612571a14/python/lsst/pipe/base/graphBuilder.py", line 1831, in makeGraph
scaffolding.resolveDatasetRefs(
File "/users/idellant/data/Clusters/gen3_processing/lsst_stack_v26_0_0/stack/miniconda3-py38_4.9.2-7.0.1/Linux64/pipe_base/g8798d61f7d+6612571a14/python/lsst/pipe/base/graphBuilder.py", line 1348, in resolveDatasetRefs
raise RuntimeError(
RuntimeError: 1 dataset(s) of type 'deepCoadd_deblendedFlux_schema' was/were present in a previous query, but could not be found now. This is either a logic bug in QuantumGraph generation or the input collections have been modified since QuantumGraph generation began.
Looking at the Github, I’m not sure why the pipeline is still looking for a dataset type deepCoadd_deblendedCatalog despite the inputCatalog being updated. I suspect that the trouble lies in line 298 and the pipeline is still expecting this dataset to exist and be defined… I’ve also already verified that both the deepCoadd_deblendedFlux and deepCoadd_deblendedFlux_schema dataset types have been created and are stored in my input collections, so I’m not sure where this is going wrong.
Any ideas? Thanks for the help!