Error while warping images onto a skymap in the Getting Started Tutorial

Support
#1

Hi,
I’m running this part of the LSST pipeline tutorial on the rc_subset with the lastest weekly (w_2022_23). When I run the pipetask to warp the images onto the skymap, I get following error:

RuntimeError: QuantumGraph is empty.

Complete error log can be found below.

When I check the status of the qgraph, it gives the following output:

lsst.ctrl.mpexec.cmdLineFwk INFO: QuantumGraph contains 3 quanta for 3 tasks, graph ID: ‘1654520157.0883536-108308’

I’m not sure as to what might be causing this error. Any help with this is appreciated!

Complete error log:

lsst.pipe.base.graphBuilder WARNING: Dataset type finalized_psf_ap_corr_catalog is not registered.
py.warnings WARNING: /jbodstorage/data_ms19054/lsst_stack/stack/miniconda3-py38_4.9.2-4.0.1/Linux64/ctrl_mpexec/g5ad6d16500+c534962079/python/lsst/ctrl/mpexec/cli/script/qgraph.py:187: UserWarning: QuantumGraph is empty
  qgraph = f.makeGraph(pipelineObj, args)

lsst.daf.butler.cli.utils ERROR: Caught an exception, details are in traceback:
Traceback (most recent call last):
  File "/jbodstorage/data_ms19054/lsst_stack/stack/miniconda3-py38_4.9.2-4.0.1/Linux64/ctrl_mpexec/g5ad6d16500+c534962079/python/lsst/ctrl/mpexec/cli/cmd/commands.py", line 130, in run
    qgraph = script.qgraph(pipelineObj=pipeline, **kwargs)
  File "/jbodstorage/data_ms19054/lsst_stack/stack/miniconda3-py38_4.9.2-4.0.1/Linux64/ctrl_mpexec/g5ad6d16500+c534962079/python/lsst/ctrl/mpexec/cli/script/qgraph.py", line 190, in qgraph
    raise RuntimeError("QuantumGraph is empty.")
RuntimeError: QuantumGraph is empty.
#2

You don’t report the command line you typed, but are you sure you included the --register-dataset-types option to pipetask?

#3

Hi @prajakta1-618, just doing some late follow-up on this issue. Did Tim’s suggestion to include the --register-dataset-types in your command line help with the reported issue?

#4

I get the same error at the same place in the tutorial. I have look at the FAQ on “QuantumGraph is empty” entry, but it didn’t help. The exact command I used was:

pipetask run -b $RC2_SUBSET_DIR/SMALL_HSC/butler.yaml
-d “tract = 9813 AND skymap = ‘hsc_rings_v1’ AND patch in (38, 39, 40, 41)”
-p $RC2_SUBSET_DIR/pipelines/DRP.yaml#makeWarp
-i u/$USER/jointcal,u/$USER/fgcm
-o u/$USER/warps
–register-dataset-types

#5

Thanks. I have gone through the tutorial myself with the current weekly and I also get the empty graph. This has been filed as DM-36354 and @jeffcarlin will look into it.

#6

We’ve had a look at this and the problem is caused by the tutorial not being updated when the “apply calibration” step was added. If you change the input collection to:

-i u/$USER/jointcal,u/$USER/fgcm,u/$USER/source_calibration

then the command should work.

1 Like
#7

I tried the substitution for the input and get the below error message:

lsst.daf.butler.registry._exceptions.MissingCollectionError: No collection with name ‘u/plah/source_calibration’ found.

The only directories in u/$USER/ are:
fgcm jointcal single_frame warps

When does the source_calibration directory get made? At present I am at the warping of the images step. In the next tutorial is when we measure sources.

#8

The source_calibration collection should have been created as the last step (“Apply the calibrations”) of part 4 (here).

#9

That worked. Thanks for your help.

#10

Hi, I am doing Getting started tutorial part 5: coadding images. But I am having the same error as previous mentioned RuntimeError: QuantumGraph is empty.

For the wrap part the command that I used
pipetask run -b $RC2_SUBSET_DIR/SMALL_HSC/butler.yaml -d "tract = 9813 AND skymap = 'hsc_rings_v1' AND patch in (38, 39, 40, 41)" -p $RC2_SUBSET_DIR/pipelines/DRP.yaml#makeWarp -i u/$USER/jointcal,u/$USER/fgcm,u/$USER/source_calibration -o u/$USER/warps --register-dataset-types

I added the u/$USER/source_calibration as @timj mentioned. But I still have the error.

For the last step “Apply the calibrations” in Getting started tutorial part 4: Using all the data to calibrate . I was having an error with the command ValueError: Not all supplied labels (specified or named subsets) are in the pipeline definition, extra labels: {'source_calibration'}. I changed the subset #source_calibration for #calibrate (as mentioned in the Wrap Up part of the same tutorial), the command was:

pipetask run -b $RC2_SUBSET_DIR/SMALL_HSC/butler.yaml -p $RC2_SUBSET_DIR/pipelines/DRP.yaml#calibrate -i u/$USER/single_frame,u/$USER/fgcm,u/$USER/jointcal -o u/$USER/source_calibration --register-dataset-types

and that work well.

Thank you

#11

Hi Tatiana -

Could you post more of the traceback that came with the RuntimeError: QuantumGraph is empty. message (the full text of the error that was printed to the screen)? Some additional information would help to figure out why you’re getting the error.