Pipetask - QuantumGraph issues

Support
#1

Hi all,

I am trying to debug a notebook that should be reproducing a file that runs successfully when run from the main RSP Terminal.

My issue is when running the following code line:

pipetask run -b /project/gmegias/aos/lsst_stack/perturbations/phosimData -i refcats,LSSTComCam/raw/all,LSSTComCam/calib --instrument lsst.obs.lsst.LsstComCam --register-dataset-types --output-run ts_phosim_1151 -p comcamPipeline.yaml -d "exposure IN (4021114101151, 4021114101152)" -j 2

I tried running this line of code from the LSP terminal to see if I could get an insight on what was going on, but the following error prompts up:

    py.warnings WARN: /opt/lsst/software/stack/stack/miniconda3-py38_4.9.2-0.6.0/Linux64/ctrl_mpexec/21.0.0-33-g1dfc228+1cc4008e43/python/lsst/ctrl/mpexec/cli/script/qgraph.py:145: UserWarning: QuantumGraph is empty
  qgraph = f.makeGraph(pipelineObj, args)

Traceback (most recent call last):
  File "/opt/lsst/software/stack/stack/miniconda3-py38_4.9.2-0.6.0/Linux64/ctrl_mpexec/21.0.0-33-g1dfc228+1cc4008e43/bin/pipetask", line 29, in <module>
    sys.exit(main())
  File "/opt/lsst/software/stack/stack/miniconda3-py38_4.9.2-0.6.0/Linux64/ctrl_mpexec/21.0.0-33-g1dfc228+1cc4008e43/python/lsst/ctrl/mpexec/cli/pipetask.py", line 43, in main
    return cli()
  File "/opt/lsst/software/stack/conda/miniconda3-py38_4.9.2/envs/lsst-scipipe-0.6.0/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/opt/lsst/software/stack/conda/miniconda3-py38_4.9.2/envs/lsst-scipipe-0.6.0/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/opt/lsst/software/stack/conda/miniconda3-py38_4.9.2/envs/lsst-scipipe-0.6.0/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/lsst/software/stack/conda/miniconda3-py38_4.9.2/envs/lsst-scipipe-0.6.0/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/lsst/software/stack/conda/miniconda3-py38_4.9.2/envs/lsst-scipipe-0.6.0/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/opt/lsst/software/stack/conda/miniconda3-py38_4.9.2/envs/lsst-scipipe-0.6.0/lib/python3.8/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/opt/lsst/software/stack/stack/miniconda3-py38_4.9.2-0.6.0/Linux64/ctrl_mpexec/21.0.0-33-g1dfc228+1cc4008e43/python/lsst/ctrl/mpexec/cli/cmd/commands.py", line 102, in run
    qgraph = script.qgraph(pipelineObj=pipeline, **kwargs)
  File "/opt/lsst/software/stack/stack/miniconda3-py38_4.9.2-0.6.0/Linux64/ctrl_mpexec/21.0.0-33-g1dfc228+1cc4008e43/python/lsst/ctrl/mpexec/cli/script/qgraph.py", line 148, in qgraph
    raise RuntimeError("QuantumGraph is empty.")
RuntimeError: QuantumGraph is empty.

If anyone has any idea of what could I be doing wrong, I would appreciate a lot your support!

#2

Here is a discussion on empty quantum graphs that might help you.

https://pipelines.lsst.io/v/daily/middleware/faq.html?highlight=empty#how-do-i-fix-an-empty-quantumgraph

Can you post a link to your comcamPipeline.yaml ?

1 Like
#3

Can you also run butler query-datasets to check that those exposures really are present in that LSSTComCam/raw/all collection?

1 Like
#4

I can only see exposures 4021123101151 and 4021123101152 in LSSTComCam/raw/all collection .

#5

If this is phosim on-sky data then not running butler define-visits is a possibility (that is covered in the FAQ).

1 Like
#6

I think that was the issue. The problem is now solved. See follow-up issue in: