Hi all,
I’m testing the LSST Science Pipelines v26 for time domain analysis. I’m using the GW170817 kilnova exposures of DECam (25 visits of all ccds). There’s no dithering in those exposures. I wonder if that caused the following jointcal error.
ERROR 2024-06-18T15:06:17.621-07:00 lsst.ctrl.mpexec.singleQuantumExecutor (jointcal:{instrument: 'DECam', skymap: 'decam_rings_v1', physical_filter: 'g DECam SDSS c0001 4720.0 1520.0', tract: 5813, ...})(singleQuantumExecutor.py:260) - Execution of task 'jointcal' on quantum {instrument: 'DECam', skymap: 'decam_rings_v1', physical_filter: 'g DECam SDSS c0001 4720.0 1520.0', tract: 5813, ...} failed. Exception RuntimeError: Large chi2 increase between steps: fit likely cannot converge. Try setting one or more of the `writeChi2*` config fields and looking at how individual star chi2-values evolve during the fit.
Process task-{instrument: 'DECam', skymap: 'decam_rings_v1', physical_filter: 'g DECam SDSS c0001 4720.0 1520.0', tract: 5813, ...}:
Traceback (most recent call last):
File "/data0/shenming/lsst_stack_v26_0_0/conda/miniconda3-py38_4.9.2/envs/lsst-scipipe-7.0.1/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/data0/shenming/lsst_stack_v26_0_0/conda/miniconda3-py38_4.9.2/envs/lsst-scipipe-7.0.1/lib/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/data0/shenming/lsst_stack_v26_0_0/stack/miniconda3-py38_4.9.2-7.0.1/Linux64/ctrl_mpexec/g218a3a8f53+ca4789321c/python/lsst/ctrl/mpexec/mpGraphExecutor.py", line 162, in _executeJob
quantumExecutor.execute(taskDef, quantum)
File "/data0/shenming/lsst_stack_v26_0_0/stack/miniconda3-py38_4.9.2-7.0.1/Linux64/ctrl_mpexec/g218a3a8f53+ca4789321c/python/lsst/ctrl/mpexec/singleQuantumExecutor.py", line 161, in execute
result = self._execute(taskDef, quantum)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data0/shenming/lsst_stack_v26_0_0/stack/miniconda3-py38_4.9.2-7.0.1/Linux64/ctrl_mpexec/g218a3a8f53+ca4789321c/python/lsst/ctrl/mpexec/singleQuantumExecutor.py", line 258, in _execute
self.runQuantum(task, quantum, taskDef, limited_butler)
File "/data0/shenming/lsst_stack_v26_0_0/stack/miniconda3-py38_4.9.2-7.0.1/Linux64/ctrl_mpexec/g218a3a8f53+ca4789321c/python/lsst/ctrl/mpexec/singleQuantumExecutor.py", line 460, in runQuantum
task.runQuantum(butlerQC, inputRefs, outputRefs)
File "/data0/shenming/lsst_stack_v26_0_0/stack/miniconda3-py38_4.9.2-7.0.1/Linux64/jointcal/g635b316a6c+8d6b3a3e56/python/lsst/jointcal/jointcal.py", line 624, in runQuantum
outputs = self.run(**inputs, tract=tract)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data0/shenming/lsst_stack_v26_0_0/stack/miniconda3-py38_4.9.2-7.0.1/Linux64/jointcal/g635b316a6c+8d6b3a3e56/python/lsst/jointcal/jointcal.py", line 727, in run
astrometry = self._do_load_refcat_and_fit(associations, defaultFilter, center, radius,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data0/shenming/lsst_stack_v26_0_0/stack/miniconda3-py38_4.9.2-7.0.1/Linux64/jointcal/g635b316a6c+8d6b3a3e56/python/lsst/jointcal/jointcal.py", line 1064, in _do_load_refcat_and_fit
result = fit_function(associations, dataName)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data0/shenming/lsst_stack_v26_0_0/stack/miniconda3-py38_4.9.2-7.0.1/Linux64/jointcal/g635b316a6c+8d6b3a3e56/python/lsst/jointcal/jointcal.py", line 1375, in _fit_astrometry
chi2 = self._iterate_fit(associations,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data0/shenming/lsst_stack_v26_0_0/stack/miniconda3-py38_4.9.2-7.0.1/Linux64/jointcal/g635b316a6c+8d6b3a3e56/python/lsst/jointcal/jointcal.py", line 1499, in _iterate_fit
raise RuntimeError(msg)
RuntimeError: Large chi2 increase between steps: fit likely cannot converge. Try setting one or more of the `writeChi2*` config fields and looking at how individual star chi2-values evolve during the fit.
ERROR 2024-06-18T15:06:17.812-07:00 lsst.ctrl.mpexec.mpGraphExecutor ()(mpGraphExecutor.py:600) - Task <TaskDef(lsst.jointcal.jointcal.JointcalTask, label=jointcal) dataId={instrument: 'DECam', skymap: 'decam_rings_v1', physical_filter: 'g DECam SDSS c0001 4720.0 1520.0', tract: 5813, ...}> failed, exit code=1; processing will continue for remaining tasks.
The goal here is to make a coadded image as a template for image subtraction.
In the first step of coadd processing, makeWarp requires finalVisitSummary, and finalVisitSummary comes from updateVisitSummary which needs the results of jointcal.
I have run a few tests but couldn’t move forward without jointcal results. For the purpose of time domain analysis, is it possible to build a coadd template without information from jointcal? Because we don’t perform astrometric/photometric measurement on that coadded image, but just do image subtraction.
Could you give me some suggestions on how to skip jointcal results at makeWarp? In v23 we can set doApplyExternalPhotoCalib, doApplyExternalSkyWcs to False. Can we do similar things in v26?
Thanks.