DECam Gen 3 VR band: RuntimeError: No reference tables could be found for input region

Support
#1

I am trying to process some VR band DECam images from raw images with flats and bias images. I appear to be making good postISRCCD images. Ingest, bias and flat processing, certification, isr and characterizeImage tasks appear to run properly, but I have an error on the calibrate task. The yaml file I call is

description: The AP template building pipeline specialized for DECam
instrument: lsst.obs.decam.DarkEnergyCamera
imports:
  - location: $AP_PIPE_DIR/pipelines/ProcessCcd.yaml
    exclude:
      - isr
  - location: $AP_PIPE_DIR/pipelines/DarkEnergyCamera/RunIsrWithCrosstalk.yaml
tasks:
  characterizeImage:
    class: lsst.pipe.tasks.characterizeImage.CharacterizeImageTask
    config:
      refObjLoader.ref_dataset_name: 'ps1_pv3_3pi_20170110'
  calibrate:
    class: lsst.pipe.tasks.calibrate.CalibrateTask
    config:
      photoCal.match.referenceSelection.magLimit.fluxField: 'i_flux'
      photoCal.match.referenceSelection.magLimit.maximum: 22.0
      astromRefObjLoader.ref_dataset_name: 'cal_ref_cat'
      #astromRefObjLoader.anyFilterMapsToThis: 'phot_g_mean'  # not yet implemented in gen3, DM-27843
      python: |
        config.astromRefObjLoader.filterMap = {}
        config.astromRefObjLoader.filterMap['g'] = 'phot_g_mean'
        config.astromRefObjLoader.filterMap['r'] = 'phot_g_mean'
        config.astromRefObjLoader.filterMap['i'] = 'phot_g_mean'
        config.astromRefObjLoader.filterMap['z'] = 'phot_g_mean'
        config.astromRefObjLoader.filterMap['y'] = 'phot_g_mean'
        config.astromRefObjLoader.filterMap['VR'] = 'phot_g_mean'
      photoRefObjLoader.ref_dataset_name: 'ps1_pv3_3pi_20170110'
      photoCal.photoCatName: 'ps1_pv3_3pi_20170110'
      connections.astromRefCat: 'cal_ref_cat'
      connections.photoRefCat: 'ps1_pv3_3pi_20170110'

run with pipetask run -j30 --register-dataset-types -d "exposure IN (855899, 855900, 855901) and detector IN (10)" -b . -i DECam/raw/all,DECam/calib,DECam/calib/cert-biases,DECam/calib/cert-flats,tmp/isr-prep -o tmp/calibrate_2 -p yamls/calibrate.yaml.

cal_ref_cat is Gaia DR2. I have also tried using pan-starrs and Gaia DR1 with the same error. The error is

Process task-8:
Traceback (most recent call last):
  File "/epyc/users/smotherh/lsst_latest/conda/miniconda3-py38_4.9.2/envs/lsst-scipipe-0.5.0/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/epyc/users/smotherh/lsst_latest/conda/miniconda3-py38_4.9.2/envs/lsst-scipipe-0.5.0/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/epyc/users/smotherh/lsst_latest/stack/miniconda3-py38_4.9.2-0.5.0/Linux64/ctrl_mpexec/21.0.0-26-gb7643ca+642d343ffb/python/lsst/ctrl/mpexec/mpGraphExecutor.py", line 127, in _executeJob
    quantumExecutor.execute(taskDef, quantum, butler)
  File "/epyc/users/smotherh/lsst_latest/stack/miniconda3-py38_4.9.2-0.5.0/Linux64/ctrl_mpexec/21.0.0-26-gb7643ca+642d343ffb/python/lsst/ctrl/mpexec/singleQuantumExecutor.py", line 103, in execute
    self.runQuantum(task, quantum, taskDef, butler)
  File "/epyc/users/smotherh/lsst_latest/stack/miniconda3-py38_4.9.2-0.5.0/Linux64/ctrl_mpexec/21.0.0-26-gb7643ca+642d343ffb/python/lsst/ctrl/mpexec/singleQuantumExecutor.py", line 274, in runQuantum
    task.runQuantum(butlerQC, inputRefs, outputRefs)
  File "/epyc/users/smotherh/lsst_latest/stack/miniconda3-py38_4.9.2-0.5.0/Linux64/pipe_tasks/21.0.0-72-gffa78901+5340526ff2/python/lsst/pipe/tasks/calibrate.py", line 622, in runQuantum
    outputs = self.run(**inputs)
  File "/epyc/users/smotherh/lsst_latest/stack/miniconda3-py38_4.9.2-0.5.0/Linux64/pipe_base/21.0.0-18-g546cdbd+4e75b35dac/python/lsst/pipe/base/timer.py", line 157, in wrapper
    res = func(self, *args, **keyArgs)
  File "/epyc/users/smotherh/lsst_latest/stack/miniconda3-py38_4.9.2-0.5.0/Linux64/pipe_tasks/21.0.0-72-gffa78901+5340526ff2/python/lsst/pipe/tasks/calibrate.py", line 722, in run
    astromRes = self.astrometry.run(
  File "/epyc/users/smotherh/lsst_latest/stack/miniconda3-py38_4.9.2-0.5.0/Linux64/pipe_base/21.0.0-18-g546cdbd+4e75b35dac/python/lsst/pipe/base/timer.py", line 157, in wrapper
    res = func(self, *args, **keyArgs)
  File "/epyc/users/smotherh/lsst_latest/stack/miniconda3-py38_4.9.2-0.5.0/Linux64/meas_astrom/21.0.0-4-g591bb35+9e2f6ff707/python/lsst/meas/astrom/astrometry.py", line 152, in run
    res = self.solve(exposure=exposure, sourceCat=sourceCat)
  File "/epyc/users/smotherh/lsst_latest/stack/miniconda3-py38_4.9.2-0.5.0/Linux64/pipe_base/21.0.0-18-g546cdbd+4e75b35dac/python/lsst/pipe/base/timer.py", line 157, in wrapper
    res = func(self, *args, **keyArgs)
  File "/epyc/users/smotherh/lsst_latest/stack/miniconda3-py38_4.9.2-0.5.0/Linux64/meas_astrom/21.0.0-4-g591bb35+9e2f6ff707/python/lsst/meas/astrom/astrometry.py", line 191, in solve
    loadRes = self.refObjLoader.loadPixelBox(
  File "/epyc/users/smotherh/lsst_latest/stack/miniconda3-py38_4.9.2-0.5.0/Linux64/meas_algorithms/21.0.0-20-g8cd22d88+eb66f50d9f/python/lsst/meas/algorithms/loadReferenceObjects.py", line 396, in loadPixelBox
    return self.loadRegion(outerSkyRegion, filtFunc=_filterFunction, epoch=epoch, filterName=filterName)
  File "/epyc/users/smotherh/lsst_latest/stack/miniconda3-py38_4.9.2-0.5.0/Linux64/meas_algorithms/21.0.0-20-g8cd22d88+eb66f50d9f/python/lsst/meas/algorithms/loadReferenceObjects.py", line 464, in loadRegion
    raise RuntimeError("No reference tables could be found for input region")
RuntimeError: No reference tables could be found for input region

Early in the processing, I get a warning about VR filter of

py.warnings WARN: /epyc/users/smotherh/lsst_latest/stack/miniconda3-py38_4.9.2-0.5.0/Linux64/obs_base/21.0.0-50-gfd878b2+50d1e73eb4/python/lsst/obs/base/formatters/fitsExposure.py:502: UserWarning: Reading file:
///astro/store/epyc2/projects2/smotherh/DEEP/Process_Raw/DECam/calib/constructed-flats/20210512T011045Z/flat/VR/VR_DECam_c0007_6300.0_2600.0/flat_DECam_VR_VR_DECam_c0007_6300_0_2600_0_S22_DECam_calib_constructed
-flats_20210512T011045Z.fits with data ID {instrument: 'DECam', detector: 10, physical_filter: 'VR DECam c0007 6300.0 2600.0', ...}: filter label mismatch (file is None, data ID is FilterLabel(band="VR", physica
l="VR DECam c0007 6300.0 2600.0")).  This is probably a bug in the code that produced it.
  warnings.warn(f"Reading {self.fileDescriptor.location} with data ID {self.dataId}: "

which makes me think I may be handling the VR filter processing incorrectly somehow. Are there any suggestions on how I might troubleshoot this issue?

#2

The problem was not linking the refcats with -i refcats/gen2. However, after fixing that issue, I am having an error potentially related to VR band.

calibrate INFO: Converted refcat flux fields to nJy (name, units): (g_flux, ''); (r_flux, ''); (i_flux, ''); (z_flux, ''); (y_flux, ''); (i_fluxSigma, ''); (y_fluxSigma, ''); (r_fluxSigma, ''); (z_fluxSigma, '')
; (g_fluxSigma, '')
Process task-8:
Traceback (most recent call last):
  File "/epyc/users/smotherh/lsst_latest/conda/miniconda3-py38_4.9.2/envs/lsst-scipipe-0.5.0/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/epyc/users/smotherh/lsst_latest/conda/miniconda3-py38_4.9.2/envs/lsst-scipipe-0.5.0/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/epyc/users/smotherh/lsst_latest/stack/miniconda3-py38_4.9.2-0.5.0/Linux64/ctrl_mpexec/21.0.0-26-gb7643ca+642d343ffb/python/lsst/ctrl/mpexec/mpGraphExecutor.py", line 127, in _executeJob
    quantumExecutor.execute(taskDef, quantum, butler)
  File "/epyc/users/smotherh/lsst_latest/stack/miniconda3-py38_4.9.2-0.5.0/Linux64/ctrl_mpexec/21.0.0-26-gb7643ca+642d343ffb/python/lsst/ctrl/mpexec/singleQuantumExecutor.py", line 103, in execute
    self.runQuantum(task, quantum, taskDef, butler)
  File "/epyc/users/smotherh/lsst_latest/stack/miniconda3-py38_4.9.2-0.5.0/Linux64/ctrl_mpexec/21.0.0-26-gb7643ca+642d343ffb/python/lsst/ctrl/mpexec/singleQuantumExecutor.py", line 274, in runQuantum
    task.runQuantum(butlerQC, inputRefs, outputRefs)
  File "/epyc/users/smotherh/lsst_latest/stack/miniconda3-py38_4.9.2-0.5.0/Linux64/pipe_tasks/21.0.0-72-gffa78901+5340526ff2/python/lsst/pipe/tasks/calibrate.py", line 622, in runQuantum
    outputs = self.run(**inputs)
  File "/epyc/users/smotherh/lsst_latest/stack/miniconda3-py38_4.9.2-0.5.0/Linux64/pipe_base/21.0.0-18-g546cdbd+4e75b35dac/python/lsst/pipe/base/timer.py", line 157, in wrapper
    res = func(self, *args, **keyArgs)
  File "/epyc/users/smotherh/lsst_latest/stack/miniconda3-py38_4.9.2-0.5.0/Linux64/pipe_tasks/21.0.0-72-gffa78901+5340526ff2/python/lsst/pipe/tasks/calibrate.py", line 737, in run
    photoRes = self.photoCal.run(exposure, sourceCat=sourceCat, expId=exposureIdInfo.expId)
  File "/epyc/users/smotherh/lsst_latest/stack/miniconda3-py38_4.9.2-0.5.0/Linux64/pipe_base/21.0.0-18-g546cdbd+4e75b35dac/python/lsst/pipe/base/timer.py", line 157, in wrapper
    res = func(self, *args, **keyArgs)
  File "/epyc/users/smotherh/lsst_latest/stack/miniconda3-py38_4.9.2-0.5.0/Linux64/pipe_tasks/21.0.0-72-gffa78901+5340526ff2/python/lsst/pipe/tasks/photoCal.py", line 450, in run
    matchResults = self.match.run(sourceCat, filterLabel.bandLabel)
  File "/epyc/users/smotherh/lsst_latest/stack/miniconda3-py38_4.9.2-0.5.0/Linux64/meas_astrom/21.0.0-4-g591bb35+9e2f6ff707/python/lsst/meas/astrom/directMatch.py", line 109, in run
    refData = self.refObjLoader.loadSkyCircle(circle.center, circle.radius, filterName, epoch=epoch)
  File "/epyc/users/smotherh/lsst_latest/stack/miniconda3-py38_4.9.2-0.5.0/Linux64/meas_algorithms/21.0.0-20-g8cd22d88+eb66f50d9f/python/lsst/meas/algorithms/loadReferenceObjects.py", line 540, in loadSkyCircle
    return self.loadRegion(circularRegion, filterName=filterName, epoch=epoch)
  File "/epyc/users/smotherh/lsst_latest/stack/miniconda3-py38_4.9.2-0.5.0/Linux64/meas_algorithms/21.0.0-20-g8cd22d88+eb66f50d9f/python/lsst/meas/algorithms/loadReferenceObjects.py", line 508, in loadRegion
    fluxField = getRefFluxField(schema=expandedCat.schema, filterName=filterName)
  File "/epyc/users/smotherh/lsst_latest/stack/miniconda3-py38_4.9.2-0.5.0/Linux64/meas_algorithms/21.0.0-20-g8cd22d88+eb66f50d9f/python/lsst/meas/algorithms/loadReferenceObjects.py", line 805, in getRefFluxFiel
d
    raise RuntimeError("Could not find flux field(s) %s" % (", ".join(fluxFieldList)))
RuntimeError: Could not find flux field(s) VR_camFlux, VR_flux

Any ideas how to troubleshoot this new error?

#3

Aha, sorry I missed that you didn’t explicitly have the refcat collection listed as an input.

Hmm, something in meas_algorithms/loadReferenceObjects.py at master · lsst/meas_algorithms · GitHub doesn’t expect VR as a flux field, but I don’t see offhand what code to edit to accept it. I think @parejkoj may be the best person to ask (he should be around next week).

#4

You need a config line that defines what photometric reference filter to use with the VR filter. For example, use this config

config.photoRefObjLoader.filterMap['VR'] = 'g'

to use the PS1 g band as your VR photometric reference. I don’t actually know which filter would be most appropriate here, though; VR is a very broad band filter, so the correct photometric reference would be some synthetic combination of g and r, I think? We don’t have a way to make such synthetic filters, though we could get closer if we had colorterms for DECam (including for VR).

Note that the DECam VR filter docs explicitly state that it is ‘not an “accurate photometry” filter’, so any photometric results you get from it should be treated with caution.

1 Like
#5

Thank you, that config allows the processing to complete. Looking at the pan-starrs bands, it appears that the r band is approximately centered on the DECam VR band, but much narrower on either side. Approximate photometry is ok for our use-case at this point (ideally accurate to approx. 0.1 mag).

If we had color terms, is there a way we could include them in the processing?

#6

Here is the HSC colorterm config. You’d need a ColortermDict for ps1*, giving the refcat filters and the coefficients. See colorterms.py in pipe_tasks for how those coefficients are used.

If you do have access to such colorterms, we would love to be able to incorporate them into obs_decam.