Backward-incompatible changes to ProcessCcdTask and subtasks

Data Management
#1

DM-4692 and DM-5348 overhaul ProcessCcdTask and introduce some backwards-incompatible changes.

Code changes

  • ProcessCoaddTask is gone, along with all bin scripts that run it. Use the new Multi-Band code, instead.

  • ProcessCcdTask has been rewritten, so its config has changed. Config override files will need to be updated. This will be done for the obs_ packages as part of the merge, but if you have personal config override files then you will probably need to update them.

  • Camera-specific variants of ProcessCcdTask are gone. You will run bin/processCcdTask.py to process images for all cameras.

  • For DECam bin/processCcdTask.py will use the LSST stack’s ISR by default. To read instcal files from the community pipeline, replace the ISR task with DecamNullIsrTask by using a config override file containing the following:

    from lsst.obs.decam.decamNullIsr import DecamNullIsrTask
    config.isr.retarget(DecamNullIsrTask)

  • A new dynamic dataset type is available for adding data ID arguments to the argument parser for command-line tasks: ConfigDatasetType obtains the dataset type from a config parameter.

  • Various subtasks have changed, including:

    • New camera-specific ISR task variants for SDSS and DECam: SdssNullIsrTask and DecamNullIsrTask.
    • New task DetectAndMeasureTask detects and deblends sources and performs single-frame measurement.
    • New task CharacterizeImageTask measures PSF and aperture correction, among other things.
    • CalibrateTask has been rewritten. It now performs deep detection and measurement, astrometry and photometry.
    • Camera-specific variants of CalibrateTask are gone.
    • ProcessImageTask (formerly a base class for ProcessCcdTask and ProcessCoaddTask) is gone.

Data product changes

  • icSrc no longer includes RA/Dec coordinate data, because the fit WCS is not available when the catalog is constructed. You will have to set the coord field yourself if you need it.
  • icExp and icExpBackground can optionally be written by CharacterizeImageTask. They are so close to calexp and calexpBackground that they are not written by default.
  • icMatch is no longer being written.

Algorithm changes

  • PSF is fit somewhat differently. The new task fits the PSF in using a configurable number of iterations. By default each iteration starts with a simple Gaussian PSF whose sigma matches the PSF of the previous fit, but you can use the actual PSF each time. Using a Gaussian causes convergence in 2 iterations. Using the fit PSF slows convergence.
  • Sources in the icSrc catalog should have a more consistent minimum SNR for varying seeing. The old code detected once, using a Gaussian PSF with FWHM set by a config parameter. The new code performs detection using the PSF in the final PSF iteration.
  • The default star selector for MeasurePsfTask is objectSize rather than sizeMagnitude. The objectSize star selector is preferred and was already being specified as an override by HSC.
  • The icSrc catalog is not matched to an astrometric reference catalog unless the star selector used to measure PSF can use the matches (which is unusual).
  • The astrometric and photometric solution now use the deeper src catalog instead of the shallower icSrc catalog, though with a new SNR cutoff whose default provides a depth similar to the icSrc catalog.
  • Fake source handling is temporarily missing; it will be re-added in DM-5310.
DM Highlights Week of 2016-03-02 to 2016-03-08
Error of using obs_decam