PSF matching failure during image difference task

Support
#1

I am running into an error when running the ImageDifferenceTask. There seems to be an issue with solving for the PSF-matching kernel when performing the template subtraction. Here are one of the logs for a failing detector/visit. These are DECam images of the galactic bulge. Processing with version w.2022.21 of the pipelines:

INFO 2022-05-29T03:59:02.345-07:00 lsst.imageDifference (imageDifference:{instrument: 'DECam', skymap: 'discrete_decaps_east', detector: 44, visit: 1000790, ...})(imageDifference.py:1406) - Processing {instrument: 'DECam', skymap: 'discrete_decaps_east', detector: 44, visit: 1000790, ...}
INFO 2022-05-29T03:59:02.381-07:00 lsst.imageDifference (imageDifference:{instrument: 'DECam', skymap: 'discrete_decaps_east', detector: 44, visit: 1000790, ...})(imageDifference.py:1338) - template has 8285283 good pixels (97.3%)
INFO 2022-05-29T03:59:02.465-07:00 lsst.imageDifference (imageDifference:{instrument: 'DECam', skymap: 'discrete_decaps_east', detector: 44, visit: 1000790, ...})(imageDifference.py:999) - Subtracting images
INFO 2022-05-29T03:59:02.467-07:00 lsst.imageDifference.subtract (imageDifference:{instrument: 'DECam', skymap: 'discrete_decaps_east', detector: 44, visit: 1000790, ...})(imagePsfMatch.py:995) - Template Wcs : 4.740502,-0.518401 -> 4.746541,-0.521015
INFO 2022-05-29T03:59:02.467-07:00 lsst.imageDifference.subtract (imageDifference:{instrument: 'DECam', skymap: 'discrete_decaps_east', detector: 44, visit: 1000790, ...})(imagePsfMatch.py:998) - Science Wcs : 4.740517,-0.518414 -> 4.746527,-0.521003
INFO 2022-05-29T03:59:02.468-07:00 lsst.imageDifference.subtract (imageDifference:{instrument: 'DECam', skymap: 'discrete_decaps_east', detector: 44, visit: 1000790, ...})(imagePsfMatch.py:410) - Astrometrically registering template to science image
INFO 2022-05-29T03:59:05.579-07:00 lsst.imageDifference.subtract (imageDifference:{instrument: 'DECam', skymap: 'discrete_decaps_east', detector: 44, visit: 1000790, ...})(imagePsfMatch.py:429) - templateFwhmPix: 3.129928335002518
INFO 2022-05-29T03:59:05.582-07:00 lsst.imageDifference.subtract (imageDifference:{instrument: 'DECam', skymap: 'discrete_decaps_east', detector: 44, visit: 1000790, ...})(imagePsfMatch.py:436) - scienceFwhmPix: 2.9629587680777902
INFO 2022-05-29T03:59:05.584-07:00 lsst.imageDifference.subtract (imageDifference:{instrument: 'DECam', skymap: 'discrete_decaps_east', detector: 44, visit: 1000790, ...})(imagePsfMatch.py:915) - Target psf fwhm is the greater, deconvolution mode
INFO 2022-05-29T03:59:07.782-07:00 lsst.imageDifference.subtract.selectDetection (imageDifference:{instrument: 'DECam', skymap: 'discrete_decaps_east', detector: 44, visit: 1000790, ...})(detection.py:586) - Detected 13533 positive peaks in 6661 footprints to 10 sigma
INFO 2022-05-29T03:59:08.874-07:00 lsst.imageDifference.subtract.selectMeasurement (imageDifference:{instrument: 'DECam', skymap: 'discrete_decaps_east', detector: 44, visit: 1000790, ...})(sfm.py:298) - Measuring 6661 sources (6661 parents, 0 children) 
INFO 2022-05-29T03:59:19.498-07:00 lsst.imageDifference.subtract (imageDifference:{instrument: 'DECam', skymap: 'discrete_decaps_east', detector: 44, visit: 1000790, ...})(diffimTools.py:426) - Growing 6661 kernel candidate stars by 21 pixels
INFO 2022-05-29T03:59:20.095-07:00 lsst.imageDifference.subtract (imageDifference:{instrument: 'DECam', skymap: 'discrete_decaps_east', detector: 44, visit: 1000790, ...})(diffimTools.py:472) - Selected 3707 / 6661 sources for KernelCandidacy
INFO 2022-05-29T03:59:20.409-07:00 lsst.imageDifference.subtract (imageDifference:{instrument: 'DECam', skymap: 'discrete_decaps_east', detector: 44, visit: 1000790, ...})(imagePsfMatch.py:545) - Matching Psf FWHM 3.13 -> 2.96 pix
INFO 2022-05-29T03:59:20.411-07:00 lsst.imageDifference.subtract (imageDifference:{instrument: 'DECam', skymap: 'discrete_decaps_east', detector: 44, visit: 1000790, ...})(imagePsfMatch.py:915) - Target psf fwhm is the greater, deconvolution mode
ERROR 2022-05-29T04:00:01.875-07:00 lsst.imageDifference.subtract (imageDifference:{instrument: 'DECam', skymap: 'discrete_decaps_east', detector: 44, visit: 1000790, ...})(psfMatch.py:1035) - ERROR: Unable to calculate psf matching kernel
ERROR 2022-05-29T04:00:01.877-07:00 lsst.ctrl.mpexec.singleQuantumExecutor (imageDifference:{instrument: 'DECam', skymap: 'discrete_decaps_east', detector: 44, visit: 1000790, ...})(singleQuantumExecutor.py:214) - Execution of task 'imageDifference' on quantum {instrument: 'DECam', skymap: 'discrete_decaps_east', detector: 44, visit: 1000790, ...} failed. Exception Exception: 
  File "src/KernelSumVisitor.cc", line 129, in void lsst::ip::diffim::detail::KernelSumVisitor<PixelT>::processKsumDistribution() [with PixelT = float]
    Unable to determine kernel sum; 0 candidates {0}
lsst::pex::exceptions::Exception: 'Unable to determine kernel sum; 0 candidates'

I’m wondering if anyone has tips for debugging this issue.

Thanks,
Steven

#2

This sounds like the PSF determiner for the matching kernel was not able to find enough isolated and unflagged stars. Check DMTN-171: Fall 2020 status of crowded field processing with the LSST Alert Production Pipelines for some settings that might help, and failing that you can try greatly increasing the detection threshold used in image differencing. The config parameter of ImageDifferenceTask to modify would be config.subtract['al'].selectDetection.thresholdValue.

#3

Thanks, Ian, I will take a look.

#4

Hi @stevenstetzler, just doing some late follow-up here. Did Ian’s suggestion help? Could we mark Ian’s response as the solution for this Topic?