Cosmic ray detection issue in dense fields

Support
#1

I’m running the characterizeImage task on data from DECam using version w.2022.06 of the pipelines. The images are of very dense fields in the galactic plane. I am finding that for some of the visits and detectors there are a very large number of cosmic rays being detected and repaired. Typically, this causes the pipeline to fail as the number of cosmic ray pixels exceeds the obs_decam default of 100,000:

Execution of task 'characterizeImage' on quantum {instrument: 'DECam', detector: 4, visit: 975920, ...} failed. Exception LengthError: 
  File "src/CR.cc", line 724, in std::vector<std::shared_ptr<lsst::afw::detection::Footprint> > lsst::meas::algorithms::findCosmicRays(MaskedImageT&, const lsst::afw::detection::Psf&, double, const lsst::daf::base::PropertySet&, bool) [with MaskedImageT = lsst::afw::image::MaskedImage<float>]
    Too many CR pixels (max 100000) {0}
lsst::pex::exceptions::LengthError: 'Too many CR pixels (max 100000)'

I’ve tried increasing this number to get the processing to succeed, but find that I have to increase it so far as to make some of the data unusable or suspect: for example, 1,000,000 is ~12% of the detector.

I’m finding that this effect is variable with visit and detector number, which makes me think it is due to an observational effect. To explore this effect, I’ve plotted the same detector from 2 visits of the same field with cosmic ray masks overplotted:

cr
cr1566×1170 273 KB

Sorry if they are not very visible with color, size, and compression, here is a full resolution image: https://epyc.astro.washington.edu/~stevengs/cr.png

What I find is that the cosmic rays seem to trace the background i.e. where the density of sources drops. I also find that the number of cosmic rays detected (9794 and 22488) are much, much higher than what I’d expect (no surprise given the error). For a less dense field and similar exposure time, the CR repair process finds 20-120 cosmic rays.

This effect is quite common, causing processing failures in ~18% of the detectors from visits to the dense field. So my solution is to turn off cosmic ray detection and repair when processing dense fields. But I am posting here in case anyone has insight about what causes this, potential fixes, and expected behavior when processing dense fields.

#2

Section 3.1 of DMTN-171 discusses the issue of cosmic ray detection in crowded fields and proposes modifying configurables like the the detection thresholds for sources (for background measurement and PSF modelling) and cosmic rays. However it seems that the nCrPixelMax is set high.

The final component that required modification is the cosmic ray detection and repair algorithm. As noted above, the assumptions behind the pixel value statistics are incorrect in crowded fields. We set the detection thresholds to the same values as for source detection ( Table 3 ), and while this works in most cases, for just under 1% of the exposures processCcd.py fails with a fatal error. In these cases the failure appears to be due to every pixel in the image being identified as a cosmic ray. This failure suggests that our cosmic ray detection algorithm needs improvement and should be investigated further, but because of the low number of exposures affected we simply increased the number of pixels required to trigger the failure. This does not solve the problem, but it allows us to continue processing these exposures to make sure that there are no additional problems

Seems to concur with what I’ve reported, although with disparity in number of CCD failures, 1% vs my 18%. Although I haven’t changed charImage.repair.cosmicray.min_DN or upped charImage.repair.cosmicray.nCrPixelMax to 10,000,000 per that report, which may reduce the failure rate to 1%.

#3

Hi @stevenstetzler , I was wondering if these DECam images of crowded fields are from our DDF program, because some of the exposure times we use (96 and 50 seconds in g and r bands) are much longer than others (30 seconds in i and z bands) – and the latter is closer to what LSST would obtain. So if the issue with CRs and how characterizeImage task fails at times might be because that number of CRs is just not expected for LSST images?