Negative peaks in calexp files

[raphaelshirley]

Hi,

Some people may have followed our debate about whether to use VISTA pipeline six exposure stacks or the ‘to some extent raw’* exposures in our development of the obs_vista package. When we pass the exposures to processCcd the resultant calexp images have large negative peaks following background subtraction (right below). I think these are being propagated to very strange negative peaks with positive fringes in the warps and subsequent coadds (left below, different region). I think the reason these are not present in the VISTA six exposure stacks is they are based on a sigma clipped mean on dithered exposures so they disappear from the mean.

One suggestion has been to find these defects using the LSST stack to create a list of defective pixels.

Is there a simpler means to for instance interpolate pixels that are highly negative following background subtraction? Or is this a more fundamental problem that needs to be addressed by going back to the original dark/flat subtraction?

I think the most thorough course of action is to enable processing of both the original VISTA pipeline stacks and the ‘to some extent raw’* exposures and to compare performance of the final photometry. At present the stacks are producing coadds that appear to be far better behaved but there are lingering concerns about measuring the PSF from a coadd rather than the exposures.

Many thanks for everyone’s ongoing support,

Raphael.

*To some extent raw in that I am currently using exposures that have had dark, flat and had the original pipeline ISR performed but can go further back to the true raw exposures if required all under the proviso that previous processing involved much work over many years and I have exactly two years of FT to produce the best possible product.

[price (Paul Price)]

It’s possible that these negative pixels are simply reflecting a feature in the calibs (bias, dark or flat) that have been applied. If that’s the case, then you can either fix and reapply the calibs, or you can treat them as defects. If they are static defects (present at the same position all the time), you should definitely mask them in your VistaIsrTask (there’s support for static masks in our ip_isr package; see obs_subaru_data for an example of static masks). If they are not static defects, you should fix them in your calibs and regenerate your “to some extent raw” (what we would call postISRCCD) exposures, because you don’t want to have to be modifying the static defect lists every time you make new calibs.

If these negative pixels are in a different place for every exposure, one way of catching them would be to run the cosmic-ray detection code on an inverted image; to do that, sub-class lsst.pipe.tasks.RepairTask, add your feature, and use config.charImage.repair.retarget(VistaRepairTask) in the processCcd configuration.

Check that the mask plane you identify these pixels with is respected in the coaddition (I think the config parameter is called badMaskPlanes), and they should vanish from your coadds. If they somehow manage to leak through and you get desperate, you can try turning on the sigma clipping in the coadd (doSigmaClip=True?), but that destroys the PSF (which is why it’s off by default) so you want to avoid that if you can. And there should be no need to resort to that since they stand out so clearly in the individual exposures.