Image Difference Task using DECam images [gen3]

Support
, ,
#61

Hello Paula,

As @mrawls mentioned we’re still doing some evaluation and commissioning studies, but if you don’t want to wait for those, the new code will certainly run. I’m not sure which pipeline you’re exactly using, but the idea is to replace the imageDifference with two new tasks: subtractImages and detectAndMeasure. If you want to do a test with the defaults, a minimally-sufficient modification to your pipeline would be something like this:

 subtractImages:
    class: lsst.ip.diffim.subtractImages.AlardLuptonSubtractTask
    config:
      connections.coaddName: parameters.coaddName
  detectAndMeasure:
    class: lsst.ip.diffim.detectAndMeasure.DetectAndMeasureTask
    config:
      connections.coaddName: parameters.coaddName

The parts of your pipeline before and after imagedifference would remain the same (note you may need to modify some contract names in that case). If you already have an imageDifference task in your pipeline, you will need to exclude that. In this example I am using parameters.coaddName as defined in the ApPipe.yaml in the ap_pipe package; your pipeline might be slightly different. If you want to run the new tasks in compatibility mode, which should give you results very close to the existing imageDifference task, you can add

forceCompatibility: True

to the config section of the subtractImages task. Let us know if you run into any difficulties.

1 Like
#62

Sorry, you might also need doSkySources: true in the config section for detectAndMeasure.

#63

Thanks Ken, I will now be trying the configs you suggest!

I have a question now specific to the code of Image Subtraction. From reading the available documentation on the implementation code of Alard and Lupton in the pipelines, I’m not clear about the treatment of scaling the flux of the template (coadd) image to match the science image.

Checking the template (R), science (I) and difference image, I see that the three of them have different calibration zero points. I understand that the template is convolved with a kernel k that minimizes the equation R Ⓧ k - I. Does the kernel account for the mismatch in flux between the template and the science image? And if it does, is the convoluted template saved by the pipeline at any point?

I assume that the photometric zero-point mismatch between the science and difference Image is due to the latter showing fainter signals.

#64

I wonder where I should add the configuration of pre-convolution of the science images with their own psf. I checked that in subtractImages.py there’s a function called finalize:

def finalize(self, template, science, difference, kernel,
                 templateMatched=True,
                 preConvMode=False,
                 preConvKernel=None,
                 spatiallyVarying=False)

I was thinking of solely changing the parameter preConvMode to True… but maybe that’s not the way to do it :full_moon_with_face:

#65

If you look at the docstring for the finalize function you’ll see that the preConvMode option is just telling the function whether the science image was pre-convolved or not; it doesn’t actually do the pre-convolution itself. The new diffim workflow doesn’t actually do it yet, though it is still in the plans.

1 Like
#66

I see… I would really like to try to pre-convolute my science images, though! to check if, by doing so, the flux dipoles for some of the sources minimize…
If there’s any mock/preliminary script that could do the pre-convolution, I would happily try it out. Maybe using the _convolveExposure function in a prior step is feasible?

#67

At the moment there is unfortunately no version of image differencing that will properly support preconvolution. If you just want to see what the images would look like with preconvolution, you could an extra convolution step as you suggest, though you would also need to measure the PSF of the convolved image (or analytically calculate the new PSF FWHM) in order to get the right basis functions for the matching kernel. Also note that this would result in an incorrect mask and variance plane, so if you try to run any of the rest of the pipeline after image subtraction you will not get reasonable results. We plan to support preconvolution with the new image differencing in the next few months.

1 Like
#68

Hi :slight_smile:

I just wanted to ask again if there is a way to obtain the convoluted template?

#69

Yes, the PSF-matched template is persisted with the name {coaddName}Diff_matchedExp, e.g. deepDiff_matchedExp.

1 Like
#70

Thanks Ian :slight_smile:
a question about the deepDiff_matchedExp, this image should correspond to the astrometrically aligned, convolved, and scaled template that matches the psf and flux counts of the science image to which it’s later subtracted, right?

#71

Yes, that’s correct. Except that the flux calibration is not automatically applied. You will still have to use the photoCalib attached to the exposure to apply the photometric calibration.

1 Like
#72

Thanks Ian :slight_smile: a question about this. So, photocalib would transform the ADU counts to nJy flux by multiplying the former to a scaling number obtained from the stars selected by the pipeline. But does photocalib account for the zero-point of the images as well? or should that be applied separately?

#73

The PhotoCalib object converts from instrumental counts to nanoJansky. The zero-point of the images is irrelevant, since the resulting flux in nJy is the actual flux on-sky, whether in the science, difference, or template image.

If you’re wondering about surface brightness images vs. fluence images, we don’t currently deal with that distinction in a coherent manner; see the discussion here for more: Photometric calibration and astrometric Jacobians