I’m attempting to use LSST Science Pipeline configure for HSC data to create an image template. I have good success to the point of making the stacked image.
When using the assembleCoadd.py tasks I get a coadded image that clearly hasn’t clipped outliers.
Here is my command: assembleCoadd.py basedir --rerun 03093_coadd:plantOutputs --selectId filter=HSC-R2 visit=220416^220414^220412^220410^220406^220392^220390^220388^220386^220384^220382^220380^220378^220376^220374^220364^220362^220360^220358^220356^220354^220352^220350^220348^220344^220342^220340^220334^220332^220330^220328^220326^220324^220322^220320^220318^220314^220302^220300^220298^220296^220294^220292^220290^220288^220286^220284^220282^220280^220278^220276^220274^220272^220270^220268^220266^220264^220262^220260 --id filter=HSC-R2 tract=0 patch=4,4
but the coadded image has satelite streaks which I expected the safeClipAssembleCoadd task would clip out.
Clearly I’m missing something about how this task works…
The safe clipping attempts to minimise unnecessary clipping in order to preserve the PSF in the coadd (because clipping destroys the coadd PSF). I believe one way it attempts to minimise clipping is by checking if the footprint of the pixels to be clipped overlaps a footprint of something static, which might indicate that the candidate artifact is actually a subtraction residual and not a real artifact. This is more likely to be triggered when dealing with satellite streaks (since they can go through many objects), so they can be tricky to remove. This is even worse if the stellar density is high.
My first suggestion would be to play with the coadd parameters. I might be able to offer more advice if you share images of the field and your reduction goals.
As an aside, you may want to consider migrating to Gen3 middleware now. We have effectively abandoned gen2 and v23 will be the final release to include it.
You can use the same clipping algorithm used in HSC PDR2 and PDR3 (https://dmtn-080.lsst.io/) , using the gen2 commandline script by adding a parameter:
Thanks for the pointers on HSC PDR2 / 3 process. I will have a look.
I am looking at Gen3 but can’t switch today as we are nearly complete in our data analysis (just doing some cross-checks of some referee comments) and I don’t want to redo too much.
@yusra Is --warpCompare option to assembleCoadd.py part of the LSST Pipeline release or only in the HSCPipe? If in the LSST Pipeline, which version? I’m running LSST V19 and don’t see the --warpCompare option.
the option looks very powerful… thanks for tell me about this. I hope to be able to use.