Questions when skyCorr, scale factor too large

Yuanyu · January 15, 2026, 1:28am

Hello, recently, I am dealing with some HSC’s data, and when I do the skyCorr, and this is some of the log file:

lsst.skyCorr INFO: Detector 103: Original background restored; BG median = 2319.6 counts, BG IQR = 197.9 counts
lsst.skyCorr.maskObjects.detection INFO: Setting factor for negative detections equal to that for positive detections: 1.000000
lsst.skyCorr.maskObjects.detection INFO: Detected 2287 positive peaks in 383 footprints to 2.5 +ve and 2.5 -ve sigma
lsst.skyCorr.maskObjects.detection INFO: Setting factor for negative detections equal to that for positive detections: 1.000000
lsst.skyCorr.maskObjects.detection INFO: Detected 2259 positive peaks in 371 footprints to 2.5 +ve and 2.5 -ve sigma
lsst.skyCorr.maskObjects.detection INFO: Setting factor for negative detections equal to that for positive detections: 1.000000
lsst.skyCorr.maskObjects.detection INFO: Detected 2258 positive peaks in 370 footprints to 2.5 +ve and 2.5 -ve sigma
lsst.skyCorr INFO: Detector 104: Original background restored; BG median = 2127.4 counts, BG IQR = 10764.4 counts
lsst.skyCorr.maskObjects.detection INFO: Setting factor for negative detections equal to that for positive detections: 1.000000
lsst.skyCorr.maskObjects.detection INFO: Detected 3519 positive peaks in 390 footprints to 2.5 +ve and 2.5 -ve sigma
lsst.skyCorr.maskObjects.detection INFO: Setting factor for negative detections equal to that for positive detections: 1.000000
lsst.skyCorr.maskObjects.detection INFO: Detected 3163 positive peaks in 432 footprints to 2.5 +ve and 2.5 -ve sigma
lsst.skyCorr.maskObjects.detection INFO: Setting factor for negative detections equal to that for positive detections: 1.000000
lsst.skyCorr.maskObjects.detection INFO: Detected 3085 positive peaks in 431 footprints to 2.5 +ve and 2.5 -ve sigmalsst.skyCorr INFO: Detector 110: Original background restored; BG median = 11723.3 counts, BG IQR = 115436.8 counts
lsst.skyCorr.maskObjects.detection INFO: Setting factor for negative detections equal to that for positive detections: 1.000000
lsst.skyCorr.maskObjects.detection INFO: Detected 3805 positive peaks in 410 footprints to 2.5 +ve and 2.5 -ve sigma
lsst.skyCorr.maskObjects.detection INFO: Setting factor for negative detections equal to that for positive detections: 1.000000
lsst.skyCorr.maskObjects.detection INFO: Detected 3742 positive peaks in 403 footprints to 2.5 +ve and 2.5 -ve sigma
lsst.skyCorr.maskObjects.detection INFO: Setting factor for negative detections equal to that for positive detections: 1.000000
lsst.skyCorr.maskObjects.detection INFO: Detected 3717 positive peaks in 399 footprints to 2.5 +ve and 2.5 -ve sigma
lsst.skyCorr INFO: Detector 111: Original background restored; BG median = 5599.2 counts, BG IQR = 41039.7 counts
lsst.skyCorr INFO: Focal plane background model constructed using 122.88 x 122.88 mm (8192 x 8192 pixel) superpixels; FP BG median = 2236.5 counts, FP BG IQR = 334.9 counts
lsst.skyCorr INFO: Sky frame subtracted with a scale factor of 3457.34548
lsst.skyCorr INFO: Focal plane background model constructed using 3.84 x 3.84 mm (256 x 256 pixel) superpixels; FP BG median = -27.3 counts, FP BG IQR = 71.5 counts

This is the detector 104,

seems like the bg level of 103 is normal while 104 not, and scale factor is very large, and in your code of the SkyCorrectionTask, you use this scale for the whole focal plane,

for calExp, skyFrameBgModel, calBkg in zip(calExps, skyFrameBgModels, calBkgs):
            # subtract the scaled sky frame model from each calExp in-place,
            # also updating the calBkg list in-place
            self.sky.subtractSkyFrame(calExp.getMaskedImage(), skyFrameBgModel, scale, calBkg)
        self.log.info("Sky frame subtracted with a scale factor of %.5f", scale)
        return scale

seems like the CCDs with problem will influence other normal CCDs seriously, And my question is, is that true? And can is there are config that can get rid of this unnormal CCDs?

Another question is, after I see the calexpBackground_skyCorr_visit_mosaic, I find, the M31 will be reduced as bg, so I want to verify if it’s necessary to turn to doSky to False, I would also like to know what criteria you use to decide when to set doSky = False during data processing, aside from obvious extended sources like M31.
The bg when dealing with M31,

Thank you!

price · January 19, 2026, 10:34pm

ccd=104..111 are focus CCDs, and not used for regular imaging. You should not include them in your processing.

As for whether or not to use doSky=False, that’s a question that very much depends on your science goals.

lskelvin · January 20, 2026, 1:54pm

Adding to @price’s comments above, I’d also note that use of SkyCorrectionTask on something like M31 may not provide optimal outputs. M31 is vast, and spans the entire field of view. I expect that each background mesh element will be significantly biased by true flux from the galaxy, which is why you’re finding that the background “model” looks somewhat like M31 itself.

You may want to experiment with turning off bgModel2 with doBgModel2=False, and increasing the xSize and ySize of bgModel1 to something larger than the standard HSC default of 8k pixels on a side. It might be worth considering a lower-order polynomial fit to the background too, to prevent high-frequency signal from entering your background model. Even with these edits however, reduction of M31 may prove tricky, and might require some bespoke background modelling in order to progress.