Question about base_SdssShape_instFlux_yy_Cov in the pipeline source catalog

sfu · March 26, 2024, 8:55pm

Hi, I’m working on real/bogus classification, and I’m testing different quantities in the LSST pipeline source catalog. I find that the performance of a quantity called “base_SdssShape_instFlux_yy_Cov” seems to be quite difference from “base_SdssShape_instFlux_xx_Cov” and “base_SdssShape_instFlux_xy_Cov”. It is a quantity in both the src catalog of a direct image, or the diaSrc catalog of a difference image.
The information in the FITS table header is like this.

TTYPE54 = 'base_SdssShape_instFlux_yy_Cov' / uncertainty covariance between base                                               
TFORM54 = '1E      '           / format of field    
TDOC54  = 'uncertainty covariance between base_SdssShape_instFlux and base_Sds&'
CONTINUE  'sShape_yy'    
TUNIT54 = 'count*pixel^2'    
TCCLS54 = 'Scalar  '           / Field template used by lsst.afw.table

I notice that bright transients usually have large values at “base_SdssShape_instFlux_xx_Cov” and “base_SdssShape_instFlux_xy_Cov” but small values at “base_SdssShape_instFlux_yy_Cov”. However, in theory covariances should not be strongly related to direction.
How does the pipeline compute those SDSS shape flux - 2nd moment covariances? I was not able to find much information about that.
Any suggestions will be appreciated.

parejkoj · March 26, 2024, 11:20pm

All of the documentation about SdssShape is here. Unforunately, there’s not much to go on.

http://doxygen.lsst.codes/stack/doxygen/x_mainDoxyDoc/classlsst_1_1meas_1_1base_1_1_sdss_shape_algorithm.html#details

This is the C++ code for the algorithm, most of which goes back to ~2014.

github.com

lsst/meas_base/blob/main/src/SdssShape.cc

// -*- lsst-c++ -*-
/*
 * LSST Data Management System
 * Copyright 2008-2016 AURA/LSST.
 *
 * This product includes software developed by the
 * LSST Project (http://www.lsst.org/).
 *
 * This program is free software: you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation, either version 3 of the License, or
 * (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 *
 * You should have received a copy of the LSST License Statement and
 * the GNU General Public License along with this program.  If not,

This file has been truncated. show original

Note that we are moving away from SdssShape to HsmShape. I’ve asked about the documentation for that (it was just rewritten in python). That package is here:

parejkoj · March 26, 2024, 11:34pm

Thinking about this some more: what do you mean by “the performance of”, in your question? Because I could believe that the xx and yy shape covariances do encode information about which sources are bogus. y is the column direction, so bad columns and bleed trails would probably have quite different yy_Cov than less pathological sources.

sfu · March 27, 2024, 12:48am

Thank you for pointing me to those webpages. The SdssShape.cc page shows those flux - moments covariances are derived from a 4D Fisher matrix. In my question, “behavior” could be a better word than “performance”. What I found is that bright sources usually had large Flux_xx_Cov and Flux_xy_Cov values, but small/random Flux_yy_Cov values, and I’m trying to understand why. Thanks for pointing out the issues of bad columns and bleed trails – I think that’s probably the reason.