Using the base_Sdss_shape measurements - errors missing?

richardxdubois · August 7, 2020, 6:31pm

In analyzing spot projector data, we’re using CharacterizeImageTask to do the source finding. I’d like to get the errors on the centroids (base_Sdss_shape_x, _y), but am finding that columns for base_Sdss_shape_xErr and _yErr are absent. Here is the code used (by Adam Snyder, in fact) to find the spots:

github.com

lsst-camera-dh/eotest/blob/master/python/lsst/eotest/sensor/spotTask.py

"""
@brief Source identification and characterization of
spot images.
"""
from __future__ import print_function
from __future__ import absolute_import
import os
import numpy as np
from astropy.io import fits

import lsst.eotest.image_utils as imutils
import lsst.afw.image as afwImage
import lsst.pex.config as pexConfig
import lsst.pipe.base as pipeBase
import lsst.pipe.tasks
from lsst.pipe.tasks.characterizeImage import CharacterizeImageTask, CharacterizeImageConfig
from lsst.pipe.tasks.calibrate import CalibrateTask, CalibrateConfig
import lsst.meas.extensions.shapeHSM

from .MaskedCCD import MaskedCCD

This file has been truncated. show original

I was thinking I could make do with the moments, _xx and _yy, and then use InstrFlux to estimate the centroid error. I imagine there must be some scale factor needed on InstrFlux to properly get the centroid error.

So, two questions - should we have expected the centroid errors to be produced; and if not, how to get them.

Thanks!

Richard

jbosch · August 9, 2020, 9:55pm

Computing a complete uncertain covariance matrix in SdssShape has been a known issue since we inherited from the code from SDSS before the start of construction; it’s just never been a high priority.

That’s mostly because we don’t use the centroid that comes out of SdssShape much; it’s mostly there as a diagnostic - the centroid the second moments were computed about. If you want a real centroid (with errors), use SdssCentroid instead.

In fact, you’re probably best off looking at the slot_Centroid_* fields, rather than depending explicitly on SdssCentroid - those are aliases that will always point to what we consider the best centroid measurement for that image (which is currently SdssCentroid in all single-epoch processing, but may not always be).

richardxdubois · August 9, 2020, 10:24pm

Thanks, Jim. I do see SdssCentroid columns for x and y; the xErr and yErr columns are present, but are all zeroes. There are no moments or flux columns. Is there a setting to enable them?

Is there a writeup somewhere describing these various measurement types?

Richard

jbosch · August 10, 2020, 1:07am

I think this probably means that your script is not setting the variance plane of the image before passing it off to CharacterizeImageTask. In our pipelines that’s the responsibility of IsrTask; if you’re not running that because you’ve got your own variant, you’ll need to add something that sets the variance according to the gains and pixel values.

Different algorithms are responsible for different measurements. For second moments, you’ll probably want to continue to use SdssShape instead of SdssCentroid (or use the slot_Shape alias).

For photometry, it depends on what kind of photometry. base_PsfFlux and base_CircularApertureFlux are probably the simplest choices in that category. I think all of these should be included in the default configuration of CharacterizeImageTask; they’re definitely included in the default configuration of CalibrateTask, so a config dump of that could be used to show a relatively complete set. The syntax for enabling algorithms is something like this:

# Assumes 'config' is an instance of CharacterizeImageTask.ConfigClass.
# The .names attribute behaves like a Python set.
config.measurement.names |= {"base_SdssShape", "base_PsfFlux"}

I’m afraid we don’t have any high-level docs that covers all algorithms. And the docs for individual algorithms are at best inconsistent in quality. Asking here is still probably the best way to get (and effectively add to) documentation.