Question about adding cuts on the sources before matching them with a reference catalog

Data Management
#1

Dear all,

For the astrometric/photometric calibration in DM processCcd , I’m trying to make some cuts on the sources detected by DM before they are matched with an external reference catalog.

I notice that there are several terms in the config file:

config.calibrate.astrometry.sourceSelector[‘science’]
config.calibrate.astrometry.sourceSelector[‘references’]
config.calibrate.astrometry.sourceSelector[‘objectSize’]
config.calibrate.astrometry.sourceSelector[‘astrometry’]
config.calibrate.astrometry.sourceSelector[‘matcher’]
config.calibrate.astrometry.referenceSelector

It seems they all include flux cut or S/N cut. Can I ask the difference among them? Which one(s) should I use to clean the sources (detected by DM rather than the objects in the external reference catalog)? Or what document should I refer to?

Thank you.

Best,
Shenming

#2

Hi Shenming,

Yes, I added the ability to select on S/N and/or flux for various star selectors. Please have a look at DM-17043 for details.

As for the various fields, each task can set its own sourceSelector algorithm based on the specific needs for the given task, e.g. for measurePsf, the default is set to objectSize here.

They can also be overridden in configs as in obs_subaru for measureApCorr here.
and measurePsf here.

If you want to have a closer look into the various types of selectors (i.e. what values they consider in their selection process), have a look at https://github.com/lsst/meas_algorithms/blob/master/python/lsst/meas/algorithms/sourceSelector.py. E.g. the ScienceSourceSelector is defined here and the ReferenceSourceSelector is defined here).

Also note that in the persisted config files, you may see multiple entries for different algorithms for a given task, but it’s only the settings for the algorithm set in [taskName].sourceSelector.name that gets effected. As an example, the latest RC2 HSC run has:

$ grep Selector /datasets/hsc/repo/rerun/RC/w_2020_14/DM-24359-sfm/config/singleFrameDriver.py | grep name
config.processCcd.charImage.measureApCorr.sourceSelector['science'].unresolved.name='base_ClassificationExtendedness_value'
config.processCcd.charImage.measureApCorr.sourceSelector['references'].unresolved.name='base_ClassificationExtendedness_value'
config.processCcd.charImage.measureApCorr.sourceSelector.name='science'
config.processCcd.charImage.ref_match.sourceSelector['science'].unresolved.name='base_ClassificationExtendedness_value'
config.processCcd.charImage.ref_match.sourceSelector['references'].unresolved.name='base_ClassificationExtendedness_value'
config.processCcd.charImage.ref_match.sourceSelector.name='matcher'
config.processCcd.charImage.ref_match.referenceSelector.unresolved.name='base_ClassificationExtendedness_value'
config.processCcd.charImage.measurePsf.starSelector['science'].unresolved.name='base_ClassificationExtendedness_value'
config.processCcd.charImage.measurePsf.starSelector['references'].unresolved.name='base_ClassificationExtendedness_value'
config.processCcd.charImage.measurePsf.starSelector.name='objectSize'
config.processCcd.calibrate.astrometry.sourceSelector['science'].unresolved.name='base_ClassificationExtendedness_value'
config.processCcd.calibrate.astrometry.sourceSelector['references'].unresolved.name='base_ClassificationExtendedness_value'
config.processCcd.calibrate.astrometry.sourceSelector.name='matcher'
config.processCcd.calibrate.astrometry.referenceSelector.unresolved.name='base_ClassificationExtendedness_value'

so, despite there being entries for algorithms other than matcher for config.processCcd.calibrate.astrometry.sourceSelector[] (e.g. science, references, objectSize, etc.), it is only these for the “named” matcher that apply:

config.processCcd.calibrate.astrometry.sourceSelector['matcher'].sourceFluxType='Psf'
config.processCcd.calibrate.astrometry.sourceSelector['matcher'].minSnr=40.0
config.processCcd.calibrate.astrometry.sourceSelector['matcher'].excludePixelFlags=True

Hope this helps (as opposed to causing further confusion!)

2 Likes
#3

You might also notice some redundancy in the source selectors. This is basically because of the issue best exemplified by https://xkcd.com/927/. However, there are some notable differences in that, for example, some selectors (like the psf selector) can’t select properly unresolved objects (since we don’t have a psf model yet!) and must rely on cruder object size to start.

1 Like