New Tasks for Fake Source Insertion


(Sophie Reed) #1

There are now two new tasks for inserting fake sources. One for coadds (insertFakes.py) and one for single epoch images (processCcdWithFakes.py). Both tasks will look for a deepCoadd_fakeSourceCat as the standard input to draw the fakes from. I have been using one of these for the COSMOS patch downloaded from the UW sims database, the column names of which are the defaults set in the config. The required parameters are:

raColName: The column name for the RA coordinate, default: raJ2000, in radians
decColName: The column name for the Dec. coordinate, default: decJ200, in radians
diskHLR: The column that contains the disk half light radius, default: DiskHalfLightRadius, in arcsecs
bulgeHLR: The column that contains the bulge half light radius, default: BulgeHalfLightRadius, in arcsecs
magVar: The column name for the magnitude calculated taking variability into account, default: magVar (eg. imagVar), in AB mags
nDisk: The column that contains the sersic index of the disk component, default: disk_n
nBulge: The column that contains the sersic index of the bulge component, default: bulge_n
aDisk: The column that contains the semi major axis length of the disk component, default: a_d
aBulge: The column that contains the semi major axis length of the bulge component, default: a_b
bDisk: The column that contains the semi minor axis length of the disk component, default: b_d
bBulge: The column that contains the semi minor axis length of the bulge component, default: b_b
paDisk: The column that contains the PA of the disk component, default: pa_disk, in degrees
paBulge: The column that contains the PA of the bulge component, default: pa_bulge, in degrees
sourceType: If set to galaxy will use the previous config options to insert a galaxy, if set to star will add a star using the PSF from the image.

These can be specified via the command line as config options or in a config file. The config option fakeType allows the user to specify the path to a fits file that contains the above columns (fakeType=/path/to/fits/file.fits). This config option can also be used to specify whether a static (fakeType=static) or snapshot (fakeType=snapshot) catalogue is used. The snapshot catalogues for each calexp in a visit have magnitudes dependent on the MJD of the calexp they were generated for. They can be used for variability studies. The default is static, which contains only one magnitude which is independent of the date of the exposure. Stars and galaxies to be input are supplied via the same file. For stars all the parameters other than position, magnitude and sourceType can be set to anything as they will not be used.

Adding into single epoch images.

You need a directory with the outputs from singleFrameDriver and jointcal (if you want to use the improved calibrations). Then run:

> processCcdWithFakes.py /datasets/hsc/repo/ --rerun where/your/inputs/are:where/you/want/the/outputs --id visit=1228 ccd=0..8^10..103 filter=HSC-I tract=9813

Note that unlike processCcd or singleFrameDriver the id argument requires the tract key. If you want to use the calibs from jointcal then set the config option useUpdatedCalibs to True with the syntax:

> -c useUpdatedCalibs=True

This task also runs the processing steps from processCcd to produced a fakes_calexp and a fakes_src which are equivalent to the normal calexp and src but contain the fake sources and, for the calexp, the mask plane FAKE which covers the area that the image of the fake source was added to. This mask plane is propagated into two flags in the fakes_src (and subsequent coadd catalogues) base_PixelFlags_flag_fake for objects where any of their footprint overlaps the inserted fake and base_PixelFlags_flag_fakeCenter for any object whose centre pixel overlaps the fake.

Subsequent processing steps are run as normal except for the specification of config options.

skyCorrection.py: -c calexpType=fakes_calexp

For example:

> skyCorrection.py /datasets/hsc/repo --rerun where/you/put/the/output/from/processCcdWithFakes --id visit=1228 filter=HSC-I tract=9813 -c calexpType=fakes_calexp

For coaddDriver.py a config file containing the following config options is required:

config.hasFakes=True
config.makeCoaddTempExp.useMeasMosaic=False
config.detectCoaddSources.hasFakes=True
config.makeCoaddTempExp.hasFakes=True
config.assembleCoadd.hasFakes=True
config.calexpType=fakes_calexp

For example:
> coaddDriver.py /datasets/hsc/repo --rerun where/you/put/the/output/from/skyCorrection --id tract=9813 patch=2,2 filter=HSC-I --selectId visit=1228 filter=HSC-I tract=9813 -C cdConfigFakes.py

Where cdConfigFakes.py is the file containing the required config options listed above. coaddDriver produces fakes_deepCoadd_calexp when run in this way (rather than deepCoadd_calexp).

For multiBandDriver.py the options below need to be in a config file:

> config.measureCoaddSources.propagateFlags.flags={'calib_psf_candidate': 0.2, 'calib_psf_used': 0.2, 'calib_psf_reserved': 0.2, 'calib_photometry_used': 0.2, 'calib_photometry_reserved': 0.2}
> config.hasFakes=True
> config.measureCoaddSources.measurement.plugins['base_PixelFlags'].masksFpAnywhere.append('FAKE')
> config.measureCoaddSources.measurement.plugins['base_PixelFlags'].masksFpCenter.append('FAKE')
> config.forcedPhotCoadd.measurement.plugins['base_PixelFlags'].masksFpAnywhere.append('FAKE')
> config.forcedPhotCoadd.measurement.plugins['base_PixelFlags'].masksFpCenter.append('FAKE')
> config.deblendCoaddSources.hasFakes=True
> config.measureCoaddSources.hasFakes=True
> config.forcedPhotCoadd.hasFakes=True

For example:

> multiBandDriver.py /datasets/hsc/repo --rerun where/you/put/the/outputs/from/coaddDriver --id tract=9813 filter=HSC-I patch=2,2 -C mbdConfigFakes.py

Where mbdConfigFakes.py contains the config options specified above.

After these steps you should have a coadd with fakes in it and measurement performed on it. Following the examples above, this will only be for one visit in the i band.

Adding into coadds.

This requires everything up to the production of deepCoadds to have been done. deepCoadds are produced during coaddDriver.py which can be run with doDetection=False to produce these but not run all the subsequent processing steps. This is faster.

The first thing to run is:

> insertFakes.py /datasets/hsc/repo --rerun where/the/deepCoadds/are:where/you/want/the/outputs/to/go --id tract=9813 patch=2,2 filter=HSC-G^HSC-R^HSC-I^HSC-Z^HSC-Y

This adds fakes to the deepCoadds (producing fakes_deepCoadd) and has the same options listed at the top of these instructions. Inserting variable sources into coadds doesn’t make much sense so it isn’t really supported for this step. Specifying your own catalogue name still works though.

After fakes have been added, multiBandDriver can be run with doDetection=True to perform the the processing steps not done by coaddDriver previously. This will produce fakes_deepCoadd_calexp rather than the deepCoadd_calexp normally produced.

multiBandDriver needs the config options below included in a config file.

> config.measureCoaddSources.propagateFlags.flags={'calib_psf_candidate': 0.2, 'calib_psf_used': 0.2, 'calib_psf_reserved': 0.2, 'calib_photometry_used': 0.2, 'calib_photometry_reserved': 0.2}
> config.hasFakes=True
> config.measureCoaddSources.measurement.plugins['base_PixelFlags'].masksFpAnywhere.append('FAKE')
> config.measureCoaddSources.measurement.plugins['base_PixelFlags'].masksFpCenter.append('FAKE')
> config.forcedPhotCoadd.measurement.plugins['base_PixelFlags'].masksFpAnywhere.append('FAKE')
> config.forcedPhotCoadd.measurement.plugins['base_PixelFlags'].masksFpCenter.append('FAKE')
> config.deblendCoaddSources.hasFakes=True
> config.measureCoaddSources.hasFakes=True
> config.forcedPhotCoadd.hasFakes=True
> config.doDetection=True
> config.detectCoaddSources.hasFakes=True
> config.forcedPhotCoadd.measurement.plugins['base_PixelFlags'].masksFpAnywhere.append('FAKE')
> config.forcedPhotCoadd.measurement.plugins['base_PixelFlags'].masksFpCenter.append('FAKE')

For example:
> multiBandDriver.py /datasets/hsc/repo --rerun where/you/put/the/outputs/from/coaddDriver --id tract=9813 filter=HSC-I patch=2,2 -C mbdConfigFakesCoadd.py

Where the needed config options are in mbdConfigFakesCoadd.py.

Sample fakes data can be found in /datasets/hsc/repo/rerun/rivate/sreed/hscRerun/cosmosFakes/fakesAddedVisit for data with fakes added into the calexps. Please take care not o overwrite things in this repo if using it for testing purposes.


(John Parejko) #2

Would it be possible to rename insertFakes and processFakes to something that better represents what they do? Assuming I’m understanding the above text correctly, maybe something like insertFakesInCoadds and insertFakesInExposures, respectively? The existing names were not helpful in the UW group understanding the difference during today’s SourceDetection brownbag.


(John Swinbank) #3

I happened to just stumble across https://github.com/lsst/pipe_tasks/pull/301.


(Sophie Reed) #4

This has been renamed and the instructions updated.