There are now two new tasks for inserting fake sources. One for coadds (insertFakes.py) and one for single epoch images (processCcdWithFakes.py). Both tasks will look for a deepCoadd_fakeSourceCat as the standard input to draw the fakes from. I have been using one of these for the COSMOS patch downloaded from the UW sims database, the column names of which are the defaults set in the config. The required parameters are:
raColName: The column name for the RA coordinate, default: raJ2000, in radians
decColName: The column name for the Dec. coordinate, default: decJ200, in radians
diskHLR: The column that contains the disk half light radius, default: DiskHalfLightRadius, in arcsecs
bulgeHLR: The column that contains the bulge half light radius, default: BulgeHalfLightRadius, in arcsecs
magVar: The column name for the magnitude calculated taking variability into account, default: magVar (eg. imagVar), in AB mags
nDisk: The column that contains the sersic index of the disk component, default: disk_n
nBulge: The column that contains the sersic index of the bulge component, default: bulge_n
aDisk: The column that contains the semi major axis length of the disk component, default: a_d
aBulge: The column that contains the semi major axis length of the bulge component, default: a_b
bDisk: The column that contains the semi minor axis length of the disk component, default: b_d
bBulge: The column that contains the semi minor axis length of the bulge component, default: b_b
paDisk: The column that contains the PA of the disk component, default: pa_disk, in degrees
paBulge: The column that contains the PA of the bulge component, default: pa_bulge, in degrees
sourceType: If set to galaxy will use the previous config options to insert a galaxy, if set to star will add a star using the PSF from the image. Update: This can be a string/bytes of “star”/“galaxy” or 0 (star)/ 1 galaxy.
These can be specified via the command line as config options or in a config file. The config option fakeType allows the user to specify the path to a fits file that contains the above columns (fakeType=/path/to/fits/file.fits). This config option can also be used to specify whether a static (fakeType=static) or snapshot (fakeType=snapshot) catalogue is used. The snapshot catalogues for each calexp in a visit have magnitudes dependent on the MJD of the calexp they were generated for. They can be used for variability studies. The default is static, which contains only one magnitude which is independent of the date of the exposure. Stars and galaxies to be input are supplied via the same file. For stars all the parameters other than position, magnitude and sourceType can be set to anything as they will not be used.
The additions of galaxies is done using galsim as such it inherits so restrictions from galsim. There are restrictions on the minimum and maximum Sersic index that can be used; 0.3 <= nBulge/nDisk <= 6.2.
Adding into single epoch images.
You need a directory with the outputs from singleFrameDriver and jointcal/fgcm (if you want to use the improved calibrations). Then run:
> processCcdWithFakes.py /datasets/hsc/repo/ --rerun where/your/inputs/are:where/you/want/the/outputs --id visit=1228 ccd=0..8^10..103 filter=HSC-I tract=9813
Note that unlike processCcd or singleFrameDriver the id argument requires the tract key.
If you want to use any of the external calibration options (jointcal or fgcm for photometry and jointcal for astrometry) then you can do that by setting some config options. If you want to use jointcal instead then replace “fgcm” in the example below with “jointcal”.
config.makeCoaddTempExp.doApplyExternalPhotoCalib=True
config.assembleCoadd.externalPhotoCalibName="fgcm"
config.assembleCoadd.assembleStaticSkyModel.doApplyExternalPhotoCalib=True
config.assembleCoadd.assembleStaticSkyModel.externalPhotoCalibName="fgcm"
This task also runs the processing steps from processCcd to produced a fakes_calexp and a fakes_src which are equivalent to the normal calexp and src but contain the fake sources and, for the calexp, the mask plane FAKE which covers the area that the image of the fake source was added to. This mask plane is propagated into two flags in the fakes_src (and subsequent coadd catalogues) base_PixelFlags_flag_fake for objects where any of their footprint overlaps the inserted fake and base_PixelFlags_flag_fakeCenter for any object whose centre pixel overlaps the fake.
There is also a driver script to run this that works like singleFrameDriver; processCcdWithFakesDriver. It takes the same arguments and configs as processCcdWithFakes.
Subsequent processing steps are run as normal except for the specification of config options.
skyCorrection.py: -c calexpType=fakes_calexp
For example:
> skyCorrection.py /datasets/hsc/repo --rerun where/you/put/the/output/from/processCcdWithFakes --id visit=1228 filter=HSC-I tract=9813 -c calexpType=fakes_calexp
For coaddDriver.py a config file containing the following config options is required:
config.hasFakes=True
config.makeCoaddTempExp.useMeasMosaic=False
config.detectCoaddSources.hasFakes=True
config.makeCoaddTempExp.hasFakes=True
config.assembleCoadd.hasFakes=True
config.calexpType=fakes_calexp
For example:
> coaddDriver.py /datasets/hsc/repo --rerun where/you/put/the/output/from/skyCorrection --id tract=9813 patch=2,2 filter=HSC-I --selectId visit=1228 filter=HSC-I tract=9813 -C cdConfigFakes.py
Where cdConfigFakes.py is the file containing the required config options listed above. coaddDriver produces fakes_deepCoadd_calexp when run in this way (rather than deepCoadd_calexp).
For multiBandDriver.py the options below need to be in a config file:
config.hasFakes=True
config.measureCoaddSources.measurement.plugins['base_PixelFlags'].masksFpAnywhere.append('FAKE')
config.measureCoaddSources.measurement.plugins['base_PixelFlags'].masksFpCenter.append('FAKE')
config.forcedPhotCoadd.measurement.plugins['base_PixelFlags'].masksFpAnywhere.append('FAKE')
config.forcedPhotCoadd.measurement.plugins['base_PixelFlags'].masksFpCenter.append('FAKE')
config.deblendCoaddSources.hasFakes=True
config.measureCoaddSources.hasFakes=True
config.forcedPhotCoadd.hasFakes=True
For example:
> multiBandDriver.py /datasets/hsc/repo --rerun where/you/put/the/outputs/from/coaddDriver --id tract=9813 filter=HSC-I patch=2,2 -C mbdConfigFakes.py
Where mbdConfigFakes.py contains the config options specified above.
After these steps you should have a coadd with fakes in it and measurement performed on it. Following the examples above, this will only be for one visit in the i band.
Adding into coadds.
This requires everything up to the production of deepCoadds to have been done. deepCoadds are produced during coaddDriver.py which can be run with doDetection=False to produce these but not run all the subsequent processing steps. This is faster.
The first thing to run is:
> insertFakes.py /datasets/hsc/repo --rerun where/the/deepCoadds/are:where/you/want/the/outputs/to/go --id tract=9813 patch=2,2 filter=HSC-G^HSC-R^HSC-I^HSC-Z^HSC-Y
This adds fakes to the deepCoadds (producing fakes_deepCoadd) and has the same options listed at the top of these instructions. Inserting variable sources into coadds doesn’t make much sense so it isn’t really supported for this step. Specifying your own catalogue name still works though.
After fakes have been added, multiBandDriver can be run with doDetection=True to perform the the processing steps not done by coaddDriver previously. This will produce fakes_deepCoadd_calexp rather than the deepCoadd_calexp normally produced.
multiBandDriver needs the config options below included in a config file.
config.measureCoaddSources.propagateFlags.flags={'calib_psf_candidate': 0.2, 'calib_psf_used': 0.2, 'calib_psf_reserved': 0.2, 'calib_photometry_used': 0.2, 'calib_photometry_reserved': 0.2}
config.hasFakes=True
config.measureCoaddSources.measurement.plugins['base_PixelFlags'].masksFpAnywhere.append('FAKE')
config.measureCoaddSources.measurement.plugins['base_PixelFlags'].masksFpCenter.append('FAKE')
config.forcedPhotCoadd.measurement.plugins['base_PixelFlags'].masksFpAnywhere.append('FAKE')
config.forcedPhotCoadd.measurement.plugins['base_PixelFlags'].masksFpCenter.append('FAKE')
config.deblendCoaddSources.hasFakes=True
config.measureCoaddSources.hasFakes=True
config.forcedPhotCoadd.hasFakes=True
config.doDetection=True
config.detectCoaddSources.hasFakes=True
For example:
> multiBandDriver.py /datasets/hsc/repo --rerun where/you/put/the/outputs/from/coaddDriver --id tract=9813 filter=HSC-I patch=2,2 -C mbdConfigFakesCoadd.py
Where the needed config options are in mbdConfigFakesCoadd.py.
Adding Sources Selectively
If you wish to add a subselection of the sources from your given catalog then you can do so by specifying the config option sourceSelectionColName. For example:
-c fakeType="/my/favourite/source/catalogue/full/of/things.fits" -c sourceSelectionColName="fakeQuasars"
This will only add the objects that are set to True in this column in the input catalogue.
Adding Images Directly
If you want to add images directly then you need to set the config options:
-c insertImages = True and -c fakeType="/catalogue/with/image/details.fits".
The catalogue given needs to have the following columns:
band + imFilename: The filename of the image to be inserted at the specified position, e.g. gimFilename
sourceType: Whether it is a star or galaxy
band + magVar: The column name for the magnitude calculated taking variability into account, e.g. imagVar, in AB mags
raJ2000: The R.A to insert the centre of the image at.
decJ2000: The declination to insert the centre of the image at.
This will add the images specified at the locations specified by the ra and dec given. At the moment you cannot add both images and sources from a catalogue to the same image.
Sample Data
Sample fakes data can be found in /datasets/hsc/repo/rerun/rivate/sreed/hscRerun/cosmosFakes/fakesAddedVisit for data with fakes added into the calexps. Please take care not to overwrite things in this repo if using it for testing purposes.