Hello,
I would like to know if DM Science Pipelines Design Document is up to date, in particular, the photometric calibration plan. Is it still anticipated that the Data Release Production will be based on photometric calibration with Gaia with FGCM in place for Quality Assurance? I ask because that’s the impression I get from skimming the text contained in the DM Science Pipelines Design doc; however, the Algorithms workshop in 2020 (same year as publication) made it seemed that the DRP would be based on FGCM. Thanks!
I think it is fair to say that we have certainly shifted towards relying more on FGCM. Using Gaia to tie together FGCM runs that cover large areas - but not the full survey - is perhaps still in the picture, but I think it’s much more likely for DR1 that we will run FGCM on the full survey.
A lot of how we will approach photometric calibration will depend on the degree to which calibration hardware like the Collimated Beam Projector and LATISS work when they are integrated with the rest of the system; we don’t have any precursor survey with anything like that to help us practice integrating that kind of data at present.
Thanks for the info. I now realise there will be lots of different types of raw calibration products: bias, darks, and dome flats for both LSSTCam and LATISS, along with star flats, and CBP data. Will the DRs include master calibration frames? Alternatively, would the ISR data products be available in DR1?
I’m not sure what the policy is for how readily they will be available (some of our image data products will available on a regenerate-as-needed basis only), but they will certainly be included at some level.
Unless there’s something very unexpected with the stability of the system, we expect to have all derived calibration data available on disk (or equivalent) via the Data Butler and other access APIs. Raw calibration data will also be available.
Hello,
I have a question regarding the procedures for the astrometric calibration described in this document versus the latest implementation with DM Science pipeline. On page 135, Section 6.13.2 Single Visit describes performing full visit astrometric fitting. In the case that a user would like to do a full visit astrometric fit, how can that be specified with the current set of pipetasks during the Single Visit Processing?
We don’t have any tasks that do full-visit astrometry; it’s something we’ve long considered doing, but since we do run multi-visit astrometry a few steps later, it’s never become important enough to work on.
Hi all 
I also have some questions regarding the data products and algorithm processes of the LSST software, specifically the estimation of the photometric zero-point for calibrated exposures (calexp) and difference-image products (diffexp).
I know for sure that for calexp the photoCal subtask finds ngood stars to measure the magnitude corresponding to one count in the image, and this is registered on the header as MAGZERO_NOBJ and MAGZERO, respectively. I wonder if there’s a more thorough description that explains step by step how this is internally calculated? I’ve read some of the documents provided by DM, but haven’t found one detailed enough u.u. Maybe there is a reference on which this process is based that can be enlightening.
diffexp also has these parameters In the header, I assume the photoCal subtask is used once again since the MAGZERO of this product differs from its science calexp
Thanks
See this Community post for information about the PhotoCalib object, which is our recommended interface for our photometric calibrations, which may be spatially varying. The MAGZERO header is provided for compatibility with the HSC pipeline.
1 Like
Thanks John for the references
I’ve been checking the difference-image images that I’ve produced with the pipeline, and there seems to be a disagreement between what photoCalib saves in the header and what the output of the task says it finds.
Here are some examples.
detector: 52, visit: 410905
# header
MAGZERO = 25.0818354594265
HIERARCH MAGZERO_RMS = 0.000249326325817118
HIERARCH MAGZERO_NOBJ = 78
# output
lsst.calibrate.photoCal INFO: Magnitude zero point: 29.918082 +/- 0.000249 from 78 stars
lsst.calibrate INFO: Photometric zero-point: 29.918
####################
detector: 36, visit: 410905
# header
MAGZERO = 25.0360128372399
HIERARCH MAGZERO_RMS = 0.000329066118719761
HIERARCH MAGZERO_NOBJ = 64
# output
lsst.calibrate.photoCal INFO: Magnitude zero point: 29.903869 +/- 0.000293 from 65 stars
lsst.calibrate INFO: Photometric zero-point: 29.903869
#####################
detector: 22, visit: 410905
# header
MAGZERO = 25.0395856849364
HIERARCH MAGZERO_RMS = 0.000259564956541494
HIERARCH MAGZERO_NOBJ = 65
# output
lsst.calibrate.photoCal INFO: Magnitude zero point: 30.218122 +/- 0.000330 from 62 stars
lsst.calibrate INFO: Photometric zero-point: 30.218122
#####################
detector: 15, visit: 410905
# header
MAGZERO = 25.0412930978766
HIERARCH MAGZERO_RMS = 0.000254564650586176
HIERARCH MAGZERO_NOBJ = 65
# output
lsst.calibrate.photoCal INFO: Magnitude zero point: 29.877539 +/- 0.000255 from 65 stars
lsst.calibrate INFO: Photometric zero-point: 29.877539
#####################
detector: 36, visit: 410961
# header
MAGZERO = 25.0473318845239
HIERARCH MAGZERO_RMS = 0.00032509749124665
HIERARCH MAGZERO_NOBJ = 69
# output
lsst.calibrate.photoCal INFO: Magnitude zero point: 29.883578 +/- 0.000325 from 69 stars
lsst.calibrate INFO: Photometric zero-point: 29.883578
#####################
detector: 15, visit: 410961
# header
MAGZERO = 25.0551914688232
HIERARCH MAGZERO_RMS = 0.000248615640912845
HIERARCH MAGZERO_NOBJ = 78
# output
lsst.calibrate.photoCal INFO: Magnitude zero point: 29.891438 +/- 0.000249 from 78 stars
lsst.calibrate INFO: Photometric zero-point: 29.891438
#####################
detector: 22, visit: 410961
# header
MAGZERO = 25.0510168576166
HIERARCH MAGZERO_RMS = 0.000255961282553049
HIERARCH MAGZERO_NOBJ = 69
# output
lsst.calibrate.photoCal INFO: Magnitude zero point: 29.887263 +/- 0.000256 from 69 stars
lsst.calibrate INFO: Photometric zero-point: 29.887263
#####################
detector: 52, visit: 410961
# header
MAGZERO = 25.0922992148099
HIERARCH MAGZERO_RMS = 0.000241004986662514
HIERARCH MAGZERO_NOBJ = 75
# output
lsst.calibrate.photoCal INFO: Magnitude zero point: 29.928545 +/- 0.000241 from 75 stars
lsst.calibrate INFO: Photometric zero-point: 29.928545
For the zero point magnitude it constantly shows an unfixed offset of 4 points approx. between the header and the output value. For the RMS/err value there’s more consistency, and occasionally there’s a disagreement on the number of stars used for the calibration.
I haven’t found a reason for this in the photocalib documentation, it looks like there’s an internal step that changes this value when it’s saved in the header.
Just in case, these are DECam images.
Happy to read any comments about this
You should be looking at the Exposure.photoCalib object (e.g. butler.get("calexp").photoCalib): that is what we use for all of our photometric calibration calculations (see the Community post I linked above). It has methods to convert between instrument flux and flux (nJy) or magnitude (AB system). It is not stored in the FITS header; as I said, that value is there for compatibility with older parts of the HSC pipeline, and we don’t use it for anything.
I’ll ask internally about the discrepancy between the printed PhotoCalTask value and what is stored in the MAGZERO header: the latter is scaled by the exposure time (but again, we don’t use that value anywhere in our pipelines). The number of sources and the error should be exactly the same: those are just carried through in the code.
Do please use the PhotoCalib object for photometric calculations, though. There is no guarantee that the photometric calibration of an exposure can be encompassed by a single number; that is true for most of our calexps right now, but may not be true in general.
1 Like
@Paula : I looked into the MAGZERO header column: it is a scaled zero point (scaled by the exposure time), and is no longer needed for HSC QA. We’ll be removing it in the future (see [DM-35534] Remove MAGZERO FITS header and related - Jira).
1 Like
It can be faster to do:
butler.get("calexp.photoCalib")
if you only want that and not the pixel data.
1 Like
Thanks for the clarification