This post summarizes the first set of Photo-z Virtual Forums in Feb, Mar, and Apr 2021 which discussed Rubin Observatory Data Management team’s Roadmap to Photometric Redshifts for the LSST Object Catalog (DMTN-049). The second set of Photo-z Virtual Forums will be held in May, June, and July 2021, and will focus primarily on the call for Letters of Recommendation from the community regarding their photo-z science needs and which existing photo-z estimators might best serve them. All are welcome to start a new topic thread in the Photometric Redshifts category to continue the discussion of any of the following.
Suggestions that directly inspired revisions to the PZ Roadmap Document (DMTN-049)
Template fitting vs. machine learning algorithms.
Suggestion: Section 4 mentions a preference for template fits, citing that they can provide additional galaxy characteristics (SFR, mass) but ML methods are becoming comparable to template fits in this respect. Remove the preference for template methods from S.4 of the roadmap document.
Implementation: Added to Section 4.2 “Output data products”: “…estimators that can provide galaxy properties other than photo-z, such as stellar mass or star formation rate, would provide more information to scientists and should be prioritized.”
PZ flags.
Suggestion: Attendees expressed general agreement that these are essential for all users because they provide indications of, e.g., algorithm failure, result reliability, input abnormality. Flags will be especially important for non-expert users of the Object PZ data products to have, and to understand, because they will easily facilitate appropriate use of this data product. The roadmap document should explain clearly what a flag is.
Implementation: Added to Section 4.2 “Output Data Products”: “Flags are often type boolean or integer, and they provide an indication of, e.g., algorithm failure modes, result reliability, input abnormality. Flags (and uncertainties, above) are especially useful to help the novice user avoid misinterpreting the results or overestimating their significance.”
Science Priorities.
Suggestion: Attendees reported that the roadmap seems to assume that PZ estimates for inactive galaxies are prioritized, and it is unclear whether an estimator which is capable of providing PZ for AGN and quasars is a possibility. It was clarified in the discussion that it is, so long as there is an existing PZ estimator that could provide it, and it should be made clearer in the roadmap document.
Implementation: Added to Section 4.1 “Scientific Utility”: “Photo-z estimators that can return reliable results not just for galaxies but also for, e.g., AGN and quasars, and/or that can also classify stellar types, should be considered favorably.”
PZ-Related Outputs.
Suggestion: Attendees expressed interest in PZ estimators that can separate sources of error, e.g., calibration of the input data vs. systematics from templates or spec-z training sets. Attendees suggested that this capability be considered as a priority when DM is selecting potential PZ estimators, and that Section 4 of the roadmap document be updated to mention this.
Implementation: Added to section 4.2 “Output Data Products”: “Photo-z estimators that can separate sources of error, e.g., calibration of the input data vs. systematics from templates or spec-z training sets, provide more information to scientists and should be prioritized.”
Comments on Object catalog PZ data products.
Attendees discussed how PZ estimators could use more from the Object catalog than just the photometry, such as the shape measurements, deblending parameters, and star/galaxy scores.
Attendees expressed interest in Object catalog PZ data products that have multiple types of point estimates which, e.g., use different types of priors that are suited for different types of science applications. There was also general agreement there should still be one “best” or “combo” point estimate for general use.
Attendees suggested that the type of PZ estimator applied could be different for different types of objects. It was clarified that this is beyond the scope of the DM-provided LSST Object PZ estimates, in part because it starts to cross the line between measurement (what DM does) and science characterization/analysis. PZ estimates that are specialized for certain Object types are considered best left to the science community. The exception here might be to have some specialized PZ data products for stars vs. galaxies, as star/galaxy classification will already be part of the Object catalog, and could be an input to the PZ estimates.
Comments on the PZ Validation Cooperative
Many comments were more applicable to the future PZ Validation Cooperative stage of the roadmap, the joint DM-community efforts to quantitatively evaluate PZ estimator performance with Rubin commissioning data. These topics will be discussed in more detail in the future.
Attendees expressed interest in evolving the evaluation criteria and developing metrics to assess PZ performance across a wide variety of science use-cases. For example, metrics that compare PZ quality in crowded/sparse fields.
DESC representatives described how they have been working on how to vet photo-z estimators, including PZ for applications outside of DE science. DESC has public code that could be used to build a framework for PZ metrics, and to generate datasets and compare PZ results from different estimators in a controlled environment.
Attendees expressed concern that there is a risk in doing a lot of work, with very little outcome, if we try to evaluate different PZ estimators using too broad a variety of inputs (i.e., training/template sets, priors). It was suggested that sharing and standardizing inputs should be a primary focus of the PZ Validation Cooperative, and that most of the testing should be done within a shared infrastructure to enable direct comparisons. General agreement on this.
Other PZ-related comments
Attendees suggested that DM work with the Rubin Education and Public Outreach (EPO) team to produce data products to enable public-facing visualizations of the PZ data products, such as large-scale structure. (MLG note: EPO is already working on this.)