DIA processing between DRP and AP will be different. Calibrations of PVIs will be better in DRP, and templates will potentially be updated. The difference will likely be greatest in the Year-One Alerts and the Data Release 1,2 processing of the first 6,12 months of the survey.
- Even meeting the same purity and efficiency requirements may not result in the same set of objects detected between AP and DRP. How much of a difference should the users expect?
- Will there be a matching between AP DIAObject IDs and DRP DIAObject IDs? Will this matching be provided by Project+Operations or will it be up to interested science groups to do this themselves?
- How will the DIAObject ID labels be different between AP and DRP. Will they be unique between AP and DRP. Will they be unique across data releases?
- Does it make even sense to refer to the set of AP DIAObject IDs “within” a Data Release? AP doesn’t formally fit into a Data Release. AP will of course be run between data releases, and it doesn’t make sense to think of AP on an image as “belonging” to a data release prior to that data. But it doesn’t clearly make sense for AP to be considered part of the next Data Release, either.
- Will DRP provide forced-photometry on all AP DIAObjects? Imagine, e.g., an object that was detected, studied, and somebody went and took a spectrum and confirmed it as an interesting FoozaWhatsIt. But imagine that in DRP, the masks are grown a bit to match where photometry and detection are more reliable and that object is no longer detected. Will this object be referred to only by its AP DIAObject ID?
- Perhaps the former question is really asking: what’s the relationship between Rubin Observatory/LSST DIAObject IDs and community transient and variable clearinghouses, such as the IAU SN, IAU MPC, or the Transient Name Server tracking?
[The intent of this topic is to summarize and generalize a discussion that occurred over Slack.]
 DRP: Data Release Processing
 AP: Alert Processing
 PVI: Processed Visit Image. Also referred to as a
 DIAObject: Difference Image Analysis Object. The result of associating individual detections on subtracted image.
 MPC: Minor Planet Center.
All DIAObject IDs will be unique across all AP alert, AP Prompt Products Database, and Data Release (DMS-REQ-0292 (or see Sec 2.4 of the DPDD)). For this reason any astrophysical object will end up with several Rubin identifiers by the end of the survey. I expect a TNS-style naming approach external to Rubin will be necessary and valuable for many science cases.
[While individual science groups will need to keep their own lists, it does seem like there should be a scale that is less than “all of Rubin/LSST DIAObjects” which is sufficiently useful and less prone to decisions based on particular science goals.
Thanks so much to @mwv for summarizing these questions which arose in Slack but are certainly appropriate for this Data Q&A topic. A response which reflects the current state of DM’s plans has been prepared, and additional discussion and follow-up questions are welcome.
The Populations of AP/DRP DIA Objects: Users should expect some differences between the sets of real time-variable DIA Objects identified by the AP and DRP pipelines. There will be real transients and variables that are detected and included in the DIA Objects table of one pipeline but not the other. For example, faint sources at the detection limit might have their signal-to-noise ratio shifted slightly above or below the detection threshold in AP or DRP, depending on the template images, conservativeness of masks around bright and saturated stars real/bogus classifier, etc. Such instances will be quantified and characterized by verification and validation processes, but there is currently no simulation or estimate for Rubin data to which we can point users at this time.
DIA Object Identifiers: All DIA Source and Object IDs will be globally unique across all pipelines and data releases. The Project will not provide a direct matching of DIA Object IDs between AP and DRP (e.g., S.3.3 of ls.st/dpdd), nor between data releases (e.g., S.2.4 of ls.st/dpdd). The AP DIA Objects should be considered as independent of the DRP DIA Objects (and as a continuously-generated data product), and not as a part of any Data Release. It is left for the scientific community to decide whether and how Rubin identifiers are incorporated into community platforms and databases (and Target Observation Managers).
DRP Forced Photometry: It is not the plan for the DRP pipelines to perform forced photometry for all AP-detected DIA Objects, but it will do this for all DRP-detected DIA Objects. (More information about DRP forced photometry can be found in S.4.2.4 and S.4.3.3 of ls.st/dpdd.) The Rubin Science Platform will provide users the ability to generate DRP DIA forced photometry at any given sky location, which would accommodate the example of a spectroscopically-classified FoozaWhatsIt