Community input to the on-sky observing strategy during commissioning

Science
#1

The Project Commissioning team welcomes input from the Rubin science community regarding considerations for the on-sky observing strategy during commissioning that would enhance opportunities for science validation studies. As discussed at the PCW 2020, this is a “shared risk” or “best effort” situation in which input from the community will be taken into consideration alongside Project needs to perform technical and scientific verification of the formal system requirements to demonstrate construction completeness. The detailed schedule of on-sky observing periods during commissioning is still to be determined, and the Project cannot ensure that any particular observations will be taken during commissioning. The Project Commissioning Team has already been planning to acquire on-sky observations that would enable science validation studies for the four main science drivers of the LSST. Additional input from the community will be valuable to further develop these plans. The Project Commissioning team will also be in regular contact with the Survey Cadence Optimization Committee and Operations Team.

The Project is particularly interested in collecting observations that would help to (1) demonstrate the scientific capability of the as-built system and (2) inform early operations of the LSST.

We anticipate at least two phases of collecting community input.

Phase 1:

To provide guidance for the on-sky observing strategy during commissioning, the community are encouraged to produce summary documents, “commissioning notes”, that are placed into the public domain and can be considered by the Commissioning Team. The structure of these commissioning notes is flexible; there is no specific template required. A brief informal document (e.g., 2-3 pages) could be sufficient. More information is welcome. There is no length restriction for the notes. Supplemental digital resources are welcome.

Potential topics to be addressed in the commissioning notes:

  • What observations would be most helpful for science validation studies during the early integration and test periods with ComCam and the full LSSTCam? Note that during these periods, the configuration of the system might not be fully optimal for science performance. What studies would be possible with, for example, an hour or a few hours of on-sky observations?
  • Are there specific target fields and/or objects that would be most valuable for science validation studies? It is most helpful to have a prioritized list of targets that span a range of RA coordinates.
  • What external reference datasets are most valuable for science validation studies? Are specific observations needed to fully utilize these external reference datasets?
  • The Project has been planning to conduct one or more multi-night science verification surveys. There is a trade-space of depth, area coverage, and band coverage for a given amount of observing time. What factors should be taken into consideration when designing these science verification surveys?
  • What observing cadences are needed to support science validation studies?
  • What dither patterns should be tested?
  • Are there other specific observations that should be considered not in the categories above?

Any materials posted by 4 December 2020 could be taken into consideration as part of the re-plan exercise currently being undertaken by the Project. These commissioning notes can be updated at any time. Anyone may submit a commissioning note. Commissions liaisons designated by their respective Science Collaborations are encouraged to curate the guidance from their Science Collaborations.

We request that commissioning notes be submitted as PDF documents using the following form. Submissions will be made publicly visible in the following google drive folder.

Phase 2:

Based on the commissioning notes received in Phase 1, the Project will iterate with the community to refine proposals. The details of this second phase will be determined closer to the time of first light.

Resources:

  • Parallel session on community input to the commissioning observing strategy at the PCW 2020, including many contributed presentations
  • Commissioning Plan (LSE-79; note that current version does not reflect the de-scope options exercised over the past year in LCR-1966 and LCR-2045; namely the reduction of sustained observing periods with both ComCam and LSSTCam)
  • LSST System Requirements (LSR: LSE-29)
  • Observatory System Specifications (OSS: LSE-30)

Many thanks in advance for sharing your expertise and ideas.

Keith Bechtol and Chuck Claver

1 Like
#2

We wanted to provide some additional guidance for the “commissioning notes” based on recent discussion with commissioning liaisons from the Science Collaborations.

First, a relevant passage on the scope of the science verification activities to be conducted during commissioning, quoted from LSE-79.

The science verification activities are structured around demonstrating that the system functional and survey performance comply with the specifications given in the Science Requirements Document (SRD, LPM-17) and LSST System Requirements (LSR, LSE-29). In addition, these activities mark the beginning of an extended effort to thoroughly characterize the capabilities of the “as-built” system, and to optimize system operation. The scope of science verification includes:

1. Determining whether the specifications defined in the LSR and SRD are being met;

2. Characterizing other system performance metrics in the context of the four primary science drivers;

3. Studying environmental dependencies and technical optimization that inform early operations;

4. Documenting system performance and verifying mechanisms to monitor system performance during operations; and

5. Validating data delivery, derived data products, and data access tools (e.g. LSST Science Platform) that will be used by the science community.

The overall goal during Science Verification is to quantify the range of demonstrated performance by using a combination of on-sky data,informed simulations of the LSST system, and external datasets.

A few definitions:

  • Verification: did we build the thing we said we were going to build? (i.e., formal verification of OSS, LSR, and SRD requirements)
  • Validation: does the thing we built do what we want / expect it to do? (i.e., science user perspective)
  • Characterization: do we understand how / why the thing we built works the way that it does? (i.e., how can we optimize operational performance of the as-built Rubin observatory)

The Project needs to take on-sky observations that will allow us to do the science verification analyses to formally demonstrate construction completeness. Where flexibility exists, our goal is to take these observations in such a way that also enables science validation (e.g., through careful selection of target fields, cadence, depth, area, and band coverage, overlap with external reference datasets). The science validation component is where we particularly appreciate help from the community who understand the detailed data quality needs for their science. In the call for “commissioning notes”, we had been thinking about gathering input particularly on the science validation because this is where the community input is especially needed to understand the user perspective. That being said, we would appreciate thoughts that could help on the science verification side as well. Hopefully the system requirements have been written in such a way that there is substantial overlap between these goals.

We are trying to optimize data-taking during the commissioning period to maximize our understanding the system capabilities, to make informed decisions about observatory operations, and to prioritize ongoing developments/refinements of the science pipelines and data access services. We are not necessarily trying to optimize science output from the commissioning data itself. Rather, we are trying to collect data that will help us to make the first data releases of the LSST survey as useful as possible for the science community. This could include answering questions such as

  • Are we routinely acquiring raw pixel-level data of sufficient quality to be used for template generation to support difference image analysis in the early years of the survey?
  • Are we routinely acquiring raw pixel-level data of sufficient quality to be used for data release processing, both in the early data releases, and subsequent years?
  • If there are data quality anomalies, can they be corrected in software? Do they need to be addressed in hardware / observing strategy / observatory operations?
  • To what extent would the nominal planned observing strategy for LSST support various science cases, considering both for the deep drilling fields and wide-fast-deep survey? How many images are needed for effective template generation? What dither pattern should be used?
  • Where do we need to focus effort on the science pipelines in the months leading up to the first data releases?
  • Are we prepared to produce photometric redshifts from the early data releases? How will we scientifically validate the performance of photometric redshifts?
  • Is our strategy for photometric and astrometric calibration sufficient?
  • How can we optimize delivered image quality for various science cases? Is our modeling of the PSF sufficient to support various science cases?
  • How can we scientifically validate and improve performance with respect to object detection, (de)blending, and background treatment? As a function of object density? As a function of image quality and sky brightness?
  • Are there systematics associated with instrument coordinates, atmospheric conditions, etc.?
  • Are the data access services meeting the needs of science users for realistic scientific workflows?

We are intentionally trying to gather input from a broad range of science perspectives. It would be tremendously useful if each of the Science Collaborations could prepare a 2-3 page document that briefly summarizes how they would approach the questions above, and that clearly states the highest priorities (since we will likely not have much time). The commissioning notes do not necessarily need to motivate some high-profile science result that could be obtained. We are interested in covering a representative range of science cases that test the system in different ways.

How will the commissioning notes be used by the Project? A few examples:

  1. Based on internal planning and community suggestions, we plan to assemble and publicly distribute a menu of candidate high-priority target fields that could be observable throughout the calendar year, depending on the exact schedule for commissioning
  2. We plan to assemble a list of external reference datasets that could be valuable for science verification and validation purposes
  3. Working with the community and Data Management, we will further develop a plan for producing and characterizing photometric redshifts in the early data releases
  4. Suggested science validation metrics and analyses could potentially be incorporated into analysis pipelines being developed by the Project to produce data quality characterization reports

We aim to refine these ideas during the second phase.

We would like to emphasize again that we very much appreciate the time that you all put into these efforts, and recognize that responding to these calls from the Project are not always supported in a formal way. We highly value your expertise and ideas, and helping us to understand the translation of data quality to your high-level science goals.