DM Monthly Status Report for April 2019

The DM monthly status report covering March’s activities has been posted to DocuShare, collection-773. For convenience, the High-level Summary is pasted below. Direct link to the full report (pdf): http://ls.st/71k

High-level Summary

Community Interactions, Meetings and Workshops

A very successful LSST Science Platform Final Design Review was held in Tucson. Nine external reviewers representing a cross-section of LSST stakeholders were given excellent presentations and demonstrations by the DM team. The response to the review was extremely positive, and the reviewers made a number of valuable recommendations.

The DM team also supported the AURA “member reps” meeting during April by providing demonstrations of the Science Platform at the three LSST breakouts.

DM team members also attended the 2019 IAA Planetary Defense Conference, where they presented on LSST’s plans for moving objects processing.

Finally, a first version of the white paper resulting from the Kavli-funded “Petabytes to Science” workshop series has now been published, containing many contributions from members of the DM team. It is available from the Petabytes to Science website.

Technical Progress

This month saw the publication of, or updates to, a number of important technical notes describing key aspects of the DM system. These include:

DMTN-094, describing the authentication system being put in place for the LSST Science Platform.

DMTN-106, describing the DM software release process.

DMTN-107, discussing options for producing science alerts during the first year of operations (before a complete set of deep templates has been compiled.

DMTN-110, a proposal for a mechanism for handling Conda environments in support of the LSST Science Pipelines.

DMTN-111, on the use of DM systems in observatory operations.

DMTN-113, summarizing performance testing of the Prompt Products Database.

A number of critical updates to DM software and services were also released this month. In particular, the Firefly system has been updated to behave properly when a VOTable with International Virtual Observatory Alliance (IVOA) ObsCore information is displayed, enabling image display if the correct metadata is included in the VOTable table entry. Further, an IVOA SODA image cutout/stitching service was deployed.

Important upgrades were also made to the Science Pipelines. Critical among these is the adoption across the codebase of the modernized, spatially-variant “PhotCalib” system for representing photometric calibration information. Further, the autograd package was integrated into the SCARLET deblender, resulting in better performance, more stable operation, and higher quality outputs. Also worth noting is a significant step towards scientific validation of the DM system, as we validate that coadds can be built correctly using photometric calibration calculated using DM’s Jointcal tool.

Major progress was made on the DM-EFD system. Having received telemetry data from the M1M3 subsystem simulator, we were able to benchmark of the DM-EFD implementation. The results show we are very close to achieving the level of performance which will be required on the summit.

The first computer hardware for the Base Data Center in Chile was procured by NCSA. The NCSA L1 Test Stand configurations was brought up to date for integration and testing.

On the summit, the control network switch in the Auxiliary Telescope is ready for the Auxiliary Instruments. We started installing the fibers in the Base Data Center, and the first hallway of the Base Office Building, and we completed installation of the fiber connecting the existing computer room with the Base Data Center. We also completed configuration of the Summit Call Manager for the new voice-over-IP telephone system.

Observation Data Model Core Components

Server-side Operations for Data Access

Engineering Facilities Database; SQR-029 High-level Summary

Community Interactions, Meetings and Workshops

A very successful LSST Science Platform Final Design Review was held in Tucson. Nine external reviewers representing a cross-section of LSST stakeholders were given excellent presentations and demonstrations by the DM team. The response to the review was extremely positive, and the reviewers made a number of valuable recommendations.

The DM team also supported the AURA “member reps” meeting during April by providing demonstrations of the Science Platform at the three LSST breakouts.

DM team members also attended the 2019 IAA Planetary Defense Conference, where they presented on LSST’s plans for moving objects processing.

Finally, a first version of the white paper resulting from the Kavli-funded “Petabytes to Science” workshop series has now been published, containing many contributions from members of the DM team. It is available from the Petabytes to Science website.

Technical Progress

This month saw the publication of, or updates to, a number of important technical notes describing key aspects of the DM system. These include:

DMTN-094, describing the authentication system being put in place for the LSST Science Platform.

DMTN-106, describing the DM software release process.

DMTN-107, discussing options for producing science alerts during the first year of operations (before a complete set of deep templates has been compiled.

DMTN-110, a proposal for a mechanism for handling Conda environments in support of the LSST Science Pipelines.

DMTN-111, on the use of DM systems in observatory operations.

DMTN-113, summarizing performance testing of the Prompt Products Database.

A number of critical updates to DM software and services were also released this month. In particular, the Firefly system has been updated to behave properly when a VOTable with International Virtual Observatory Alliance (IVOA) ObsCore information is displayed, enabling image display if the correct metadata is included in the VOTable table entry. Further, an IVOA SODA image cutout/stitching service was deployed.

Important upgrades were also made to the Science Pipelines. Critical among these is the adoption across the codebase of the modernized, spatially-variant “PhotCalib” system for representing photometric calibration information. Further, the autograd package was integrated into the SCARLET deblender, resulting in better performance, more stable operation, and higher quality outputs. Also worth noting is a significant step towards scientific validation of the DM system, as we validate that coadds can be built correctly using photometric calibration calculated using DM’s Jointcal tool.

Major progress was made on the DM-EFD system. Having received telemetry data from the M1M3 subsystem simulator, we were able to benchmark of the DM-EFD implementation. The results show we are very close to achieving the level of performance which will be required on the summit.

The first computer hardware for the Base Data Center in Chile was procured by NCSA. The NCSA L1 Test Stand configurations was brought up to date for integration and testing.

On the summit, the control network switch in the Auxiliary Telescope is ready for the Auxiliary Instruments. We started installing the fibers in the Base Data Center, and the first hallway of the Base Office Building, and we completed installation of the fiber connecting the existing computer room with the Base Data Center. We also completed configuration of the Summit Call Manager for the new voice-over-IP telephone system.