DM Monthly Status Report for January 2020

The DM monthly status report covering December activities has been posted to DocuShare, collection-972. For convenience, the High-level Summary is pasted below. Direct link to the full report

High-level Summary

Community Interactions, Meetings and Workshops

DM Project Manager Wil O’Mullane attended the 235th meeting of the American Astronomical Society in Hawai’i, at which the renaming of the project to the Vera C. Rubin Observatory was announced. This useful meeting provided the opportunity for a number of useful interactions with the scientific community and the project’s funding agencies.

Several members of the DM team attended the Dark Energy Science Collaboration winter meeting in Tucson. Leanne Guy and Robert Lupton were there as official liaisons for the DM System Science Team, while Wil O’Mullane, Chris Morrison, Melissa Graham and Simon Krughoff also participated in the meeting.

Much of the DM team is now focused on preparations for the Rubin Observatory Algorithms Workshop, a key part of our strategy to engage with the scientific community, which will take place in Princeton in March.

Technical Progress

This month saw the DM Middleware Team transition to new leadership under Middleware Manager Tim Jenness (AURA) and Product Owner Robert Gruendl (NCSA). We thank Fritz Mueller (SLAC) for his leadership of this effort over the last several years. This new team got rapidly to work, enabling ingest of precursor data from DECam into the “Generation 3” Data Butler system.

The Architecture (primarily Lim and Jenness), Data Release Production (Fisher-Levine and Plazas), System Science (Lupton), and Data Facility (Pietrowicz, Morganson, Menanteau, Win, and others) teams spent much of this month directly supporting System Integration, Test, and Commissioning for LATISS and the Commissioning Camera.

The Architecture team also documented the impacts of LCR-1923 — descoping the Camera Data Acquisition System crosstalk correction capabilities — on the DM requirements and system design.

The Pipelines team made a number of algorithmic improvements, including releasing a prototype implementation of the HelioLinC algorithm (Holman et al, 2018) for linking solar system objects; substantial improvements to photometric calibration through improved aperture corrections; and better brighter-fatter effect correction kernel generation (Antilogus et al, 2014), with many thanks to Craig Lage (UC Davis) for help with the last of those. We have also tested the new SCARLET deblender release on precursor data.

The first acceptance test campaign for a Science Pipelines release is currently being run by the DM System Science team. Many performance requirements and other metrics have been verified; the results are reported in DMTR-201.

A Python helper package that wraps some complexity encountered by users of our Engineering and Facility Database (EFD) has been demonstrated to stakeholder and released as a prototype.

New hardware has been deployed at the Data Facility and is ready for developers to use. This includes a number of new Kubernetes nodes, 3 PB of additional data storage, and extra test machines for the Qserv database system. Furthermore, the NCSA camera test stand is now configured with Kubernetes.

This new hardware is already being for evaluating Apache Cassandra as a platform for implementing the Alert Production Database (APDB), which has extreme performance demands.

Puppet automated deployments are now stable in Chile. All network links remain operational and we are preparing to migrate LATISS traffic off the general AURA network to the LSST data network in February.

The LSST Atmospheric Transmission Imager and Slitless Spectrograph; the primary instrument on the Auxiliary Telescope.