DM Monthly Status Report

The DM monthly status report covering February activities has been posted to DocuShare, collection-1022. For convenience, the High-level Summary is pasted below. Direct link to the full report https://docushare.lsst.org/docushare/dsweb/Get/report-1022/DM%20Monthly%20Progress%20Report%20202002.pdf

High-level Summary

Community Interactions, Meetings and Workshops

The DM Leadership Team held a successful “virtual” face-to-face meeting on BlueJeans. This covered a variety of technical topics, which highlights including the status of middleware development and discussion of ways to streamline the image capture system following LCR-1923 (which removes support for live crosstalk correction from the Camera data acquisition system).

Across the subsystem, preparation continues for the Rubin Observatory Algorithms Workshop, which will take place in March. The DM System Science Team held a “virtual” face-to-face meeting to prepare, while members of the DM-SST and the Science Pipelines teams have based their work around material which will be presented at the workshop. This included a two-week data analysis sprint which took place at Princeton University

Technical Progress

The first full week of data produced by the Auxiliary Telescope spectrograph has been transferred over the AURA network to the data facility at NCSA. Members of the DM team have been working hard to analyze this data, which has driven a number of improvements to instrument signature removal and pipeline integration. During April, we expect to move away from the AURA network to dedicated Rubin Observatory networking.

Many documents were issued or updated. Of particular notes:

Draft outlines of DM construction papers were provided to the PST.

SQR-037 reports on a security analysis of SQuaRE services.

DMTN-133 describes data processing systems which may be triggered by the Observatory Control System (OCS).

A prototype ObsTAP service with providing WISE data was deployed for testing. Qserv profiling, tuning, and performance optimization was carried out in preparation for scale testing at the level of 50% of the first Rubin Observatory data release. In the process, full-sky query dispatch times were reduced from ~30 minutes to ~1 minute, and a long-standing shared-scan concurrency regression was addressed. Evaluation of a clustered Cassandra implementation of the Alert Production Database continued in earnest at NCSA.

Science Pipelines activity focused around preparing for the Algorithms Workshop. This included an “analysis sprint” in which a number of careful analyses of our data products were performed: the results will be presented at the workshop next month. For example, the use of “sky objects” to quantify over-subtraction of backgrounds around bright objects was demonstrated, and a prototype system for tracking simulated sources through the Alert Production pipeline was rolled out. Substantial improvements to reliability of both galaxy model fitting and deblending were made.

A Kubernetes (“K8s”) cluster was built, installed, and configured from common build scripts at the NCSA camera test stand. The Engineering and Facility Database environment was then deployed.

Project Science Team.

The implementation of the International Virtual Observatory Alliance Observation Data Model through the Table Access Protocol; http://www.ivoa.net/documents/ObsCore/

Wide-field Infrared Survey Explorer