The DM monthly status report covering Janaury activities has been posted to DocuShare, collection-8803. For convenience, the High-level Summary is pasted below. Direct link to the full report:
High-level Summary
Community Interactions, Meetings and Workshops
DM supported the AURA Management Council for Rubin (AMCR) Jan 19, 20.
The DM Scientist worked with the SAC to set up the Community Broker review committee. Eric Bellm and Colin Slater are ex-officio members of this committee to provide technical guidance.
Technical Progress
Several documents were issued or had major updates:
DMTN-049, the DM Roadmap for serving photometric redshifts. The document was shared with the community and virtual forums to collect community input will start in February
SQR-051, a discussion of how we can better handle credentials in order to improve security isolation between services.
DMTN-178 was written by Gabriele Comoretto on his departure from the project to describe how to operate the verification document generation process.
The new year finally saw AuxTel back on sky, with some images taken in January which can be ingested with both the older Gen2 and the newerGen3 middleware at the summit and at NCSA. A longer run is planned for February.
To improve testing, a mechanism was added to our authentication system to allow our test “monkey” service to impersonate users in order to emulate simultaneous user load in the Science Platform.
RFC-748 was implemented to replace the unsupportable afw::image::Filter class with the new FilterLabel, which will enable multi-instrument Gen 3 repositories. An integrated pipeline for inserting fake sources in alert production was written and documented, allowing the use of fake sources for image differencing, measurement, and source association.
Conversion to generation 3 middleware was completed for two more DRP Tasks: global photometric calibration (FGCM) and the photon transfer curve (PTC) Task. Also, the default deblending algorithm was switched from the SDSS deblender to Scarlet
An ~80 TB synthetic test catalog was created and successfully ingested into the Qserv test cluster at NCSA, to be used for a Qserv load testing campaign at the scale of 50% DR0. This catalog conforms closely in structure to the core of the anticipated LSST DRP schema, and contains ~13.5 billion Objects, ~133 billion Sources, and ~444 billion Forced Sources.
Design validation for the performance-sensitive APDB (Alert Production Database) continued with Cassandra in the Google cloud, and is expected to conclude in February.
The use of rigidly fixed third-party dependencies was replaced with a conda metapackage (rubin-env) that allows users to easily install other software alongside the Science Pipelines; also upgraded to Python 3.8.
The Engineering Facilities Database has been made resilient against being sent NaNs (Not a Number) in telemetry values thus improving robustness against misbehaving devices.
DDS network problems at the summit were solved after extensive troubleshooting. A short term solution was provided to re-enable operations, and a long term solution was also provided.