The DM monthly status report covering September 2021 activities has been posted to DocuShare, collection-8803. For convenience, the High-level Summary is pasted below. Direct link to the full report: DocuShare Authorization Error
Community Interactions, Meetings and Workshops
DM had a good Joint Directors Review September 7-9. Several suggestions were taken on
board to prepare for the Joint Status Review in October. DM also supported the Aura
Management Council for Rubin September 23/24.
Technical Progress
DMTN-208 presents an architecture for the image cutout service.
All remaining LSP-FDR recommendation tickets were resolved with a defined plan of work
where needed.
A new test dataset for monitoring scientific performance has been deployed. The
rc2_subset contains a subset or HSC RC2 data and provides a Gen3 compliant repository
for measuring metrics in nightly CI. The faro package for computing science performance
metrics now runs nightly in CI on this dataset .
An online database schema browser was developed (available at
Science Data Model Schemas | sdm_schemas) which automatically tracks the project’s latest
schema definition files as they are committed to Github.
The Qserv Kubernetes operator was made generally available at operatorhub.io, providing
a streamlined and standardized way to obtain and deploy Qserv on top of Kubernetes.
The Execution Butler (eliminating batch worker interaction with the Butler Registry
database) and quantum clustering (grouping multiple pipeline steps into one workflow job)
were released. The Butler also now caches data retrieved from object stores.
The OCS-Controlled Pipeline Service (OCPS) was officially released and deployed, with
separate instances for LATISS, LSSTComCam, and LSSTCam.
The Solar System object association pipeline was integrated with AP. A pure Gen 3
reference catalog ingester was created. The Science Pipelines now produce Parquet
ForcedSource (i.e. lightcurve) tables conformant with the standard data model for Data
Release Production.
LDM-503-EFDa, a Level 2 milestone representing operational functionality for the summit
Engineering Facilities Database, has been formally claimed.
Rucio testing across all sites has started for data movement to In2P3, UK and SLAC.