The DM monthly status report covering October activities has been posted to DocuShare, collection-873. For convenience, the High-level Summary is pasted below. Direct link to the full report (pdf): https://docushare.lsst.org/docushare/dsweb/Get/report-891/DM%20Monthly%20Progress%20Report%20201910.pdf
High-level Summary
Community Interactions, Meetings and Workshops
In conjunction with the Operations Team, members of DM have begun evaluating the SLAC National Accelerator Laboratory, Brookhaven National Laboratory, and Fermi National Accelerator Laboratory as potential data processing partners for LSST operations. During October, we provided all of the labs with information packs, held initial discussions by teleconference, and conducted our first site visit (to SLAC); this process will continue with site visits to the remaining two labs in November, and delivery of a report and recommendations to the funding agencies by mid-December.
The DM Leadership Team and System Science Team met at SLAC during the final week of the month. The DMLT combined regular status and planning activities with discussion of specific technical issues including the development of DM’s new (“generation 3”) middleware and a comprehensive refresh of our sizing and costing model. The SST focused on preparing for the Algorithms Workshop, which will be held during March 2020 in Princeton.
NCSA hosted and participated in a workshop on the Camera Data Acquisition (DAQ) v4.0 API, and which brought together members of the DM and Camera subsystems. This event also saw the API installed on the NCSA Test Stand.
During October, members of the DM team attended and presented at the InfluxDays conference in San Francisco, CA; at the Astronomical Data Analysis and Software & Systems (ADASS) XXIX meeting in Groningen, the Netherlands; at the International Virtual Observatory Interoperability Meeting, also in Groningen; at the SCARLET deblender workshop, in Naples, Italy; at a joint meeting of the Transients and Milky Way Science Collaborations, at the University of Delaware; and at the Harvard & Smithsonian Center for Astrophysics.
Technical Progress
Notable documents published or updated this month include:
DMTN-091: Test Datasets for Scientific Performance Monitoring
DMTN-135: DM sizing model and purchase plan for the remainder of construction (draft version)
DMTN-120: Improving Extensibility in afw.image.Exposure and Replacing afw.table.io
SQR-034: Engineering & Facility Database (EFD) Operations Model (proposal)
The DM team produced an astrometric reference catalog based on Gaia Data Release 2. This is a closer match to the system we expect to deploy in the operational era than the catalogs that have been available to date. We also produced LSST Science Data Model-compliant Object tables in Apache Parquet based on processing substantial volumes of Hyper Suprime-Cam data. This is an important step towards making our test processing of precursor data into a realistic analog of LSST data releases, and will enable ingest and serving of this data through DM’s standard data access interfaces.
The IVOA TAP service is now integrated with LSST authentication systems, and has been upgraded to support asynchronous queries. Qserv multi-master configuration now boots successfully, and has basic coverage in our continuous integration test suite.
We successfully installed the version 1.0 release candidate of the Archiver, Forwarder, and Header Service on the Auxiliary Telescope test stand in Tucson. We also set up and configured the NCSA test stand with multiple Configurable SAL Components (CSC) including Telescope and Site environments. The NCSA test stand is now part of our deployment strategy, as new deployments are tested at NCSA before rolling out to the Tucson test stand environment.
Our Continental United States (CONUS) Network Implementation Team (CNIT) led by Paul Wefel of ESnet continues working on implementation of our Miami–Atlanta-Chicago network. The links between Atlanta and Chicago are now ready, and we received a quote from Internet2 for the Miami–Atlanta link.
Table Access Protocol; http://www.ivoa.net/documents/TAP/