The DM monthly status report covering November activities has been posted to DocuShare, Collection-1645. For convenience, the High-level Summary is pasted below. Direct link to full report (pdf): http://ls.st/40j.
High-level Summary
Community Interactions, Meetings and Workshops
The DM System Science Team and Leadership Team met in Princeton during the week of November 5. The SST addressed topics including a review of the recently-released DESC science requirements document; technologies for “next-to-database” processing of LSST data; the proposed LSST Science Data Model; and updates to the Data Products Definition Document. The DMLT held wide ranging discussions, including changes to the DM scope; plans for transitioning to commissioning and operations; future software releases; and planning for the upcoming development cycle (Spring 2019, starting 2018-12-01). Notes, presentations and action items from both meetings are available on Confluence.
The following week, several DM members attended ADASS XXVIII in College Park, MD. Jim Bosch, DRP Science Lead, gave an invited talk on the LSST Science Pipelines, which was very well received.
Simultaneously, a very successful DM-Commissioning-Camera “Bootcamp” was held at SLAC. The technical focus of this meeting was on Instrument Signature Removal, but, beyond that, it served an important role as a team integration exercise.
During the same hectic week, project members participated in the Supercomputing 2018 conference. A highlight of the meeting was a 100 gigabit/second network demonstration between La Serena and and the LSST Data Facility, which was conducted on November 14. The NCSA booth displayed real-time network information and ingest of transferred DECam image files from the Data Transfer Node (DTN) into a shared file system at NCSA.
Technical Progress
A number of important updates and enhancements have been made to the LSST Science Platform. A highlight is the ability to visualize source “footprints” within the Firefly tool.
The first queries were processed from the Presto distributed query engine to LSST’s Qserv system. This is a critical step in interfacing the CADC TAP service to Qserv. Meanwhile, metadata for WISE test dataset was converted to TAP_Schema format and loaded at the PDAC, leveraging the “Felis” schema translation tool provided by the DM Architecture team. A prototype tool capable of accessing the TAP_Schema to browse the database was provided. We have also deployed a proof of concept prototype for rapidly getting EFD data to the Science Platform, and introduced resource quotas in the Notebook Aspect of the Platform.
The ”Jointcal” calibration tool, which forms a key part of plans for both photometric and astrometric calibration of the LSST survey, is now fully integrated with the LSST Science Pipelines. It is now possible to execute all major stages of coadd construction and measurement based on Jointcal-derived calibration data. In addition, integration of the Forward Global Calibration Model (FGCM) system with the LSST Pipelines was completed, providing further important photometric calibration and QA monitoring capabilities.
Relating to the SST discussion of the nascent Science Data Model, a system for transforming arbitrary pipeline outputs into a form compliant with the SDM was prototype and demonstrated at the DMLT meeting. Further, a new system for collecting metric values from running pipelines and providing them to the SQuaSH monitoring system was designed and prototyped.
Finally, we completed provisioning for the second phase of the L1 Complete Test Stand in Illinois and initial setup and verification of the Summit Authentication and Authorization systems.