DM Monthly Status Report for June 2018

The DM monthly status report covering June activities has been posted in DocuShare, Collection-6145. For your convenience, the High-level Summary is pasted below. Direct link to full report (pdf): http://ls.st/1m2.

High-Level Summary

Several DM team members attended the highly successful LSST@Europe3 meeting. A number of talks were delivered, and we organized a tutorial session on using the LSST Software Stack with the Science Platform. The tutorial resulted in 50+ simultaneous users of the Science Platform Notebook Aspect, the heaviest usage it has seen to date. We observed no scaling issues on our Google Cloud Platform deployment. The tutorial was supported remotely by many DM team members who were not at the meeting who fixed bugs and addressed infrastructural issues in response to user feedback. It was a demonstration of excellent teamwork, and users were both happy and impressed.

DM team members were also involved in the SPIE meeting in Austin, Texas, and participated in the Scheduler Review in Tucson and, of course, the LSST Directors Review at SLAC. The latter included demonstrations of both the Science Platform and Qserv. DM received useful feedback: the general tone was extremely positive, but with a number of actionable suggestions for improvement.

A key event this month was completion of the FITS File Assembly Pathfinder Early Integration Activity in conjunction with the Systems Engineering and Camera teams. This is a precursor to the upcoming spectrograph integration tests.

The DM System Science Team started developing the DM Acceptance Test Specification (LDM-639). This is being produced entirely using the JIRA system based on the tools and guidelines developed by LSST System Engineering. The document generation was supported by tools created by the DM Architecture team. The DM Subystem Scientist is actively working with the Commissioning Team to develop high-level plans for how DM and the Commissioning Team will work together the commissioning phase.

The Science Pipelines teams adopted the new “Pessimistic B” astrometric matcher, described in DMTN-031, as the system-wide default. This provides a substantial boost to both reliability and performance. The prototype Prompt Products Database (PPDB) was included in the Alert Production Pipeline and a prototype mini-broker capable of executing multiple alert filters in parallel was produced. An early version of DMTN-080, a technical note describing the new “Warped Image Comparison” artifact rejection code tech note was published. Further, the Pipelines team worked with the Data Facility and SQuaRE groups to transmit metrics of biweekly HSC processing runs into SQuaSH, the Data Management QC service.
The Data Access and Database team completed the Qserv data distribution/replication framework completed and deployed it in the Prototype Data Access Center (PDAC). This instance is now running with a data replication factor of 3: it can withstand up to two simultaneous worker node failures and self-heal without service interruption.

The Data Facility completed initial testing of the Consolidated Database System, based on Oracle RAC technology, and began working with developers to tune performance and queries.

We completed installation of the core network equipment, including spine and leaf switches, KVMs, and cabling in Summit Computer Room. A winter storm caused three power poles and three fiber poles to be knocked over in the San Carlos Valley. Power to the summit was interrupted but communications remained operational in spite of the fiber being on the ground. Weather prevented immediate repair, but eventually both were repaired and the power restored.