DM Monthly Status Report for September

The DM monthly status report covering September has been posted in Docushare, Collection-4640. For your convenience I am pasting high-level summary and project status below. Direct link to the full report (pdf):

High-level Summary

Data Management development continues to track the Fall 2016 rolling-wave cycle plan.
Mid-cycle release (12.1) was made. Work on setting up the Prototype Data Access Center
(PDAC), and on Python3 migration continued. Notable advancements were made in difference
image decorrelation. Work on conversion from SWIG to pybind11 started. Work on the fiber
cable installation from the AURA Gatehouse to the Summits commenced.

The work on replan continued and is on track for completion by the end of this calendar year.

Project Status

Selected technical accomplishments:

  • The AP Team:
    • Onboarded two new hires
    • Worked on Python 3 support
    • Implemented difference decorrelation with preconvolution turned on
    • Improved portability of the DCR free template models.
  • The DRP Team:
    • Led the decision making process and started work on the conversion of the LSST
      stack from SWIG to pybind11.
    • Began investigating the Piff package for PSF estimation.
    • Added initial code for tracking the covariance introduced by warping and
      coaddition to the stack.
    • Updated the default pipeline configuration to match the “battle-tested” defaults
      from Hyper Suprime-Cam.
  • The SUIT Team started understanding the Strip82 tables and sample image files, started
    designing the PDAC UI. Several members of SUIT team also visited SLAC camera
    group to discuss design and functions to support camera image visualization.
  • The DAX Team worked with NCSA to get the PDAC cluster configured and deployed
    Qserv to the cluster, finished containerization of DAX Web Services for the PDAC, and
    continued Stripe 82 data retrieval/staging in preparation for PDAC load.
  • Under supervision of the International Communications and Base Site Team,
    Cobra/Telefonica commenced work on the fiber cable installation from the AURA
    Gatehouse to the Summits, primarily working on the first leg from the gatehouse to the
    fork. The DWDM equipment procurement process is in its final stage, and keeps in
    evaluation two proposals (CORIANT and ADVA). Along with representatives from DM,
    Systems Engineering, OCS, TCS, CCS, and DAQ, we formed the Summit Base ITC
    Tiger Team to rapidly advance the state of design of the Information Technology and
    Communications (ITC) at the Summit Base complex.
  • The NCSA Team:
    • Completed deployment of the Verification Cluster, and placed it in production
      mode following a “friendly user” period. Future expansion of the cluster will be
      dependent on need.
    • Hosted a two-day operations planning workshop where we reviewed the DPP
      operations concepts, work breakdown, and staffing model with operational
    • Completed identified work in python3 migration and transition to the new logging
      framework for common middleware.
    • Produced a draft roadmap for phased integration of services according to
      functionalities needed for commissioning phases of the project (e.g., arrival of the
      Auxiliary Telescope, ComCam, Full Camera commissioning, etc.).
  • The SQuaRE Team made the first mid-cycle “official” release (12.1) including OSX
    binaries and is now doing weekly release-and-tag automatically through its CI system.