Science Pipeline release 16.0 - Status and discussion

Data Management DM Notifications
#1

Here is where we currently are in the release process. Current step in bold.

Summary

Establishing if Science Pipeline is ready for next release.
Tentative target date to close the release June 21st 2018.

Release Precursor Steps

  1. Identify any pre-release blockers (“must-have features”) :tools:
    To all science pipeline contributors, please check if there are outstanding issues that have to be included in next 16.0 release. If no outstanding issues are highlighted, and no problems are found, we plan to use next weekly build (w_2018_22) as starting point for the 16.0 major release.
  2. Wait for them to clear

Release Engineering Steps

  1. Git Tag v16.0.rc1
  2. Eups publish rc1 candidate (based on b3638) (also w_2018_22)
  3. Branch v16 of newinstall.sh
  4. Github release lsst_demo v16
  5. Wait for first round of bugs to clear
  6. Repeat last 2 steps, -rc2 and rc3 candidates <-- final candidate is rc2
  7. Confirm DM Externals are at stable tags
  8. Full OS testing (see https://ls.st/faq )
  9. Tag DM Auxilliary (non-lsst_distrib) repos
  10. Git Tag 16.0, rebuild, eups publish

Binary release steps

  1. Produce factory binaries (produced using tarball-matrix)
  2. Test factory binaries
  3. Gather contributed binaries

Documentation Steps

Integration on v16.0 branch of pipelines_lsst_io

  1. Update Prereqs/Install
  2. Gather Release notes
  3. Gather Metrics report
  4. Update Known Issues
  5. Email announcement
1 Like
#2

@gcomoretto I’d like to include updates to the validation_data_* packages so that they have a set of reduced data in them processed by the v16.0 stack in their own v16.0 tags.

I’m currently taking v16.0-rc1 set and running processing for each of those data sets. I anticipate that I can be done with this by EoB Monday, June 11, modulo getting them reviewed.

1 Like
#3

validation_data_cfht and validation_data_decam now done.

Still working on validation_data_hsc – this might take a few days.

1 Like
#4

I agree it would be nice to get newer processing included in the reduced version of these test datasets. This should not require rebuilding any packages since these test data re purely self contained. This will require an rc2 tag.

#5

Right after the rc1 tag, preparation for the workshops in Lyon and at NCSA resulted in some bug fixes for display_firefly and firefly_client. If there is an rc2 tag, could these be included? I think it is fair to call this activity testing of the rc1 candidate.

#6

Once the issue have been merge into master we can take them in account for the RC2.
Can you please specify which bug fixes are we talking about?

#7

Michael, once all the changes are merge into master, we can then take them in account for the RC2.
Can you please specify which issues (ticket branches) are involved?

#8

For display_firefly, DM-14732, DM-14734, DM-14763.

For firefly_client, DM-14238.

These have all been merged.

#9

The validation_data_{cfht,decam,hsc} packages are now all done and merged to master. This work was done under DM-14716. Specifically the SHA1s of what should be tagged as 16.v16.0.rc2 are

validation_data_cfht  d5cd87f
validastion-data_decam 5fdc4fd
validation_data_hsc ebda37e
1 Like
#10

Thank you @mwv and @davidshupe, a RC2 should be available soon.

#11

Version 16.0 is out. The announcement is available at: