Stack release 14.0 - Status and discussion

DM-1357.

Where is the Python 2 deprecation notice going?

That should go in the release notes. I believe @swinbank is working on that (there isn’t a Git branch for that yet as far as I know).

The term “release notes” could cover a variety of sins, but my intention is simply to produce some sort of glorified changelog covering the Science Pipelines code, rather than a statement of policy or procedures for covering future releases. That is, the equivalent of this, but probably in a little less detail. I was expecting anything which doesn’t need specialist Pipelines knowledge to come from SQuaRE.

That’s fine. After you merge those release notes I can add an announcement to both the v14 release notes and a small mention on the homepage where we link to the release notes. Does that sound good?

:+1:

Is @mwv doing the characterization test report? I can issue the DMTR number for it.

Is @mwv doing the characterization test report?

He produced some numbers that I put in the document. See the 14.0 branch of pipelines_lsst_io.

Ok. DMTR-41 will be where this ends up.

@KSK I’m done with the conversion to DMTR-41. Can you take a look before I upload it to Docushare? I assume you know this but we should include a note where we report a metric is significantly worse than the previous release’s value.

I’m not entirely convinced by the performance argument that we don’t expect any difference from a year ago. Firstly, the performance target has changed in the past year. Secondly, we have received reports of performance changes over the year.

If you need to edit the characterization report please edit the Latex in https://github.com/lsst-dm/DMTR-41

Thanks for the conversion. I’m in discussions with @mwv to see if we can compose explanations for the changes in KPM numbers between v13 and the release candidate.

I suppose it’s too late to ask to wait on DM-11905 (in review) and DM-9195 (should go into review tomorrow) to get into the build? This would get the new jointcal photometric model and a better photoCalib into 14.0, which would be a big win.

I believe the answer is no. Critical hot fixes only. 14 release candidate is based off weekly 33 I think so is from quite a long time ago. You might be able to make the case for it if you could cherry pick these changes back to jointcal from that epoch but that would still trigger an entirely new round of operating system build validations and you’d have to make a really good argument. The release is essentially done pending the release notes.

Got it, no worries. 33 is far enough back that this would be complicated to pick back in.

I think it’d be very good to include DM-11625, a very serious algorithmic bug that barely missed the cut (it went into w_2017_34).

The fix is tiny and highly localized, and the effect of the bug was serious enough to cause the HSC team to restart our official data release processing (and delay the release by over a month); it results in tons of missing objects in the coadd catalogs in very deep data.

Note that this will be the first release with pybind11.

1 Like

The distrib server has a v14_0_rc2. Is RC1 really the final candidate? If not, which weekly is RC2 based on? I think the versions are a bit different from 33.

rc_2 is rc_1 plus a hot fix from Jim. I’m still struggling to produce binaries from it, i’ll update the status soon.

as in more than one package being different? Did the third parties change between rc1 and rc2? It should be a single change to one package with that cherry pick.

As in:

$ diff v14_0_rc1.list v14_0_rc2.list | diffstat
 unknown |  181 ++++++++++++++++++++++++----------------------------------------
 1 file changed, 70 insertions(+), 111 deletions(-)

Some of those look like trivial changes. RC1 appears to include Sims, which has been dropped from RC2. I didn’t look closely.