Deploying rubin-env 5.0.0

On Tuesday, 2022-11-08, we updated our build infrastructure to use version 5.0.0 of the rubin-env conda-forge metapackage. The d_2022_11_09 daily and w_2022_46 weekly will be the first tagged releases to incorporate this change.

This new environment adds the parsl (RFC-871) and htcondor (DM-36920) workflow interfaces and pytorch (RFC-874) for machine learning. Several version restrictions (“pins”) that were in place for previous versions have been removed (galsim, matplotlib-base, photutils, pybind11, scipy, spherematch), and the versions of compilers and the boost libraries have been updated to current conda-forge standards.

The rubin-env-rsp metapackage added in 4.1.0 now has jupyterlab-variableinspector.

A new rubin-env-developer metapackage has been added with non-RSP developer tools (RFC-869), including postgresql (RFC-885). The USDF shared stack environment will use this going forward.

The rubin-env-extras metapackage (not intended for user installation but ensuring compatibility) is now based on the above metapackages plus the independently-maintained rubin-sims package, except on ARM architectures.

Source and binary packages that built under rubin-env 4.1.0 or earlier versions will generally not continue to function under rubin-env 5.0.0: last-minute source changes by @erykoff were necessary to get the stack to build. Weekly releases on or after w_2022_46 are not guaranteed to build under previous rubin-env versions, but w_2022_46 at least is intended to be backward-compatible to 4.1.0.

FYI, there’s a bug in the conda that comes with this release that it doesn’t work correctly out-of-the-box with bash prompts. See conda 22.9.0 breaks bash command prompt · Issue #11885 · conda/conda · GitHub

The solution that worked for me, which fixed the problem and avoided putting rubin-env conda stuff directly in my .bashrc (which I did not want) was:

cp ~/.bashrc ~/.bashrc-bak
conda init bash
cp ~/.bashrc-bak ~/.bashrc