Deploying rubin-env 8.0.0

On Tuesday, 2023-12-19, we updated our build infrastructure to use version 8.0.0 of the rubin-env conda-forge metapackage. The d_2023_12_20 daily and w_2023_51 weekly will be the first tagged releases to incorporate this change.

The biggest changes with this new environment are the transition to Pydantic 2 (DM-41156) and the update to new versions of the conda compilers (12 for Linux, 16 for macOS on the C/C++ side, 12 for FORTRAN on all architectures). Packages added include meson (RFC-978), ngmix-core (RFC-955), defusedxml (RFC-952), nest-asyncio (RFC-948), and lsst-ts-xml (RFC-942).

Dependency version updates include yaml-cpp = 0.8, wcslib = 8, skyproj >= 1.1.0 (DM-40920), ruff = 0.1.7, matplotlib >= 3.8.0, log4cxx = 1.1.0, cfitsio = 4.3.1, boost = 1.82 (via libboost-python-devel), and astropy >= 5.3 (astropy 6.0 will typically be installed by default), most of which are needed to keep up with conda-forge. pandas has been allowed to update to version 2.1, while pins of pyarrow and scipy have been released. htcondor has been pinned to the range >=23.0, <23.1 to ensure long-term stability. parsl and pybind11 remain pinned pending further testing.

cython was removed, as it has not actually been used anywhere, and it should be a developer tool, not a runtime requirement. mkl was also removed to allow usage of various BLAS libraries according to conda-forge standards.

The rubin-env-rsp environment has had mamba, which was unnecessary, removed, and the pin on dask-core was released. pyvo has been allowed to update to version 2 in both the rubin-env-rsp and rubin-env-developer environments.

Binary packages that built under rubin-env 7.0.1 will not function under rubin-env 8.0.0, as indicated by the semantic versioning. Sources will often continue to work, mostly depending on their usage of pydantic. Similarly, because of the compiler and library changes, binaries built under rubin-env 8.0.0 will not function under previous versions, and sources for 8.0.0 may not build under previous versions.

Thanks to @erykoff (as is becoming a bit too usual) for last-minute bug squashing, and @timj and Matthias Wittgen for earlier input and fixes.

1 Like