The lsstinstall tool, which creates conda environments for installing the LSST Science Pipelines, is now available for beta testing. It will eventually replace the newinstall tool, but during this beta test period both are still available.
If you have any questions about or experience any problems with lsstinstall, please post in the Support category, file a Jira ticket, or ask in the #dm-build-problems Slack channel.
After this user beta testing period and an RFC, uses of newinstall in the release process, container builds, and documentation are expected to be converted to lsstinstall.
It uses mambaforge rather than miniconda, giving much faster environment solutions and installations. (5 minutes vs. more than 60 currently on CentOS.)
It generates a loadLSST.sh script that can be used in any non-csh-based shell (at least those supported by conda). loadLSST.bash can still be used (it is a link). csh/tcsh remain unsupported.
It can be used for any conda-based LSST Science Pipelines release (anything after w_2020_20), so there is no need to use a release-specific script. The main version should be used in all cases.
It can install the latest rubin-env version, the version for a given release, or the exact dependency versions that were used to build a given release (e.g. for production purposes).
Multiple rubin-env versions can be maintained in the same directory using different environment names. The environment name can be passed to loadLSST.sh. User-customized environments can be used (but the user must ensure compatibility).
After sourcing loadLSST.sh, users can do so again to get a different environment and corresponding LSST Science Pipelines packages. (Using a fresh shell may still be useful to ensure cleanliness.)
LSST Science Pipelines packages are installed in an eups stack within the conda environment to ensure proper dependencies.
Usage: lsstinstall [-n]
[-T EUPS_TAG | -X EUPS_TAG | -v RUBINENV_VERSION]
[-e ENV_NAME] [-u] [-P CONDA_PATH] [-E EUPS_URL]
[-B] [-S] [-h]
Installs the Rubin software conda environment.
Enables the eups distrib install command for Science Pipelines packages.
-n -- No-op. Echo commands instead of running.
-T EUPS_TAG
-- Select the rubin-env version used to build the given EUPS_TAG.
-X EUPS_TAG
-- Select the exact environment used to build the given EUPS_TAG.
-v RUBINENV_VERSION
-- Select a particular rubin-env version (default=latest).
-e ENV_NAME
-- Specify the environment name to use; if it exists, assume that
it is compatible and should be used.
-u -- Update rubin-env in an existing environment to the latest build.
-P CONDA_PATH
-- Use an existing conda installation (default=create a new one).
If a conda is activated, it will be used, ignoring this option.
-E EUPS_URL
-- Select a different EUPS distribution server root URL
(default=https://eups.lsst.codes/stack).
-B -- DO NOT use binary "tarball" eups packages.
-S -- DO NOT use source eups packages.
-b -- ignored for backward compatibility.
-c -- ignored for backward compatibility.
-t -- ignored for backward compatibility.
-h -- Display this help message.
An update: as of 2022-02-10, the current version of lsstinstall has changed the meaning of the -P option and added a -p option to replace it.
-p CONDA_PATH
-- Specify the path to the conda installation.
If a conda installation already exists there, it will be used.
If it does not exist, it will be created.
If a conda is activated, it will be used, ignoring this option.
-P -- DO NOT use an existing activated conda; always install a new one.
In addition, the command line argument that used to be accepted by the loadLSST.sh file output by lsstinstall has now been removed, as it was causing problems when loadLSST was included in other scripts. Instead, the conda environment to be initialized by loadLSST can be specified using the LSST_CONDA_ENV_NAME environment variable if the default of the first environment created using lsstinstall is not desired.
Another update: as of 2022-03-28, the current version of lsstinstall has added a -C option to allow one or more additional conda channels to be specified. In particular, this allows a development channel such as conda-forge/label/rubin-env_dev to be added. The conda-forge channel will now always be included explicitly, overriding settings in .condarc.
-C CHANNEL
-- Use the given conda channel before the conda-forge channel.
May be repeated; first has highest priority.
Hi KT,
thanks for the nice tool. At in2p3, I see that if someone tries to active my local installation they get the following error msg:
bash lsstinstall
Using existing conda at /path/to/conda
Selected rubin-env=4.1.0
Using existing environment lsst-scipipe-4.1.0
/path/to/conda/envs/lsst-scipipe-4.1.0/share/eups/pkgroot exists; overwriting
/path/to/lsstinstall: line 336: /path/to/lsst/conda/envs/lsst-scipipe-4.1.0/share/eups/pkgroot: Permission denied
Unable to write pkgroot filecode here
I changed the permissions with no success. Is this expected/intended? or, might it be specific to in2p3?
It shouldn’t be necessary to run lsstinstall for each user, or even more than once. The purpose of the tool is to create the conda installation and environment; thereafter, others can share it (if you have appropriate permissions set). They should only need to execute
source loadLSST.sh
in order to use your installation.
A couple of cautions: you may need to make $EUPS_PATH writable for others to be able to install eups packages, and if you don’t want others to install their own conda environments and packages in your space, you’ll not only need to set the permissions on the conda/envs and conda/pkgs directories but also remove group write permission from the conda/pkgs/urls* files.
looks like a typo (especially since you are likely in the latest environment at the time). It’s surprising that, unless you type the command line exactly as written, you get rolled back to whichever environment was new when lsstinstall was first run.
When setting up the Stack on login, you need to hardcode the latest environment if you want that one to be activated. source loadLSST.sh by itself gives unexpected behavior.
Allowing loadLSST.sh to be overwritten on install would address both issues.
I believe I designed it to not be overwritten so as not to potentially break anything that was currently working. I had some thoughts of using command-line options to loadLSST.sh to control its behavior, but the fact that it has to be sourced and not executed makes that very difficult.
We might be able to redesign it in a way (similar to conda) that would allow options. Instead of:
The idea then would be to use the options to select a particular environment, or the last-installed environment, or the highest-numbered environment, or whatever.