I need to install a package which is not included in lsst_apps on top of a read-only, functional installation of the LSST stack v12.1. The package I need to install (e.g. obs_subaru or obs_cfht) actually exists in LSST EUPS repository (i.e. http://sw.lsstcorp.org/eupspkg) with the appropriate tag.
The concrete use case I have is that some users of the CernVM-FS binary distribution (which is read-only) need to add some specific LSST packages to their individual software stack.
I don’t know if this is a supported use case by EUPS. After re-reading the EUPS documentation, I thought that I could set the EUPS_PATH variable to include both a R/W area (e.g. $HOME) and the R/O area where the stack is already installed.
So this is the sequence of commands I use:
# Tell EUPS to also use $HOME
$ mkdir -p $HOME/.eups && export EUPS_PATH=$HOME/.eups
# Initialize my working environment to use LSST v12.1
$ source /cvmfs/lsst.in2p3.fr/software/linux-x86_64/lsst-v12.1/loadLSST.bash
# EUPS_PATH looks good
$ echo $EUPS_PATH
/cvmfs/lsst.in2p3.fr/software/linux-x86_64/lsst-v12.1:/home/fabio/.eups
# This is the LSST software distribution server. So far, so good.
$ eups distrib path
http://sw.lsstcorp.org/eupspkg
# EUPS documentation says that one use the 'eups path' command to
# 'Print the product stack directories given via EUPS_PATH'.
# In this case, it does not print /home/fabio/.eups, which is actually in $EUPS_PATH
$ eups path
/cvmfs/lsst.in2p3.fr/software/linux-x86_64/lsst-v12.1
# Try to install 'obs_subaru'
# I was expecting the command below to work, but instead, it prints an error message
# which does not really help me understanding what is wrong 8-[
$ eups distrib install -I $HOME/.eups -t v12_1 obs_subaru
eups distrib: coercing to Unicode: need string or buffer, list found
# As expected, the command below does not work because
# /cvmfs/... is a read-only file system
$ eups distrib install -t v12_1 obs_subaru
[ 1/66 ] apr 1.5.2 (already installed) done.
eups distrib: You don't have permission to assign a global tag b2307 in /cvmfs/lsst.in2p3.fr/software/linux-x86_64/lsst-v12.1/ups_db
Is this use case supported by EUPS? If so, what would be the recommended way to achieve my goal?
Is this eups version 2.0.2? It’s possible that we still have some python3 portability bugs in there. It’s also possible that it’s fixed in current eups master.
Thanks for your feedback. I should have thought that $HOME/.eups had a special meaning for EUPS. Indeed, if I use another directory for installing my EUPS packages, the command eups path works as expected. For the record:
Here is what I get using the option --debug=raise :
$ eups --debug=raise distrib install -I $HOME/myeupspackages -t v12_1 obs_subaru
Traceback (most recent call last):
File "/cvmfs/lsst.in2p3.fr/software/linux-x86_64/lsst-v12.1/eups/bin/eups_impl.py", line 57, in <module>
status = cmd.run()
File "/cvmfs/lsst.in2p3.fr/software/linux-x86_64/lsst-v12.1/eups/python/eups/cmd.py", line 245, in run
return self.execute()
File "/cvmfs/lsst.in2p3.fr/software/linux-x86_64/lsst-v12.1/eups/python/eups/cmd.py", line 172, in execute
return ecmd.run()
File "/cvmfs/lsst.in2p3.fr/software/linux-x86_64/lsst-v12.1/eups/python/eups/cmd.py", line 1903, in run
return self.execute()
File "/cvmfs/lsst.in2p3.fr/software/linux-x86_64/lsst-v12.1/eups/python/eups/cmd.py", line 1923, in execute
return ecmd.run()
File "/cvmfs/lsst.in2p3.fr/software/linux-x86_64/lsst-v12.1/eups/python/eups/cmd.py", line 245, in run
return self.execute()
File "/cvmfs/lsst.in2p3.fr/software/linux-x86_64/lsst-v12.1/eups/python/eups/cmd.py", line 2213, in execute
if not utils.isDbWritable(self.opts.installStack) and \
File "/cvmfs/lsst.in2p3.fr/software/linux-x86_64/lsst-v12.1/eups/python/eups/utils.py", line 291, in isDbWritable
return os.access(dbpath, (os.F_OK|os.R_OK|os.W_OK))
TypeError: coercing to Unicode: need string or buffer, list found
Modifying EUPS_PATH works around the error but does not seem to help making the command ultimately succeed:
$ eups --version
EUPS Version: 2.0.2
$ echo $EUPS_PATH
/home/fabio/myeupspackages:/cvmfs/lsst.in2p3.fr/software/linux-x86_64/lsst-v12.1
$ eups path
/home/fabio/myeupspackages
/cvmfs/lsst.in2p3.fr/software/linux-x86_64/lsst-v12.1
$ eups distrib install -t v12_1 obs_subaru
[ 1/66 ] apr 1.5.2 (already installed) done.
eups distrib: You don't have permission to assign a global tag b2307 in /cvmfs/lsst.in2p3.fr/software/linux-x86_64/lsst-v12.1/ups_db
It seems to me that EUPS still wants to write to the /cvmfs/... component of EUPS_PATH which is read-only. Does EUPS require that I have read-write permissions to all the components of my EUPS_PATH?
OK, I don’t know that -I will work, but the problem with reordering the path is due to someone adding a setup eups to the setups.[c]sh script (which can reorder EUPS_PATH). I have pushed a fix, but it’s to the mksetup script which is used to generate the setups.* scripts.
So you can pull eups from github and run configure, then “setup -r .” to use a version of eups which solves this problem.
That version of eups has the -I bug fixed, but I’m not sure that it ever worked; that’s probably a question for @mjuric (but I might get a chance to look at it).
An alternative to pulling a new eups would be to edit your current eups.table file (you can find it with eups list eups -m) and changing the pathAppend(EUPS_PATH, ...) to pathPrepend(EUPS_PATH, ...) – untested, but I think it’ll work as a short-term alternative.
However, installing an additional package in a R/W area does not work, neither using the -I option nor without:
$ eups distrib install -t v12_1 obs_subaru
[ 1/66 ] apr 1.5.2 (already installed) done.
eups distrib: You don't have permission to assign a global tag b2307 in /cvmfs/lsst.in2p3.fr/software/linux-x86_64/lsst-v12.1/ups_db
$ eups distrib install -I $HOME/myeupspackages -t v12_1 obs_subaru
[ 1/66 ] apr 1.5.2 (already installed) done.
eups distrib: You don't have permission to assign a global tag b2307 in /cvmfs/lsst.in2p3.fr/software/linux-x86_64/lsst-v12.1/ups_db