Change of behavior in eups

Support
,
#1

Up to recently I was used to use the “magic” tag in eups (V1.5.9) with something like :
cd meas_simastrom
eups declare -r . -t boutigny
then I had :
eups list | grep simastrom
meas_simastrom tag:boutigny boutigny
so I could do setup meas_simastrom -t boutigny and everything was perfect

Recently I made a full fresh installation of the stack and got eups V2.0.1
The first time I make eups declare I get the same thing :
eups list | grep simastrom
meas_simastrom tag:boutigny boutigny
But if do (for instance after re-login) :
. $LSSTSW/bin/setup.sh
I get :
eups list | grep simastrom
meas_simastrom tag:boutigny
and when I try to setup meas_simastrom -t boutigny eups thinks that there is no acceptable version of meas_simastrom

What happens ? Is there a way to make the very useful “-t username” working again ?

#2

These are the changes that went into 2.0.1:

2016-01-21 mjuric (2.0.1)dentical to v1.5.6 and 1.5.5
    - Fixed the breakage caused by the introduction of Python 3.x support
    - Completed Python 3.5 compatibility changes; EUPS should now be fully 
      functional under Python 3.5
    - All unit tests now pass on Python 2.6, 2.7 and 3.5
    - Fixed a number of bugs revealed by the unit tests
    - Framework for writing functional tests as scripts
    - eupspkg: reintroduced the TAP_PACKAGE envvar mistakenly removed 
      in 1.5.8, and deprecated the .tap_package mechanism
    - release-new-version.sh learned to autogenerate draft release notes

    dev stats:
      - 62 files changed, 968 insertions(+), 718 deletions(-)
      - contributors: Russell Owen, Mario Juric, Miguel de Val-Borro, Tim Jenness

2016-01-09 mjuric (2.0.0)
    - Update code to support both python 2.6+ and 3.3+ (dropping 2.5 and older).
      Due to the complexity involved in accomplishing this, this release
      should be considered EXPERIMENTAL. Please let us know of any issues or
      bugs. (patchset contributed by Miguel de Val-Borro)
    - Ensure DYLD_LIBRARY_PATH survives setup/unsetup on OS X 10.11 El Capitan
    - Rework setups.c?sh scripts to use maximally re-use eups.table file

    bug fixes:
      - make eupspkg more robust on machines with >= ~24 cores
      - 'unsetup eups' didn't work under (t)csh
      - don't depend on cPickle-specific behavion
      - bin/eups_setup_impl.py.in: move imports so PYTHONPATH recovery works
      - configure.ac: unescape ${prefix} so it gets expanded
      - Makefile.in: create $EUPS_PATH after clearing $EUPS_DIR
      - on non-GNU systems options must come before arguments
      - cStringIO and io.StringIO are not interchangeable
      - fix unit tests broken by Python 3 compatibility code

    dev stats:
      - 60 files changed, 1438 insertions(+), 1285 deletions(-)
      - 20 commits by Miguel de Val-Borro
      - 18 commits by Mario Juric

2015-12-24 mjuric (1.5.10)
      jhoblitt: allow https:// URLs in EUPS_PKGROOT
      timj: Allow spaces in path names
      rhl: Use a python2 compatible version of commonpath to replace commonprefix
           The problem is paths such as "/foo/bar" and "/foo/bar_XXX" for which commonprefix returns "/foo/bar"
      rhl: Allow user to specify a product name with "eups tags --clone”
#3

The first thing to do is check which packages are setup with your username tag. It is possible to lose the tags when updating your stack (in fact before 2.0.1, I would lose theme very time I logged out, but I know that was not common). E.g. eups list -v -t boutigny where the -v just gives more information.

I am hoping you just lost the eups tags (and possibly declaration of the packages themselves) in which case redeclaring them should do you.

As an aide, but something that was causing me a lot of grief recently:

If you have more than one stack or copy your stack between machines then you should work to avoid having packages declared in ~/.eups/ups_db. To do this you must declare these packages in two steps, first without your username as a tag, then (once it is safely declared) add the tag:

  • eups declare -r . foo git
  • eups declare -r . foo git -t boutigny

I made ~/.eups/ups_db unwritable to avoid allowing eups to put anything in there, so if I mess up at least eups rejects the attempt instead of cluttering up that database.

#4

Thanks @rowen. I don’t know why but I was probably exactly in the case you describe with ~/.eups/ups_db and after trying a lot of things I ended up (by chance) declaring the packages with “eups declare -r . foo git” and then, the “eups declare -r . foo git -t boutigny” did the job correctly and the tag was permanently recorded ( I don’t know if I am explaining correctly, but this is what I observed).

I want to stress that this happened with a fresh installation of the stack (new laptop) and did not try to copy the stack between different machines.

#5

@boutigny When you do a fresh installation of your stack you will always lose all of your personally declared packages. I personally prefer this, at least most of the time; I tend to perform a clean installation after finishing a project, at which point the personally declared packages are no longer needed. But if you want to preserve them you could write a simple shell script to redeclare them. (I actually did that for DM 4692 as I needed to rebuild several times to catch up with changes on master).

#6

This is a new laptop so I am not loosing previously defined tags, I am loosing my new tags.

#7

I am actually having the same problem.
Within the same shell, you can do (e.g.)
eups declare -r . -t $USER sims_maf
and it ‘sticks’ as long as you are within the same shell (you can use the -t $USER tag to set up the version you want).
[ewok:~/lsstRepos/sims_maf] lynnej% eups list sims_maf
sims_2.2.2+1 sims b1916 sims_2_2_2
tag:lynnej lynnej setup

However if you go to another shell, you find:
[ewok:~] lynnej% eups list sims_maf
sims_2.2.2+1 sims b1916 sims_2_2_2 setup
tag:lynnej
and eups doesn’t ‘see’ the lynnej version.
setup sims_maf -t $USER -t sims results in the sims version being set up instead, and if you have packages marked -t $USER only (I have some that are not official LSST packages yet), they are completely not found.

#8

And yes – it looks like it’s because the .eups directory is now being used for these packages (It wasn’t previously for some reason).

#9

If you want to avoid having eups use the .eups/ups_db directory then I suggest you set it unwritable. eups will then complain if it tries to declare anything there. In particular it will complain if you do this:

eups declare -r . mypackage version -t myusername

all at once. If you do the declaration first, then repeat adding the tag, all will be well.

I strongly recommend this technique for a private stack, but I think you must let eups declare your private tags in ~/.eups/ups_db on a shared stack.

#10

@boutigny, @ljones, @rowen – this is a real bug, I’ve opened an issue upstream.

#11

I see that the issue is closed upstream. How can I update eups in the lsstsw directory tree ? Should I reinstall it from the master branch somewhere else ?

#12

@boutigny Does the follow post help in updating an eups within lsstsw? (and instead use Eups 2.0.2)

#13

@jsick Yes, this is indeed very useful if we want to update to a newer tagged version. In the present case, I think that the fix is only available in the master branch, so I guess that I have to go through a manual installation of eups and to update the current link by hand as suggested by some people. Unless someone releases a new tagged version…

#14

You can actually set your EUPS_VERSION environment variable to a git SHA1 and just re-run ./bin/deploy.

How do we update EUPS?