Installing stack 12.1 with conda

Support
#1

Hello,

I want to install pipeline of version 12.1 with conda, and have just tried the channel http://conda.lsst.codes/stack

It seems that the conda packages have been upgraded to 12.1 version.

I have deleted all my 12.0 packages and re-install v12.1, but a lot of packages were enabled by conda with the following message:

Package http://conda.lsst.codes/stack::lsst-meas-extensions-psfex-0.12.1.post1-0 uses a pre-link script.
Pre-link scripts are potentially dangerous  discouraged.

I’d be grateful if you can help me with this. In addition, I was wondering if the demo test (which currently is still compatible with v12.0) will work for v12.1?

Thank you

#2

Hi mbl,

Thanks for bringing this up. The pre-link script is there to ensure eups is configured correctly before installing. At this time we don’t have a work around to avoid this warning message. It should not affect your Conda environment other than configuring LSST’s Conda install of eups.

The previous DM stack Conda binaries are still available in version specific Conda channels. Here is the v12.0 channel: http://conda.lsst.codes/stack/0.12.0. These old channels are not actively maintained but are still publicly available. So there may be issues. I’d recommend using a single Conda channel and environment at a time. This will avoid problems mixing incompatible packages from different LSST Conda binary releases.

The demo does change based on the current release. Similar to other binary artifacts older demos are still available but need to be specified. The 12.0 demo is available from git here: https://github.com/lsst/lsst_dm_stack_demo/tree/12.0 or archived here: https://github.com/lsst/lsst_dm_stack_demo/archive/12.0.tar.gz.

#3

My attention was drawn recently to the fact that the v12.0 demo, that is, the one available at

https://github.com/lsst/lsst_dm_stack_demo/archive/12.0.tar.gz

does not fully work when you run it using v12.1 software, at least not when the stack is installed from source via newinstall.sh as I do. There seems to be an issue in the verification phase. Here is what I get:

$ ./bin/demo.sh
<snip> this produces the usual messages

$ ./bin/compare detected-sources.txt 
Traceback (most recent call last):
  File "./bin/compare", line 176, in <module>
    if compareWithNumPy(args.filename, args.reference, args.tolerance):
  File "./bin/compare", line 104, in compareWithNumPy
    table1, table2 = get_array(filename), get_array(reference)
  File "./bin/compare", line 51, in get_array
    array = np.loadtxt(f, dtype=DTYPE)
  File "/cvmfs/lsst.in2p3.fr/software/linux-x86_64/lsst-v12.1/Linux64/miniconda2/3.19.0.lsst4/lib/python2.7/site-packages/numpy/lib/npyio.py", line 928, in loadtxt
    items = [conv(val) for (conv, val) in zip(converters, vals)]
  File "/cvmfs/lsst.in2p3.fr/software/linux-x86_64/lsst-v12.1/Linux64/miniconda2/3.19.0.lsst4/lib/python2.7/site-packages/numpy/lib/npyio.py", line 658, in floatconv
    return float.fromhex(asstr(x))
ValueError: invalid hexadecimal floating-point string

I have not tried against a v12.1 stack installed via conda, but I would be surprised if the demo fully worked.

The v12.1 demo, available at

https://github.com/lsst/lsst_dm_stack_demo/archive/12.1.tar.gz

does work against the v12.1 stack.

#4

In general, we do not expect that a demo from one release will work with code from another. The demo package includes comparison scripts and “expected output” files that we use to guard against unexpected changes in the outputs of the pipeline, but small expected changes are quite frequent, and those involve changes to the demo package.

#5

Thank you.

Now the issue has been fixed now.

#6

I was also wondering if obs_base was included with stack 12.1.

I now have stack 12.1 installed but I am unable to find obs_base.

I need it to setup/declare for instance, obs_decam.

I believe it was something included in v12.0 though.

#7

Unfortunately, the creation of obs_base post-dates the 12.1 release. You can see the complete contents of 12.1 here: https://sw.lsstcorp.org/eupspkg/tags/v12_1_2.list (sorry, I’m not aware of a nicer online interface to that information).

obs_base was originally created by renaming daf_butlerUtils, which was in 12.1. However, I don’t think you can expect a modern obs_decam to work with the 12.1 vintage daf_butlerUtils.

Your best bet is probably to use an older version of obs_decam in conjunction with the 12.1 stack release. I’m guessing that 2818ce61 might work, but I haven’t confirmed it.

The good news is that this should become a lot less painful for the next release, which will include a version of obs_decam (DM-8256).

#8

obs_decam has a version tagged as 12.1, which will work with the 12.1 stack. If you check out that tag it will know to use daf_butlerUtils and not obs_base.