New installation documentation for Pipelines v14.0

Data Management
#1

In conjunction with the upcoming v14.0 release of the LSST Science Pipelines, I’ve been working on improving and expanding the installation documentation. These docs cover:

  • The newinstall.sh installation method and the brand-new binary packages.
  • The lsstsw installation method.
  • The Docker images.
  • How to build individual packages from source for development.

I’d like to hear your feedback on these new docs:

https://pipelines.lsst.io/v/DM-12009/index.html#installation

Feel free to reply to this topic or add comments on the open PR at https://github.com/lsst/pipelines_lsst_io/pull/61.

Important: The draft newinstall.sh documentation refers to a v14.0 tag that doesn’t exist yet. You can still try out the EUPS binary installation with a recent tag:

eups distrib install lsst_distrib -t w_2017_38

p.s. While researching this documentation I came to appreciate how awesome the Docker images are. Whether you’re a user or a developer, I encourage you to try out the Docker images instead of installing your own stack.

2 Likes
When do we need shebangtron?
#2

Note: to get the demo to run, I had to edit line 1 of processCcd.py to point to my miniconda python path instead of the /Users/square/… path.

Second – should I be concerned if the demo outputs yield the following “compare” results?

$ ./bin/compare detected-sources.txt
Failed (absolute difference 1e-06, relative difference 3.39802e-06 over tolerance 0) in column base_SdssCentroid_xSigma.
Failed (absolute difference 1e-07, relative difference 1.09556e-06 over tolerance 0) in column base_SdssCentroid_xSigma.
.
.
.

#3

If you are using the v14.0 demo then it should pass if you are building the v14.0 stack.

#4

I am using both the v14.0 demo and v14.0 stack, so something’s amiss?

#5

I think that means that you are on a mac and you have managed to get the binary distribution. There is an extra step needed to rewrite the shebangs on macs. I assume that would be mentioned in the v14.0 release notes (that aren’t completely finished).

#6

Hey @jeffcarlin, thanks for the report.

As @timj says, did you run the “shebangtron” step described here? I think that’s supposed to fix that. Are you using your own miniconda, or the miniconda provided by newinstall.sh?

Interesting, we should look into this. It looks like the demo’s v14.0 tag was made on September 4th, corresponding to an earlier release candidate tag for the Stack. A v14.0-rc2 tag came out on October 6th that probably changed the measurement outcomes slightly.

#7

Oops - thought I did, but apparently forgot the shebangtron step. Thanks!

#8

It turns out this is a difference between the release candidate and the weekly mentioned in the original post. I’m not sure what the specific difference is, but @jeffcarlin and I have shown that the demo completes if using the v14_0_rc2 version of the stack.

#9

It was DM-10902. This fix appeared between weekly 33 and weekly 34. v14.0 is derived from weekly 33.