How to use the stack with TravisCI

Data Management
#1

Hi everyone,

I’m at the DESC meeting, and a number of people (@drphilmarshall, Richard Dubois, Jim Chiang, among others) mentioned the difficulty of running unit tests (e.g., in DESC or other analysis codes) that depend on the stack. The problem is that it would take hours just to build the stack (if installed with newinstall.sh).

This is something that binary installs with conda (that I experimented with and @josh is working on productizing) can solve. As a demo hack, I put together a TravisCI .travis.yml file plus an install script, that demos how to do this:

With this, it takes ~11 minutes for an (empty) test to run the first time, and ~2 minutes on subsequent times. This is a just quick demo, so many additional improvements can be added to the install.sh script (e.g., specifying the conda channel from which to install, so one could use the builds of sims from master, etc.).

#2

After some prodding over coffee, I made the script smarter so one can specify a different conda channel or a different set of packages. It’s also now smart about caching (will know to refresh the installed product cache if you change the channel or add/remove products). To demonstrate all this, the .travis.yml file now installs the most recent lsst-sims build from the dev channel that @danielsf maintains.

I hope this’ll makes it more useful! Declaring mission accomplished & moving on :).