New install of lsst_stack, numpy not found


(Bob Blum) #1

Fresh install of lsst_stack.

Running sdss demo and I get this error:

(lsst-scipipe-4d7b902) MacBook-Pro-79:lsst_dm_stack_demo-19.0.0 blum$ ./bin/demo.sh

Traceback (most recent call last):

File “/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/pipe_tasks/19.0.0+2/bin/processCcd.py”, line 23, in <module>

from lsst.pipe.tasks.processCcd import ProcessCcdTask

File “/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/pipe_tasks/19.0.0+2/python/lsst/pipe/tasks/processCcd.py”, line 22, in <module>

from lsst.ip.isr import IsrTask

File “/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/ip_isr/19.0.0/python/lsst/ip/isr/init.py”, line 27, in <module>

from .isrFunctions import *

File “/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/ip_isr/19.0.0/python/lsst/ip/isr/isrFunctions.py”, line 23, in <module>

import numpy

ModuleNotFoundError: No module named ‘numpy’

Then

(lsst-scipipe-4d7b902) MacBook-Pro-79:lsst_dm_stack_demo-19.0.0 blum$ conda install numpy

Collecting package metadata (current_repodata.json): done

Solving environment: done

All requested packages already installed.


(K-T Lim) #2

Did you have the LSST conda environment activated when you ran shebangtron? What’s the first line of /Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/pipe_tasks/19.0.0+2/bin/processCcd.py ?


(Bob Blum) #3

#!/Users/blum/lsst_stack/python/miniconda3-4.7.10/bin/python


(Bob Blum) #4

#!/Users/blum/lsst_stack/python/miniconda3-4.7.10/bin/python

I believe I did have LSST miniconda active, yes.


(Bob Blum) #5

I did this before shebangtron: source loadLSST.bash # for bash


(K-T Lim) #6

That’s the LSST-installed miniconda, but not the one from the scipipe_conda_env environment, which should be something like /Users/blum/lsst_stack/python/miniconda3-4.7.10/envs/lsst-scipipe-4d7b902/bin/python

But if you followed these directions:

source loadLSST.bash
eups distrib install -t v19_0_0 lsst_distrib
curl -sSL https://raw.githubusercontent.com/lsst/shebangtron/master/shebangtron | python

I’m not sure how things could have gone wrong.

What is the output of this?

source loadLSST.bash
command -v python

(Bob Blum) #7

/Users/blum/lsst_stack/python/miniconda3-4.7.10/bin/python


(K-T Lim) #8

That’s interesting. How about the output of echo $PATH; echo $LSST_CONDA_ENV_NAME?


(Bob Blum) #9

(base) MacBook-Pro-79:lsst_stack blum$ echo $PATH

/Users/blum/lsst_stack/python/miniconda3-4.7.10/bin:/Users/blum/lsst_stack/python/miniconda3-4.7.10/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/obs_sdss/19.0.0+2/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/dax_ppdb/19.0.0/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/verify/19.0.0+2/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/ip_isr/19.0.0/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/pipe_tasks/19.0.0+2/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/meas_astrom/19.0.0/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/meas_base/19.0.0/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/pykg_config/1.3.0+6/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/daf_butler/19.0.0/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/astro_metadata_translator/0.1.0-5-g1039c3c+8/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/starlink_ast/lsst-dev-gd6cc4e835a/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/gsl/2.6/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/cfitsio/3360.lsst6/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/fftw/3.3.4.lsst2/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/pex_config/19.0.0/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/afw/19.0.0/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/pex_policy/19.0.0/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/apr_util/1.5.4.lsst1/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/apr/1.5.2/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/pyyaml/5.1+2/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/python_psutil/5.4.3+4/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/boost/1.69.lsst1/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/doxygen/1.8.13.lsst3/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/pep8_naming/0.8.2+3/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/pyflakes/2.1.0+2/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/flake8/3.7.5+3/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/scons/3.0.0.lsst1+5/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/sconsUtils/19.0.0/bin:/Users/blum/lsst_stack/eups/2.1.5/bin:/Users/blum/lsst_stack/python/miniconda3-4.7.10/bin:/Users/blum/lsst_stack/python/miniconda3-4.7.10/bin:/opt/anaconda3/condabin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/opt/X11/bin:/usr/local/git/bin:/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/meas_algorithms/19.0.0/bin


(Bob Blum) #10

and
lsst-scipipe-4d7b902


(K-T Lim) #11

Ah, something has gotten confused here, and the pair of minicondas at the beginning of the path is another indication of this. You’ll need to start in a fresh shell. The first entries in your PATH after source loadLSST.bash should be /Users/blum/lsst_stack/eups/2.1.5/bin:/Users/blum/lsst_stack/python/miniconda3-4.7.10/envs/lsst-scipipe-4d7b902/bin:/Users/blum/lsst_stack/python/miniconda3-4.7.10/bin

I guess it might be good to look at the contents of your loadLSST.bash as well.

I’m also wondering if /opt/anaconda3/condabin could be part of the problem, although it shouldn’t be.


(Bob Blum) #12

In a fresh shell, after source loadLSST.bash the second and third paths are reversed from what you write above.


(K-T Lim) #13

Ah! I think I may have found the problem (which would be in newinstall.sh). I’ll try to reproduce and then file a bug.


(K-T Lim) #14

Nope, that wasn’t it, things look fine in my brand new (but not macOS) installation. I will need the output of cat loadLSST.bash


(Bob Blum) #15

cat loadLSST.bash

This script is intended to be used with bash to load the minimal LSST

environment

Usage: source loadLSST.bash

export PATH="/Users/blum/lsst_stack/python/miniconda3-4.7.10/bin:${PATH}"

export LSST_CONDA_ENV_NAME=${LSST_CONDA_ENV_NAME:-lsst-scipipe-4d7b902}

shellcheck disable=SC1091

source activate “$LSST_CONDA_ENV_NAME”

LSST_HOME="$( cd “$( dirname “${BASH_SOURCE[0]}” )” && pwd )"

Bootstrap EUPS

EUPS_DIR="${LSST_HOME}/eups/2.1.5"

source “${EUPS_DIR}/bin/setups.sh”

export -f setup

export -f unsetup

export EUPS_PKGROOT=${EUPS_PKGROOT:-https://eups.lsst.codes/stack/osx/10.9/clang-1000.10.44.4/miniconda3-4.7.10-4d7b902|https://eups.lsst.codes/stack/src}


(Bob Blum) #16

I did a sudo \rm -r lsst_stack and reinstalled. The reversal of the two paths noted above was corrected, but the result of running the demo script was still the same. Seems numpy must be in the wrong place.


(Bob Blum) #17

(lsst-scipipe-4d7b902) MacBook-Pro-79:lsst_dm_stack_demo-19.0.0 blum$ conda list |grep numpy
numpy 1.16.2 py37hacdab7b_0
numpy-base 1.16.2 py37h6575580_0


(Bob Blum) #18

(lsst-scipipe-4d7b902) MacBook-Pro-79:lsst_dm_stack_demo-19.0.0 blum$ ./bin/demo.sh
Traceback (most recent call last):
File “/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/pipe_tasks/19.0.0+2/bin/processCcd.py”, line 23, in
from lsst.pipe.tasks.processCcd import ProcessCcdTask
File “/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/pipe_tasks/19.0.0+2/python/lsst/pipe/tasks/processCcd.py”, line 22, in
from lsst.ip.isr import IsrTask
File “/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/ip_isr/19.0.0/python/lsst/ip/isr/init.py”, line 27, in
from .isrFunctions import *
File “/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/ip_isr/19.0.0/python/lsst/ip/isr/isrFunctions.py”, line 23, in
import numpy
ModuleNotFoundError: No module named ‘numpy’