New install of lsst_stack, numpy not found

Support
#10

and
lsst-scipipe-4d7b902

#11

Ah, something has gotten confused here, and the pair of minicondas at the beginning of the path is another indication of this. You’ll need to start in a fresh shell. The first entries in your PATH after source loadLSST.bash should be /Users/blum/lsst_stack/eups/2.1.5/bin:/Users/blum/lsst_stack/python/miniconda3-4.7.10/envs/lsst-scipipe-4d7b902/bin:/Users/blum/lsst_stack/python/miniconda3-4.7.10/bin

I guess it might be good to look at the contents of your loadLSST.bash as well.

I’m also wondering if /opt/anaconda3/condabin could be part of the problem, although it shouldn’t be.

#12

In a fresh shell, after source loadLSST.bash the second and third paths are reversed from what you write above.

#13

Ah! I think I may have found the problem (which would be in newinstall.sh). I’ll try to reproduce and then file a bug.

#14

Nope, that wasn’t it, things look fine in my brand new (but not macOS) installation. I will need the output of cat loadLSST.bash

#15

cat loadLSST.bash

This script is intended to be used with bash to load the minimal LSST

environment

Usage: source loadLSST.bash

export PATH="/Users/blum/lsst_stack/python/miniconda3-4.7.10/bin:${PATH}"

export LSST_CONDA_ENV_NAME=${LSST_CONDA_ENV_NAME:-lsst-scipipe-4d7b902}

shellcheck disable=SC1091

source activate “$LSST_CONDA_ENV_NAME”

LSST_HOME="$( cd “$( dirname “${BASH_SOURCE[0]}” )” && pwd )"

Bootstrap EUPS

EUPS_DIR="${LSST_HOME}/eups/2.1.5"

source “${EUPS_DIR}/bin/setups.sh”

export -f setup

export -f unsetup

export EUPS_PKGROOT=${EUPS_PKGROOT:-https://eups.lsst.codes/stack/osx/10.9/clang-1000.10.44.4/miniconda3-4.7.10-4d7b902|https://eups.lsst.codes/stack/src}

#16

I did a sudo \rm -r lsst_stack and reinstalled. The reversal of the two paths noted above was corrected, but the result of running the demo script was still the same. Seems numpy must be in the wrong place.

#17

(lsst-scipipe-4d7b902) MacBook-Pro-79:lsst_dm_stack_demo-19.0.0 blum$ conda list |grep numpy
numpy 1.16.2 py37hacdab7b_0
numpy-base 1.16.2 py37h6575580_0

#18

(lsst-scipipe-4d7b902) MacBook-Pro-79:lsst_dm_stack_demo-19.0.0 blum$ ./bin/demo.sh
Traceback (most recent call last):
File “/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/pipe_tasks/19.0.0+2/bin/processCcd.py”, line 23, in
from lsst.pipe.tasks.processCcd import ProcessCcdTask
File “/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/pipe_tasks/19.0.0+2/python/lsst/pipe/tasks/processCcd.py”, line 22, in
from lsst.ip.isr import IsrTask
File “/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/ip_isr/19.0.0/python/lsst/ip/isr/init.py”, line 27, in
from .isrFunctions import *
File “/Users/blum/lsst_stack/stack/miniconda3-4.7.10-4d7b902/DarwinX86/ip_isr/19.0.0/python/lsst/ip/isr/isrFunctions.py”, line 23, in
import numpy
ModuleNotFoundError: No module named ‘numpy’

#19

Hi @rblum5, just checking in to see if you found a way to resolve the issue of numpy not being found, or if you’re still in need of assistance on this topic?

#20

Thanks for asking @MelissaGraham!

I have not had time to get back to this and its still an open issue. Important but not urgent.

#21

Sorry, had this bookmarked but didn’t have any fresh ideas.

The fundamental problem is that you don’t appear to have a directory with envs/lsst-scipipe-4d7b902/bin in it in your PATH. This is supposed to be set by the source activate line in loadLSST.bash. If it’s not, then something is wrong with your conda installation or possibly your shell startup files (.bash_profile, .bash_login, .profile, .bashrc).

You mentioned a fresh install. What version of newinstall.sh (what URL) did you use, and what version of the stack did you install?

1 Like
#22

Thanks @ktl.

I do have the directory: /Users/blum/lsst_stack/python/miniconda3-4.7.10/envs/lsst-scipipe-4d7b902/bin/

But numpy is not there.

numpy conda package is here:

ls /Users/blum/lsst_stack/python/miniconda3-4.7.10/envs/lsst-scipipe-4d7b902/pkgs/nump*

numpy-1.16.2-py37hacdab7b_0
numpy-1.16.2-py37hacdab7b_0.conda
numpy-base-1.16.2-py37h6575580_0
numpy-base-1.16.2-py37h6575580_0.conda

Appears not to be installed in the bin directory?

#23

numpy does not belong in the bin directory. The point is for you to use the python in the envs/.../bin directory which will have access to the numpy in the pkgs directory.

#24

Check.

Yes, so the path to the envs version of python is in PATH but it comes after the miniconda/bin in the PATH. This appears after I execute setup obs_sdss.

#25

So I guess the issue is why does the path end up that way after running setup obs_sdss. If I always want the lsstpipe version of python while using the stack, I guess this is easy to fix manually.

#26

setup obs_sdss should prepend .../obs_sdss/.../bin: to your PATH. If it’s also prepending .../miniconda3-4.7.10/bin, something is quite strange.

Basically, which python or command -v python should always give a path with envs/lsst-scipipe in it. If it’s not, we need to figure out what is overriding it.

#27

as soon as I source loadLSST.bash, the paths are reversed and the envs/lsst-scipipe is second

#28

Interesting that is go to csh and source loadLSST.csh, the paths come out right.

#29

I seem to have resolved this. I disabled .bash_profile file that was setting up a conda environment from an independent conda installation.

3 Likes