Issue installing lsst_apps and _distrib, OSX


(Wesley Fraser) #1

I am hoping to run through the pipeline tutorial, but am unable to install the pipeline, following the installation steps.

I have provided a complete terminal session copy to show what happens when I attempt to install using miniconda as recommended, but the essential part is, that is seems eups as installed is broken. I get the following traceback during the source loadLSST.bash.

Traceback (most recent call last):
File “/Users/fraserw/lsst_stack/eups/2.1.5/bin/eups_setup”, line 42, in
import eups.setupcmd
ModuleNotFoundError: No module named ‘eups.setupcmd’
-bash: export: setup: not a function
-bash: export: unsetup: not a function

Any ideas on how to fix this?

Details:
OSX 10.14
Anaconda installed
cmake from homebrew
Note - PYTHONPATH is set to ‘’ before installation is attempted

Terminal output:
lsst_stack : fraserw $ export PYTHON_PATH=’’
lsst_stack : fraserw $ !bash
bash newinstall.sh -ct

LSST Software Stack Builder

############################## 42.2%!!! This script differs from the official version on the distribution server.
If this is not intentional, get the current version from here:
https://raw.githubusercontent.com/lsst/lsst/master/scripts/newinstall.sh
######################################################################## 100.0%
(23) Failed writing body
Detected git version 2.21.0 (Apple Git-122). OK.

LSST stack requires Python >= 3.6; you seem to have Python 2.7.10 on your path
(/usr/bin/python).

In addition to Python >= 3.6, some LSST packages depend on recent versions of
numpy, matplotlib, and scipy. If you do not have all of these, the
installation may fail. Using the Miniconda Python distribution will ensure
all these are set up.

Would you like us to install the Miniconda Python distribution (if unsure, say
yes)?yes
::: Deploying Miniconda3-4.5.12-MacOSX-x86_64.sh
######################################################################## 100.0%
PREFIX=/Users/fraserw/lsst_stack/python/miniconda3-4.5.12
installing: python-3.7.1-haf84260_7 …
Python 3.7.1
installing: ca-certificates-2018.03.07-0 …
installing: conda-env-2.6.0-1 …
installing: libcxxabi-4.0.1-hcfea43d_1 …
installing: xz-5.2.4-h1de35cc_4 …
installing: yaml-0.1.7-hc338f04_2 …
installing: zlib-1.2.11-h1de35cc_3 …
installing: libcxx-4.0.1-hcfea43d_1 …
installing: openssl-1.1.1a-h1de35cc_0 …
installing: tk-8.6.8-ha441bb4_0 …
installing: libffi-3.2.1-h475c297_4 …
installing: ncurses-6.1-h0a44026_1 …
installing: libedit-3.1.20170329-hb402a30_2 …
installing: readline-7.0-h1de35cc_5 …
installing: sqlite-3.26.0-ha441bb4_0 …
installing: asn1crypto-0.24.0-py37_0 …
installing: certifi-2018.11.29-py37_0 …
installing: chardet-3.0.4-py37_1 …
installing: idna-2.8-py37_0 …
installing: pycosat-0.6.3-py37h1de35cc_0 …
installing: pycparser-2.19-py37_0 …
installing: pysocks-1.6.8-py37_0 …
installing: python.app-2-py37_9 …
installing: ruamel_yaml-0.15.46-py37h1de35cc_0 …
installing: six-1.12.0-py37_0 …
installing: cffi-1.11.5-py37h6174b99_1 …
installing: setuptools-40.6.3-py37_0 …
installing: cryptography-2.4.2-py37ha12b0ac_0 …
installing: wheel-0.32.3-py37_0 …
installing: pip-18.1-py37_0 …
installing: pyopenssl-18.0.0-py37_0 …
installing: urllib3-1.24.1-py37_0 …
installing: requests-2.21.0-py37_0 …
installing: conda-4.5.12-py37_0 …
installation finished.
WARNING:
You currently have a PYTHONPATH environment variable set. This may cause
unexpected behavior when running the Python interpreter in Miniconda3.
For best results, please verify that your PYTHONPATH only points to
directories of packages that are compatible with the Python interpreter
in Miniconda3: /Users/fraserw/lsst_stack/python/miniconda3-4.5.12
######################################################################## 100.0%
Solving environment: done

==> WARNING: A newer version of conda exists. <==
current version: 4.5.12
latest version: 4.8.0rc0

Please update conda by running

$ conda update -n base -c defaults conda

Downloading and Extracting Packages
pytest-doctestplus-0 | 23 KB | ##################################### | 100%
scikit-learn-0.20.3 | 5.2 MB | ##################################### | 100%
libpng-1.6.36 | 296 KB | ##################################### | 100%
mkl_random-1.0.2 | 358 KB | ##################################### | 100%
lzo-2.10 | 190 KB | ##################################### | 100%
pytest-openfiles-0.3 | 11 KB | ##################################### | 100%
intel-openmp-2019.1 | 1.1 MB | ##################################### | 100%
python-3.7.2 | 21.6 MB | ##################################### | 100%
future-0.17.1 | 700 KB | ##################################### | 100%
bzip2-1.0.6 | 149 KB | ##################################### | 100%
wrapt-1.11.1 | 42 KB | ##################################### | 100%
pytables-3.5.1 | 1.4 MB | ##################################### | 100%
scipy-1.2.1 | 15.6 MB | ##################################### | 100%
h5py-2.9.0 | 942 KB | ##################################### | 100%
numexpr-2.6.9 | 130 KB | ##################################### | 100%
pip-19.0.3 | 1.8 MB | ##################################### | 100%
bottleneck-1.2.1 | 113 KB | ##################################### | 100%
cython-0.29.6 | 2.0 MB | ##################################### | 100%
pytest-astropy-0.5.0 | 6 KB | ##################################### | 100%
ca-certificates-2019 | 126 KB | ##################################### | 100%
pluggy-0.9.0 | 30 KB | ##################################### | 100%
pytz-2018.9 | 268 KB | ##################################### | 100%
mkl_fft-1.0.10 | 153 KB | ##################################### | 100%
pyyaml-5.1 | 169 KB | ##################################### | 100%
pytest-arraydiff-0.3 | 15 KB | ##################################### | 100%
hdf5-1.10.4 | 4.5 MB | ##################################### | 100%
setuptools-40.8.0 | 643 KB | ##################################### | 100%
kiwisolver-1.0.1 | 56 KB | ##################################### | 100%
blosc-1.15.0 | 460 KB | ##################################### | 100%
astropy-3.1.2 | 6.8 MB | ##################################### | 100%
python-dateutil-2.8. | 281 KB | ##################################### | 100%
numpy-1.16.2 | 49 KB | ##################################### | 100%
pytest-4.3.1 | 352 KB | ##################################### | 100%
cycler-0.10.0 | 14 KB | ##################################### | 100%
mkl-2019.1 | 154.4 MB | ##################################### | 100%
libgfortran-3.0.1 | 495 KB | ##################################### | 100%
more-itertools-6.0.0 | 89 KB | ##################################### | 100%
matplotlib-3.0.3 | 6.6 MB | ##################################### | 100%
certifi-2019.3.9 | 155 KB | ##################################### | 100%
attrs-19.1.0 | 35 KB | ##################################### | 100%
tornado-6.0.1 | 640 KB | ##################################### | 100%
atomicwrites-1.3.0 | 9 KB | ##################################### | 100%
freetype-2.9.1 | 864 KB | ##################################### | 100%
sqlite-3.27.2 | 2.3 MB | ##################################### | 100%
sqlalchemy-1.3.1 | 1.7 MB | ##################################### | 100%
pyopenssl-19.0.0 | 82 KB | ##################################### | 100%
pytest-remotedata-0. | 13 KB | ##################################### | 100%
libedit-3.1.20181209 | 159 KB | ##################################### | 100%
numpy-base-1.16.2 | 4.2 MB | ##################################### | 100%
pandas-0.24.2 | 9.9 MB | ##################################### | 100%
py-1.8.0 | 140 KB | ##################################### | 100%
cffi-1.12.2 | 213 KB | ##################################### | 100%
wheel-0.33.1 | 39 KB | ##################################### | 100%
cryptography-2.6.1 | 583 KB | ##################################### | 100%
snappy-1.1.7 | 40 KB | ##################################### | 100%
blas-1.0 | 5 KB | ##################################### | 100%
openssl-1.1.1b | 3.4 MB | ##################################### | 100%
pyparsing-2.3.1 | 105 KB | ##################################### | 100%
psutil-5.6.1 | 328 KB | ##################################### | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Collecting deprecated==1.2.5 (from -r /var/folders/p6/hw3mrjd54sx5txgvtwbvtvx00000gr/T/condaenv.v68bqfz3.requirements.txt (line 1))
Using cached https://files.pythonhosted.org/packages/9f/7a/003fa432f1e45625626549726c2fbb7a29baa764e9d1fdb2323a5d779f8a/Deprecated-1.2.5-py2.py3-none-any.whl
Requirement already satisfied: wrapt<2,>=1 in /Users/fraserw/lsst_stack/python/miniconda3-4.5.12/envs/lsst-scipipe-1172c30/lib/python3.7/site-packages (from deprecated==1.2.5->-r /var/folders/p6/hw3mrjd54sx5txgvtwbvtvx00000gr/T/condaenv.v68bqfz3.requirements.txt (line 1)) (1.11.1)
Installing collected packages: deprecated
Successfully installed deprecated-1.2.5

To activate this environment, use

$ conda activate lsst-scipipe-1172c30

To deactivate an active environment, use

$ conda deactivate

name: lsst-scipipe-1172c30
channels:

  • http://ssb.stsci.edu/astroconda
  • http://conda.lsst.codes/sims
  • http://conda.lsst.codes/stack/0.13.0
  • https://conda.anaconda.org/NLeSC
  • defaults
  • http://repo.continuum.io/pkgs/free
    dependencies:
  • asn1crypto=0.24.0=py37_0
  • astropy=3.1.2=py37h1de35cc_0
  • atomicwrites=1.3.0=py_0
  • attrs=19.1.0=py_0
  • blas=1.0=mkl
  • blosc=1.15.0=hd9629dc_0
  • bottleneck=1.2.1=py37h1d22016_1
  • bzip2=1.0.6=h1de35cc_5
  • ca-certificates=2019.1.23=0
  • certifi=2019.3.9=py37_0
  • cffi=1.12.2=py37hb5b8e2f_1
  • chardet=3.0.4=py37_1
  • cryptography=2.6.1=py37ha12b0ac_0
  • cycler=0.10.0=py37_0
  • cython=0.29.6=py37h0a44026_0
  • freetype=2.9.1=hb4e5f40_0
  • future=0.17.1=py37_0
  • h5py=2.9.0=py37h3134771_0
  • hdf5=1.10.4=hfa1e0ec_0
  • idna=2.8=py37_0
  • intel-openmp=2019.1=144
  • kiwisolver=1.0.1=py37h0a44026_0
  • libcxx=4.0.1=hcfea43d_1
  • libcxxabi=4.0.1=hcfea43d_1
  • libedit=3.1.20181209=hb402a30_0
  • libffi=3.2.1=h475c297_4
  • libgfortran=3.0.1=h93005f0_2
  • libpng=1.6.36=ha441bb4_0
  • lzo=2.10=h362108e_2
  • matplotlib=3.0.3=py37h54f8f79_0
  • mkl=2019.1=144
  • mkl_fft=1.0.10=py37h5e564d8_0
  • mkl_random=1.0.2=py37h27c97d8_0
  • more-itertools=6.0.0=py37_0
  • ncurses=6.1=h0a44026_1
  • numexpr=2.6.9=py37h7413580_0
  • numpy=1.16.2=py37hacdab7b_0
  • numpy-base=1.16.2=py37h6575580_0
  • openssl=1.1.1b=h1de35cc_1
  • pandas=0.24.2=py37h0a44026_0
  • pip=19.0.3=py37_0
  • pluggy=0.9.0=py37_0
  • psutil=5.6.1=py37h1de35cc_0
  • py=1.8.0=py37_0
  • pycparser=2.19=py37_0
  • pyopenssl=19.0.0=py37_0
  • pyparsing=2.3.1=py37_0
  • pysocks=1.6.8=py37_0
  • pytables=3.5.1=py37h5bccee9_0
  • pytest=4.3.1=py37_0
  • pytest-arraydiff=0.3=py37h39e3cac_0
  • pytest-astropy=0.5.0=py37_0
  • pytest-doctestplus=0.3.0=py37_0
  • pytest-openfiles=0.3.2=py37_0
  • pytest-remotedata=0.3.1=py37_0
  • python=3.7.2=haf84260_0
  • python-dateutil=2.8.0=py37_0
  • pytz=2018.9=py37_0
  • pyyaml=5.1=py37h1de35cc_0
  • readline=7.0=h1de35cc_5
  • requests=2.21.0=py37_0
  • scikit-learn=0.20.3=py37h27c97d8_0
  • scipy=1.2.1=py37h1410ff5_0
  • setuptools=40.8.0=py37_0
  • six=1.12.0=py37_0
  • snappy=1.1.7=he62c110_3
  • sqlalchemy=1.3.1=py37h1de35cc_0
  • sqlite=3.27.2=ha441bb4_0
  • tk=8.6.8=ha441bb4_0
  • tornado=6.0.1=py37h1de35cc_0
  • urllib3=1.24.1=py37_0
  • wheel=0.33.1=py37_0
  • wrapt=1.11.1=py37h1de35cc_0
  • xz=5.2.4=h1de35cc_4
  • yaml=0.1.7=hc338f04_2
  • zlib=1.2.11=h1de35cc_3
  • pip:
    • deprecated==1.2.5
    • tables==3.5.1
    • trippy==1.1.1
      prefix: /Users/fraserw/lsst_stack/python/miniconda3-4.5.12/envs/lsst-scipipe-1172c30

Configured EUPS_PKGROOT:
Using python at /Users/fraserw/lsst_stack/python/miniconda3-4.5.12/envs/lsst-scipipe-1172c30/bin/python to install EUPS
Installing EUPS (2.1.5)… done.
Creating startup scripts (bash) … done.
Creating startup scripts (ksh) … done.
Creating startup scripts (csh) … done.
Creating startup scripts (zsh) … done.

Bootstrap complete. To continue installing (and to use) the LSST stack type
one of:

source “/Users/fraserw/lsst_stack/loadLSST.bash” # for bash
source “/Users/fraserw/lsst_stack/loadLSST.csh” # for csh
source “/Users/fraserw/lsst_stack/loadLSST.ksh” # for ksh
source “/Users/fraserw/lsst_stack/loadLSST.zsh” # for zsh

Individual LSST packages may now be installed with the usual eups distrib install command. For example, to install the science pipeline elements
of the LSST stack, use:

eups distrib install lsst_apps

Next, read the documentation at:

https://pipelines.lsst.io

and feel free to ask any questions via the LSST Community forum:

https://community.lsst.org/c/support

                                   Thanks!
                                            -- The LSST Software Teams
                                                   http://dm.lsst.org/

lsst_stack : fraserw $ eups distrib install lsst_app
-bash: eups: command not found
lsst_stack : fraserw $ source
_build/ eupsbuild.log loadLSST.csh loadLSST.zsh python/
eups/ loadLSST.bash loadLSST.ksh newinstall.sh stack/
lsst_stack : fraserw $ source loadLSST.bash
Traceback (most recent call last):
File “/Users/fraserw/lsst_stack/eups/2.1.5/bin/eups_setup”, line 28, in
import eups.setupcmd
ModuleNotFoundError: No module named ‘eups.setupcmd’

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/Users/fraserw/lsst_stack/eups/2.1.5/bin/eups_setup”, line 42, in
import eups.setupcmd
ModuleNotFoundError: No module named ‘eups.setupcmd’
-bash: export: setup: not a function
-bash: export: unsetup: not a function


(Simon Krughoff) #2

It seems like your shell may be confused. Try opening a fresh terminal and do:
source “/Users/fraserw/lsst_stack/loadLSST.bash”, assuming you are using bash.

I guess I would be a little surprised if that works, but we use a lot of environment variables, so having a fresh env may help.


(John Swinbank) #3

See also Problem when loading loadLSST.bash — the problem there seems to be related to $PYTHONPATH.