New install fails on `python_d2to1` from missing `jinja2` requirement

Support
#1

I tried a clean install on my Mac OS X 10.12.5 laptop this morning.

It failed with a missing requirement for jinja2 when installing python_d2to1.

Is this a missing dependency that should be installed by miniconda?

[Calm ~] git clone git@github.com:lsst/lsstsw
Cloning into 'lsstsw'...
remote: Counting objects: 1583, done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 1583 (delta 1), reused 1 (delta 0), pack-reused 1578
Receiving objects: 100% (1583/1583), 305.10 KiB | 0 bytes/s, done.
Resolving deltas: 100% (1038/1038), done.
Checking connectivity... done.
[Calm ~] cd lsstsw
[Calm lsstsw] ./bin/deploy
[...]
::: Deploying lsst_build
Cloning into 'lsst_build'...
remote: Counting objects: 718, done.
remote: Total 718 (delta 0), reused 0 (delta 0), pack-reused 718
Receiving objects: 100% (718/718), 129.18 KiB | 0 bytes/s, done.
Resolving deltas: 100% (278/278), done.
Checking connectivity... done.
Done. Run the following:

    . /Users/wmwv/lsstsw/bin/setup.sh

to begin using it.
[Calm lsstsw]     . /Users/wmwv/lsstsw/bin/setup.sh
notice: lsstsw tools have been set up.
[Calm lsstsw] rebuild python_d2to1
        python_d2to1:  ok (0.5 sec).
              python:  ok (0.4 sec).
# BUILD ID: b2978
              python: 0.0.6 (already installed).
        python_d2to1: 0.2.12.lsst2+1 ERROR (2 sec).
*** error building product python_d2to1.
*** exit code = 1
*** log is in /Users/wmwv/lsstsw/build/python_d2to1/_build.log
*** last few lines:
:::::  [2017-08-03T21:59:41.806029Z]   File "/Users/wmwv/lsstsw/miniconda/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg/pkg_resources/__init__.py", line 2264, in resolve
:::::  [2017-08-03T21:59:41.806159Z]   File "/Users/wmwv/local/lib/python2.7/site-packages/sphinx/setup_command.py", line 23, in <module>
:::::  [2017-08-03T21:59:41.807978Z]     from sphinx.application import Sphinx
:::::  [2017-08-03T21:59:41.808212Z]   File "/Users/wmwv/local/lib/python2.7/site-packages/sphinx/application.py", line 31, in <module>
:::::  [2017-08-03T21:59:41.808315Z]     from sphinx.roles import XRefRole
:::::  [2017-08-03T21:59:41.808469Z]   File "/Users/wmwv/local/lib/python2.7/site-packages/sphinx/roles.py", line 21, in <module>
:::::  [2017-08-03T21:59:41.808577Z]     from sphinx.util import ws_re
:::::  [2017-08-03T21:59:41.808770Z]   File "/Users/wmwv/local/lib/python2.7/site-packages/sphinx/util/__init__.py", line 30, in <module>
:::::  [2017-08-03T21:59:41.808843Z]     import jinja2
:::::  [2017-08-03T21:59:41.808900Z] ImportError: No module named jinja2
#2

Really should be using the -3 option and I really need to make that the default for the next release.

That’s weird. It’s doing some Sphinx stuff so that’s a Sphinx dependency that is missing and pulled in from setuptools. I haven’t seen that before.

#3

Note that there doesn’t seem to be a generic problem here — building python_d2to1 does not depend on jinja2.

[jds@magpie lsstsw (master)]$ rebuild python_d2to1                                                                                                                                                          
        python_d2to1:  ok (0.8 sec).               
              python:  ok (0.5 sec).               
# BUILD ID: b1945                                  
              python: 0.0.6 (already installed).   
        python_d2to1: 0.2.12.lsst2+1 ok (2.0 sec). 
# BUILD b1945 completed.                           
[jds@magpie lsstsw (master)]$ python -c'import jinja2'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named jinja2

In fact, the issue seems to be that @mwv has installed his own copy of Sphinx outside the LSST environment (in /Users/wmwv/local) but hasn’t installed the Sphinx dependencies. When setuptools sees this, it gets confused and gives up. I don’t think there’s an LSST-specific issue here, other than the general fragility of our build system when confronted by diverse (& possibly broken!) end-user environments.

#4

Thanks, @swinbank!

I have sphinx pip-installed in my local $PYTHONUSERBASE, but the jinja2 dependency is installed in my Fink python install. When running the miniconda python version from lsstsw, it picks up my local sphinx packages in $PYTHONUSERBASE but not its dependencies in the Fink install (because it’s correctly using the miniconda PYTHONPATH).

I had to both unset PYTHONUSERBASE and remove the .local directory that I had pointing to local to get python_d2to1 to successfully install.