Building obs_decam

[price (Paul Price)]

The dates of files in that conda channel are from 2015. You are not going to be able to run a modern obs_decam (straight out of github) with that.

[price (Paul Price)]

Try building a more modern version of the stack by following the install guide at https://confluence.lsstcorp.org/display/LSWUG/Building+the+LSST+Stack+from+Source .

[swinbank (John Swinbank)]

Try building a more modern version of the stack by following the install guide at https://confluence.lsstcorp.org/display/LSWUG/Building+the+LSST+Stack+from+Source .

Careful: that suggests you run

$ eups distrib install -t v11_0 lsst_apps

which will not get you anything substantially newer than the Conda packages. Using -t v12_0_rc1 is probably a better bet.

[mbl]

Hi, thank you for you suggestion.

I did have tried building from the source at very beginning but encountered some issues. One of them was that why it was checking for CC instead of gcc complier? Was this only resulted from the fact that my gcc version was too old?

Thanks

Error:

scons: Reading SConscript files …
EUPS integration: enabled
Checking who built the CC compiler…error: no result
CC is unknown version unknown
Checking for C++11 support
Checking whether the C++ compiler works… no
C++11 extensions could not be enabled for compiler ‘unknown’

• exit -4
  eups distrib: Failed to build sconsUtils-2016_01.0-4-g49bc714.eupspkg: Command:
  source /home/mleng/miniconda2/opt/eups/bin/setups.sh; export EUPS_PATH=/home/mleng/LSST:/home/mleng/miniconda2/var/opt/eups; (/home/mleng/LSST/EupsBuildDir/Linux64/sconsUtils-2016_01.0-4-g49bc714/build.sh) >> /home/mleng/LSST/EupsBuildDir/Linux64/sconsUtils-2016_01.0-4-g49bc714/build.log 2>&1 4>/home/mleng/LSST/EupsBuildDir/Linux64/sconsUtils-2016_01.0-4-g49bc714/build.msg
  exited with code 252

[rowen (Russell Owen)]

The reported error is:

TypeError: can only concatenate list (not “tuple”) to list

and appears to be due to concatenating scripts.DEFAULT_TARGETS (presumably a list) and the tuple (“decam”,). This seems to be a change introduced by @rowen rather recently.

This probably needs a ticket and a fix.

I think this code is correct. obs_decam builds fine for Jenkins and on my machine and I emulated the following code in obs_subaru's SConsctruct file:

from lsst.sconsUtils import scripts
scripts.BasicSConstruct("obs_subaru", defaultTargets=scripts.DEFAULT_TARGETS + ("hsc",))

[timj (Tim Jenness)]

Was this only resulted from the fact that my gcc version was too old?

Yes. See my original reply. You need to install GCC 4.8 or newer.

[mbl]

Thank you for your reply. I have installed the latest the version of the stack but while installing obs_decam, the same error persists:

scons: Reading SConscript files …
EUPS integration: enabled
Checking who built the CC compiler…error: no result
CC is gcc version 4.8.5
Checking for C++11 support
Checking whether the C++ compiler worksyes
C++11 supported with '-std=c++11’
Checking for C++ header file tr1/unordered_map… yes
TypeError: can only concatenate list (not “tuple”) to list:
File “/home/mleng/lsst/Linux64/obs_decam/SConstruct”, line 3:
scripts.BasicSConstruct(“obs_decam”, defaultTargets=scripts.DEFAULT_TARGETS + (“decam”,))

Any suggestions on ways to fix this is very much appreciated.

[swinbank (John Swinbank)]

I have installed the latest the version of the stack

Could you be more specific about exactly which version this means, and how you installed it (through Anaconda, using the guide @price suggested, or some other method)?

Please set up your system as if you were about to build obs_decam, then try running both

eups list -s sconsUtils

and

grep DEFAULT_TARGETS $SCONSUTILS_DIR/python/lsst/sconsUtils/scripts.py

and post the output here.

[mbl]

Hello jdswinbank,

Thank you for your reply. I installed the version 12 as what you
suggested. I have the stack installed this time from the source. I have
run the test data and it proceeded successfully.

To build obs_decam, after loading LSST environment I setup pipe_tasks
first. I tried the commands you suggested and the followings were the
outputs:

bash-4.1$ eups list -s sconsUtils
11.0 current b1711 v11_0 setup

grep DEFAULT_TARGETS scripts.py
DEFAULT_TARGETS = (“lib”, “python”, “tests”, “examples”, “doc”, “shebang”)
defaultTargets=DEFAULT_TARGETS,
def finish(defaultTargets=DEFAULT_TARGETS,

Thanks for your help.

[swinbank (John Swinbank)]

I installed the version 12

but

bash-4.1$ eups list -s sconsUtils
11.0 current b1711 v11_0 setup

This worries me: version 12 of the stack should not be using sconsUtils version 11.

I suggest starting from a clean environment – ie, remove anything relevant to LSST from your shell initialization files (.profile, .bashrc, etc) if you’ve added it, then open a fresh new terminal. Start by sourcing loadLSST, as described in the instructions @price linked you to earlier. Then check which versions of both lsst_apps and sconsUtils you have installed:

eups list lsst_apps
eups list sconsUtils

I’m confused by the rest of your reply: it looks like you edited it while I was typing this. At one point, it seemed like $SCONSUTILS_DIR wasn’t set to anything useful, which could be the source of your problems, but you’ve now removed the text that indicates that. Can you confirm what the contents of $SCONSUTILS_DIR is (ie, echo $SCONSUTILS_DIR) immediately before you try to build obs_decam?

[mbl]

Hi, here is the update:

grep DEFAULT_TARGETS scripts.py
DEFAULT_TARGETS = (“lib”, “python”, “tests”, “examples”, “doc”, “shebang”)
defaultTargets=DEFAULT_TARGETS,
def finish(defaultTargets=DEFAULT_TARGETS,

I will try removing everything related to version 11 and update the outputs. Thank you.

[mbl]

Hello jdswinbank,

After cleaning everything related to version 11 and after loading lsst environment, I checked the versions of followings:

bash-4.1$ eups list -s sconsUtils
2016_01.0-8-ge052bd8 b2060 v12_0_rc1 b2058 qserv-dev setup

bash-4.1$ eups list lsst_apps
2016_01.0-1-gd7afef2+24 current v12_0_rc1 b2060

bash-4.1$ eups list sconsUtils
2016_01.0-4-g49bc714 sims_2_2_4c dax_2016_03 b1990 current b1962 w_2016_15 w_2016_12 b1997 qserv_2016_03 b1969
2016_01.0-8-ge052bd8 b2060 v12_0_rc1 b2058 qserv-dev setup

bash-4.1$ grep DEFAULT_TARGETS scripts.py
DEFAULT_TARGETS = (“lib”, “python”, “tests”, “examples”, “doc”, “shebang”)
defaultTargets=DEFAULT_TARGETS,
def finish(defaultTargets=DEFAULT_TARGETS,

When I tried building obs_decam I received the following:

bash-4.1$ setup pipe_tasks
bash-4.1$ setup -k -r .
bash-4.1$ scons install declare
scons: Reading SConscript files …
EUPS integration: enabled
Checking who built the CC compiler…(cached) error: no result
CC is gcc version 4.8.5
Checking for C++11 support
Checking whether the C++ compiler works… (cached) yes
C++11 supported with '-std=c++11’
Setting up environment to build package ‘obs_decam’.
scons: done reading SConscript files.
scons: Building targets …
python decam/makeLinearizer.py decam/linearity_table_v0.4.fits --force
Traceback (most recent call last):
File “decam/makeLinearizer.py”, line 13, in
from lsst.ip.isr import LinearizeLookupTable
ImportError: cannot import name LinearizeLookupTable
scons: *** [decam/linearizer] Error 1
scons: building terminated because of errors.

Thanks

[timj (Tim Jenness)]

When I tried building obs_decam I received the following

That’s because obs_decam is newer than the 12.0 release. You need to checkout the git version that matches the 12.0 ip_isr. I don’t think obs_decam got a tag because it wasn’t part of the release. @frossie can confirm.

[swinbank (John Swinbank)]

So it looks like we’ve overcome the earlier problem (probably caused by an old version of sconsUtils): this new error is different, and @timj has given you the correct diagnosis: you need to find a version of obs_decam corresponding to the stack v12 release. Based on the dates, I’m guessing 737bd0d might do the trick.

[mbl]

Hello jdswinbank,

Thanks for your reply. Yes the earlier problem seems to be solved. Thank you!
To search for a compatible version of obs_decam that matches stack 12.0, I have been trying most of the packages under the branches but none of them actually worked. I received the same error message for all of them.

I have also tried the one you suggested tickes/DM-6147, but unfortunately, it did not work either. Please help.

Thanks

[price (Paul Price)]

To search for a compatible version of obs_decam that matches stack 12.0, I have been trying most of the packages under the branches but none of them actually worked.

Use the mainline (not the branches) at different points in time. The 12.0 snapshot appears to be about 4 weeks ago between May 23 and May 30 (@frossie will know). Maybe try 7bf2411?

When you say that “none of them actually worked”, what is the actual error that shows it’s not working?

[ctslater (Colin Slater)]

I got obs_decam to work with v12 by checking out 2c2dc43. Not sure if that’s precisely the right one or not.

More broadly, I think it’s a problem that obs_decam is not “in the release” (I don’t know what criteria sets that). I put a note that in what I think is the relevant ticket, DM-5661.

[mbl]

Hello price,

Thank you for your reply. What I meant was that it showed the same error message as before: ImportError: cannot import name LinearizeLookupTable.

[mbl]

Thank you so much ctslater, as you suggested I found 2c2dc43 is working properly.

Thank you guys for all you helps!

Warm regards,

Leng

[price (Paul Price)]

More broadly, I think it’s a problem that obs_decam is not “in the release” (I don’t know what criteria sets that).

For obs_subaru, we simply put obs_subaru into lsst_distrib (all it takes is an RFC). The problem with obs_decam is that there’s no package owner — no-one overseeing its development, no-one ensuring that it’s always working, no-one making it easy to use for users, no-one to write the RFC.

Is obs_decam important for LSST or not? If so, it needs to be regularly CI-ed and it needs an owner who is regularly using it himself.