Installation of `log` package fail with eups

Support
,
#1

Hello all,

I’m trying to install the lsst stack, not using binaries, but by recompiling the whole stack. My goal is to explore the compilation options, and see if there is a performance improvement when compiled natively.

As I’m not yet familiar with the whole ecosystem, I’m starting slowly by installing the stack using the lssinstall method

I’ve executed the following commands :

curl -OL https://ls.st/lsstinstall
chmod u+x lsstinstall
./lsstinstall -B

And then :

source loadLSST.zsh
export NCORES=40
export MAKESFLAGS="-j $NCORES"
export EUPSPKG_SCONSFLAGS="-j $NCORES"
eups distrib install -t w_latest lsst_distrib

The installation process fails at the logpackage installation, due to a test failing.
I’ve included the resulting log file pytest-log.xml.failed (5.9 KB)

I’ve checked the path of the directory, in the logfile, and it was in fact, empty. Could be a permissions issues but everything seemed alright on my side.

Thanks

#2

This did not fail when I did it on locally on my machine.

Only the server I work on seems to be affected.
I have tried different option for the installation but it still fail.

#3

Hi @abernard, I don’t have an answer for you but I suspect it would help people to help you if you could provide more information about the server (type, OS, etc.), and what kind of permissions you do have there?

#4

Hi @MelissaGraham ,

I figured it out. I was working on an unknown system, and mostly NFS-mounted. The installation script was keeping a .nfsXXX file alive, and thus preventing the deletion of the temporary directory.

To fix it, I listed NFS mountpoints :

nfsstat -m
results

I found a path that wasn’t on this list and set it up as my temporary directory (as suggested by
@FabioHernandez ):

export TMPDIR=/path/to/localdir
mkdir -p $TMPDIR

See this thread on StackOverflow : python 3.x - shutil.rmtree() error when trying to remove NFS-mounted directory - Stack Overflow

According to SO, it is fixable upstream. Either by adding a logging.shutdown() or testing within a with statement (in the testFileAppender function in log/tests/test_log.py at main · lsst/log · GitHub).

I will test it next week and keep you updated.

#5

Glad to hear it @abernard, thanks for posting the solution it is appreciated.

#6

Tried the solution specified earlier but did not succeed fixing the issue. There is still issue with NFS and I don’t know how to solve that yet, but is easily circumvented.

#7

For the record I fixed the test code so this won’t happen again.