Build on Scientific Linux 7 fails for afw

Support
, ,
#1

Hi All,

Anybody mananged to build LSST on Scientific Linux 7.2?

$ eups distrib install -t v13_0 lsst_distrib
…
[ 54/93 ] daf_persistence 13.0 (already installed) done.
[ 55/93 ] pex_config 13.0 (already installed) done.
[ 56/93 ] afw 13.0 …

***** error: from /scratch/midway2/ivy2/LSST/EupsBuildDir/Linux64/afw-13.0/build.log:
with preprocessed source if appropriate.
See http://bugzilla.redhat.com/bugzilla for instructions.
scons: *** [src/detection/Footprint.os] Error 4
c++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See http://bugzilla.redhat.com/bugzilla for instructions.
scons: *** [src/formatters/ExposureFormatter.os] Error 4
c++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See http://bugzilla.redhat.com/bugzilla for instructions.
scons: *** [src/formatters/MaskFormatter.os] Error 4
c++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See http://bugzilla.redhat.com/bugzilla for instructions.
scons: *** [src/formatters/ImageFormatter.os] Error 4
scons: building terminated because of errors.

  • exit -4
    eups distrib: Failed to build afw-13.0.eupspkg: Command:
    source /scratch/midway2/ivy2/LSST/eups/bin/setups.sh; export EUPS_PATH=/scratch/midway2/ivy2/LSST; (/scratch/midway2/ivy2/LSST/EupsBuildDir/Linux64/afw-13.0/build.sh) >> /scratch/midway2/ivy2/LSST/EupsBuildDir/Linux64/afw-13.0/build.log 2>&1 4>/scratch/midway2/ivy2/LSST/EupsBuildDir/Linux64/afw-13.0/build.msg
    exited with code 252

Thank you,
Igor

#2

Does the rest of the log file have anything more helpful than “internal compiler error”?

#3

I see your c++ executables are getting killed. Perhaps this is the OOM killer? If you’re building on a machine with less than 8 GB of memory, see Build Issue when building from newstall.sh et al for tips on how to make this work.

1 Like
#4

Well, I am building on a cluster under Slurm. By default I got 2G in sinteractive. Now I tried 10G and the job is still killed. I was looking what’s going on with top: the job was using all the available CPUs but not that much memory, much less than 10G. However, this time I did get out of memory message:

[ 55/93 ] pex_config 13.0 (already installed) done.
[ 56/93 ] afw 13.0 …

***** error: from /scratch/midway2/ivy2/LSST/EupsBuildDir/Linux64/afw-13.0/build.log:
}}}
^
compilation terminated.
c++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See http://bugzilla.redhat.com/bugzilla for instructions.
scons: *** [src/detection/GaussianPsf.os] Error 4
src/coord/Observatory.cc:93:3: fatal error: error writing to /tmp/ccFgFx8l.s: Cannot allocate memory
}}} // namespace lsst::afw::coord
^
compilation terminated.
The bug is not reproducible, so it is likely a hardware or OS problem.
scons: *** [src/geom/AffineTransform.os] Error 1
The bug is not reproducible, so it is likely a hardware or OS problem.
scons: *** [src/cameraGeom/CameraPoint.os] Error 1
The bug is not reproducible, so it is likely a hardware or OS problem.
scons: *** [src/coord/Observatory.os] Error 1
scons: building terminated because of errors.

  • exit -4
    eups distrib: Failed to build afw-13.0.eupspkg: Command:
    source /scratch/midway2/ivy2/LSST/eups/bin/setups.sh; export EUPS_PATH=/scratch/midway2/ivy2/LSST; (/scratch/midway2/ivy2/LSST/EupsBuildDir/Linux64/afw-13.0/build.sh) >> /scratch/midway2/ivy2/LSST/EupsBuildDir/Linux64/afw-13.0/build.log 2>&1 4>/scratch/midway2/ivy2/LSST/EupsBuildDir/Linux64/afw-13.0/build.msg
    exited with code 252

=======================
I guess, as a next step I can try to request all CPU cores and all the memory in the machine (64G) but it is weird that a package might require that much memory to get compiled.

#5

OK that helped with this package (the rest is still in progress). I did not realize that it would use all the CPUs. I guess, it was SLURM that was killing it for using too many cores.