I have the following error when installing mariadbclient 10.1.21.lsst2 (hscPipe 6.5.1):
-- Performing Test SUPPORTS_VERSION_IN_LINK_SCRIPT
-- Performing Test SUPPORTS_VERSION_IN_LINK_SCRIPT - Failed
CMake Error at libmysql/CMakeLists.txt:289 (MESSAGE):
Your current linker does not support VERSION command in linker scripts like
a GNU ld or any compatible linker should. Perhaps you're using gold?
Either switch to GNU ld compatible linker or run cmake with
-DDISABLE_LIBMYSQLCLIENT_SYMBOL_VERSIONING=TRUE to be able to complete the
build
the configuration is:
CentOS release 6.9 (Final)|
gcc (GCC) 5.4.0
cmake version 3.9.1
All the pre-requisites are installed. However the cluster sys. admin uses easy build instead of devtoolset. And it seems to be the source of the error: cmake is apparently using ld.gold by default (not ld) and one option is not supported.
The sys. admin is not really keen to switch to a different module system. The solution I see is to adapt the script to use ld.gold on your side or force cmake to use ld (and not ld.gold) on our side. But I don’t know how to do it…
Since the error message gives a possible way to run cmake, you could try adding that flag to the eupspkg.cfg.sh file (along with the other ARGS entries already there).
I’m a little worried that your use of CentOS 6.9 (vs. our baseline of 7) will cause you more problems down the road, however.
The plan is to switch to centOS 7 this summer, but we’d like to have the pipeline working before then. I’m also worried about the fact we’re using easy build instead of devtoolsets.
I know you’re trying to install hscPipe and we don’t have binaries for that, but since you’re hung up on the third-party packages you might try the LSST binaries (which should be the same for these third-party packages), which would get you around the need to build these packages with the different linker.
(for py3; change to .../miniconda2-... for py2). Similarly for any other third-party package you have trouble with. And then go back to installing hscPipe.
It should identify your system as “el6” and install the appropriate binaries (it should select the same EUPS_PKGROOT as I quoted for the first solution).
I managed to install most packages after a complete installation of the LSST distribution. For some packages the version differed with the one needed for hscPipe so I had to switch back to the LSST EUPS_PKGROOT to install them manually.
However I have a compilation error with pex_exceptions and since it’s tagged with hsc it has no binaries. It says:
scons: Reading SConscript files ...
EUPS integration: enabled
Checking who built the CC compiler...unknown
CC is unknown version unknown
Checking for C++14 support
Checking whether the C++ compiler works... yes
C++14 supported with '-std=c++14'
Setting up environment to build package 'pex_exceptions'.
Running pytest with 12 processes
pytest: automated test discovery mode enabled.
scons: done reading SConscript files.
scons: Building targets ...
c++ -o src/Exception.os -c -std=c++14 -g -DLSST_LITTLE_ENDIAN=1 -O3 -fPIC -Iinclude -I/home/coupon/local/source/lsst_stack/stack/miniconda3-4.3.21-10a4fa6/Linux64/base/6.0b1-hsc+7/include -isystem /home/coupon/local/source/lsst_stack/stack/miniconda3-4.3.2
Cannot guess fingerprint without .git directory; will be set to '0x0'.
c++ -o python/lsst/pex/exceptions/exceptions.os -c -std=c++14 -g -DLSST_LITTLE_ENDIAN=1 -O3 -fPIC -Iinclude -I/home/coupon/local/source/lsst_stack/stack/miniconda3-4.3.21-10a4fa6/Linux64/base/6.0b1-hsc+7/include -isystem /home/coupon/local/source/lsst_stac
c++ -o tests/testLib.os -c -std=c++14 -g -DLSST_LITTLE_ENDIAN=1 -O3 -fPIC -Iinclude -I/home/coupon/local/source/lsst_stack/stack/miniconda3-4.3.21-10a4fa6/Linux64/base/6.0b1-hsc+7/include -isystem /home/coupon/local/source/lsst_stack/stack/miniconda3-4.3.2
c++ -o tests/test_Exception_1.o -c -std=c++14 -g -DLSST_LITTLE_ENDIAN=1 -O3 -Iinclude -I/home/coupon/local/source/lsst_stack/stack/miniconda3-4.3.21-10a4fa6/Linux64/base/6.0b1-hsc+7/include -isystem /home/coupon/local/source/lsst_stack/stack/miniconda3-4.3
buildConfig(["doc/doxygen.conf"], ["doc/doxygen.conf.in"])
doxygen /home/coupon/local/source/lsst_stack/stack/miniconda3-4.3.21-10a4fa6/EupsBuildDir/Linux64/pex_exceptions-6.0b0-hsc+10/pex_exceptions-6.0b0-hsc+10/doc/doxygen.conf
c++ -o lib/libpex_exceptions.so -Wl,-rpath-link -Wl,/home/coupon/local/source/lsst_stack/stack/miniconda3-4.3.21-10a4fa6/EupsBuildDir/Linux64/pex_exceptions-6.0b0-hsc+10/pex_exceptions-6.0b0-hsc+10/lib:/home/coupon/local/source/lsst_stack/stack/miniconda3-
c++ -o tests/test_Exception_1 -Wl,-rpath-link -Wl,/home/coupon/local/source/lsst_stack/stack/miniconda3-4.3.21-10a4fa6/EupsBuildDir/Linux64/pex_exceptions-6.0b0-hsc+10/pex_exceptions-6.0b0-hsc+10/lib:/home/coupon/local/source/lsst_stack/stack/miniconda3-4.
/home/coupon/local/source/lsst_stack/stack/miniconda3-4.3.21-10a4fa6/Linux64/boost/1.60.lsst1+2/include/boost/test/tree/test_unit.hpp:249: error: undefined reference to 'boost::unit_test::ut_detail::normalize_test_case_name[abi:cxx11](boost::unit_test::bas
collect2: error: ld returned 1 exit status
scons: *** [tests/test_Exception_1] Error 1
scons: building terminated because of errors.
+ exit -4
The error message isn’t complete because the long lines have been truncated, so it’s difficult to tell.
I know that boost plays some naming tricks, and it’s possible that they include the name of the linker in there, so that you’re in trouble if you’re using a different linker than boost was built with.
One way to step around all of this completely would be to use a container. Presumably Docker is out of the question because you’re running on a cluster where you don’t have root, but Singularity doesn’t require root and it can run Docker as well. I would be happy to build either a Docker or Singularity image containing hscPipe 6.5.1. If this appeals to you, you could take the LSST stack image out for a test drive:
lsst_latest_weekly () {
date +'w_%Y_%U'
}
singularity run -e -H ~/docker:/home/lsst -B ~/LSST:/home/lsst/LSST docker://lsstsqre/centos:7-stack-lsst_distrib-$(lsst_latest_weekly)
This uses the latest LSST weekly Docker (sometimes that may not point to anything if there’s been some trouble building the weekly as there has a couple of times recently, but I think that’s been resolved), maps the local ~/docker directory into the container’s homedir (to keep the real homedir and container homedir separate, in case that’s useful to you) and maps your real ~/LSST directory (which contains all my git clones) into the container’s ~/LSST so it picks up changes you make with your editor. Unlike Docker, Singularity lets you see the external world as well, so your cluster filesystem mounted at /clusterfs (or whatever) exists even when you’re in the Singularity container.
Thanks Paul but I don’t know if it will really make my life easier, I tend not to like containers very much when developing things (like the external data module).
Would it be difficult to add binaries for HSC in the LSST PKGROOT ?
I use containers a lot when developing (for LSST/HSC and PFS) on my MacBook, as they provide a simple binary distribution that should work regardless of the host. Mapping real directories into the container is essential for this to work (because you don’t want to have to download your editor in the container and run it from there; instead, you want to run your editor in its native environment).
I think there is zero chance of binaries for HSC being added to LSST’s EUPS_PKGROOT. We could potentially instead add an HSC binary distribution (one of our clusters at Princeton still runs an el6-like system), but I would need to discover how to set it up. I would prefer to distribute containers since that’s a “one size fits all” approach using a technology I’m already comfortable with and I think more likely to succeed faster.
The system admin finally installed devtoolset and it solved the problem. We also had an issue with the file sharing system when compiling packages using several threads, but setting EUPSPKG_NJOBS=1 fixed it.