Build failure for meas_algorithms 17.0.1

Support
#1

I’ve had a quick rummage but can’t find a mention of the problem I’ve just encountered. Apologies if I’ve missed something relevant buried in the vast amount of information available.

My system runs a stock Ubuntu 19.04. “bash newinstall.sh -ct” doesn’t want to install binary tarballs. Disappointing, but so be it. The script ran to completion.

“eups distrib install -t v17_0_1 lsst_distrib” threw up 2 errors which were easily fixed. The first run complained about the lack of cmake. “apt install cmake” did the job. Likewise, “apt install flex”. The third one has me stumped.

The a portion error report is pasted below. I can provide much more …

Thanks for any assistance which may get me going again.

Paul
…/…/…/…/Linux64/esutil/0.6.2.5.lsst2+1/lib/python/esutil/htm/htmc.py:152: SystemError
---------------------------- Captured stderr setup -----------------------------
RuntimeError: FATAL: module compiled as little endian, but detected different endianness at runtime
_____________ ERROR at setup of HtmIndexTestCase.testLoadPixelBox ______________
[gw6] linux – Python 3.7.2 /usr/local/src/lsst_stack/python/miniconda3-4.5.12/envs/lsst-scipipe-1172c30/bin/python3.7

#2

This is a bit odd given that you would presumably have built esutil from source (we don’t have ubunutu binaries). Are you building on x86_64 or something else?

#3

Thanks Tim,

I thought it was a little odd. Yes, this is a perfectly ordinary X86_64 box running a perfectly ordinary installation of Ubuntu.

I’m guessing that esutil was built from source, though I’ve not checked. All I did wat kick off a couple of programs and then install cmake and flex which appeared to be essential. (TBH, I was mildly surprised they weren’t part of the stock Ubuntu install.)

#4

You may need more than just cmake and flex. See https://pipelines.lsst.io/install/prereqs/debian.html

#5

can you import esutil.htm.htmc ?

#6

Checked that and tried the apt-get command. The only package from that list which is required is libcurl4-openssl-dev and I’m a little perturbed that iraf-dev iraf-noao-dev & libcurl4-gnutls-dev will be removed. So I didn’t proceed.

#7

Not without further assistance. I’m completely new to this game and I’m sure the environment hasn’t yet been set up properly. For instance:

pcl@thoth:~/Astro$ python
Python 2.7.16 (default, Apr 9 2019, 04:50:39)
[GCC 8.3.0] on linux2
Type “help”, “copyright”, “credits” or “license” for more information.

import esutil.htm.htmc
Traceback (most recent call last):
File “”, line 1, in
ImportError: No module named esutil.htm.htmc

or
pcl@thoth:~/Astro$ python3
Python 3.6.8 (default, Apr 9 2019, 04:59:38)
[GCC 8.3.0] on linux
Type “help”, “copyright”, “credits” or “license” for more information.

import esutil.htm.htmc
Traceback (most recent call last):
File “”, line 1, in
ModuleNotFoundError: No module named ‘esutil’

which, to be honest, is exactly what I expected.

#8

You need to “setup” esutil for this to work. Assuming you have got to the point where eups distrib is working then something like:

$ eups list esutil
   0.6.2.5.lsst2+3 	b4164 b4165 b4166
$ setup -t b4166 esutil
$ python
$ import esutil.htm.htmc

should work. Where the build number you use in the setup is the one reported by eups list (it will be different to my build numbers above).

#9

The import worked. The eups list command returned many strings so w_2019_12 was chosen at it appears to be the most recent.

The full list is

(lsst-scipipe-1172c30) # eups list esutil
0.6.2.5.lsst2+1 sims_w_2018_51 test_tag w_2019_03 w_2018_49 sims_w_2019_11 sims_2_13_1 w_2019_10 sims_w_2018_48 sims_w_2018_49 sims_w_2019_02 sims_w_2018_50 sims_w_2019_10 w_2019_11 w_2018_52 w_2019_09 v17_0_1 v17_0 w_2018_43 w_2019_06 w_2018_45 sims_w_2019_10_1 w_2019_01 sims_w_2018_47 sims_w_2019_12 sims v17_0_rc2 w_2018_47 sims_w_2018_45 current v17_0_1_rc1 w_2019_12 sims_w_2019_04 sims_w_2018_44 sims_w_2019_06 sims_w_2019_05 sims_2_13_0 w_2018_50 sims_w_2019_08 o_latest w_2018_46 w_2018_51 w_2019_04 sims_w_2019_09 w_2019_07 sims_w_2019_01 sims_w_2019_03 sims_w_2018_52 w_2018_44 w_2019_02 v17_0_rc1 w_2019_05 w_2018_48 w_2019_08 simon_test sims_w_2018_46

#10

Hmm. Ok. That means we need to see more of the pytest output. I need to see the contents of the .failed file in the meas_algorithms tests/.tests/ directory. I’m not sure it’s going to help much though since it’s not clear why Htm importing is failing.

#11

Okey-doke, here you are.pytest-meas_algorithms.xml.failed (45.9 KB)

Hope uploading a file is preferable to pasting 757 lines of text. I don’t know the culture here but fora elsewhere prefer that approach.

#12

That’s great. Turns out that every test involving htm is failing so here is one snippet:

============================= test session starts ==============================
platform linux -- Python 3.7.2, pytest-3.6.2, py-1.7.0, pluggy-0.6.0
rootdir: /usr/local/src/lsst_stack/stack/miniconda3-4.5.12-1172c30/EupsBuildDir/Linux64/meas_algorithms-17.0.1/meas_algorithms-17.0.1, inifile: setup.cfg
plugins: flake8-1.0.4, xdist-1.20.1, forked-0.2, session2file-0.1.9, cov-2.5.1, remotedata-0.3.1, openfiles-0.3.2, doctestplus-0.3.0, arraydiff-0.3
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I / gw6 I / gw7 I / gw8 I / gw9 I / gw10 I / gw11 I
gw0 [282] / gw1 [282] / gw2 [282] / gw3 [282] / gw4 [282] / gw5 [282] / gw6 [282] / gw7 [282] / gw8 [282] / gw9 [282] / gw10 [282] / gw11 [282]

scheduling tests via LoadScheduling
........................................................................................................E....E..EE....E...E..s....E.........E........ss..s..s...E......
.EE..E....E.s......................................................................................................
==================================== ERRORS ====================================
________________ ERROR at setup of HtmIndexTestCase.testIngest _________________
[gw6] linux -- Python 3.7.2 /usr/local/src/lsst_stack/python/miniconda3-4.5.12/envs/lsst-scipipe-1172c30/bin/python3.7

cls = <class 'test_htmIndex.HtmIndexTestCase'>

    @classmethod
    def setUpClass(cls):
        cls.outPath = tempfile.mkdtemp()
        cls.testCatPath = os.path.join(os.path.dirname(os.path.realpath(__file__)), "data",
                                       "testHtmIndex.fits")
        # arbitrary, but reasonable, amount of proper motion (angle/year)
        # and direction of proper motion
        cls.properMotionAmt = 3.0*lsst.geom.arcseconds
        cls.properMotionDir = 45*lsst.geom.degrees
        cls.properMotionErr = 1e-3*lsst.geom.arcseconds
        cls.epoch = astropy.time.Time(58206.861330339219, scale="tai", format="mjd")
        ret = cls.make_skyCatalog(cls.outPath)
        cls.skyCatalogFile, cls.skyCatalogFileDelim, cls.skyCatalog = ret
        cls.testRas = [210., 14.5, 93., 180., 286., 0.]
        cls.testDecs = [-90., -51., -30.1, 0., 27.3, 62., 90.]
        cls.searchRadius = 3. * lsst.geom.degrees
        cls.compCats = {}  # dict of center coord: list of IDs of stars within cls.searchRadius of center
        cls.depth = 4  # gives a mean area of 20 deg^2 per pixel, roughly matching a 3 deg search radius
    
        config = IndexerRegistry['HTM'].ConfigClass()
        # Match on disk comparison file
        config.depth = cls.depth
>       cls.indexer = IndexerRegistry['HTM'](config)

tests/test_htmIndex.py:135: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
python/lsst/meas/algorithms/indexerRegistry.py:46: in makeHtmIndexer
    return HtmIndexer(depth=config.depth)
python/lsst/meas/algorithms/htmIndexer.py:36: in __init__
    self.htm = esutil.htm.HTM(depth)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <esutil.htm.htm.HTM;  >, depth = 4

    def __init__(self, depth=10):
>       this = _htmc.new_HTMC(depth)
E       SystemError: <built-in function new_HTMC> returned a result with an error set

../../../../Linux64/esutil/0.6.2.5.lsst2+1/lib/python/esutil/htm/htmc.py:152: SystemError
---------------------------- Captured stderr setup -----------------------------
RuntimeError: FATAL: module compiled as little endian, but detected different endianness at runtime
______ ERROR at setup of HtmIndexTestCase.testLoadIndexedReferenceConfig _______

It seems that this line esutil.htm.HTM(depth) is causing the problem. Can you see if you can run that line? (maybe the import wasn’t enough). I can guess that depth is an integer but I’m not sure of the API.

#13

A simple copy and paste told me, correctly, that depth wasn’t defined. Using an explicit 4 in the call gave this output:

esutil.htm.HTM(4)
RuntimeError: FATAL: module compiled as little endian, but detected different endianness at runtime
ImportError: numpy.core.multiarray failed to import

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File “”, line 1, in
File “/usr/local/src/lsst_stack/stack/miniconda3-4.5.12-1172c30/Linux64/esutil/0.6.2.5.lsst2+1/lib/python/esutil/htm/htmc.py”, line 152, in init
this = _htmc.new_HTMC(depth)
SystemError: returned a result with an error set

#14

Aha. Numpy in your conda env becomes a smoking gun?

#15

That may be significant. I seem to remember the build process discovering the system installed numpy and using it. Unfortunately that message has long since scrolled off the screen. It may yet be n a log file and I’ll go looking. Would you suggest (temporarily) uninstalling numpy and letting the miniconda build process install its own?

(Edit: removing numpy from the system is not going to happen. It’s an essential component of the local installation of astrometry.net and I can’t afford to lose that.)

I confess I dislike anaconda (miniconda in this instance) and would prefer to use native tools where possible. I thought going with the recommendations would make life easier. Ah, the irony.

#16

Despite the above, I just flushed the system numpy and have restarted the install. I can always re-get the Ubuntu packages when it next becomes necessary to do some plate solving.

Fingers crossed …

#17

Hmm. That made not the slightest difference. :cry:

#18

After doing some research, I find that we’ve actually seen this exact problem before in DM-9457 and DM-9796.

As the latter ticket says, this has been fixed upstream, but there hasn’t been a release yet, so we never included the fix in our stack. Your alternatives would seem to be to downgrade to gcc 7 or to manually patch esutil, since presumably you don’t want to wait for us to issue a new release.

#19

Patching holds no terrors. After a bit of digging into DM-9796 I found the patch. I’ll give it a go later today and report back.

Thanks,

#20

You will probably want to look at https://developer.lsst.io/stack/packaging-third-party-eups-dependencies.html#testing-the-package to see how to rebuild after patching.