Access to "root" in butler

Data Management
#1

It comes up fairly often that code that uses butler would like to be able to get the root of the butler’s repository. This worked in “old” butler because it used only one repository.

New(er) butler has the ability to manage multiple repositories, which introduces the possibility of multiple roots. Do we want to add a getter to butler to get a list of repository roots?

In all the current use cases the length of the list would be 1, and scripts that assume a single repo can (should?) assert if the list is not exactly 1 item long.

#2

What kinds of things need the root dir? I can think of ingest (ingestImages.py, ingestCalibs.py). What else is there?

#4

What is the use-case for wanting to get the root? It appears to break encapsulation.

The example raised on hipChat was an ingest script, and this is rather non-typical.

#5

I don’t really know except as I encounter issues. It seems like the ingest scripts have all(?) been fixed now by using the input value from args instead of fetching it back out of the butler’s mapper (which IMO is a good thing)

I tried grepping my build dir for
grep -rn --include \*.py "\mapper.root\b" ., these two results were returned.

./obs_subaru/python/lsst/obs/subaru/ccdTesting.py:912: mat = re.search(r"/rerun/(.*)", butler.mapper.root) I don’t think this will have worked any time recently, I wonder if it’s old test code that does not run currently.

./pipe_drivers/python/lsst/pipe/drivers/constructCalibs.py:585: header.add("CALIB_CREATION_ROOT", butler.repository._mapper.root), does constructCalibs do something similar to an ingest script?

#6

So I think that the conclusion is that, rather than providing “root” access, we should file issues asking to get the codes that use it fixed.

#7

Should we schedule this cleanup for one of the AHM hackathons?