Recent Butler changes: Repository class and 'cfg' configuration

On Friday I checked in changes to butler that add a Repository class, and allow butler to be instantiated using a nested dictionary that describes the butler configuration including what mapper to use, mapper policy, what storage type to use, etc.

Documentation that describes these things is in LDM-463. It’s not in master yet, but you can read the version in the ticket branch at http://ldm-463.lsst.io/en/tickets-dm-5091/

Of note are sections 2.3 Repository and 2.7 Butler Configuration

Please let me know if you have any questions/comments/etc.

If you’re running into failures that might be related to this change there are 2 API changes to look for:

The signature to Butler.queryMetadata has changed slightly, from
def queryMetadata(self, datasetType, key, format=None, dataId={}, **rest)
to
def queryMetadata(self, datasetType, format=None, dataId={}, **rest)
(note that key has been removed)
This is particularly nasty: because of **rest the execution won’t see the wrong number of arguments being passed to queryMetadata. (One good way to avoid this is to use they keywords when passing arguments).

The signature of the init function of ButlerLocation has changed, from
def __init__(self, pythonType, cppType, storageName, locationList, dataId)
to
def __init__(self, pythonType, cppType, storageName, locationList, dataId, mapper, access=None)
(note that mapper and access=None were added)
So when creating a ButlerLocation, code must now pass in a Mapper that found the info being passed into the ButlerLocation. Access is optional (for now at least), and if you don’t have one to pass in then let it be None.

Do I assume by “checked in” you mean “merged to master” and is this the work that was part of DM-4683?

yes

(this post must be at least 20 characters)

Ok thanks. Just a note that the LDM-463 document linked from here ends up with some text saying that things will change with DM-4683. Did those things change or is that text needing a new ticket?

The changes were made to ButlerSubset to accommodate multiple repositories. I’ll pull that note out of the document.