Centrally-defined butler datasets

As part of my work on DM-3372 (“stack provenance”), I added a new feature to the CameraMapper that allows us to provide a default specification for datasets. This allows us to add datasets that will be defined in the same way for all cameras. This meant that I didn’t have to add a packages dataset in all the obs_* packages (to save the package version information), but by putting it in a single place it takes effect for all the obs_* packages. Beyond that convenience, I hope it will also allow us to simplify the existing obs_* packages by moving most of the coadd products into a single place.

The default dataset specifications are stored in daf_butlerUtils/policy as images.yaml, exposures.yaml, calibrations.yaml and datasets.yaml, according to which category of dataset it is. They have the same information as what would go in our *Mapper.paf files, but (as the file extensions suggest) are in YAML. Files that don’t currently exist are treated as empty.

@price, why did you break the different categories into separate yaml files, instead of having one yaml file, named something like “defaultPolicy.yaml”?

It was my first foray into YAML, and it seemed dividing the namespace using files was neater and/or easier than having multiple namespaces in a single file (e.g., all elements in a file have the same format). You’re welcome to refactor it if you want.

understand. thanks.

(this post must be at least 20 characters)