Migrating from afw.image.Filter to FilterLabel

Tags: #<Tag:0x00007f27b919f1a8> #<Tag:0x00007f27b919ebb8> #<Tag:0x00007f27b919e280> #<Tag:0x00007f27b919da38>

Today we expect to merge DM-27169, which will let you retrieve filter information from the Exposure class as a FilterLabel. This class, introduced in RFC-730, replaces Filter's system of names, canonical names, and aliases with just two names: a band (e.g., “g” or “r”) and a physical filter (e.g., “g DECam SDSS c0001 4720.0 1520.0” or “HSC-R2”). Note that not all FilterLabel objects have both a band and a physical filter, especially during the transition period, so please program defensively.

The Exposure and ExposureInfo classes are gaining methods hasFilterLabel, getFilterLabel, and setFilterLabel; it will also be possible to ask the Butler for an exposure_filterLabel in Gen 2 and an exposure.filterLabel in Gen 3. The Filter-based methods and components are still useable, except in the special cases described below.

Backward-Compatibility

Existing code that uses Filter will continue to work, with the following caveats.

Filter-Based APIs

Exposure now stores filter information as a FilterLabel internally, but continues to provide methods such as getFilter() for backward-compatibility. The same goes for the Butler component exposure_filter.

Unfortunately, the conversion from FilterLabel back to Filter is imperfect: the getName() method of the returned filter will always return the band or, for HSC data, the disambiguating names “i2” or “r2”. This is consistent with Filter's past behavior for calexps, but a breaking change for raws, which previously used the physical filter as getName(). This inconsistency only affects code that calls getFilter().getName() on raw Exposures. Code that operates on calexps (a.k.a. PVIs), warps, or coadds, or that calls getCanonicalName(), should see no change.

We’ve done our best to migrate all Science Pipelines code that depended on the old raw behavior to FilterLabel. If we missed something, or you have third-party code that is affected by this change, we recommend replacing getFilter().getName() with getFilterLabel().physicalLabel.

Filter-Based Files

Any old-style Exposure files that used Filter will be loaded as Exposure objects that use FilterLabel. Where possible, these FilterLabel objects will be populated using information provided by official filter registrations.

However, there may be cases where the code wasn’t able to unambiguously determine the correct names. The most likely form will be a band-only FilterLabel created from a Filter that has many aliases (of which only one is the physical filter), but it may also occur if the conversion code was unable to access the Filter registry. This problem should be minimized once we implement DM-27178, which will be able to gather supplementary information from obs packages. Thank you for your patience while we get that done.

Transition

The deprecation cycle for Filter will be a bit complicated:

  • We will be phasing out use of Filter within Science Pipelines over the next few weeks (DM-27170). This will affect dailies, weeklies, and GitHub builds.
  • We expect Filter to be officially deprecated for third-party users in version 22 of Science Pipelines.
  • Once version 22 is out, we will remove Filter entirely from development builds, and create new APIs (Exposure.getFilter(), butler.get(exposure.filter), etc.) that return a FilterLabel (DM-27177).
  • These new filter-named APIs will become the official ones in version 23, and methods like getFilterLabel() will be deprecated.
  • In version 24 of Science Pipelines, getFilterLabel(), exposure.filterLabel, etc. will be removed (DM-27811), and the only APIs remaining will use the “filter” name, returning a FilterLabel.

We apologize for the back-and-forth nature of the transition, but we deemed it worth the extra disruption to end up with a natural-sounding API in the end while retaining backward-compatibility at each step.

2 Likes

Just to clarify, while new Science Pipelines code should be able to read old Exposure files as described above, the reverse is not the case: new-style Exposure files cannot be read with versions older than weekly w_2020_50 (the first version to write such files). The recent version 21 release reads and writes old-style Exposure files only.

1 Like

One case of incomplete filters we’ve run into is calexps written before w_2021_06, which updated most tasks to use FilterLabel internally (as described in afw.image.Filter is now deprecated). Though the Exposure format for these files used FilterLabel, these labels were populated using old-style Filter code, and therefore kept the old behavior of being identified only by their band.

This should not be a major problem in Gen 2, as these files can be patched on read following DM-27178; a Gen 3 fix is pending on DM-28583.