New approach and tools for pybind11 wrappers

I’ve just merged DM-10384, which implements RFC-529 and establishes a new approach to organizing our pybind11 code to address three issues with our current approach:

  • the total size of our binary modules is unnecessarily large due to symbol duplication;
  • much of our wrapper code (especially in afw) is fragile due to poor handling of circular dependencies between classes;
  • our __module__ strings are too long, as they include implementation-detail packages and modules that should not be exposed to users.

Unfortunately, the new approach cannot easily be applied incrementally within a package; in general one either needs to convert the entire package or none of it. Until the new approach is in place across the board (only utils and geom have been converted so far), it’s fine to continue using the old one in packages that have not yet been converted.

If you’re adding bindings for a new package, converting an old one, or just want to understand the code you’re seeing, please check out the new pybind11 how-to section in the DM developer guide.