Switch from Swig to Pybind11 imminent

We plan to merge DM-8467 to master on Monday morning (after the latest weekly, w_2017_10, has been tagged). This is a comprehensive change affecting almost all packages in lsst_distrib.

From that point on, all C++ to Python wrappers will be generated with Pybind11 Instead of Swig.

At the same time a new DM Pybind11 Style Guide (DM-8577) and a new step-by-step Pybind11 wrapping tutorial (DM-9089) will be merged. Please see these ticket branches for documentation.

How does this affect me?

  • If you are using, or developing against, a stable release you won’t be affected at all. The latest, 13.0, release is still based on Swig.
  • If you are developing against the latest weekly, consider sticking to w_2017_10 for a while. Alternatively you can use the 13.0 release instead.
  • If you are developing in Python against master you might see small API changes (that will be documented in the release notes on Monday), but in general should not notice a major difference. The current Pybind11 switch attempts to minimize API changes from Swig. But Pybind11 enables more Pythonic API’s for wrapped C++ classes, so you may get nicer interfaces over time.
  • If you are developing in C++ and your code needs to be called from Python, you will have to add Pybind11 wrappers. See the aforementioned documentation.

I’m about to merge some code, what should I do?

Even if it is a pure Python package (which will probably just work). If it is part of lsst_distrib, please let us know before merging. You can reach us on Slack at #dm-pybind11-porting, or reply to this post. We are also happy to assist you at the JTM hack session to help you add Pybind11 wrappers.

Can I try it already?

Yes. Using lsstsw you can issue:

rebuild -r tickets/DM-8467 lsst_distrib

to build a Pybind11 stack. Note that there will be quite some churn the coming days due to cleanup, commit squashing, etc. But please feel free to try it out!

2 Likes