This is in some sense an alternate proposal to the one on the table for RFC-81, but I didn’t want to hijack that issue page (and I wanted to get some more Discourse experience).
Before we start a big, stack-wide effort to improve our Python interfaces with a fundamental change to how we map C++ to Python, as proposed in RFC-81, I think we should consider actually implementing a number of changes that are either easier or less controversial, and then reassess where we stand with respect to providing a quality Python interface.
My list of such changes:
- Turn on Swig’s support for keyword arguments in Python. This should give us keyword argument support for all non-overloaded methods, for very little additional effort.
- Add properties (via Swig
%extendblocks) for the most frequently-used getters and setters, including (but not limited to):
- Replace heavily-overloaded constructors in
afw.imagewith static method factories. This should produce more readable code in both languages, and avoiding overloading here will allow us to use keyword arguments automatically.
- Replace usage of
std::vector<T>for numeric scalar types in frequently used C++ classes with
ndarray::Array(which will naturally convert to
- Identify the top N (N=5?) most important unpythonic interfaces in need of improvement, and either:
- attempt a C++ refactor with the aim of improving both interfaces (if the C++ interface is in bad shape, too)
- add Swig
%extendblocks to customize individual methods. This can include the addition of new helper methods or adding support for keyword arguments via
- Generate reference documentation for Python code by introspecting the Python modules themselves (which would allow us to include docstrings added in the Swig layer, which are ignored entirely by Doxygen). I think this is relatively straightforward to do with Sphinx, at least as a proof-of-concept.
My personal hope for our C++/Python bindings is to move away from Swig towards Boost.Python or something like it, so I’m not suggesting we go crazy in trying to use Swig
%extend blocks across the full stack. But I think we should go far enough to compare that approach to a shim or Boost.Python approach to customizing the mapping. More importantly, I think it’s very important we disentangle the multiple different factors that currently contribute to our poor Python interfaces:
- interfaces that are bad in both languages
- interfaces that are natural in C++ that simply don’t translate well to Python (regardless of tools)
- interfaces that are bad because of the limitations of the tools we’re using to map from C++ to Python
- interfaces that are bad because of the way we’re using those tools