Python Style Guide Update: PEP 8, flake8 & Numpydoc

Last week I mentioned that work was underway to improve the usability of our Python style guide (RFC-162). This work in DM-5456 is now merged (:tada:) and I encourage you to take a look:

https://developer.lsst.io/coding/python_style_guide.html

With one key exception (numpydoc, see below), nothing about our Python style has changed; we’ve only made it clearer how our style relates to the Python community’s style guide, PEP 8. If you haven’t already, I encourage you to read through PEP 8 and then read our DM Python Style Guide to learn about DM-specific guidelines.

Flake8 and autopep8

A key feature in the revised DM Python Style Guide is clear guidance on how to lint your Python for stylistic errors using flake8 and how to automatically fix many style issues with autopep8. See the sections:

SQuaRE plans to provide flake8 linting as part of CI (DM-5637). As flake8 becomes a regular part of our workflow, it becomes important to declare all lines of code that purposefully deviate from the style guide for a good reason. For this, see the guideline:

Props to @parejkoj, @timj and @frossie for the research behind this feature. :clap:

Numpydoc

This style guide update also documents the decision in RFC-214 to officially adopt Numpydoc as the docstring format in all DM Python code. Previously we used Doxygen formatting.

For documentation on the Numpydoc format, see: https://developer.lsst.io/docs/py_docs.html

What does this Numpydoc change mean?

RFC-214 has essentially given DM developers permission to use the Numpydoc format for docstrings. That is, you aren’t required to dedicate effort now to change all docstrings; rather, you’re allowed to write (or re-write) docstrings in numpydoc so that you won’t need to re-format those docstrings in a month when Sphinx and Numpydoc become our core documentation stack.

This does mean that doxygen will be broken for numpydoc’d docstrings. But in RFC-214 we decided this was an acceptable interim state.

Also, we haven’t rolled out the capability to render Sphinx/numpydoc documentation for Stack packages. You may end up writing numpydoc docstrings that contain syntax errors. Again, this will only be a temporary problem. I’m already beta-testing the new documentation stack on some packages and it shouldn’t be long before documentation builds are rolled out across the Stack.

Learning numpydoc

The key to Numpydoc is that it specifies a well-defined pattern of sections in reStructuredText. Docstrings for classes, methods/functions and modules are composed from these sections.

The Numpydoc style guide can be a lot to digest in one reading. In the beginning, I recommend that you master the summary sentence, parameters, and returns sections. This will be sufficient for basic API documentation. Once you’ve mastered this, you can look at incorporating other sections into your docstrings.

I’m looking into providing some numpydoc tutorials and examples so that we can all work together to build amazing API docs.