Use python linters!

Data Management
#1

Please run a linter on your Python code before checking it into the stack. Unlike C++, many Python errors will not show up until a particular branch of code is executed. Running a linter can catch many errors that might otherwise be missed.

The two most common linters are pyflakes and flake8. The latter adds PEP-8 style checking, which can be useful once you configure it to eliminate false warnings. pyflakes catches important errors such as trying to use a variable that has never been defined. It also warns you of unused imports and setting variables that are never used, and rightly complains about from foo import *

Many editors support automatic live linting of Python code, including Emacs, Sublime Text and Atom. I strongly using automatic linting if your editor supports it, the linting is more likely to occur and you will catch errors earlier.

1 Like
#2

Sublime Text details: install packages SublimeLinter and SublimeLinter-plyflakes. Then configure SublimeLinter as follows (until we switch to Python 3, presumably with future for backwards compatibility):

    "@python": 2,

I also recommend that you bring up a dialog with errors on save, though this can drive you nuts if you are editing code that includes warnings you want to leave for some reason (that will abate as we start insisting that our code passes pyflakes:

    "show_errors_on_save": true,
#3

For vim, I use and advocate the syntastic plugin. It requires you to install flake8 separately. The README has details on how to customize the error codes that flake8 cares about (e.g., exceptions to PEP8).

For lines of code that you want to be exceptions to flake8 rules, you can append a

# NOQA

comment to that line of code. Then Flake8 will ignore errors on that line.

I often find this useful in __init__.py files where I import modules to setup a namespace, but of course do not use those modules in the __init__ module itself. (Pyflakes catches unused imports, among other things)

#4

Atom details: install package linter and linter-flake8. (Note that there is a linter-pyflakes but last I checked it was broken with no sign of development.)

You will probably want to configure linter-flake8 to reduce spurious warnings. I have Max Line Length set to 110 and Ignore Error Codes set as follows (which may be too aggressive, but you will at least want a subset):

C901, E112, E113, E121, E122, E123, E124, E125, E126, E127, E128, E129, E131, E133, E201, E202, E203, E211, E221, E222, E225, E226, E227, E228, E231, E241, E251, E261, E262, E265, E271, E272, E301, E302, E303, E502, W292, W293, W391
#5

Emacs users should speak for their own setups, but https://github.com/flycheck/flycheck seems to be the way to use Flake8 from Emacs. Search this page of the docs for “Python”.

#6

Atom users also have another Flake8 plugin: https://atom.io/packages/flake8

… but linter-flake8 that @rowen mentions above seem more comprehensive at first glance: https://github.com/AtomLinter/linter-flake8

#7

Yes, linters are amazing. SublimeLinter also supports pylint, which is slightly different from pyflakes in a way that I don’t quite understand.

This does bring up a good point, though: we’ve likely got plenty of from foo import * scattered around. I would be of the opinion that such things should go away, since the make certain linting checks impossible and it generally makes code less clear. But we should specify where (outside of _init_.py, if anywhere) it’s allowed to import * in our standards.

#8

We already disallow from foo import * except in __init__.py files.
https://confluence.lsstcorp.org/display/LDMDG/Python+Coding+Standard
see Consistency with the DM C++ Coding Guide namespaces SHOULD be followed