New tools for documenting ad hoc scripts in the LSST Science Pipelines

Tags: #<Tag:0x00007f61a4080b30>

(Jonathan Sick) #1

With the LSST Science Pipelines, we include many useful ad hoc scripts. These scripts are included in the bin.src/ and bin/ directories of packages but aren’t command-line tasks. Examples are of the verify package and (potentially) of the skymap package.

Today we’re releasing tools and guidelines to help you write documentation for these scripts that appears on our documentation sites like This post will quickly introduce you to the new templates.

Argparse-based Python scripts

Most of our ad hoc scripts are written in Python and use argparse to define their command-line interfaces. We’ve included the autoprogram directive to make documenting these scripts a snap. autoprogram reflects the arguments, options, and help messages you’ve already added to your argparse.ArgumentParser instance into the documentation.

See the Argparse-based script topic type page in the Developer Guide to get started.

You can see an argparse-based script topic in action for the script.

As the docs state, you need to arrange your script so that an importable function generates the argparse.ArgumentParser instance. In general, this means that your script in the package’s bin.src/ directory will be a stub that imports and runs a function like main() within your package’s Python modules. This pattern has many benefits, though, including enabling unit testing of your script’s business logic.

General ad hoc scripts

You can also document other types of scripts, such as shell scripts. See the Command-line script topic type page in the Developer Guide to get started.

Use sqrbot-jr to get a head start

Our Slackbot, sqrbot-jr, makes starting documentation pages easy.

In Slack, open a direct message with sqrbot-jr, then type

create file

Pick either Script topic (argparse) or Script topic (generic) from the list of templates. Fill in the variables, copy the generated boilerplate, and add the new file to the scripts/ subdirectory of the module documentation directory.

More documentation improvements to look forward to

Script documentation is just the first of several documentation tooling features that we plan to ship this summer and fall. In SQR-032 we’ve outlined several essential features and our plans for delivering them, including: doctest-based documentation testing, Jupyter Notebook-based tutorials, integration of the Sphinx build with the scons harness, and integration of uploads with the stack-os-matrix Jenkins job. If you’re at all interested in these features, please look over SQR-032 and let us know what you think in this topic.