As you may know, we’ve been building a new approach to documenting the LSST Science Pipelines (the DMTN-030 plan) and LSST’s EUPS-based products in general. In this plan, we’re writing docstrings in the Numpydoc format, writing guides in reStructuredText, and building the pipelines.lsst.io Sphinx site as part of our Jenkins CI continuous integration pipelines.
We’re now on the verge of flipping the switch and fully adopting this new documentation system. This post will get you up to speed on the latest developments and our next steps.
New documentation on contributing documentation
If you’re a DM developer writing documentation, you’ll want to look at the documentation on the layout of doc/ directories and page templates (called topic types). Our goal with these topic types is to give you starting points for documentation that make it easier to contribute documentation, make it easier to organize and maintain the documentation, and provide a more consistent experience for our readers. These templates also appear in our Stack package template project.
If you want a broader look at how the documentation system works, there’s an Overview of the Stack documentation system.
Recent changes to the templates and documentation build system
If you’ve participated in the doc directory upgrade and numpydoc conversion activities, you’ve already got some experience writing and building documentation in the new system. We’ve improved upon the templates and tooling you’ve used, so I want to summarize the key changes.
New build tools
The stack-docs and package-docs tools replaced the
build-stack-docs tool and the direct use of
sphinx-build. These tools are implemented with the terrific Click package and include both
clean subcommands. Again, the Developer Guide has tutorials on how to use both of these tools.
Most package documentation directories are going away
A package documentation directory is something like
doc/pipe_base, as opposed to a module documentation directory like
doc/lsst.pipe.base. We no longer use package documentation directories for packages that provide Python modules or C++ namespaces.
We made this change based on feedback that the packaging system can be confusing to non-developers. Since most users will be working with documentation related to the Python module (see the Module homepage), it makes sense to provide links to the associated GitHub repo and Jira component from there. Over the new few days I’ll be PRing these changes to our
We updated the recommended usage of
automodapi. See the template for the module homepage. By default, we recommend using the
no-inheritance-diagram options. You’re welcome to still use inheritance diagrams in cases where it’s useful.
doc/_static directory is no longer required. In fact, you probably don’t need it for your packages.
Where you find the new docs right now
Documentation following the new system is being built and published daily by our Jenkins CI (currently by the documenteer job attached to the release pipelines).
You can find the latest daily release at https://pipelines.lsst.io/v/daily.
The latest weekly release is at https://pipelines.lsst.io/v/weekly.
Also, you can find documentation for specific daily and weekly release tags. For example: https://pipelines.lsst.io/v/w_2018_29.
Switching over to Jenkins-based builds
The next step is to entirely switch the build for https://pipelines.lsst.io from Travis CI to Jenkins CI. This change likely won’t become visible on the homepage until the next major release.
As a side effect, that means changes to the
master branch of the pipelines_lsst_io repository aren’t automatically built and published to the homepage of https://pipelines.lsst.io — only a new major
lsst_distrib release will update the https://pipelines.lsst.io homepage.
We (or at least, I) don’t have a clear plan of how we’ll introduce patches to the documentation of published releases. I’ll be talking to the release manager to see how we can introduce such a process.