RFC-225 reminded me that we still haven’t come up with a fix for the single biggest problem (IMO) with our C++ documentation system: the huge number of Doxygen warnings that are due to Doxygen parsing failures rather than actual problems in our markup. This makes problems in the markup a lot harder to spot, of course, allowing those to proliferate as developers give up on warning-free builds.
Most of these come from Doxygen’s failure to reliably match function definitions in .cc files with their declarations in .h files, especially when templates and/or namespace aliases are involved. The easiest fix for that is to just disable Doxygen parsing of .cc files, at least once the Doxygen comments have been moved to .h files (where they should be for all new code). But that also prevents the .cc files from being included in the Doxygen outputs. I’m not personally convinced we should be including header files (let alone .cc files) in Doxygen; that’s the third place I’d go to look for source code (after my local copies and the GitHub web interface), but I do agree that having links from the reference documentation to the code where they’re defined is useful.
So, where do we go from here?
Does anyone have enough experience with Doxygen to have any ideas on how we could fix this sort of problem by configuring it better? I couldn’t even figure out how to turn off warnings about this particular problem, but that was a long time ago, and Doxygen made have gotten better at this.
Now that we’re using GitHub much more, is having the .cc files in the Doxygen outputs still useful? How many people are currently using Doxygen HTML to look at source code?
There is an open ticket for updating doxygen (DM-4891) but it’s blocked because of a cmake version issue on CentOS7 and I haven’t spent any time working out whether that’s a hard version requirement or a soft one.
I checked wth Jeff a year or two ago, and we could send some money to Dimitri the Doxygen to get the bugs fixed. This is orthogonal to the which-files-do-we-doxygenate, but still worth thinking about IMHO.
No idea, but it seems a trivial thing to try and you can just build doxygen using that ticket branch and see what happens if you have a reasonably modern cmake.
How much work would it take to move all Doxygen comments from .cc to .h files? It’d be nice to add this to RFC-225’s standards update and maybe do this at a meeting hack week at the same time as we’re also updating Python docstrings. cc: @KSK@swinbank
My plan is to not have source code inlined with the API reference. Instead, my plan is to link to GitHub’s source pages (linked for that branch/tag). Same for Python API references.
Separately, @timj pointed out to me that a lot of doxygen warnings happen in the per-package doxygen builds from links to APIs that exist in other API packages. This may be a problem for me since I plan to harvest Doxygen’s XML output from per-package builds for the overall Sphinx documentation build.
There are a lot of older classes in afw and meas_algorithms that still have Doxygen markup in the .cc files, but I bet the transition could still be done by a single person in a week.
This sounds excellent, and I imagine it’d be quite doable with probably just some regex parsing of the Doxygen XML, but I think you’d still have to make use of Doxygen parsing those source files in order to know where a particular class or function is declared and defined if you wanted to link to it at that level of granularity.
As long as the references obey the eups dependency ordering (packages can refer to packages they depend on, but not packages that depend on them), these should be handled by Doxygen’s “tag” mechanism. That doesn’t always work as well as it should, though, and I don’t know if that’s an intrinsic limitation or because we’re doing something wrong. And you’re right that it will never correctly handle references that don’t obey the eups dependency ordering.