What is the philosophy of using setupOptional?
When should one include a package as setupOptional? When should one not?
What are the downsides of including too many packages as setupOptional?
What are the downsides of not including a potentially relevant, but not required, package as setupOptional?
I personally dislike optional dependencies in EUPS, and would generally like to decrease our use of them. While they work fine with EUPS itself (and can be quite useful there), they cause problems for package distribution systems (whether that’s eups distrib or something else), since distributions will always have to make a boolean decision on whether to include the optional package or not. And at least with eups distrib, once that decision has been made by the packager, I the installed package’s dependency isn’t optional anymore - it’s either required or not. That makes sense from the perspective that eups distrib wants to replicate the packager’s environment on the user’s system exactly for reproduceability, but it really limits the utility of optional dependencies.
In SDSS ops, we used them for things like numpy/scipy/astropy that might have been installed and managed at the system level, and might have been tracked by EUPS. Since LSST tracks some of those things (e.g. numpy) with conda, while other external packages are explicitly managed by EUPS always, this probably isn’t a usecase we have need for.
One thing that may have prompted this question is whether validate_drp should list pipe_tasks, obs_cfht, and/or obs_decam dependencies, even though they are only used in examples and in commands the documentation suggests that a user run. They are not used by executables, library code or unit tests.
At present obs_cfht and obs_decam are listed as optional dependencies and pipe_tasks is omitted. I personally would be more comfortable omitting obs_* packages and listing pipe_tasks as a dependency (optional or otherwise), since users are typically only interested in one camera and can be expected to set that one up. But maybe they should all be ditched. It will work as long as the documentation is clear about what the user must set up.
The example by @rowen is of course what prompted thinking about this, but I’m asking for a general statement of policy/recommendation/explanation of philosophy to help guide developers.
setupOptional has sometimes non-obvious interactions with building, packaging, and use. Some examples were highlighted by @jbosch in his first response. There is currently no guidance about what to do and there’s a variety of opinions.
For example, I would be interested in the (potentially differing) perspectives from Apps, Packaging, and Architecture.