What's the real intention of the installation test/demo?

In prototyping new docs for the Stack, I’ve been seeding it with our existing source code installation instructions. The final step is to test the stack installation by running a demo project. After looking closely at our installation instructions, I’ve started to wonder about how things can be improved/streamlined, and one item has to do with the test process.

My issue is: we test the Stack by running a demo, and sort of use it to teach people about using the Stack, but don’t really test the stack because we don’t actually give a one-liner command to run and validate the output (I know we provide a bin/compare script, but it’s buried in the lead of the instructions).

My thought is that

  • the test should be a one liner command after installation with easy to understand validation (In fact, should users be running a unit test suite (or subset thereof) instead of this smoke test?),
  • teaching the stack should be saved for dedicated tutorials in the docs.

Before I get too deep into thinking about streamlining the user installation docs, I was wondering if people are actually using the demo project in ways that aren’t clear to me as a newcomer.

I thought that for LSST sconsUtils packages we do usually run the per-package unit tests on installation.

The problem with running extensive integration tests on the installation is that they will typically require much larger datasets that we don’t want to force users to download. Having some minimal integration test is a good thing, though, in my opinion. The demo is not structured as such, but I would think it should eventually be turned into that kind of test.

Dedicated separate tutorials is fine with me.