I’ve recently worked on a pair of small tickets (DM-5084 and DM-5086) that can’t really be tested in isolation, but are quite easy to test once the outputs of a full run of the pipeline (i.e. through multi-band coadd processing) are available. They’re more or less trivial checks to make sure that some new feature was in fact enabled, or that some configuration change is having the expected effect. These are precisely the kind of issues we’ve had trouble consistently getting right on the HSC side, and I think it’d be very valuable for us to start adding checks of this sort as we port that code to LSST.
I’m currently planning to drop those tests into ci_hsc, because that’s very convenient and it helps me confirm specifically that things are working with HSC so I can validate the HSC merge, and even if it’s not being run automatically, it is being run manually often enough that it’ll still be a big improvement in test coverage.
But I know ci_hsc is likely to be refactored quite a bit before it’s included with the rest of the CI, and there are some other validate_* packages that I think play a similar role that maybe I should be thinking about as well.
So, @frossie, @mwv, @nidever: what can I do to make these tests more useful to you in the future? I don’t think there’s any work I need from SQuaRE right now, but I’d like to make sure these tests are captured long-term and expanded to run on more cameras than just HSC.