Changes to default handing of `AnnotatedPartialOutputsError`

Data Management DM Notifications
#1

I’ve just merged DM-44488, which belatedly finishes off the implementation of RFC-958 by introducing special handling for the new lsst.pipe.base.AnnotatedPartialOutputsError exception:

  • by default, when this exception is caught during PipelineTask execution, executors will log the error message at ERROR level and then proceed, just as they do with NoWorkFound. This allows (but also requires) downstream tasks to be prepared to handle partial outputs (e.g. an Exposure with no WCS).
  • the --raise-on-partial-outputs option may be passed to pipetask run and pipetask run-qbb to raise this exception (or, rather, the exception it chains, since that’s how it’s supposed to be used) immediately instead.

Since the new CalibrateImageTask is the only place we currently raise this exception type and that’s currently only in use in AP, and we expect to use the equivalent of --raise-on-partial-outputs in prompt processing, the immediate effects of this will be fairly narrow. But I do expect us to have to spend some time improving the failure modes of downstream tasks to at least improve error messages. On the other hand, we will no longer be blocked from trying to “recover” from earlier failures in later steps that often have access to more data, especially once CalibrateImageTask is used in the DRP pipelines as well.

1 Like
#2

Thanks for finishing this part of RFC-958.

One thing we don’t currently have is good documentation on what the use cases of AnnotatedPartialOutputsError and AlgorithmError are. That’s not helped by the fact that our sphinx doc system won’t show methods on exceptions (see Astropy Issue), so the explanatory docstring–and in fact the entire method!–for annotate isn’t in our docs. You can at least see the docs for it on the method itself.

We’ll try to put together some high level docs on how PipelineTasks and Tasks should handle different kinds of error.

Note that one requirement of getting AnnotatedPartialOutputsError is that the failing subtask has to raise an AlgorithmError; we have several tickets in progress to convert e.g. RuntimeError and other more general exceptions into Task-specific errors. For example, AstrometryError and its subclasses are raised by AstrometryTask, and thus CalibrateImageTask will fail with partial outputs in the manner described by Jim above, whereas PsfexPsfDeterminer will just fail with RuntimeError until DM-44034 is completed.

1 Like