I actually think it’s getting to be time to drop
ProcessCoaddTask. We should now be using the multiband processing tasks on coadds whenever possible, and to the extent we still want to support single-band processing on coadds, I think we should reimplement that as a simplified version of the multiband tasks.
That will eliminate the need for
ProccessImageTask, which only existed to share code between
ProcessCoaddTask, and I’m sort of happy about that, because I think
ProcessImageTask's relationship to it subclasses is a very big reason why the logic in
ProcessCcdTask is hard to follow. Essentially, I think we’ve reached the point where processing CCDs and coadds is different enough that it doesn’t make sense to try to share code above detection/deblending/measurement subtask level.
But this obviously requires an RFC, and that would probably require coming up with a plan on how to make the multiband tasks more user-friendly on generic single-band coadd images. That’s otherwise fairly orthogonal to the
CalibrateTask work and a fair amount of effort in its own right. So I guess I’ll just say that we shouldn’t worry too much about code duplication in
ProcessCoaddTask, as I think we can consider it deprecated even if we can’t remove it. That might mean we can just remove
ProcessImageTask right now, but I’m happy to leave that as an implementation question to @rowen.
What about making
IsrTask responsible for installing an initial default
Psf? That will make the
ImageCharacterizationTask outputs the same in terms of which objects are present in the
Exposure (they’d differ only in the quality/correctness of those objects).
And if we have a script for running these tasks on images that didn’t go through our own ISR, that’s going to need to do some other manipulation of the external images anyway to get
Exposure objects; it might as well install a default PSF (or whatever better one might be available) as well.
Yeah, I don’t want to dive too far into philosophy on this right now either, but I will say that I agree that this is a worthwhile goal as long as it doesn’t get in the way of making our pipelines as good as they can be on well-characterized data, and I do think the more we do on that the less relevant it will be to completely generic data.
Given the desire to be able to run
CalibrateTask on the outputs of either
ImageCharacterization (and, downstream, a desire to run coaddition on the outputs of any of these), I think we want to think of all the datasets produced by these tasks (including the one we currently call
postIsrCcd) to be versions of
calexp. I think we may want to think about what sort of features we’d want in a “versioning” feature intrinsic to the Butler, and put that in as a feature request to @npease. At present, I think we have two options, and I’m not a huge fan of either of them:
- Just add a string to the front or back of “calexp”, indicating at what stage it was produced, and use a configuration parameter on the tasks to tell them which ones to read and write. This is obviously quite similar to how we deal with types of coadds.
- Add a “version” data ID key (maybe a string or a number), and just use the “calexp” dataset for all of these inputs and outputs.
One key question would be whether we want to consider any kind of overwriting existing files, and whether we can consider any updates to subcomponents (e.g. do we have to write the whole image if we’re just updating one mask plane?).