ingestImages.py is failing on some CFHT images downloaded by @idellant from the CADC portal. This is coming from the method : getExtensionName(md) in pipe_taks/…/ingest.py which is looking for 2 separate “EXTNAME” entries in the metadata and only found one in @idellant exposures.
In a regular CFHT exposure we have:
EXTNAME = ‘COMPRESSED_IMAGE’ / name of this binary table extension
EXTNAME = 'ccd02 ’ / CCD number in the mosaic
While in the faulty one we only have the second entry.
Does this double “EXTNAME” entry mandatory in the fit standard, meaning that CADC has screwed up something during image processing ? Or is it non mandatory, in which case we have to fix pipe_tasks/…/ingest.py in order to accept the single “EXTNAME” ?
I don’t even understand what two EXTNAME values in a single header means. Do you mean that you have a MEF with a table and an image and those are the two EXTNAME headers. That makes more sense. If that’s the case, is the binary table there but lacking EXTNAME or is it missing completely?
It looks to me like this is an image HDU that has been tile-compressed (which turns it into a binary table). That transformation probably added the “COMPRESSED_IMAGE” EXTNAME, and didn’t remove the existing ccd02 one. I have no idea if that’s valid, though I wonder if ccd02 is an invalid EXTNAME and that caused a compression library/utility to miss it.
Yes, EXTNAME = COMPRESSED_IMAGE is for tile image compression. I guess two EXTNAME headers does make sense (the ccd02 is the actual name and then compression has to add a second one). Is it possible that the file without the COMPRESSED_IMAGEEXTNAME is a file that has not been compressed?
A quick grep of the FITS tile compression convention doesn’t turn up EXTNAME anywhere, so it seems adding EXTNAME = 'COMPRESSED_IMAGE' is not necessary. I’ve recommended to @boutigny that we change the ingest code to check for 2 EXTNAME entries and discard any containing COMPRESSED_IMAGE (instead of blindly grabbing the second entry), but I’m worried that that is still pretty fragile. It might be better to force individual obs packages to override this.