ingestImages.py is failing on some CFHT images downloaded by @idellant from the CADC portal. This is coming from the method : getExtensionName(md) in pipe_taks/…/ingest.py which is looking for 2 separate “EXTNAME” entries in the metadata and only found one in @idellant exposures.
In a regular CFHT exposure we have:
EXTNAME = ‘COMPRESSED_IMAGE’ / name of this binary table extension
EXTNAME = 'ccd02 ’ / CCD number in the mosaic
While in the faulty one we only have the second entry.
Does this double “EXTNAME” entry mandatory in the fit standard, meaning that CADC has screwed up something during image processing ? Or is it non mandatory, in which case we have to fix pipe_tasks/…/ingest.py in order to accept the single “EXTNAME” ?
I don’t even understand what two
EXTNAME values in a single header means. Do you mean that you have a MEF with a table and an image and those are the two
EXTNAME headers. That makes more sense. If that’s the case, is the binary table there but lacking
EXTNAME or is it missing completely?
It looks to me like this is an image HDU that has been tile-compressed (which turns it into a binary table). That transformation probably added the “
EXTNAME, and didn’t remove the existing
ccd02 one. I have no idea if that’s valid, though I wonder if
ccd02 is an invalid
EXTNAME and that caused a compression library/utility to miss it.
EXTNAME = COMPRESSED_IMAGE is for tile image compression. I guess two
EXTNAME headers does make sense (the
ccd02 is the actual name and then compression has to add a second one). Is it possible that the file without the
EXTNAME is a file that has not been compressed?
A quick grep of the FITS tile compression convention doesn’t turn up
EXTNAME anywhere, so it seems adding
EXTNAME = 'COMPRESSED_IMAGE' is not necessary. I’ve recommended to @boutigny that we change the ingest code to check for 2
EXTNAME entries and discard any containing
COMPRESSED_IMAGE (instead of blindly grabbing the second entry), but I’m worried that that is still pretty fragile. It might be better to force individual obs packages to override this.
Right, but I don’t think it’s required, so we can’t claim that CADC has generated malformed files here.
Just to close this topic : The fix has been implemented in DM-10487