Does DM need absolute FITS metadata standardization?

How should DM deal with the fact that the different camera test stands deliver similar data with different headers. An obvious thing would be to force the different teams to use the same standard, but that seems to be difficult logistically. We could also have different mappers for different test stands. Finally, we could have a post burner that sanitizes everything before ingestions.

@gpdf this came up as an action item for us at the last DMLT.

Why can the camera team not provide consistent headers? Has this been discussed with the proper people at slac?

1 Like

Maybe I don’t know who the proper people are. I have talked with several people and it still hasn’t happened.

Would this involve EPO as well, assuming the public portal may display object data from the FITS headers?

I’m not sure what you mean by object data. We will certainly have to standardize our published data format.

This question is more about whether we force the current distributed research groups to come to consensus now, or deal with the differences in software.

It definitely seems like we should standardize on the headers. I don’t see how they could contain qualitatively different data. By the time we’ve written a separate mapper for each of them, we’ve done all the work necessary to unify their outputs.

On the other hand, if there are important qualitative differences between the test stands, having separate mappers (or slightly separate configurations for the same mapper) might be a good idea.

I’m presuming the angst has largely been triggered by MonoCam. Which may well now be water under the bridge in that I don’t know if there will be any more observations with it. It was not done with official camera supported software…

The camera team proper has documented its headers in LCA-10140 for the (TS3) acceptance test stands at BNL. We were not able to get the vendors to comply, so we had to write afterburners to make compliant headers for their data. I’m not sure if DM has any actual interest in the acceptance data, but if there are issues with the headers it would be good to hear about them.

I’m not sure how much we can change the single sensor data headers at this stage of the game, but the raft-level testing headers are under discussion now, in LCA-13501. We’d certainly be happy to hear suggestions there.