This is a question that already arose in several occasions. What is or what will be the mechanism to extract “standard” values from the fits header independently of the instrument. I am thinking of quantities like air-mass, exposure time, Julian date, latitude of the observatory, local sidereal time.
In the simultaneous astrometry (aka joincal) we are getting those value directly from the fits header with a “if camera==” block in order to select the right keyword in the header.
I prefer to think of this as a obs_* package camera definition question than a FITS metadata question; I think it’s the job of a CameraMapper to define how to get these values for an arbitrary instrument, regardless of how they appear in the raw data images. I don’t think CameraMapper has all the hooks to do this, especially because we don’t have places (in e.g. afw.image.Calib) to put much of this information.
IMO, DM-5503 is more about what’s in the FITS files we write.
I think this is a good example of the kind of work that has been falling through the cracks since we switched from R&D to construction, and I hope @mjuric and @jbecla or one of the working groups can figure out where it belongs. This needs to be at least partially driven by Science Pipelines (as those are the people who know what kinds of things we need to extract from raw data), but designing a general mechanism to extract it gets into the Butler/Mapper designs as well.
Really? I don’t see that at all (and I just re-read it). It’s all about how me make the translated information available to our code base isn’t it? That’s definitely what I thought it was talking about. In my previous header translation system the translation was bi-directional so you could set the generic value and it would turn up with the right FITS header when written to disk.
It definitely seems like the obs_* camera mappers need to be the places where this translation happens but someone has to define the dictionary and classes to represent the translated form.
I’ll add that DM-5503 was driven entirely by a discussion we had on HipChat about the jointcal if-blocks.
I suppose I was saying what I thought DM-5503 should be about instead of what it actually is about.
I do still think the only place we should really be thinking about “FITS Headers” is when reading raw data, and at that stage we can’t expect anything to be standardized, even if it should be.
Once we get to jointcal (which should be reading images and metadata persisted by our own pipeline), we’re not talking about FITS anymore, except as a way to round-trip our in-memory objects that should be entirely hidden from the rest of our code. I understand that an easy way to get some information into jointcal is to look for specific values in the unstructured PropertySet metadata, which effectively means looking at FITS headers, but I’d much prefer for us to have structured metadata classes attached to Exposures for these quantities, and for algorithms like jointcal to only look at those.
DM-5503 is entirely about translating those to internal metadata form and not about trying to standardize what telescopes use for headers in raw data. History shows us that it’s not possible for anyone to agree on anything with regard to what they write to data files.
which is exactly the point. We want to make a structure metadata class for providing easy access to everything our algorithms need without having to understand where that information came from in a particular data file. That’s exactly what I thought DM-5503 was all about (and exactly what I did in ORAC-DR [without it being a structured class rather than a dict with defined standardized keywords]).
Ah, in that case, my apologies for cluttering up the thread; I just read the issue title and confused it with some other issue I’d seen that’s probably old and possibly dead by this point. It seems we’ve been in agreement all along.
It may be that I’m misunderstanding @boutigny’s point. I’m glad we agree on the topic that we think this is covering.
What both Tim and Jim said (I think you’re agreeing?): we shouldn’t think about FITS headers anywhere in the stack, except at the persist/unpersist step, and having a structured way to access that metadata is what we want (I want it for the same reasons Dominique does!).
As for DM-5503, I’m somewhat surprised that’s listed as an alerts pipeline thing. I would have thought the butler would handle that, and provide the metadata structure when data is requested.
It’s an obs_* camera mapper thing (which the butler handles) but it’s not a butler thing because the butler itself doesn’t care how an Exposure object is laid out. It’s probably an AP thing because @KSK was the person to file it and he also has reason to worry about obs_decam.
From my (the user) perspective the only important thing is to be able to get a quantity like the air-mass or the local sidereal time in a standard way. I would like something like exposure.get(“air-mass”) which is working in the same way for obs_cfht, obs_decam or obs_cfht
I think the goal of having more structure is that you would say something like exposure.getExposureInfo().getAirMass() in C++ or exposure.exposure_info.air_mass in Python rather than having a less-structured keyword-based interface that then requires external documentation for what the keywords are and their meanings.
We decided long-ago that no-one should be parsing metadata (derived from fits headers or elsewhere). That’s why when an Exposure is created information that is deemed useful is removed from the metadata and converted into an object; two that come to mind are Wcs and Calib. In general this is going to be camera-specific, so I think it has to go in the obs package in general.
Doesn’t Dominique’s question come down to which object contains
air-mass, exposure time, Julian date, latitude of the observatory, local sidereal time
https://lsst-web.ncsa.illinois.edu/doxygen/x_masterDoxyDoc/classlsst_1_1afw_1_1image_1_1_calib.html
tells you that exposure.getCalib().getExpTime() and exposure.getCalib().getMidTime() return the exposure time and (modified) Julian date. The (ra, dec) of the boresight is a bit more of a pain,
import lsst.afw.cameraGeom as afwCamGeom boreSight = calexp.getDetector().getCenter() pointing = calexp.getWcs().pixelToSky(boreSight.getPixels(1)) print [_.asDegrees() for _ in pointing]
and relies on a decent per-chip astrometric solution that can be used for the centre of the field. I’d be happy to see that packaged up somehow.
Converting that (ra, dec) to air-mass and LST needs the latitude/longitude of the observatory. I don’t think we have a good place to put that (maybe in the camera object?)
(But be aware that there is no getPixels() in the current cameraGeom).