I was using the htm20 method to obtain the calexps I needed, and now I’m switching to the OVERLAPS method.
Since I need all calexps from a region, I’m using the following code:
target_point = SpherePoint(Angle(self.ra, degrees), Angle(self.dec, degrees))
RA = target_point.getLongitude().asDegrees()
DEC = target_point.getLatitude().asDegrees()
circle = Region.from_ivoa_pos(f"CIRCLE {RA} {DEC} {self.radius}")
datasetRefs = butler.query_datasets("calexp", where=f"visit_detector_region.region OVERLAPS my_region AND band IN {bands_str}", bind={"ra": RA, "dec": DEC, "my_region": circle})
I understand that this should retrieve all calexps intersecting the my_region, but when I plot them, I notice that not all calexps intersect with it. I’ve attached a plot with the first 40 calexps retrieved and the region in question. There are at least 3 calexps that do not intersect with the circle.
Thanks for raising this issue. You’re not doing anything wrong here! I think this is a known issue.
Basically, the sky region for individual detectors (calexps) has some padding, and is larger than the actual image boundary. Due to this padding, queries that request all images containing a given coordinate (i.e., OVERLAPS queries) will return images for which the coordinate is in the padding zone, and not in the image. You’re right to be using OVERLAPS and then just disregarding the few for which your target coordinate was in this padding region.
Extra details about why this is happening, and where to learn more.
The boundaries of a calexp are contained in the metadata s_region parameter. It is defined at the time when the visit is defined for observation, not after the image is acquired. The s_region parameter uses the camera geometry and the telescope boresight, and since it is essentially a prediction, it has some padding. Think of it as the anticipated outer boundaries that will contain the sky region imaged by that specific detector, and not based on the image WCS or astrometric solution.
Although the image metadata will evolve from what is available for DP0.2, and a query by WCS boundaries should become possible, users should anyway be prepared for cases where a target coordinate is within the s_region but is not covered by image pixel data. For example, detector defects, lost amplifiers, etc. would not affect the s_region either, but also cause a lack of pixel data at the target coordinate.
There is a demonstration of this known issue in DP0.2 tutorial notebook 02c on Image Queries with TAP, in Section 4. But the image included in the original post of this topic also does a good job of illustrating it.
I’m pretty sure this answers your question, so I’m going to preemptively mark this post as the solution for this topic – but as always, please do reply to confirm, or let us know if it didn’t resolve the issue. Happy to keep discussing in thread.
