Hi, I noticed some weird behaviour when trying to run some metrics using the healpixel spatial slicer. There were a small number of visits that seemed to not be counted at all when running my metrics.
To investigate this I got the list of all observationId’s that are present when I use the healpixel spatial slicer. This confirmed that some observationId’s that are in the initial visit database do not appear in the dataSlices and therefore would not be counted by the metric.
Interestingly this only appears to be a problem at low healpixel resolution (nside = 32). When I increased the resolution to nside = 256 the missing visits now appear in the dataSlices. So I am asking if anyone knows what might cause this behaviour, mostly out of curiosity for how healpixel and maf are working here.
Ok, so I should poke at this more but I’m going to suggest this:
Think of the healpix slicer as identifying a grid of “here is where I am going to measure a metric”. And so at that healpix point, it pulls in visits which overlap that particular point on the sky. It doesn’t pull in visits which do not overlap that point. So you’re sampling your metric at low resolution, but you’re not making the visits cover more sky than they really do.
So yes, this is what I would expect to happen with a low resolution healpix slicer – that it would not include all visits. They didn’t overlap the healpix grid points on the sky, so they did not contribute to your metric.
Okay I think I understand a little bit, sounds like it’s is to do with the extent of the visit footprint compared to the healpixel grid?
Although even if a visit doesn’t overlap a particular healpixel at low resolution surely it should appear in different healpixel/dataSlice? Therefore I would have expected that all visits should be counted at least once when evaluating the metric across the whole sky…
If the healpix resolution is low enough that the healpix points are spaced far enough apart that none overlaps a particular visit, then that visit wouldn’t show up in any heal pixel.
At nside=64, the resolution for the healpix grid is ~0.9 deg, so a single visit should show up in on the order of (3-4)^^2 (maybe 12?) healpixels. At nside=16, the resolution is about 3.66 deg, so usually visits should show up in at least one heal pixel, but it’s also possible they would squeak between. At nside=8, the resolution is about 7.3 degrees, so I would expect visits to be routinely missed by not showing up at any healpixel because the heal pixels are spaced too far apart.
I guess I am surprised that you’re finding missing visits at nside=32, but if it’s only a few … maybe I could see that it might happen, given that MAF does apply an approximate map of the focal plane so there are corners and raft gaps. In your notebook it looked like you were missing about 2.5% of visits at nside=32, so this could be within the realm of “the healpix fell into a raft gap”.
I hacked together a visualization of this, of sorts below. The red points are the healpix points at nside=64, the green at nside=32, and the black at nside=16 (note that they aren’t generally the same points in ra/dec).
The blue circles are approximately about the size of the Rubin FOV --and I put in an approximate outline of the footprint at a random rotation angle underneath in gray. A healpix slicer with nside=16 would likely only identify one of these visits while the other two would be ‘invisible’ , while nside=64 would sample all of these visits. At nside=32, it does look like you should find all of the visits, but maybe some were unlucky? (you could imagine the corners of the footprint hitting two or so of the grid points, and then if the other one or two fell in a raft gap?).
Ah, I think I understand now. I was originally thinking of healpixels having spatial extent as opposed to being points as you have described. So I can see now how the footprint of the visit may not overlap the healpixel points.
This might explain something else I noticed recently. I was trying to get an estimate of the area of each visit footprint so I ran a CountMetric with an sql query for an individual visit. The healpixel map for a single visit seemed to have holes, but that would be explained now by a particular healpixel point lying within a visit gap.
