EDP2: blendedness is negative for ~85% of primary objects, and mostly unflagged

Forgive my post if I just completely messed something up!

In the EDP2 Object catalog, blendedness is negative far more often and far more extremely than
in DP1. It is defined as a flux fraction, so I expected it on [0, 1] apart from a small noise
floor. I would like to know whether this is known, and what to use for sample selection meanwhile.

I have data rights; everything below is aggregate counts from TAP at data.lsst.cloud. I am an
independent user, so please read this as a question rather than a data-quality report.

The comparison. 0.2 deg cone on ECDFS (53.13, −28.10), which both releases cover, restricted to
detect_isDeblendedModelSource = 1 so the populations are comparable. “Valued” excludes nulls.

Band Release Valued Negative % negative < −10
r DP1 44,588 115 0.3% 0
r DP2 21,140 17,854 84.5% 2,060
g DP1 44,194 162 0.4% 0
g DP2 21,389 17,874 83.6% 1,393

DP1 goes negative too — up to 39.5% of objects in the crowded Seagull field — but stays near zero,
which is what I would expect from a de-noised estimator where child and parent flux are nearly
equal. Across six DP1 fields (119,713 valued objects) none falls below −10. The most negative
DP2 value I have seen is −702,404. The TAP_SCHEMA description strings for blendedness and
blendedness_flag are byte-identical between the two releases, so I do not think I am comparing
different definitions.

The flag does not track it. r_blendedness_flag fires for only a third of the negatives, and
gets rarer as they get worse: 66% are unflagged below 0, 86% below −10, and 98.7% below −1000. So
the usual “filter on the failure flags” advice does not help here.

Why it matters. blendedness < 0.1 is a routine cleanliness cut, and because these values are
negative rather than large they pass it — the cut is inverted rather than merely noisy. In ECDFS,
24,949 DP2 objects pass it and 17,972 of those (72%) are negative. The same cut on DP1 is 1.7%
negative.

One lead. The extreme values track brightness, not blend size. The worst sit on bright objects
(r_psfFlux ~3e6 nJy, r ≈ 15), while parents with 39 peaks bottom out at −29 and parents with only
17 peaks reach −5568. For 1 − child/parent to reach −7151 on a child of 3.1e6 nJy, the parent’s
absolute-value-weighted flux would have to come out near 430 nJy — which looks like the parent half
of the ratio failing on bright objects rather than a deblending-topology problem. That is an
inference from the numbers; I have not looked at pixels.

-- swap dp2 for dp1 to see the control
SELECT COUNT(*) AS n FROM dp2.Object
WHERE CONTAINS(POINT('ICRS', coord_ra, coord_dec),
               CIRCLE('ICRS', 53.13, -28.10, 0.2)) = 1
  AND detect_isDeblendedModelSource = 1
  AND r_blendedness < 0

Caveats. ECDFS is common sky, but DP1 and EDP2 differ in camera (ComCam vs LSSTCam) as well as
pipeline version (v29 vs v30), so this is not a clean A/B on the deblender alone. I checked g and
r only — the sync TAP endpoint times out for me on the other bands. DP2 returns fewer objects and
more nulls in the same cone, so I have used fractions rather than raw counts throughout. EDP2 is
explicitly an early preview and this may be known or already fixed for the full release.

Possibly related: the saturation reported at the centres of bright galaxies in EDP2 deep coadds
(topic 12419). If bright-object footprints contain pixels masked or interpolated toward zero, a
parent flux measured over that footprint could come out far too small, which would produce exactly
this signature.

Questions

  1. Is this the downstream consequence of known issue 1 (poor deconvolution in crowded fields), or a
    separate problem in the blendedness measurement? The brightness correlation made me suspect the
    latter.
  2. Should blendedness_flag be catching these? The near-absence of flags below −100 is the part I
    find hardest to interpret.
  3. Is there a recommended way to select a clean sample from EDP2 meanwhile — a different column, or
    a blendedness BETWEEN 0 AND 0.1 style guard?

Happy to provide more detail if it would help: the full query set, the per-field breakdown across
all six DP1 fields, the unflagged-fraction table by severity, or counts for any other field or
band.

1 Like

We’re investigating internally. I just wanted to give you an update so that you know that your question isn’t being ignored. This does seem to be a real phenomenon that others have reported and we have some guesses as to the cause but I’ll hold off reporting until we have a better understanding.

1 Like

I can confirm that there is a bug in the science pipelines that essentially wrote nonsense into the blendedness columns. I sorry for the inconvenience but I do have a workaround for you.

There is a column called deblend_blendNChild which tells how many children were deblended with a given object in the catalog. Selecting on (deblend_blendNChild == 1) | detect_isIsolated will essentially give you all of the objects that are not heavily blended. Note that this is different from only choosing isolated objects (detect_isIsolated) and I’ll explain why because the difference is subtle and has to do with how we segmented blends in DP2.

Detection on a coadd groups all connected pixels above a given threshold as a footprint and detects peaks within each footprint. If there is only a single peak in the footprint then it is considered isolated and is not deblended at all. These are the objects marked detect_isIsolated. However, the deblender acts in a partially deconvolved space, where we deconvolve from the observed seeing to a Gaussian seeing with sigma=0.8. In that space, many objects that are blended in the observed coadds are actually isolated, as eg. point sources are just small Gaussians in the deconvolved space even though they are much larger in the observations. The deblender is initialized and runs in that partially deconvolved space, so anything with deblend_blendNChild==1 is effectively isolated in that it’s model only overlaps with other sources after it is convolved with the 35x35 pixel difference kernel. So using the suggested cuts should give you a clean sample. For example I just tested tract 9813 and found that 53% of the sources passed this cut.

Thanks, this is very helpful, and confirming that it is a pipeline bug clears up what I was seeing.

The distinction between detect_isIsolated and deblend_blendNChild == 1 also makes sense now. I had been treating the observed-coadd footprint topology as closer to the effective deblending topology than it actually is.

I’ll stop using blendedness for EDP2 sample selection and use:

(deblend_blendNChild == 1) OR detect_isIsolated

for the clean/lightly-blended sample instead.

I may also run this across the fields/bands I already tested just to characterize how much the sample changes relative to the blendedness < 0.1 selection. If useful, I’m happy to post those aggregate results back here.

Thanks again for investigating this.

I marked Fred’s response as the solution.

@lrspeiser, it would be great if you could share your findings here!

Thanks, Fred and Yumi. Fast check so forgive errors by me. I ran a follow-up comparison of the confirmed pipeline bug and the recommended workaround—this is characterization of the selection impact, not a new data-quality claim.

I used the same 0.2-degree ECDFS cone (53.13, -28.10), with A = detect_isDeblendedModelSource = 1; B = A plus r_blendedness < 0.1; C = A plus (deblend_blendNChild = 1 OR detect_isIsolated = 1).

ECDFS gives A=29,130; B=18,712; C=14,515; B∩C=8,780; B\C=9,932; C\B=5,735. Thus 53.08% of objects accepted by B are rejected by the workaround. This is selection disagreement, not a measured contamination rate. Of C\B, 4,880 have null r blendedness and 855 have values >=0.1.

The original 84.46% negative fraction reproduces (17,854/21,140 non-null values in A). One clarification to my earlier post: the 24,949 passing the old cut was an unrestricted cone count, without A; the directly comparable A-based B count is 18,712.

I queried all six bands and all six original cones. Only ECDFS and EDFS have DP2 rows in those cones, so I added three other DP2 cones. The r-band comparisons are:

Field A B C B∩C B\C C\B B\C / B
ECDFS 29,130 18,712 14,515 8,780 9,932 5,735 53.08%
EDFS 28,823 20,631 14,800 10,298 10,333 4,502 50.08%
DP2_tract_9813_cone 43,569 36,186 18,706 15,472 20,714 3,234 57.24%
DP2_tract_10053_cone 3,386 3,011 3,171 2,811 200 360 6.64%
DP2_tract_5280_cone 7,753 4,006 5,415 2,589 1,417 2,826 35.37%

The saved tables also include per-band null/negative fractions and brightness/child-count bins. Brightness uses PSF-flux AB magnitudes without extra quality cuts. These are targeted regions, not a survey-wide estimate. Reproducible ADQL, aggregate CSV/Parquet and code: GitHub - lrspeiser/rubin-edp2-blendedness: Theory-neutral follow-up of the confirmed EDP2 blendedness pipeline bug: reproducible queries and aggregate selection comparisons · GitHub.

Thanks again for confirming the issue and explaining the workaround.

1 Like