Synthetic source injection on multiple visits

Hi, I had been working on something along the lines demonstrated in the DP02_14_Injecting_Synthetic_Sources.ipynb tutorial, but applied for strong Lensing purposes, however, I have a few questions/issues that I hope I can get some help with.

  1. From the tutorial point of view, i.e. from the Jupyter notebook, is it possible to write the injected exposures back to a butler so that further analysis on the injected visits can be done?

  2. About the injection of the same synthetic image into different visits to coadd them afterwards, my impression is that the notebook tutorial would already take into account for the correct rotation of the injected image, this is specially important if its an extended image, like a simulated lens. But to verify this I would like to coadd the images, so it takes me back to point 1. Is there another way I could check the alignment is correct?

  3. Furthermore, I’ve tried to inject the same synthetic image, a lens, to several visits from the command line using the pipeline tasks, and I get the injection right but the wrong alignment. I haven’t found a way to tell the injection task how to align the image to a given visit. This is the command line I’ve used to produce the two images below, as you can see in both I get exactly the same orientation of the injected lens, therefore, if I coadd them I’ll get all the injected visits with the image injected in the exact same position.

An additional issue I just faced when trying to produce a fresh example is that the command I used to run seems not to be working, no injected image is saved, but I don’t see any error in the log. I’ll try again later to see if I can figure out what happen.

This is the command I have run in the past

pipetask --long-log --log-file inject.log run --register-dataset-types -b dp02 -i 2.2i/runs/DP0.2,u/alxogm/inject_SL_t0 -o /runs/DP0.2,u/alxogm/inject_SL -p /home/alxogm/WORK/source_injection/pipelines/inject_visit.yaml -d "instrument='LSSTCam-imSim' AND skymap='DC2' AND visit=656444 AND detector=38 AND band='r' "

This is following documentation in
Thanks for your help.

1 Like

Just wanted to let you know that a demo is being prepared, but in the meantime, anyone else can join the conversation!

1 Like

Hi @ChristinaAdair is there a repository where I can see a preliminary version or in development of this demos? Thanks!

Hello! I think we will have more information on this soon. I am not sure if there is a repository, but I will ask.

Hi @alxogm,

I can provide a partial answer for your first question, about how to write the injected exposure back to the butler.

Follow DP0.2 tutorial notebook 14 on synthetic source injection up to and including Section 4.3.2. Then use the writeable_butler to put the injected_exposure into the butler, using the line below. In this case I’ve used the INJECTION_CATALOG_COLLECTION defined in the tutorial for the run name.

writeable_butler.put(injected_exposure, 'calexp', dataId=dataId_g, run=INJECTION_CATALOG_COLLECTION)

Then instantiate a new temporary butler (tempButler) that points to the collection where the injected exposure was put. Retrieve it and plot it to prove to yourself it is your image with the sources injected.

tempButler = Butler(butler_repo, collections=[INJECTION_CATALOG_COLLECTION])
my_calexp = tempButler.get('calexp', dataId_g)
fig = plt.figure()
display = afwDisplay.Display(frame=fig)
display.scale('asinh', 'zscale')
del tempButler

Although this is a way to write to the butler, I am not sure that this is a path to being able to coadd. In DP0.2 tutorial notebook 9a, it is explained how coadding DP0.2 images has to be done with an older version of the pipelines, because the pipelines have evolved substantially since DP0.2 was processed two years ago. The source injection functionality was only added recently. So at the moment, I don’t think there is a way to make an injection pipeline that can both inject sources into, and then coadd, DP0.2 images. But this is not the final word! We will keep investigating and report back, on this and about your other reported issues.

1 Like

Hi Alma - Unfortunately, Melissa’s point in the “Caveats” above is correct. The tools in the pipelines to coadd images currently don’t work with the DP0.2 data products. Thus, even if you are able to write the injected images to a Butler, you cannot use (to my knowledge) the pipelines tools to coadd them. We are exploring whether there are other ways to create coadds from the images, but it may take a while.

Regarding the other part of your question (aligning the images) – if the image you are injecting has a WCS associated with it, the source_injection task should inject it with the correct orientation. However, because calexp images have different orientations, you will want to “register” (or “warp”) them so that they are oriented in the same direction. There is a notebook in this DP0.2 repo called “aligning_injected_images.ipynb” that shows how to do this. (That notebook also does a simple “sum” of the resulting images at the end, which is definitely not the scientifically best way to combine the images. It’s just an illustration that they are aligned.)

Hope this is helpful!

1 Like

Also, regarding writing injected images back to the Butler – @lskelvin added a new section to the FAQ for the source_injection package that explains how to do this. See this link for details.

Thanks @MelissaGraham and @jeffcarlin , yes I am aware of the caveats, but it’s ok for now. I will try your recommendations and see if I can achieve my goal.


Hi @alxogm, I just wanted to follow up here to ask if there’s anything else we can do to help?

As @jeffcarlin mentioned above, we now have a new section on our FAQs page which addresses how best to ingest injected data products into a data butler repository if you wish to continue onward processing. As @MelissaGraham outlined above, there are other tools available which will allow you to ingest data into the butler from a Python/notebook environment as well, however, it may be easier and perhaps safer for you to call on the SimplePipelineExecutor to call a pipeline YAML directly. That will ensure that all output data products from the task get correctly registered with the correct dimensions.

With regards points 2 and 3 from your original post, and just to re-iterate what has already been said here, synthetic sources that you inject should be rotated/sheared with respect to the WCS attached to the image. E.g., the injection of a Sérsic source with a given position angle “beta” should always be sheared with respect to North.

I’m very keen to hear of your progress, so please do reach out if you need any extra assistance on this.

As a brief example, here I’ve injected a simple single Sérsic into two different simulated LSSTCam-imSim detectors which overlap the same pointing (visit 204708, detector 38, and visit 211100, detector 174, both in the i-band). The Sérsic source is the large extended source in the ~center of the frame:

These two detectors have boresight rotation angles of 120 and 215, respectively. As you can see, the applied shear appears to be being applied with respect to North, producing a consistently injected source on the sky.