Hello, the community forum! Again, I’ve been trying to run the LSST science (AP) pipeline on DECam images, and especially on the DES SN observations for comparison.
Out of ignorance, I have been deleting the ./apdb/association.db files that were created from the test runs, that contained all of the detections and forced measurements. I did keep all of the output in the ./repo files, thinking that I could read in all of the output using Butler. I have not been able to find forced source measurements from the output in ./repo, except in ./apdb/association.db (which I didn’t keep from previous runs). Is there an easy way to recreate the ./apdb/association.db files from the runs?
I had tried to use this table before, but got errors like below, so I thought the table didn’t exist (I used deep coadd image as template, instead of goodSeeing coadd)
MissingDatasetTypeError(f"Dataset type with name ‘{name}’ not found.")
lsst.daf.butler.registry._exceptions.MissingDatasetTypeError: “Dataset type with name ‘deepDiff_diaForcedSrc’ not found.”
Does the following lines look like the right way to access the table?
Make sure you use the correct names that Eric provided, either goodSeeingDiff_diaForcedSrc for an older run or dia_forced_source_apdb if you are using a stack from weekly w_2025_16 or later. The problem is that you are looking for deepDiff_diaForcedSrc.
Sorry didn’t see the replies. I had changed the template setup, to use deepcoadd images as template, rather than goodseeing coadd as template, so my diasource tables were named as “deepDiff_%” instead of “goodSeeingDiff_%”.
I had checked my previous runs that used goodseeing coadd as template (so tables were named as “goodSeeingDiff_%”. But I still wasn’t able to find the table “goodSeeingDiff_diaForcedSrc”.
We were using the LSST stack version 26 (I had already processed a large amount of data with this version, so I can’t just switch versions now …) – maybe this is a version issue that the table wasn’t in version 26?
Hi @isullivan, just wanted to check in and see if you know whether stack version 26 produces this table that @ynzhang is looking for, given the alterations to the naming? Or perhaps know who else we might ping for an answer. Thanks!
Version 26 is two years old, but going back to that version I see that the association config doWriteAssociatedSources was False by default, meaning that the associated sources, forced sources, and diaObject tables would not be written as separate files unless you changed that setting. If you also deleted the association.db database file, then the only way to recover the results would be to re-run association (diaPipe.py).