Processing exposures with all necessary calibrations, the chracterize image task runs as it should and writes the outputs icSrc and icExp. However, when calibrate task runs as the next step on the pipeline, it crashes saying FileNotFoundError: [Errno 2] No such file or directory: ‘/arc/projects/NewHorizons/HSC_2024/PG2_BUTLER/u/NH/newer-processCCD/20210705/20250610T204854Z/icSrc_schema/icSrc_schema_u_NH_newer-processCCD_20210705_20250610T204854Z.fits’
I can’t find this icSrc_schema/ anywhere, I have icSrc/, icExp/, all the logs and metadata, icExpBackground/ and postISRCCD/ documentation didn’t help with this and I’m looking for any fixes to get this schema
Can you please specify the version of the software that you are using and the command you are running?
I am using d-latest and running the DRP-Prod+injection pipeline on the processCcd subset, which chains lsst.ip.isr.IsrTask → lsst.pipe.tasks.characterizeImage.CharacterizeImageTask → lsst.pipe.tasks.calibrate.CalibrateTask → lsst.source.injection.inject_visit.VisitInjectTask
Usually those schema datasets would be written by the “init-only” AKA “pre-exec-init” stage. Are you executing the pipelines with pipetask or bps?
i am using pipetask
If you’re using the latest, note that we are in the process of deprecating CharacterizeImageTask and CalibrateTask; that combination has been replaced with CalibrateImageTask, which is generally more robust and eliminates the need for the icSrc/icExp datasets.