Hello. I am following this link to process HC data. I am using DM version DMVER=“v23.0.2” which worked well with simpleSingleFrame after updating the DRP.yaml (this version DRP.yaml ). I got an error when running pipetask with jointcal : pipetask run -b $RC2_SUBSET_DIR/SMALL_HSC/butler.yaml \ -p $RC2_SUBSET_DIR/pipelines/DRP_tuto_from_git.yaml#jointcal \ -i u/$USER/single_frame \ -o u/$USER/jointcal \ --register-dataset-types. The error I got is: raise RuntimeError( RuntimeError: 40 dataset(s) of type 'sourceTable_visit' was/were present in a previous query, but could not be found now.This is either a logic bug in QuantumGraph generation or the input collections have been modified since QuantumGraph generation began. /var/spool/slurmd/job10771521/slurm_script: line 44: /pbs/home/m/myassine: Is a directory. Hope that someone can help me to understand how to fix this kind of issues. Thank you in advance,
Can you try with the current weekly build of the software? v23 is, effectively, from last October so using the weekly tutorial with v23 may cause some problems. w_2022_28 came out last night.
This error message is not the one we want to be producing here (it’s for a much rarer case that is all but impossible if you’re using your own private repo), so there’s a bug on our side somewhere. But in the past when we’ve seen this, it’s indicated a pipeline that’s inconsistent either internally or with the processing you’ve done. Given that you’re trying to use a custom pipeline with an older release, that’s pretty likely, and @timj 's suggestion is definitely the one to try first if you don’t absolutely need to be using v23.
If you do, or you still run into trouble with the latest weekly, you should probably consider this equivalent to an empty QG, and follow the steps here to debug - though unfortunately that error message bug means some of the diagnostics that FAQ entry mentions won’t be available.