How to do FGCM task in v24.0.0?

Support
#1

Hi all,

I’m working with the LSST pipeline v24.0.0 and HyperSuprimeCam data,
and trying to run the step 2c subset of DRP which contains the Forward Global Calibration Method(FGCM)
(~photometric calibration process?) with the following command :

pipetask run --register-dataset-types \
-b $REPO --instrument lsst.obs.subaru.HyperSuprimeCam \
-i /HSC/runs/step2a \ 
-o /HSC/runs/step2c \
-p $DRP_PIPE_DIR/pipelines/HSC/DRP-Prod.yaml#step2c \
-c parameters:band=g \
-d "instrument='HSC'"

It returns FileNotFoundError,

    raise FileNotFoundError(
FileNotFoundError: Not enough datasets (0) found for non-optional connection fgcmBuildStarsTable.fgcmLookUpTable (fgcmLookUpTable) with minimum=1 for quantum data ID {instrument: 'HSC'}.

and It seems like it requires “fgcm lookup table”.
So I tried to make one since I have found a pipeline tutorial for making a lookup table in https://hsc.mtk.nao.ac.jp/pipedoc/pipedoc_8_e/tutorial_e/mosaic.html
which works for Gen2 repositories.

However the fgcmMakeLut.py script described in the link does not work in my Gen3 repository.
How can I go forward?

Thank you for your help!

#2

Are you using v24 because you are required to use it? Would you be open to using a newer release?

#3

Hi @timj ,

Thank you for your response.
It’s about the reference catalogs. For now I use the older version in two reasons.

First, according to my previous post (https://community.lsst.org/t/error-in-single-frame-processing-of-hsc-data/7873?u=hjkim181213) I had to install older pipeline to recover and run ‘convert_refcat_to_nJy.py’ script to convert old PS1 catalog which is not supported anymore to newer one.

Second, in my experience of running pipetask in the v25.0.0 it required the Gaia DR3 catalog I guess. But what I own is only DR2.

#4

You can convert the catalogs using this script using the older software version but then you can use the converted scripts with the newer software.

#5

Hello, recently I am processing HSC’s data, and I met the same no fgcmLookUpTable question as you, since there are no specific cmd that used to generate the fgcmLookUpTable in this post, so I give the cmd I use in the Gen3 repository to generate the fgcmLookUpTable, I hope this cmd is correct.

pipetask run --register-dataset-types \
  -b ./butler.yaml \
  -i HSC/calib/unbounded \
  -o u/$USER/fgcmMakeLUT \
  -p /home/$USER/lsst_stack/lsst_v29_2_1/conda/envs/lsst-scipipe-10.1.0/share/eups/Linux64/fgcmcal/g21d47ad084+cbe8876073/pipelines/_ingredients/fgcmMakeLUT.yaml \
  -C fgcmMakeLut:/home/$USER/lsst_stack/lsst_v29_2_1/conda/envs/lsst-scipipe-10.1.0/share/eups/Linux64/obs_subaru/ge43c9703ef+d402bdd674/config/fgcmMakeLut.py \
  > fgcmMakeLUT.log 2>&1 &