How to do FGCM task in v24.0.0?

Support
#1

Hi all,

I’m working with the LSST pipeline v24.0.0 and HyperSuprimeCam data,
and trying to run the step 2c subset of DRP which contains the Forward Global Calibration Method(FGCM)
(~photometric calibration process?) with the following command :

pipetask run --register-dataset-types \
-b $REPO --instrument lsst.obs.subaru.HyperSuprimeCam \
-i /HSC/runs/step2a \ 
-o /HSC/runs/step2c \
-p $DRP_PIPE_DIR/pipelines/HSC/DRP-Prod.yaml#step2c \
-c parameters:band=g \
-d "instrument='HSC'"

It returns FileNotFoundError,

    raise FileNotFoundError(
FileNotFoundError: Not enough datasets (0) found for non-optional connection fgcmBuildStarsTable.fgcmLookUpTable (fgcmLookUpTable) with minimum=1 for quantum data ID {instrument: 'HSC'}.

and It seems like it requires “fgcm lookup table”.
So I tried to make one since I have found a pipeline tutorial for making a lookup table in https://hsc.mtk.nao.ac.jp/pipedoc/pipedoc_8_e/tutorial_e/mosaic.html
which works for Gen2 repositories.

However the fgcmMakeLut.py script described in the link does not work in my Gen3 repository.
How can I go forward?

Thank you for your help!

#2

Are you using v24 because you are required to use it? Would you be open to using a newer release?

#3

Hi @timj ,

Thank you for your response.
It’s about the reference catalogs. For now I use the older version in two reasons.

First, according to my previous post (https://community.lsst.org/t/error-in-single-frame-processing-of-hsc-data/7873?u=hjkim181213) I had to install older pipeline to recover and run ‘convert_refcat_to_nJy.py’ script to convert old PS1 catalog which is not supported anymore to newer one.

Second, in my experience of running pipetask in the v25.0.0 it required the Gaia DR3 catalog I guess. But what I own is only DR2.

#4

You can convert the catalogs using this script using the older software version but then you can use the converted scripts with the newer software.

#5

Hello, recently I am processing HSC’s data, and I met the same no fgcmLookUpTable question as you, since there are no specific cmd that used to generate the fgcmLookUpTable in this post, so I give the cmd I use in the Gen3 repository to generate the fgcmLookUpTable, I hope this cmd is correct.

pipetask run --register-dataset-types \
  -b ./butler.yaml \
  -i HSC/calib/unbounded \
  -o u/$USER/fgcmMakeLUT \
  -p /home/$USER/lsst_stack/lsst_v29_2_1/conda/envs/lsst-scipipe-10.1.0/share/eups/Linux64/fgcmcal/g21d47ad084+cbe8876073/pipelines/_ingredients/fgcmMakeLUT.yaml \
  -C fgcmMakeLut:/home/$USER/lsst_stack/lsst_v29_2_1/conda/envs/lsst-scipipe-10.1.0/share/eups/Linux64/obs_subaru/ge43c9703ef+d402bdd674/config/fgcmMakeLut.py \
  > fgcmMakeLUT.log 2>&1 &
#6

Thanks for your post @Yuanyu! Is it possible to be more specific about the issue you’re having so that we can help, there doesn’t seem to be anything obviously wrong with the command so more details would help us help you. Thanks!

#7

Thank you for your reply! I generate the fgcm lookup table with the cmd successfully, and the purpose I give the cmd here is to help others know how to generate a fgcm lookup table in gen3 version, since I didn’t find a tutorial about that.
Thank you!

1 Like