Getting started tutorial part 4 problems

Support
#1

Hello,

I’m trying to run FGCM part of Getting started tutorial part 4 and getting error message:sst.daf.butler.cli.utils ERROR: Caught an exception, details are in traceback:

Traceback (most recent call last):
  File "/home/venil/lsstsw/stack/lsst-scipipe-9.0.0/Linux64/daf_butler/gc03a7ffb89+49d5bde2b1/python/lsst/daf/butler/_butler_config.py", line 138, in __init__
    butlerConfig = Config(other)
                   ^^^^^^^^^^^^^
  File "/home/venil/lsstsw/stack/lsst-scipipe-9.0.0/Linux64/daf_butler/gc03a7ffb89+49d5bde2b1/python/lsst/daf/butler/_config.py", line 286, in __init__
    self.__initFromUri(other)
  File "/home/venil/lsstsw/stack/lsst-scipipe-9.0.0/Linux64/daf_butler/gc03a7ffb89+49d5bde2b1/python/lsst/daf/butler/_config.py", line 375, in __initFromUri
    content = uri.read()
              ^^^^^^^^^^
  File "/home/venil/lsstsw/stack/lsst-scipipe-9.0.0/Linux64/resources/gc62a4ab5da+25667260f6/python/lsst/resources/file.py", line 97, in read
    with open(self.ospath, "rb") as fh:
         ^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/SMALL_HSC/butler.yaml'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/venil/lsstsw/stack/lsst-scipipe-9.0.0/Linux64/ctrl_mpexec/gf3af793bb1+a78a79140c/python/lsst/ctrl/mpexec/cli/cmd/commands.py", line 221, in run
    pipeline = script.build(**kwargs, show=show)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/venil/lsstsw/stack/lsst-scipipe-9.0.0/Linux64/ctrl_mpexec/gf3af793bb1+a78a79140c/python/lsst/ctrl/mpexec/cli/script/build.py", line 118, in build
    butler = Butler.from_config(butler_config, writeable=False)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/venil/lsstsw/stack/lsst-scipipe-9.0.0/Linux64/daf_butler/gc03a7ffb89+49d5bde2b1/python/lsst/daf/butler/_butler.py", line 315, in from_config
    butler_config = ButlerConfig(config, searchPaths=searchPaths, without_datastore=without_datastore)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/venil/lsstsw/stack/lsst-scipipe-9.0.0/Linux64/daf_butler/gc03a7ffb89+49d5bde2b1/python/lsst/daf/butler/_butler_config.py", line 159, in __init__
    raise FileNotFoundError(errmsg) from e
FileNotFoundError: [Errno 2] No such file or directory: '/SMALL_HSC/butler.yaml' (given '/SMALL_HSC/butler.yaml' and no known aliases: No repository index defined.  Neither of the environment variables DAF_BUTLER_REPOSITORIES or DAF_BUTLER_REPOSITORY_INDEX was set.)

What is the source of the problem?

Thank you

Getting started tutorial python problems
#2

Did you set up the package? This looks a lot like RC2_SUBSET_DIR hasn’t been set. See the tutorial instructions: Getting started tutorial part 1: setting up the Butler data repository — LSST Science Pipelines

#3

Thanks. Now I’m getting :lsst.daf.butler.cli.utils ERROR: Caught an exception, details are in traceback:

Traceback (most recent call last):
  File "/home/venil/lsstsw/stack/lsst-scipipe-9.0.0/Linux64/ctrl_mpexec/gf3af793bb1+a78a79140c/python/lsst/ctrl/mpexec/cli/cmd/commands.py", line 229, in run
    if (qgraph := script.qgraph(pipelineObj=pipeline, **kwargs, show=show)) is None:
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/venil/lsstsw/stack/lsst-scipipe-9.0.0/Linux64/ctrl_mpexec/gf3af793bb1+a78a79140c/python/lsst/ctrl/mpexec/cli/script/qgraph.py", line 225, in qgraph
    qgraph = f.makeGraph(pipelineObj, args)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/venil/lsstsw/stack/lsst-scipipe-9.0.0/Linux64/ctrl_mpexec/gf3af793bb1+a78a79140c/python/lsst/ctrl/mpexec/cmdLineFwk.py", line 636, in makeGraph
    butler, collections, run = _ButlerFactory.makeButlerAndCollections(args)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/venil/lsstsw/stack/lsst-scipipe-9.0.0/Linux64/ctrl_mpexec/gf3af793bb1+a78a79140c/python/lsst/ctrl/mpexec/cmdLineFwk.py", line 392, in makeButlerAndCollections
    butler, inputs, self = cls._makeReadParts(args)
                           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/venil/lsstsw/stack/lsst-scipipe-9.0.0/Linux64/ctrl_mpexec/gf3af793bb1+a78a79140c/python/lsst/ctrl/mpexec/cmdLineFwk.py", line 330, in _makeReadParts
    self.check(args)
  File "/home/venil/lsstsw/stack/lsst-scipipe-9.0.0/Linux64/ctrl_mpexec/gf3af793bb1+a78a79140c/python/lsst/ctrl/mpexec/cmdLineFwk.py", line 270, in check
    raise ValueError(consistencyError)
ValueError: Output CHAINED collection 'u/venil/fgcm' exists and does not include the same sequence of (flattened) input collections ('u/venil/single_frame/20250115T051610Z', 'u/venil/single_frame/20250114T191454Z', 'HSC/raw/RC2/9615', 'HSC/raw/RC2/9697', 'HSC/raw/RC2/9813', 'HSC/calib/gen2/20180117', 'HSC/calib/DM-28636', 'HSC/calib/gen2/20180117/unbounded', 'HSC/calib/DM-28636/unbounded', 'HSC/masks/s18a', 'HSC/fgcmcal/lut/RC2/DM-28636', 'refcats/DM-28636', 'refcats/DM-39298', 'skymaps') as a contiguous subsequence. Use --rebase to ignore this problem and reset the output collection, but note that this may obfuscate what inputs were actually used to produce these outputs.
#4

I’ll try to do what it says .