Hi everyone!
I’m trying to reduce some data from HyperSuprimeCam with Gen3 butler.
I followed the pipeline and most of it went well,
but I encountered an error in generating the master bias.
Right before this procedure, I have ingested all the raw frames including bias and flats.
I ran the following command:
pipetask run --register-dataset-types -b $REPO --instrument lsst.obs.subaru.HyperSuprimeCam -i HSC/raw/all,HSC/calib/curated/19700101T000000Z,HSC/calib/unbounded -o HSC/calib/bias -p $CP_PIPE_DIR/pipelines/HyperSuprimeCam/cpBias.yaml -d "instrument='HSC' AND exposure.observation_type='bias' AND exposure.day_obs=20191221"
And this error appeared:
INFO 2023-08-01T15:36:14.543+09:00 lsst.ctrl.mpexec.cmdLineFwk ()(cmdLineFwk.py:621) - QuantumGraph contains 1232 quanta for 2 tasks, graph ID: '1690871774.4181228-570568'
ERROR 2023-08-01T15:36:15.851+09:00 lsst.daf.butler.cli.utils ()(utils.py:1049) - Caught an exception, details are in traceback:
Traceback (most recent call last):
File "/home/hjkim/lsst_stack/stack/miniconda3-py38_4.9.2-4.0.5/Linux64/daf_butler/gdde7253329+1ed234098f/python/lsst/daf/butler/datastores/fileDatastore.py", line 1162, in _write_in_memory_to_artifact
formatter.write(inMemoryDataset)
File "/home/hjkim/lsst_stack/stack/miniconda3-py38_4.9.2-4.0.5/Linux64/daf_butler/gdde7253329+1ed234098f/python/lsst/daf/butler/formatters/file.py", line 259, in write
self._writeFile(inMemoryDataset)
File "/home/hjkim/lsst_stack/stack/miniconda3-py38_4.9.2-4.0.5/Linux64/pipe_base/g641d719c08+0e065976f8/python/lsst/pipe/base/formatters/pexConfig.py", line 76, in _writeFile
inMemoryDataset.save(self.fileDescriptor.location.path)
File "/home/hjkim/lsst_stack/stack/miniconda3-py38_4.9.2-4.0.5/Linux64/pex_config/gc75b51116a+846e1f9efd/python/lsst/pex/config/config.py", line 1283, in save
os.chmod(outfile.name, (~umask & 0o666))
PermissionError: [Errno 1] Operation not permitted: '/Volumes/YOUNG3/hjkim/A2443/butler0/HSC/calib/bias/20230801T063615Z/isr_config/tmpqm0ido3l'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/hjkim/lsst_stack/stack/miniconda3-py38_4.9.2-4.0.5/Linux64/ctrl_mpexec/g4d64b21cde+4a4456e283/python/lsst/ctrl/mpexec/cli/cmd/commands.py", line 131, in run
script.run(qgraphObj=qgraph, **kwargs)
File "/home/hjkim/lsst_stack/stack/miniconda3-py38_4.9.2-4.0.5/Linux64/ctrl_mpexec/g4d64b21cde+4a4456e283/python/lsst/ctrl/mpexec/cli/script/run.py", line 187, in run
f.runPipeline(qgraphObj, taskFactory, args)
File "/home/hjkim/lsst_stack/stack/miniconda3-py38_4.9.2-4.0.5/Linux64/ctrl_mpexec/g4d64b21cde+4a4456e283/python/lsst/ctrl/mpexec/cmdLineFwk.py", line 719, in runPipeline
preExecInit.initialize(
File "/home/hjkim/lsst_stack/stack/miniconda3-py38_4.9.2-4.0.5/Linux64/ctrl_mpexec/g4d64b21cde+4a4456e283/python/lsst/ctrl/mpexec/preExecInit.py", line 115, in initialize
self.saveConfigs(graph)
File "/home/hjkim/lsst_stack/stack/miniconda3-py38_4.9.2-4.0.5/Linux64/ctrl_mpexec/g4d64b21cde+4a4456e283/python/lsst/ctrl/mpexec/preExecInit.py", line 343, in saveConfigs
self.butler.put(taskDef.config, configName, {})
File "/home/hjkim/lsst_stack/stack/miniconda3-py38_4.9.2-4.0.5/Linux64/daf_butler/gdde7253329+1ed234098f/python/lsst/daf/butler/core/utils.py", line 56, in inner
return func(self, *args, **kwargs)
File "/home/hjkim/lsst_stack/stack/miniconda3-py38_4.9.2-4.0.5/Linux64/daf_butler/gdde7253329+1ed234098f/python/lsst/daf/butler/_butler.py", line 1220, in put
self.datastore.put(obj, ref)
File "/home/hjkim/lsst_stack/stack/miniconda3-py38_4.9.2-4.0.5/Linux64/daf_butler/gdde7253329+1ed234098f/python/lsst/daf/butler/core/utils.py", line 56, in inner
return func(self, *args, **kwargs)
File "/home/hjkim/lsst_stack/stack/miniconda3-py38_4.9.2-4.0.5/Linux64/daf_butler/gdde7253329+1ed234098f/python/lsst/daf/butler/datastores/fileDatastore.py", line 2185, in put
storedInfo = self._write_in_memory_to_artifact(inMemoryDataset, ref)
File "/home/hjkim/lsst_stack/stack/miniconda3-py38_4.9.2-4.0.5/Linux64/daf_butler/gdde7253329+1ed234098f/python/lsst/daf/butler/datastores/fileDatastore.py", line 1164, in _write_in_memory_to_artifact
raise RuntimeError(
RuntimeError: Failed to serialize dataset isr_config@{}, sc=Config] (id=48824e06-a8c7-4371-8818-6a6b2fdecd9f) of type <class 'lsst.ip.isr.isrTask.IsrTaskConfig'> to temporary location file:///Volumes/YOUNG3/hjkim/A2443/butler0/HSC/calib/bias/20230801T063615Z/isr_config/2lx0xs6ey_tjdxqn.py
I have no idea what this message tells 
I’d appreciate your help.