Error in defining tract for a specific coordinate (HSC data processing)

Support
,
#1

Hi all,

I’m using hscpipe v8.4 to process some data. I’m interested in a fairly small field in the middle of the HSC image, so I tried using the tract definition using WCS as defined here.

My tract_overrides.config file has the following lines:

root.skyMap = "discrete"
root.skyMap["discrete"].raList = [149.5]
root.skyMap["discrete"].decList = [68.8]
root.skyMap["discrete"].radiusList = [0.3]
root.skyMap["discrete"].pixelScale = 0.2
root.skyMap["discrete"].projection = "TAN"
root.skyMap["discrete"].tractOverlap = 0

I then ran makeSkyMap.py ./HSC --rerun M81 --configfile tract_overrides.config. However, I run into the Butler error below. I can’t find what exactly is going wrong. Can someone point me in the right direction?

(base) [shriharsh@prithvi M81]$ makeSkyMap.py ./HSC/ --rerun M81 -C tract_overrides.config 
root INFO: Loading config overrride file '/opt/hscpipe/8.4/lsst_home/stack/miniconda3-4.7.10-4d7b902/Linux64/obs_subaru/8.4-hsc/config/makeSkyMap.py'
root INFO: Loading config overrride file '/opt/hscpipe/8.4/lsst_home/stack/miniconda3-4.7.10-4d7b902/Linux64/obs_subaru/8.4-hsc/config/hsc/makeSkyMap.py'
Config override file 'tract_overrides.config' appears to use 'root' instead of 'config'; trying with 'root'
Traceback (most recent call last):
  File "/opt/hscpipe/8.4/lsst_home/stack/miniconda3-4.7.10-4d7b902/Linux64/daf_persistence/8.0-hsc/python/lsst/daf/persistence/butler.py", line 890, in _setAndVerifyParentsLists
    repoData.cfg.extendParents(parents)
  File "/opt/hscpipe/8.4/lsst_home/stack/miniconda3-4.7.10-4d7b902/Linux64/daf_persistence/8.0-hsc/python/lsst/daf/persistence/repositoryCfg.py", line 181, in extendParents
    newParents, self._parents))
lsst.daf.persistence.butlerExceptions.ParentsMismatch: The beginning of the passed-in parents list: [RepositoryCfg(root='../..', mapper=<class 'lsst.obs.hsc.hscMapper.HscMapper'>, mapperArgs={'calibRoot': '/home/shriharsh/Work/M81/HSC/CALIB'}, parents=[], policy=None), RepositoryCfg(root='../..', mapper=<class 'lsst.obs.hsc.hscMapper.HscMapper'>, mapperArgs={}, parents=[], policy=None)] does not match the existing parents list in this RepositoryCfg: [RepositoryCfg(root='/home/shriharsh/Work/M81/HSC', mapper=<class 'lsst.obs.hsc.hscMapper.HscMapper'>, mapperArgs={'calibRoot': '/home/shriharsh/Work/M81/HSC/CALIB'}, parents=[], policy=None)]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/hscpipe/8.4/lsst_home/stack/miniconda3-4.7.10-4d7b902/Linux64/pipe_tasks/8.1-hsc/bin/makeSkyMap.py", line 25, in <module>
    MakeSkyMapTask.parseAndRun()
  File "/opt/hscpipe/8.4/lsst_home/stack/miniconda3-4.7.10-4d7b902/Linux64/pipe_base/8.0-hsc+1/python/lsst/pipe/base/cmdLineTask.py", line 605, in parseAndRun
    parsedCmd = argumentParser.parse_args(config=config, args=args, log=log, override=cls.applyOverrides)
  File "/opt/hscpipe/8.4/lsst_home/stack/miniconda3-4.7.10-4d7b902/Linux64/pipe_base/8.0-hsc+1/python/lsst/pipe/base/argumentParser.py", line 684, in parse_args
    namespace.butler = dafPersist.Butler(inputs=inputs, outputs=outputs)
  File "/opt/hscpipe/8.4/lsst_home/stack/miniconda3-4.7.10-4d7b902/Linux64/daf_persistence/8.0-hsc/python/lsst/daf/persistence/butler.py", line 521, in __init__
    self._setAndVerifyParentsLists(repoDataList)
  File "/opt/hscpipe/8.4/lsst_home/stack/miniconda3-4.7.10-4d7b902/Linux64/daf_persistence/8.0-hsc/python/lsst/daf/persistence/butler.py", line 894, in _setAndVerifyParentsLists
    parents, repoData.cfg.parents, e))
RuntimeError: Inputs of this Butler:[RepositoryCfg(root='../..', mapper=<class 'lsst.obs.hsc.hscMapper.HscMapper'>, mapperArgs={'calibRoot': '/home/shriharsh/Work/M81/HSC/CALIB'}, parents=[], policy=None), RepositoryCfg(root='../..', mapper=<class 'lsst.obs.hsc.hscMapper.HscMapper'>, mapperArgs={}, parents=[], policy=None)] do not match parents of existing writable cfg:[RepositoryCfg(root='/home/shriharsh/Work/M81/HSC', mapper=<class 'lsst.obs.hsc.hscMapper.HscMapper'>, mapperArgs={'calibRoot': '/home/shriharsh/Work/M81/HSC/CALIB'}, parents=[], policy=None)] (ParentMismatch exception: The beginning of the passed-in parents list: [RepositoryCfg(root='../..', mapper=<class 'lsst.obs.hsc.hscMapper.HscMapper'>, mapperArgs={'calibRoot': '/home/shriharsh/Work/M81/HSC/CALIB'}, parents=[], policy=None), RepositoryCfg(root='../..', mapper=<class 'lsst.obs.hsc.hscMapper.HscMapper'>, mapperArgs={}, parents=[], policy=None)] does not match the existing parents list in this RepositoryCfg: [RepositoryCfg(root='/home/shriharsh/Work/M81/HSC', mapper=<class 'lsst.obs.hsc.hscMapper.HscMapper'>, mapperArgs={'calibRoot': '/home/shriharsh/Work/M81/HSC/CALIB'}, parents=[], policy=None)]
#2

Did you move the HSC directory from somewhere else? Is it the output (or output rerun) of another command? Basically, the repositoryCfg.yaml recorded in the repo disagrees with your command line arguments somehow; going back to a “clean” input repo is usually the best solution.

(Expertise on Gen2 Butler problems like this is rapidly fading, unfortunately.)

1 Like
#3

My guess is that you’re using an existing M81 rerun without using the same --calib. Try:

makeSkyMap.py /home/shriharsh/Work/M81/HSC --rerun M81 --calib /home/shriharsh/Work/M81/HSC/CALIB --configfile tract_overrides.config
#4

Hi K-T, Paul,

Thanks for the input. I didn’t realise I had to explicitly specify the calib location for makeSkyMap. I’ll try again and report back.