However I get the following error message thereafter:
# > processCcd.py input --output ${OUTPUT_DIR} --id visit=803022 ccd=13 --configfile ${VOLUME_DIR}/input/processConfig.py
: Loading config overrride file '/processing/my_packages/obs_cfht/config/processCcd.py'
Traceback (most recent call last):
File "/cvmfs/lsst.in2p3.fr/software/linux-x86_64/lsst-v12.0/Linux64/pipe_tasks/12.0/bin/processCcd.py", line 25, in <module>
ProcessCcdTask.parseAndRun()
File "/cvmfs/lsst.in2p3.fr/software/linux-x86_64/lsst-v12.0/Linux64/pipe_base/12.0/python/lsst/pipe/base/cmdLineTask.py", line 450, in parseAndRun
parsedCmd = argumentParser.parse_args(config=config, args=args, log=log, override=cls.applyOverrides)
File "/cvmfs/lsst.in2p3.fr/software/linux-x86_64/lsst-v12.0/Linux64/pipe_base/12.0/python/lsst/pipe/base/argumentParser.py", line 436, in parse_args
self._applyInitialOverrides(namespace)
File "/cvmfs/lsst.in2p3.fr/software/linux-x86_64/lsst-v12.0/Linux64/pipe_base/12.0/python/lsst/pipe/base/argumentParser.py", line 599, in _applyInitialOverrides
namespace.config.load(filePath)
File "/cvmfs/lsst.in2p3.fr/software/linux-x86_64/lsst-v12.0/Linux64/pex_config/12.0/python/lsst/pex/config/config.py", line 532, in load
self.loadFromStream(stream=code, root=root)
File "/cvmfs/lsst.in2p3.fr/software/linux-x86_64/lsst-v12.0/Linux64/pex_config/12.0/python/lsst/pex/config/config.py", line 552, in loadFromStream
exec stream in {}, local
File "/processing/my_packages/obs_cfht/config/processCcd.py", line 43, in <module>
'i2': 'i',
File "/cvmfs/lsst.in2p3.fr/software/linux-x86_64/lsst-v12.0/Linux64/pex_config/12.0/python/lsst/pex/config/configurableField.py", line 103, in __getattr__
return getattr(self._value, name)
AttributeError: 'AstrometryConfig' object has no attribute 'solver'
I do not understand why processCcd.py is not working as expected. Any help would be appreciated
This seems to be because of changes made to obs_cfht after the release of v12 of the stack. Can you try reverting to a slightly older version of obs_cfht? I think that abdb9b9 should work. That is, after cloning from GitHub, but before building and setting up obs_cfht, run
What is in this file? I suggest you try running the same command except remove --configfile ${VOLUME_DIR}/input/processConfig.py and add --show config. if that passes then your config override file has an outdated setting, one that may be clear from the displayed config.
# processCcd.py input --output /processing/output_one_ccd/ --id visit=803022 ccd=13 --configfile /processing/input/processConfig.py
: Loading config overrride file '/processing/my_packages/obs_cfht/config/processCcd.py'
: Config override file does not exist: '/processing/my_packages/obs_cfht/config/megacam/processCcd.py'
: input=/processing/input
: calib=None
: output=/processing/output_one_ccd
CameraMapper: Loading registry registry from /processing/output_one_ccd/_parent/registry.sqlite3
CameraMapper: Unable to locate calibRegistry registry in root: /processing/output_one_ccd/calibRegistry.sqlite3
CameraMapper: Unable to locate calibRegistry registry in current dir: ./calibRegistry.sqlite3
CameraMapper: Loading Posix registry from /processing/output_one_ccd
CameraMapper: Loading registry registry from /processing/output_one_ccd/_parent/registry.sqlite3
CameraMapper: Unable to locate calibRegistry registry in root: /processing/output_one_ccd/calibRegistry.sqlite3
CameraMapper: Unable to locate calibRegistry registry in current dir: ./calibRegistry.sqlite3
CameraMapper: Loading Posix registry from /processing/output_one_ccd
processCcd: Processing {'taiObs': '2005-07-14T07:26:21.14', 'extension': 14, 'object': 'D3', 'visit': 803022, 'filter': 'r', 'state': 'p', 'runId': '05AL01', 'date': '2005-07-14', 'ccd': 13, 'expTime': 300.238}
[....]
processCcd.calibrate.photoCal: Magnitude zero point: 32.139278 +/- 0.000487 from 40 stars
processCcd.calibrate: Photometric zero-point: 32.139278
So if I understand correctly, I have to revert to an older version of obs_cfht and I have to generate the processConfig.py configuration file before to execute processCcd.py with a given visit and ccd id.
Generating processConfig.py depends on what you want to achieve. If you’re just using the default configuration, there’s no need to dump it to a file with --show config then load that when you start processing: processCcd.py will run with that default configuration without your intervention.
On the other hand, if you want to modify the configuration, then you could dump it to a file like that and use it as a template. If you do that, then yes, it’s possible the configuration options will change for different versions of the stack and/or pipe_tasks, so you might find it safest to dump a fresh copy of the configuration when the versions change.
Note, though, that your config file only needs to specify the parameters that you need to override. A minimal override file is less likely to be affected by changes than one in which you list absolutely everything, even the defaults.
Ok understood. I will create an empty processCcd.py file and add configuration options later if needed (I have no idea of the meaning of those parameters). This way, I will keep the --configfile option as a reminder
One last question: is there a way to know which version of obs_cfht is compatible with a given version of the stack?
Our usual recommendation is to run without overrides unless you are sure you need them. Otherwise it means you are using a unique configuration nobody else understands, which makes it hard to help you if you run into problems.
As to version compatibility…good question and I don’t have the answer. obs_cfht is not yet part of the standard distribution (though I think there are plans to change that). I hope there is a better way, but one thing that ought to work is to check the obs_cfht history in git and pick master that matches the date of the release.
I’m not aware of a better way to find a compatible version that the one Russell suggests. This is a known problem (DM-5661): I hope we’ll be able to fix it soon.
