Testing obs_decam and obs_cfht

Data Management
#1

I just implemented a change to how reference object loaders are handled in tasks (DM-5798). The main change is that the reference object loaders are now constructed at the CalibrateTask and CharacterizeImageTask level instead of inside the AstrometryTask. This means that any config overrides applied to the refObjLoader config need to be applied one level up.

I have gone through the obs packages. Both regular CI with the SDSS integration test and ci_hsc pass, so I’m fairly confident those are mostly working. I don’t have test data on hand for DECam or CFHT. It would be great if someone could run a simple test with processCcd to make sure the configs are correct before I merge. @hsinfang, @boutigny, or @mwv do you have data to test either of these obs packages?

#2

validate_drp will process both CFHT and DECam (using data from the associated validation_data package)

#3

I am extremely busy these days and will not have time to test it myself immediately but there are also CFHT data at NCSA in /lsst8/boutigny/valid_cfht

#4

Is it a first step toward the possibility to get different reference catalogs for astrometry and photometry calibration ?

#5

One of the requirements for the new design was to make it easier to have different catalogs for photometric and astrometric calibration. We will need to rework ProcessCcdTask and possible photoCalTask to make this happen.

#6

Hello.

I’m new in LSST and I’m trying to use the stack to process a cluster (with the help of Dominique). While I was running processCcd.py, I encountered an error linked to ‘refObjLoader’ and ‘anetAstrometry’, which could apparently comes from the changes that you talk about in this post.

You can find below the error message followed by the part of the config file related to this error. Please let me know if I can help in any ways to debug or run more tests (but keep in mind that I have very little knowledge of the stack for now).

Thanks!

Nicolas

Error message:

_CameraMapper: Loading Posix registry from /home/chotard/Work/clusters/3C295/output_
_Traceback (most recent call last):_
_  File "/home/chotard/Work/lsstsw/stack/Linux64/pipe_tasks/12.0.rc1-1-gc9b72b9+1/bin/processCcd.py", line 25, in <module>_
_    ProcessCcdTask.parseAndRun()_
_  File "/home/chotard/Work/lsstsw/stack/Linux64/pipe_base/12.0.rc1+8/python/lsst/pipe/base/cmdLineTask.py", line 452, in parseAndRun_
_    resultList = taskRunner.run(parsedCmd)_
_  File "/home/chotard/Work/lsstsw/stack/Linux64/pipe_base/12.0.rc1+8/python/lsst/pipe/base/cmdLineTask.py", line 194, in run_
_    if self.precall(parsedCmd):_
_  File "/home/chotard/Work/lsstsw/stack/Linux64/pipe_base/12.0.rc1+8/python/lsst/pipe/base/cmdLineTask.py", line 281, in precall_
_    task = self.makeTask(parsedCmd=parsedCmd)_
_  File "/home/chotard/Work/lsstsw/stack/Linux64/pipe_base/12.0.rc1+8/python/lsst/pipe/base/cmdLineTask.py", line 371, in makeTask_
_    return self.TaskClass(config=self.config, log=self.log, butler=butler)_
_  File "/home/chotard/Work/lsstsw/stack/Linux64/pipe_tasks/12.0.rc1-1-gc9b72b9+1/python/lsst/pipe/tasks/processCcd.py", line 135, in __init___
_    self.makeSubtask("calibrate", butler=butler, icSourceSchema=self.charImage.schema)_
_  File "/home/chotard/Work/lsstsw/stack/Linux64/pipe_base/12.0.rc1+8/python/lsst/pipe/base/task.py", line 226, in makeSubtask_
_    subtask = configurableField.apply(name=name, parentTask=self, **keyArgs)_
_  File "/home/chotard/Work/lsstsw/stack/Linux64/pex_config/12.0.rc1-1-gc153103/python/lsst/pex/config/configurableField.py", line 77, in apply_
_    return self.target(*args, config=self.value, **kw)_
_  File "/home/chotard/Work/lsstsw/stack/Linux64/pipe_tasks/12.0.rc1-1-gc9b72b9+1/python/lsst/pipe/tasks/calibrate.py", line 279, in __init___
_    self.makeSubtask("astrometry", refObjLoader=self.refObjLoader, schema=self.schema)_
_  File "/home/chotard/Work/lsstsw/stack/Linux64/pipe_base/12.0.rc1+8/python/lsst/pipe/base/task.py", line 226, in makeSubtask_
_    subtask = configurableField.apply(name=name, parentTask=self, **keyArgs)_
_  File "/home/chotard/Work/lsstsw/stack/Linux64/pex_config/12.0.rc1-1-gc153103/python/lsst/pex/config/configurableField.py", line 77, in apply_
_    return self.target(*args, config=self.value, **kw)_
_  File "/home/chotard/Work/lsstsw/stack/Linux64/meas_astrom/12.0.rc1-3-g992ad82+1/python/lsst/meas/astrom/anetAstrometry.py", line 145, in __init___
_    pipeBase.Task.__init__(self, **kwds)_
_TypeError: __init__() got an unexpected keyword argument 'refObjLoader'_

Config file

_from lsst.meas.astrom.anetAstrometry import ANetAstrometryTask_
_config.calibrate.astrometry.retarget(ANetAstrometryTask)_
_config.calibrate.astrometry.solver.sipOrder=3_
#7

That’s a bug in the ANetAstrometryTask, since its __init__ method isn’t accepting the new refObjLoader argument. If you could file a ticket on Jira, I’m sure @KSK will have a look at it.

#8

Ticket is DM-6525.

#9

Yes indeed. The change was initially driven by a different need, but one of its advantages is that it will be much easier to pass separate catalogs to the astrometric and photometric solvers.