Calib construction scripts

In DM-3373 and a few other tickets, we ported over from HSC scripts for constructing calibs. These reside in pipe_drivers, and use the MPI process pool in ctrl_pool. Example command-lines for using these scripts are on DM-5124, but here’s a rough guide:

  • Bin scripts are constructBias.py, constructDark.py, constructFlat.py and constructFringe.py.
  • These are like a regular CmdLineTask, but with additional arguments that configure how to do the parallelism:
  • --batch-type may be smp (Symmetric Multi-Processing, i.e., run on the local machine with multiple cores), pbs (submit the batch request to your PBS cluster) or slurm (submit the batch request to your Slurm cluster).
  • --job is the job name (used by batch systems to identify the job, and in naming output log files).
  • --time is the walltime limit for batch systems, per element (in this case, the element is individual CCDs).
  • --cores (for smp or slurm modes) specifies the number of cores to use.
  • --nodes and --procs (for pbs or slurm modes) specifies the number of nodes and CPUs per node to use.
  • --batch-submit can be used to provide additional options for batch submission (i.e., qsub or sbatch, as appropriate).
  • --mpiexec can be used to provide additional options for mpiexec. On our Slurm clusters at Princeton, we’ve found we need -bind-to socket to prevent trouble.
  • --dry-run can be used to test the command-line before submitting.
  • You will probably need some configuration additions in your obs_* package.
  • The individual scripts write bias_config, dark_config, flat_config and fringe_config datasets, so your Mapper needs to support those.
  • Overrides can be added to your obs_*/config directory as bias.py, dark.py, flat.py and fringe.py.

@merlin is trying to get this working with DECam at the moment. Hopefully we can report success soon! Users of other cameras are welcome to request help here.

Steps taken so far are to splice in:

bias_config: { template: "config/bias.py" python: "lsst.pipe.drivers.constructCalibs.BiasConfig" persistable: "Config" storage: "ConfigStorage" tables: raw tables: raw_visit } dark_config: { template: "config/dark.py" python: "lsst.pipe.drivers.constructCalibs.DarkConfig" persistable: "Config" storage: "ConfigStorage" tables: raw tables: raw_visit } flat_config: { template: "config/flat.py" python: "lsst.pipe.drivers.constructCalibs.FlatConfig" persistable: "Config" storage: "ConfigStorage" tables: raw tables: raw_visit } fringe_config: { template: "config/fringe.py" python: "lsst.pipe.drivers.constructCalibs.FringeConfig" persistable: "Config" storage: "ConfigStorage" tables: raw tables: raw_visit
}

to DecamMapper.paf, just so that steps are recorded for others.

@KSK I am also about to try this for the newly created obs_monocam as well :slight_smile:

This now works with:

  • DECam, using branch tickets/DM-5899 of obs_decam; and
  • Monocam, using branch u/price/20160429 of obs_monocam.

Note that these branches will (hopefully!) be cleaned up as part of their journey to master.