In DM-3373 and a few other tickets, we ported over from HSC scripts for constructing calibs. These reside in pipe_drivers, and use the MPI process pool in ctrl_pool. Example command-lines for using these scripts are on DM-5124, but here’s a rough guide:
- Bin scripts are
constructBias.py
,constructDark.py
,constructFlat.py
andconstructFringe.py
. - These are like a regular
CmdLineTask
, but with additional arguments that configure how to do the parallelism: -
--batch-type
may besmp
(Symmetric Multi-Processing, i.e., run on the local machine with multiple cores),pbs
(submit the batch request to your PBS cluster) orslurm
(submit the batch request to your Slurm cluster). -
--job
is the job name (used by batch systems to identify the job, and in naming output log files). -
--time
is the walltime limit for batch systems, per element (in this case, the element is individual CCDs). -
--cores
(forsmp
orslurm
modes) specifies the number of cores to use. -
--nodes
and--procs
(forpbs
orslurm
modes) specifies the number of nodes and CPUs per node to use. -
--batch-submit
can be used to provide additional options for batch submission (i.e., qsub or sbatch, as appropriate). -
--mpiexec
can be used to provide additional options formpiexec
. On our Slurm clusters at Princeton, we’ve found we need-bind-to socket
to prevent trouble. -
--dry-run
can be used to test the command-line before submitting. - You will probably need some configuration additions in your obs_* package.
- The individual scripts write
bias_config
,dark_config
,flat_config
andfringe_config
datasets, so your Mapper needs to support those. - Overrides can be added to your
obs_*/config
directory asbias.py
,dark.py
,flat.py
andfringe.py
.
@merlin is trying to get this working with DECam at the moment. Hopefully we can report success soon! Users of other cameras are welcome to request help here.