batchWallTime in constructCalibs.py


(Surhud More) #1

I was trying to use the --batch-type=pbs option for constructing calibration files, and passing the options --nodes=10 --cores=32. I get a RuntimeError:

raise RuntimeError("Non-positive walltime: %s (did you forget '--time'?)" % (walltime,))

I have tracked this down to the function batchWallTime in constructCalibs.py (line 414-419)

    def batchWallTime(cls, time, parsedCmd, numCores):
        numCcds = len(parsedCmd.butler.get("camera"))
        numExps = len(cls.RunnerClass.getTargetList(
            parsedCmd)[0]['expRefList'])
        numCycles = int(numCcds/float(numCores) + 0.5)
        return time*numExps*numCycles

Because numCores can be greater than the numCcds in the case of a PBS script, numCycles becomes zero. The fix would be:

    def batchWallTime(cls, time, parsedCmd, numCores):
        numCcds = len(parsedCmd.butler.get("camera"))
        numExps = len(cls.RunnerClass.getTargetList(
            parsedCmd)[0]['expRefList'])
        numCycles = int(numCcds*numExps/float(numCores) + 0.5)
        return time*numCycles

It seems I do not have a LSST JIRA account to report it there. Should I just send a pull request without an associated LSST DM ticket?


(K-T Lim) #2

Please go ahead and file a pull request. Note the policy in https://developer.lsst.io/legal/copyright-overview.html#external-contributors