We have finished reprocessing the entire HSC PDR1 data, using the DM Software Stack w_2018_15.
For team members with access to lsst-dev, the output repositories are available at:
More details of this reprocessing can be found at:
https://confluence.lsstcorp.org/display/DM/S18+HSC+PDR1+reprocessing
Congratulations, @hsinfang!
When we processed the Wide dataset at Princeton a few years ago, our utilisation estimate was about 1k core-weeks. Your value of 6718 node-hours looks similar, but interpretation requires knowing the number of cores per node; what did you use?
For wide, it was
That sums up to ~720 core-weeks.
What’s the number if you include the cores you weren’t actively using but scheduled to steal their memory (because you couldn’t use them to do anything else)? More like 2k core weeks?
Each node of Verification Cluster has 24 physical cores or 48 virtual cores (reference here). So indeed it’s ~2k core weeks, or half if you count the physical cores.