In the course of working on DM-7292, which should be a straightforward port of
pipe_tasks from py2 to py3, we encountered an interesting problem with how thresholds set with
assertAlmostEqual are used in tests. In this specific situation,
testProcessCcd returns slightly different values for image statistics (background mean, numGoodPix, image mean, etc.) when it is run in py2 and py3.
I believe the problem relates to how image noise is simulated with pseudorandom numbers. For instance, python’s built-in
random returns different values for the same seed in py2 vs. py3, while
numpy.random returns identical values in py2 and py3. Unfortunately, this doesn’t seem to be the cause of this specific issue because the problem persisted when I switched to using
numpy.random in the few places where
random was used throughout
Two main points. One, please be aware that tests may pass but in fact have subtly different calculated values than expected, particularly if
random is involved. Two, has anyone encountered an issue like this before and have any clues where the differences in the test image statistics are being introduced?