New test decorators for iteration

Data Management DM Notifications
,
#1

I have recently merged DM-22199, which provides class and method decorators for tests to provide iteration. The intent is to make it simple to extend unit tests to cover more cases.

If you create the object being tested in the setUp method, use the class decorator, lsst.utils.tests.classParameters. It creates multiple classes with the parameters you provide. For example:

from lsst.utils.tests import classParameters

@classParameters(foo=[1, 2], bar=[3, 4])
class MyTestCase(unittest.TestCase):
    def setUp(self):
        self.fooBar = FooBar(self.foo, self.bar)

will generate two classes, as if you wrote:

class MyTestCase_1_3(unittest.TestCase):
    foo = 1
    bar = 3

    def setUp(self):
        self.fooBar = FooBar(self.foo, self.bar)
    ...

class MyTestCase_2_4(unittest.TestCase):
    foo = 2
    bar = 4

    def setUp(self):
        self.fooBar = FooBar(self.foo, self.bar)
    ...

Note that the parameter values are embedded in the class name, which allows identification of the particular class in the event of a test failure.

If you want to iterate over parameters for an individual test, use the method decorator, lsst.utils.tests.methodParameters. It loops over the parameters, using subTest to identify the parameter combination in the event of a failure. For example:

from lsst.utils.tests import methodParameters

class MyTestCase(unittest.TestCase):
    @methodParameters(foo=[1, 2], bar=[3, 4])
    def testSomething(self, foo, bar):
        ...

will run tests:

testSomething(foo=1, bar=3)
testSomething(foo=2, bar=4)

There’s also a lsst.utils.tests.debugger function decorator that will drop you into pdb when it catches an exception.

1 Like
#2

This looks great! Is there a way to make it do an itertools.product of the method parameters rather than a zip of them?

#3

Not trivially. I thought it was more important to be explicit about the parameters than make it easy to get complicated. I guess you could do something like this:

def parametersProduct(**settings):
    """Produce parameter lists corresponding to the product on the input parameters

    Parameters
    ----------
    **settings : `dict` (`str`: iterable)
        The lists of test parameters. The lists may be of different lengths.

    Returns
    -------
    parameters : `dict` (`str`: iterable)
        Lists of test parameters; the product of the inputs. The lists will be of the same length.
    """
    values = list(itertools.product(*settings.values()))
    return {name: [vv[ii] for vv in values] for ii, name in enumerate(settings)}

Then you can:

class MyTestCase(unittest.TestCase):
    @methodParameters(**parametersProduct(foo=[1, 2], bar=[3, 4]))
    def testSomething(self, foo, bar):
        ...

and it will run:

testSomething(foo=1, bar=3)
testSomething(foo=1, bar=4)
testSomething(foo=2, bar=3)
testSomething(foo=2, bar=4)

If people think it’s useful, we could add parametersProduct to lsst.utils.tests.

1 Like