Unit Testing with Python

Unit testing is considered an essential part of software development. Through unit testing, we can evaluate each code component, find out how well it performs, and determine how well it reacts to valid or invalid input. A regression suite of unit tests is also an excellent way of detecting unexpected changes in a code base caused by refactoring or writing new code.

In this article, I examine the mechanisms of unit testing in Python, starting with the unittest module and its key classes. I examine tests individually and in suites, and I discuss how to facilitate their construction and use. Readers should have a working knowledge of Python. The sample test code requires Python 2.5 or later.

The unittest Module

The unittest module started life as the third-party module PyUnit. PyUnit was a Python port of JUnit, the Java unit testing framework. Designed by Steve Purcell, PyUnit became an official Python module starting with version 2.5.

Figure 1: Core classes in unittest.

As Figure 1 shows, there are five key classes in the unittest module. The TestCase class holds the test routines and provides hooks for preparing each routine and for cleaning up after. The TestSuite class serves as a collection container. It can hold multiple TestCase objects and multiple TestSuite objects.

The TestLoader class loads test cases and suites defined locally or from an external file. It emits a TestSuite object that holds those cases and suites. The TextTestRunner class provides a standard platform to run the tests. The TestResults class provides a standard container for the test results.

Out of these five classes, only TestCase must be subclassed. The other four classes can also be subclassed, but they are generally used as is.

Preparing a Test Case

Figure 2 shows the structure of the TestCase class. In it are three sets of methods that are used most often in designing the tests. In the first set are the pre- and post-test hooks. The setUp() method fires before each test routine, the tearDown() after the routine. Override these methods when you create a custom test case.

Figure 2: The structure of a TestCase class.

The second pair of methods control test execution. Both methods take a message string as input, and both abort an ongoing test. But the skipTest() method cancels the current test, while the fail() method fails it explicitly.

The third set of methods help identify the test. The method id() returns a string containing the name of the TestCase object and of the test routine. And the method shortDescription() returns the docstr comment at the start of each test routine. If the routine has no such comment, shortDescription() returns a None.

Listing One shows the sample bare bones test case FooTest. FooTest has two test routines: testA() and testB(). Both routines get the required argument of self. Both have a docstr comment for a first line.

Listing One: Code to show the sequence of unit test execution.

#!/usr/bin/python

import unittest

class FooTest(unittest.TestCase):
    """Sample test case"""
    
    # preparing to test
    def setUp(self):
        """ Setting up for the test """
        print "FooTest:setUp_:begin"
        ## do something...
        print "FooTest:setUp_:end"
    
    # ending the test
    def tearDown(self):
        """Cleaning up after the test"""
        print "FooTest:tearDown_:begin"
        ## do something...
        print "FooTest:tearDown_:end"
    
    # test routine A
    def testA(self):
        """Test routine A"""
        print "FooTest:testA"
    
    # test routine B
    def testB(self):
        """Test routine B"""
        print "FooTest:testB"

Figure 3 shows how FooTest behaves when executed.

Figure 3: FooTest behavior.

Note the same setUp() and tearDown() methods run before and after each test routine. So how do you let setUp() and tearDown() know which routine is being run?  You must first identify the routine by calling shortDescription() or id() (See Listing Two). Then use an if-else block to route to the appropriate code. In the sample snippet, FooTest calls shortDescription() to get the routine's docstr comment, then runs the prep and clean-up code for that routine.

Listing Two: Using test descriptions.

import unittest

class FooTest(unittest.TestCase):
    """Sample test case"""
    
    # preparing to test
    def setUp(self):
        """ Setting up for the test """
        print "FooTest:setUp_:begin"
        
        testName = self.shortDescription()
        if (testName == "Test routine A"):
            print "setting up for test A"
            
        elif (testName == "Test routine B"):
            print "setting up for test B"
            
        else:
            print "UNKNOWN TEST ROUTINE" 
            
        print "FooTest:setUp_:end"
    
    # ending the test
    def tearDown(self):
        """Cleaning up after the test"""
        print "FooTest:tearDown_:begin"
        
        testName = self.shortDescription()
        if (testName == "Test routine A"):
            print "cleaning up after test A"
            
        elif (testName == "Test routine B"):
            print "cleaning up after test B"
            
        else:
            print "UNKNOWN TEST ROUTINE"
            
        print "FooTest:tearDown_:end"
    
    # see Listing One...

Designing a Test Routine

Each test routine must have the prefix "test" in its name. Without that prefix, the routine will not run. To perform a test, the test routine should use an assert method. An assert method gets one or more test arguments and an optional assert message. When a test fails, the assert halts the routine and sends the error message to stdout.

There are three sets of assert methods. In the first set (Table 1) are the basic Boolean asserts, which fire on a True or False result.

Table 1: Basic asserts in unittest.

To check for just a True or False, use assertTrue() or assertFalse(), as in Listing Three:

Listing Three: Checking for True or False.

self.assertTrue(argState, "foobar() gave back a False")
# -- fires when the instance method foobar() returns a True

self.assertFalse(argState)
# -- fires when foobar() returns a False
# Notice this one does not supply an assert message

To check whether two arguments are the same, use assertEqual() and assertNotEqual() as in Listing Four. These last two asserts check the arguments' values, as well as their data types.

Listing Four: Checking arguments.

argFoo = "narf"
argBar = "zort"

self.assertEqual(argFoo, argBar, "These are not the same")
# -- this assert will fail

self.assertNotEqual(argFoo, argBar, "These are the same")
# -- this assert will succeed

argFoo = 123
argBar = "123"

self.assertEqual(argFoo, argBar, "These are not the same")
# -- this assert will fail

To check if the arguments are the same objects, use assertIs() and assertIsNot(). Like assertEqual() and assertNotEqual(), these two asserts examine both argument values and type. To check if an argument is an instance of a specific class, use assertIsInstance() and assertIsNotInstance() as in Listing Five.

Listing Five: Checking if an argument is an instance of a specific class.

argFoo = Bar()

# checking against class Bar
self.assertIsInstance(argFoo, Bar, "The object is not an instance of class Bar")
# -- this assert will succeed

# checking against class Foo
self.assertIsNotInstance(argFoo, Foo, "The object is an instance of class Foo")
# -- this assert will fail

Both asserts get a class name as a second argument. Both behave similarly to the library function isInstance(). Finally, to check for a nil, use assertIsNone() and assertIsNotNone().

The second set of asserts are comparative (see Table 2).

Table 2: The comparative assertions.