Unit Testing with Python

To check whether one argument is greater or less than another, use assertGreater() and assertLess() as in Listing Six.

Listing Six: Greater or Less.

argFoo = 123
argBar = 452

self.assertGreater(argFoo, argBar, "Foo is less than Bar")
# -- this assert will fail

self.assertLess(argFoo, argBar, "Foo is greater than Bar")
# -- this assert will succeed

To check whether one argument is greater, less than or equal to the other, use assertGreaterEqual() and assertLessEqual(). The arguments in these four asserts can be a primitive (integer, float, character), a sequence, or a collection. Both arguments, however, must have the same data type.

To do a tolerance check, use assertAlmostEqual() and assertAlmostNotEqual(). These two asserts round off the arguments to a fixed number of decimal places before comparing their values. The number of decimal places is 7 by default. To change it, pass the new number with a places label.

To compare two sequences, use assertItemsEqual(). Its two arguments must be the same sequence type (list, tuple, set, and so on). Note that this assert sorts the sequence items prior to comparison.

Third set of asserts (Table 3) work with collection objects such as dictionaries, lists, sets, and tuples.

Table 3: Assertions for collections.

Arguments must be a collection type. Some assertions need not use arguments of the same type. All but one assert in this set have no complements.

To check whether one dictionary has some of the key/value pairs as the other, use assertDictContainsSubset() as shown in Listing Seven.

Listing Seven.

argBar = {'narf':456, 'poink':789}

self.assertDictContainsSubset(argFoo, argBar, "Foo does not have Bar")
# -- this assert will succeed

self.assertDictContainsSubset(argBar, argFoo, "Foo does not have Bar")
# -- this assert will fail

argBar = {'narf':456, 'egad':789}
self.assertDictContainsSubset(argFoo, argBar, "Foo does not have Bar")
# -- this assert will also fail

The first argument serves as reference; the second holds the pairs in question. To check if both dictionaries have the same key/value pairs, use assertDictEqual(). Each pair must have the same key labels and data values. How the pairs are arranged is irrelevant.

To check two sequence objects, use assertSequenceEqual(). Sequence types include lists, sets, tuples, even strings. For sequence objects to be same, they must have the same number of data items. The items must have the same value and they must be arranged the same. The sequence type must also be the same.

To check if two list objects are the same, use assertListEqual(). Both objects must have the same number of items. Those items must have the same values and the same order. To check two set objects, use assertSetEqual(). As with lists, both set objects must have the same number of items and item values. But item order is irrelevant, because set objects arrange their items internally.

Finally, to check if two tuples are the same, use assertTuplesEqual(). To check if two strings are the same, use assertMultilineEqual(). And to find out if one string is or is not inside another string, use assertIn() and assertNotIn().

This third set of asserts has one interesting behavior. If the collection objects are not equal, the assert will report the differences between the objects. It also adds this diff result to the assert message, if one is available.

Preparing a Test Suite

Usually, a few test cases are enough for your testing needs of a single class. But what if you have a dozen or more tests cases on hand — some you wrote yourself, some written by others? What if you want only a subset of test routines to run from the same test case? What if you want to refactor your test routines for easier cataloging and distribution? For these situations, you might need a test suite.

Figure 4 shows the basic structure of the TestSuite class.

Figure 4: The TestSuite class.

There are three sets of instance methods. The first set lets us add test cases to the suite. To add a single test case, use the addTest() method as shown in Listing Eight.

Listing Eight.

class FooTest(unittest.TestCase):
	def testA(self):
		"""Test routine A"""
		print "Running test A"

# creating a new test suite
newSuite = unittest.TestSuite()

# adding a test case
newSuite.addTest(unittest.makeSuite(FooTest))

Pass the test case (here being FooTest) to the instance method using the convenience function makeSuite(). You can also use makeSuite() to "convert" the test case into a test suite.

newSuite = unittest.makeSuite(FooTest)

To add a specific test routine, pass the test case object to the suite through the same addTest() method. Then pass the name of the test routine to the test case's constructor. Notice the routine name is passed as a string.

newSuite.addTest(FooTest("testA"))

You can also use the same method to add two or more test routines to the same suite:

To add two or more test cases, gather the names of the test cases into a list as shown in Listing Nine.

Listing Nine.

testList = [FooTest, BarTest]
testLoad = unittest.TestLoader()

caseList = []
for testCase in testList:
	testSuite = testLoad.loadTestsFromTestCase(testCase)
	caseList.append(testSuite)

newSuite = unittest.TestSuite(caseList)

Parse the list with a for loop, then use a TestLoader object (testLoad) to read each case. Add the read cases to a second list (caseList). Then create the TestSuite object (newSuite), passing to the class constructor the list object caseList.

Suppose you want to add another test suite to the suite. Simply pass the other suite to the addTest() method — no need to reuse the makeSuite() function to prepare the added suite.

fooSuite = unittest.TestSuite()
fooSuite.addTest(unittest.makeSuite(FooTest))
#...
barSuite = unittest.TestSuite()
barSuite.addTest(fooSuite)

The second set of methods run the tests in the test suite. The run() method takes a TestResult object as input, while debug() does not. But debug() does let an external debugger monitor the ongoing test.

Finally, the last set contains the method countTestCases(). This method returns the number of test cases held in the suite.

testCount = fooSuite.countTestCases()

Running the Tests

You have two ways to run your unit tests. If the test script is a single file with one or more test cases, add these lines after the last test case.

if __name__ == "__main__":
unittest.main()

The if block detects how the file is acted upon. If the file is imported into another file, the macro __name__ is unchanged. If the file is executed directly, either from the text editor, from another script, or from the command-line, the __name__ macro resolves into "__main__", and the class method main() gets called. This in turn invokes the run() methods of every test case defined or imported by the script file.

If the script file defines a test suite, first create an instance of TextTestRunner. Then pass the test suite object to the runner's run() method.

Regardless of approach, test cases and their routines run in alphanumeric order. BarTest runs before FooTest, FooTest before Test123, and test_12() before test_A(). Consider the test script in Listing Nine. In it, you have two test cases: BarTest and FooTest. BarTest has three test routines; FooTest has two.

Figure 5 shows how these two test cases run. BarTest runs first, FooTest second.

Figure 5: BarTest and FooTest.

The test routines in BarTest run in the order from A to C. Those in FooTest run from 1 to 2. The same setUp() and tearDown() methods run before and after each test routine. But the BarTest routines have their own setUp() and tearDown(). The same also holds for the FooTest routines.

Finally, you have the option to skip or fail some of the test routines. To skip a routine unconditionally, use the class method unittest.skip() as shown in Listing Ten.

Listing Ten: Skipping a routine.

@unittest.skip("Skip over the entire test routine")
def testB():
	"""Test routine B"""
	# the following code will not run
	fooA = narf()
	self.assertNotEqual(fooA, 123)

This method gets one argument: a log message describing the reason for the skip. Place the method call before the test routine, and make sure to prefix the call with a @ token. Alternatively, use the instance method skipTest(), which you can place inside the test routine as in Listing Eleven.

Listing Eleven.

	"""Test routine B"""
	self.skipTest("Skip over the rest of the routine")
	
	# the following code will not run
	fooA = narf()
	self.assertNotEqual(fooA, 123)