Channels ▼
RSS

JVM Languages

Unit Testing with Python


Unit testing is considered an essential part of software development. Through unit testing, we can evaluate each code component, find out how well it performs, and determine how well it reacts to valid or invalid input. A regression suite of unit tests is also an excellent way of detecting unexpected changes in a code base caused by refactoring or writing new code.

In this article, I examine the mechanisms of unit testing in Python, starting with the unittest module and its key classes. I examine tests individually and in suites, and I discuss how to facilitate their construction and use. Readers should have a working knowledge of Python. The sample test code requires Python 2.5 or later.

The unittest Module

The unittest module started life as the third-party module PyUnit. PyUnit was a Python port of JUnit, the Java unit testing framework. Designed by Steve Purcell, PyUnit became an official Python module starting with version 2.5.

Python Unit Tests
Figure 1: Core classes in unittest.

As Figure 1 shows, there are five key classes in the unittest module. The TestCase class holds the test routines and provides hooks for preparing each routine and for cleaning up after. The TestSuite class serves as a collection container. It can hold multiple TestCase objects and multiple TestSuite objects.

The TestLoader class loads test cases and suites defined locally or from an external file. It emits a TestSuite object that holds those cases and suites. The TextTestRunner class provides a standard platform to run the tests. The TestResults class provides a standard container for the test results.

Out of these five classes, only TestCase must be subclassed. The other four classes can also be subclassed, but they are generally used as is.

Preparing a Test Case

Figure 2 shows the structure of the TestCase class. In it are three sets of methods that are used most often in designing the tests. In the first set are the pre- and post-test hooks. The setUp() method fires before each test routine, the tearDown() after the routine. Override these methods when you create a custom test case.

Python Unit Tests
Figure 2: The structure of a TestCase class.

The second pair of methods control test execution. Both methods take a message string as input, and both abort an ongoing test. But the skipTest() method cancels the current test, while the fail() method fails it explicitly.

The third set of methods help identify the test. The method id() returns a string containing the name of the TestCase object and of the test routine. And the method shortDescription() returns the docstr comment at the start of each test routine. If the routine has no such comment, shortDescription() returns a None.

Listing One shows the sample bare bones test case FooTest. FooTest has two test routines: testA() and testB(). Both routines get the required argument of self. Both have a docstr comment for a first line.

Listing One: Code to show the sequence of unit test execution.

#!/usr/bin/python

import unittest

class FooTest(unittest.TestCase):
    """Sample test case"""
    
    # preparing to test
    def setUp(self):
        """ Setting up for the test """
        print "FooTest:setUp_:begin"
        ## do something...
        print "FooTest:setUp_:end"
    
    # ending the test
    def tearDown(self):
        """Cleaning up after the test"""
        print "FooTest:tearDown_:begin"
        ## do something...
        print "FooTest:tearDown_:end"
    
    # test routine A
    def testA(self):
        """Test routine A"""
        print "FooTest:testA"
    
    # test routine B
    def testB(self):
        """Test routine B"""
        print "FooTest:testB"

Figure 3 shows how FooTest behaves when executed.

Python Unit Tests
Figure 3: FooTest behavior.

Note the same setUp() and tearDown() methods run before and after each test routine. So how do you let setUp() and tearDown() know which routine is being run?  You must first identify the routine by calling shortDescription() or id() (See Listing Two). Then use an if-else block to route to the appropriate code. In the sample snippet, FooTest calls shortDescription() to get the routine's docstr comment, then runs the prep and clean-up code for that routine.

Listing Two: Using test descriptions.

import unittest

class FooTest(unittest.TestCase):
    """Sample test case"""
    
    # preparing to test
    def setUp(self):
        """ Setting up for the test """
        print "FooTest:setUp_:begin"
        
        testName = self.shortDescription()
        if (testName == "Test routine A"):
            print "setting up for test A"
            
        elif (testName == "Test routine B"):
            print "setting up for test B"
            
        else:
            print "UNKNOWN TEST ROUTINE" 
            
        print "FooTest:setUp_:end"
    
    # ending the test
    def tearDown(self):
        """Cleaning up after the test"""
        print "FooTest:tearDown_:begin"
        
        testName = self.shortDescription()
        if (testName == "Test routine A"):
            print "cleaning up after test A"
            
        elif (testName == "Test routine B"):
            print "cleaning up after test B"
            
        else:
            print "UNKNOWN TEST ROUTINE"
            
        print "FooTest:tearDown_:end"
    
    # see Listing One...

Designing a Test Routine

Each test routine must have the prefix "test" in its name. Without that prefix, the routine will not run. To perform a test, the test routine should use an assert method. An assert method gets one or more test arguments and an optional assert message. When a test fails, the assert halts the routine and sends the error message to stdout.

There are three sets of assert methods. In the first set (Table 1) are the basic Boolean asserts, which fire on a True or False result.

Assert Complement Assert Operation

assertTrue(a, M)

assertFalse(a, M)

a = True; a = False

assertEqual(a, b, M)

assertNotEqual(a, b, M)

a = b; a ≠ b

assertIs(a, b, M)

assertIsNot(a, b, M)

a is b; a is not b

assertIsNone(a, M)

assertIsNotNone(a, M)

a = nil; a ≠ nil

AssertIsInstance(a, b, M)

AssertIsNotInstance(a, b, M)

isinstance(a,b);
not isinstance(a,b)

Table 1: Basic asserts in unittest.

To check for just a True or False, use assertTrue() or assertFalse(), as in Listing Three:

Listing Three: Checking for True or False.

self.assertTrue(argState, "foobar() gave back a False")
# -- fires when the instance method foobar() returns a True

self.assertFalse(argState)
# -- fires when foobar() returns a False
# Notice this one does not supply an assert message

To check whether two arguments are the same, use assertEqual() and assertNotEqual() as in Listing Four. These last two asserts check the arguments' values, as well as their data types.

Listing Four: Checking arguments.

argFoo = "narf"
argBar = "zort"

self.assertEqual(argFoo, argBar, "These are not the same")
# -- this assert will fail

self.assertNotEqual(argFoo, argBar, "These are the same")
# -- this assert will succeed

argFoo = 123
argBar = "123"

self.assertEqual(argFoo, argBar, "These are not the same")
# -- this assert will fail

To check if the arguments are the same objects, use assertIs() and assertIsNot(). Like assertEqual() and assertNotEqual(), these two asserts examine both argument values and type. To check if an argument is an instance of a specific class, use assertIsInstance() and assertIsNotInstance() as in Listing Five.

Listing Five: Checking if an argument is an instance of a specific class.

argFoo = Bar()

# checking against class Bar
self.assertIsInstance(argFoo, Bar, "The object is not an instance of class Bar")
# -- this assert will succeed

# checking against class Foo
self.assertIsNotInstance(argFoo, Foo, "The object is an instance of class Foo")
# -- this assert will fail

Both asserts get a class name as a second argument. Both behave similarly to the library function isInstance(). Finally, to check for a nil, use assertIsNone() and assertIsNotNone().

The second set of asserts are comparative (see Table 2).

Assert Complement Assert Operation

assertGreater(a, b, M)

assertLess(a, b, M)

a > b; a < b

assertGreaterEqual(a, b, M)

assertLessEqual(a, b, M)

a ≥ b; a ≤ b

assertAlmostEqual(a, b, n, M)

assertNotAlmostEqual(a, b, n, M)

round(a-b, n) = 0
round)a-b, n) ≠ 0

assertItemsEqual(a, b, M)

none

sort(a) = sort(b);
sort(a) ≠ sort(b)

Table 2: The comparative assertions.


Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.
 

Video