Kirby is a senior consultant with Dataline Inc. He is also a Microsoft Certified Solution Developer, Charter Member. He can be reached at [email protected].
If you are like me, you like writing tests for your code about as much as you like writing end-user documentation. Typically, you might write a separate application or module to test a module in your software application. This type of testing (called "unit-level testing") is usually given short shrift -- if it is included in the project plan at all. I don't like writing unit-level tests in most modern structured programming languages. Since the test code is usually more complicated and harder to write than it should be, I end up not testing nearly as much as I should. Intellectually, I know testing is good, but time spent away from coding the main application seems hard to justify emotionally. Then I stumbled upon JPython. While it may not be the ultimate answer, JPython can help you quickly produce unit tests for your Java packages.
JPython is a freely available version of Python (http://www.python.org) implemented in 100 percent pure Java. Python, a cross-platform programming language implemented on most major platforms, can be used for a variety of purposes. Python's abilities as a scripting language are important when it comes to writing test applications. Features that make Python a good scripting language include:
- No separate compile and link phase.
- No type declarations.
- Dynamic loading and reloading of modules.
- Huge library of add-on modules.
- Access to the built-in compiler and bytecode interpreter.
To this list, JPython (written by Jim Hugunin and available at http://www .python.org/jpython/download) adds seamless access to Java packages and classes. (When downloading JPython, I recommend you also download the Python 1.5.2 library. Part of Python's power comes from its extensive library of add-on modules. The standard JPython distribution comes with only a subset of the Python library.) Since JPython is implemented in Java and runs in the Java Virtual Machine, using Java classes in Python code is trivial. As a scripting language with access to Java classes, JPython lets you quickly whip up a test of any Java class.
There are two issues I've run across concerning portability and JPython. First, JPython does not seem to work with the Kaffe JVM distributed with Red Hat Linux (http://www.redhat.com/). However, it does work fine with the Blackdown JVM for Linux (http://www.blackdown.org). Second, because JPython does on-the-fly bytecode loading, some Just-In-Time (JIT) compilers can have problems. Still, I've written some pretty advanced code with JPython and have only had to turn off the JIT compiler once.
JPython Console Interface
Since JPython includes a console interface, you can begin using it without writing any code. When you start JPython without any command-line options, you'll see JPython's startup message, as with Example 1(a), which uses the JVM shipped with JBuilder2. The ">>>" is JPython's command prompt. From there you can begin typing any valid JPython command. I use this console interface extensively to explore the functionality of unfamiliar Java classes. By exploring their inner workings, I save time in the edit, compile, and test phases of Java coding. Let's say I just learned that the java.math.BigInteger class might solve my integer problems, but from the spartan documentation, I haven't figured out how to use it. To explore this class in JPython, I must first make JPython aware of its existence using Example 1(b). This is actually the same syntax you would use to import a regular Python class. Next, I can try out one of the constructors for BigInteger and view the results; see Example 1(c).
The first line creates a new instance of the BigInteger class. As you'd expect, JPython looks for an appropriate constructor of BigInteger and creates a new object (bi). When displaying the contents of bi, JPython also looks for an appropriate way to display the object, by eventually calling the toString method of the object. If JPython cannot find an appropriate constructor, then it would report an appropriate error. If you wanted to test BigInteger's max method, you could try Example 1(d).
There is one JPython feature that makes scripts that use Java cleaner looking and sometimes easier to write. JPython uses the JavaBean Introspector to identify class properties either from common design patterns (methods named getX or setX) or from explicitly specified BeanInfo. Although illegal in Java, Example 2(a) lets you write this code using java.util.Date's getHours and setHours methods.
Unfortunately, I don't recommend using this inside of your own JPython test scripts. There is nothing wrong with the functionality -- it works fine -- but I like being able to use my test scripts as sample code for the consumers of my Java classes. Other developers reading my scripts really need to see the getX and setX methods that they would have to use in their own Java programs. This example without the Bean properties looks like Example 2(b). It's not as pretty, but makes for a better example of how to use the Date class.
JPython Test Script
Listing One is a JPython program to test the java.math.BigInteger class. When you examine Listing One, you will see more involved tests of BigInteger than those I just described, plus a new test for BigInteger's abs method. The structure of this program is typical of the test scripts I first created with JPython:
1. Print an introductory message (line 6).
2. Perform one or more tests (lines 7-22).
3. Print a closing message saying all is well (line 23).
Python, unlike some scripting languages, is a "real" language and supports procedures, classes, and inheritance. I created a base class (TestCase) designed to make the quick and consistent coding of tests possible. Listing Two performs the same tests as Listing One, but within a Python class derived from TestCase (shown in Listing Three). Although the BigIntegerTest class seems to be a bit longer than our first version in Listing One, it contains more functionality, and each actual test is shorter. Since each test is segregated into its own class method, you can easily identify the individual tests and add new tests without worrying about side effects from other tests.
TestCase
Before considering all of the intricacies of Python classes and the TestCase base class, look at the code involved in writing a single test. Compare lines 7-11 of Listing One and lines 9-12 of Listing Two. In Listing One, the code performs an operation, checks the result, and if the result wasn't what was expected, prints an error and exits. My problem with Listing One is that it takes some mental effort to construct the test, and this effort is duplicated in each test. The corresponding code in Listing Two replaces the If statement and the call to sys.exit(1) with assert_. This method is defined in the TestCase base class. Like the standard assert functions found in other languages, TestCase's method only jumps into action if the statement passed to it evaluates to False (0 in Python). The single call to assert_ replaces the If, print error, and exit functions in each test of the original code.
You should also notice that what was a comment in Listing One (# Test String Conversion) is now in a special quoted string called a "docstring." In Python, practically everything is an object, including the functions, and functions have a __doc__ property that returns this quoted string. Since the docstring can be accessed programmatically, the test harness can now display information about the test. In effect, a line in the test script that was useful only to the developer of the test is now doing double duty when the test is run. In TestCase, the docstring is printed to the console before the test is run, providing onscreen commentary for free.
To run the new TestCase-derived unit test for BigInteger, simply run JPython with the name of the script on the command line. Example 3 shows the results on my computer.
When JPython loads BigIntegerTest.py (available electronically; see "Resource Center," page 5), it immediately executes lines 23-27, which eventually call TestCase's runTest method. This method is responsible for executing the test methods within the class. In addition, this method is responsible for creating the output just mentioned.
runTest is the main workhorse of TestCase and any of the derived classes. Using Python's inspection capabilities, runTest first obtains a dictionary of all the methods and properties available in the class. If a method begins with "test_," runTest performs the following activities:
- Calls the class initialize method.
- Writes out the docstring.
- Executes the method.
- Calls the reset method.
When there are no more methods, runTest exits. Since the initialize and reset methods are executed before and after each test, they can hold code common to all of the tests -- rather like a function-level constructor and destructor. These methods also serve to ensure that each test is run in an environment separate from the other tests. This, together with each test being in a separate method, cuts down on side effects between tests.
Remember that TestCase's assert_ method is responsible for reporting any problems in the test. Its main code is activated only if the expression passed is False. If the expression is False, the assert_ method outputs the name of the file and line number where the assertion failed and calls the writeError method to display the error message. A raise call at the end of assert_ sends the error back to the caller. In the case of BigIntegerTest.py, the error is simply ignored and the unit test exits. My feeling is that once the unit has failed one test, the rest of the tests are suspect. Once a test fails I do not bother running any remaining tests on that unit.
TestCase contains one other method that I have not discussed. failTest keeps track of whether a test in this unit test has failed. If the unit test exits immediately after a failure, why keep track of the failed test?
Batch Testing
Once you have imposed a modicum of structure on the unit-level tests, it is not a far stretch to implement all sorts of automated, batch, and regression testing. The testsuite.py and testeng.py scripts (both available electronically) implement batch testing, but they can easily be extended to support all manner of testing, while still allowing simple test cases to be written quickly.
Testsuite.py contains a single class called TestSuite that does not really do much besides act as a collection class for the TestCase-derived class. TestSuite has the typical collection methods -- add, remove, and count -- to manage the contents of the collection. The interesting methods in TestSuite are run, and runTest actually executes all of the tests in the suite and a single test, respectively. The run method simply iterates through the collection and calls the runTest method for each test. runTest performs the same function that the "__main__" code did in Listing Two. It calls TestCase's runTest method and intercepts any exceptions generated by the test.
Testeng.py is the master integrator between TestSuite, TestCase, and the unit-level tests you design. Testeng.py loads each test class into a TestSuite, then executes the TestSuite by calling the run method. It accomplishes this magic using two cool features of JPython (and Python for that matter). First, JPython lets you dynamically build, compile, and execute Python code on-the-fly. Second, JPython provides functionality to dynamically load new modules. Near the end of testeng.py, Example 4(a) dynamically loads each test module.
The module is imported into JPython and a handle to that module is retrieved in the first two lines. If the module contains a procedure called addTests, that method is called with a TestSuite object, allowing each module to control how tests are added to the test suite. Example 4(b) is an example of an addTests method for the BigIntegerTest.py module.
Testeng.py will attempt to load any .py file in the current directory that does not begin with "test" (so testcase.py, testsuite.py, and testeng.py will not be mistaken for actual tests).
Conclusion
Since JPython is written in Java, it is straightforward to include JPython packages in a Java application and use JPython as your application's scripting engine. JPython classes can subclass Java classes. JPython also makes an excellent tool for prototyping Java applets that are embedded in a web browser.
Still, the testing framework described here is not dependent on JPython. The code works unmodified under the DOS, Windows, and UNIX versions of Python. If your bailiwick is Windows and COM, the Python "win32all" add-in lets you test COM components the same way JPython lets you test Java packages and classes.
The test harness included with this article is basic in nature, but can be extended. Its primary focus is to enable you to create unit tests with as little work as possible. In my experience, it's best to write test cases quickly, or you won't write them at all.
DDJ
Listing One
01: # 02: # BigIntegerTest.py 03: # Tests the standard java class java.math.BigInteger 04: import sys 05: from java.math import BigInteger 06: print "Test the standard java BigInteger class." 07: # Test string conversion 08: bi = BigInteger( '100' ) 09: if bi.longValue() != 100: 10: print "BigInteger string conversion failed." 11: sys.exit(1) 12: # Test max operator 13: bi1 = BigInteger( '100' ) 14: bi2 = BigInteger( '200' ) 15: if bi1.max(bi2) != 200: 16: print "BigInteger max operator failed." 17: sys.exit(1) 18: # Test absolute value operator 19: bi = BigInteger( '-100' ) 20: if bi.abs() != 100: 21: print "BigInteger abs operator failed." 22: sys.exit(1) 23: print "All tests completed."
Listing Two
01: # 02: # BigIntegerTest.py 03: # Tests the standard java class java.math.BigInteger 04: import sys 05: from java.math import BigInteger 06: from testcase import TestCase 07: class BigIntegerTest(TestCase): 08: "Test the standard java BigInteger class." 09: def test_stringConversion(self): 10: "Test string conversion" 11: bi = BigInteger( '100' ) 12: self.assert_( bi.longValue() == 100 ) 13: def test_maxOperator(self): 14: "Test max operator" 15: bi1 = BigInteger( '100' ) 16: bi2 = BigInteger( '200' ) 17: self.assert_( bi1.max(bi2) == 200 ) 18: def test_abs(self): 19: "Test absolute value operator" 20: bi = BigInteger( '-100' ) 21: self.assert_( bi.abs() == 100 ) 22: if __name__ == "__main__": 23: try: 24: c = BigIntegerTest( "BigIntegerTest" ) 25: c.runTest() 26: except: 27: pass
Listing Three
01: import sys
02: import traceback
03: import os
04: from time import *
05: from string import *
06: class TestCase:
07: "Base class for python based unit level and regression tests. \
08: To use this class, implement the runTest, initialize, and reset \
09: methods. Call the assert_ method to test for success."
10: def __init__(self, name):
11: self.testName = name
12: self.failed = 1
13: self.failureReason = "<<Not Tested>>"
14: def runTest(self):
15: "Run the test methods in this class"
16: names = dir( self.__class__ )
17: self.failed = 0
18: self.failureReason = ""
19: for name in names:
20: # if this is a test method
21: if name[:5] == "test_":
22: self.initialize()
23: try:
24: func = getattr( self, name )
25: self.writeMessage( func.__doc__ )
26: # Execute the test
27: func()
28: finally:
29: self.reset()
30: def initialize(self):
31: "Called before each test is run. Should be overridden."
32: pass
33: def reset(self):
34: "Called after each test is run. Should be overridden."
35: pass
36: def failTest(self, reason):
37: "Mark this test as having failed."
38: self.failed = 1
39: self.failureReason = reason
40: def writeMessage(self, message):
41: "Writes a message to the standard output device."
42: self.__writeMessage(sys.stdout, message )
43: def writeError(self, error):
44: "Writes an error to the standard output device."
45: self.__writeMessage( sys.stdout, "ERR " + error )
46: def writeWarning(self, warning):
47: "Writes an warning to the standard output device."
48: self.__writeMessage( sys.stdout, "WRN " + warning )
49: def __writeMessage(self, outFile, message):
50: year, month, day, hour, minute, second, weekday, julian,\
51: daylight = localtime(time())
52: outFile.write( "%04d%02d%02d %02d:%02d:%02d\t" % (year,\
53: month, day, hour, minute, second) )
54: outFile.write( "%s\t%s\n" % (self.testName, message) )
55: def assert_(self, test):
56: "Logs and generates an error if the test is false."
57: if test == 0:
58: stack = traceback.extract_stack()
59: stackFrame = stack[len(stack)-2]
60: file,line,method,call = stackFrame
61: path,file = os.path.split(file)
62: source = "%s %s %s %s" % (file, line, method, call)
63: self.failTest( "Assertion failed: %s" % source )
64: self.writeError( self.failureReason )
65: raise AssertionError, source
Copyright © 1999, Dr. Dobb's Journal
