Matt is a senior QA engineer for Zilliant and can be contacted at firstname.lastname@example.org.
JUnit is a freely available Java framework built by Erich Gamma and Kent Beck (http://www.junit.org/). In general, JUnit's design encourages good testing habits by moving much of the menial bookkeeping to the automated system. However, JUnit does not provide an easy way to write tests for testing subclasses. Some developers cut-and-paste their tests between test classes, which makes the tests hard to maintain. Others attempt to subclass one test from another, which does not scale when interfaces need testing. If you intend to make your tests maintainable and robust, then they need a design just as much as the core system.
The GroboUtils project (http://groboutils.sourceforge.net), a project that I maintain, is a collection of useful Java utilities. Included in these utilities is the GroboUtils JUnit Extension (GJE) that helps you write tests for inherited logic. Among other features, GJE helps reflect the source structure in the testsif a class extends a class and/or implements an interface, then so do its test suites.
The JUnit Extension
The standard use for JUnit Version 3.7 test cases typically involves creating a class that contains test methods covering the features for a specific development class. This test class includes a public static method named suite(), which creates a junit.framework.TestSuite instance containing all the tests covered by the test case. TestSuite contains logic to parse test classes into separate instances, one for each test method. TestSuite creates each instance of the test class by passing the test method name to the test class's constructor. (In this article, I use the term "test suite" to refer to the group of tests created by a test class's static suite() method, and "concrete test classes" to refer to traditional JUnit test classes. I also refer to the nontest classes as "project classes," which are the set of classes being tested. JUnit Version 3.8.1 has slightly different requirements.)
GJE (available electronically, see "Resource Center," page 5) lets all test classes define an inheritance hierarchy used by the project class under test. For example, a test class for the javax.swing.JComboBox class in the Java Foundation Classes could define that it inherits all tests from the java.awt.event.ActionListener interface and the javax.swing.JComponent class. I refer to these inheritance hierarchy tests as "inherited test classes;" see Figure 1.
These inherited test classes use an externally instantiated factory to create instances of the project class under test. Thus, abstract classes and interfaces can define tests that rely on the factory, and concrete tests can add specific factories to the inherited test classes to test for compatibility with the concrete class. Since concrete test classes can associate multiple factories per inherited test class, they can ensure that many different initial states of the concrete class all behave in accordance with the inherited specification. This means that each inherited test can run in the concrete class's context. Figure 2 is the public GJE UML diagram.
Sample1 (all sample code is available electronically) is a simple interface that presents a contract to all implementations, telling them that they must throw an IllegalArgumentException when receiving a Null argumentsomething that should be tested for in all implementations. The corresponding test class for this interface, Sample1TestI (Sample1TestI.java), has the same general form as a standard JUnit test. One test method, testAddString1(), passes a Null value to an instance of the interface, and ensures that an IllegalArgumentException was thrown, while the other test method, testAddString2(), ensures that no exception is thrown when the test passes an empty string to the instance. (You should not consider any of the test classes presented here a thorough test.)
But since I declared Sample1 as an interface, it cannot be instantiated. So the test class inherits from net.sourceforge.groboutils.junit.v1.iftc.InterfaceTestCase (itself a subclass of junit.framework.TestCase). The constructor for InterfaceTestCase requires as a parameter a factory that knows how to create the particular instance of the interface, and subclasses of InterfaceTestCase can access this factory through the method getImplObject(). InterfaceTestCase performs assertions on the factory-created object to ensure it does not equal Null, and it is an instance of the class passed into the constructor, which, in this case, is Sample1.
Standard JUnit provides the java.framework.TestSuite class to parse test classes into individual tests. TestSuite's parsing behavior creates a new instance of the test class for every discovered test method, passing the name of the method to the test class's constructor. Each test class should provide a static suite() method to return a collection of tests for itself. A standard JUnit test class may define the suite() method as:
return new TestSuite( Sample1TestI.class );
Instead of using TestSuite, Sample1TestI uses the GJE InterfaceTestSuite class (see Listing One; also available electronically), which can parse both standard JUnit test classes as well as InterfaceTestCase-style classes. Just as with TestSuite, test classes don't have to extend InterfaceTestCase for InterfaceTestSuite to recognize the test classes as interface tests, as long as the test class provides a constructor in the correct form: '( String, ImplFactory )'.
Sample1Impl (Sample1Impl.java) is a minimal implementation of Sample1. The corresponding test class, Sample1ImplTest (Sample1ImplTest.java), defines a concrete test class for Sample1Impl. This is identical to a standard JUnit test class, except for the suite() method (see Listing Two, also available electronically). Step by step, this suite() method:
1. Gets the inherited test suite by calling Sample1TestI.suite(), which contains the tests for Sample1.
2. Adds itself to the retrieved test suite by calling suite.addTestSuite(THIS_CLASS), just like a call to the corresponding method in the TestSuite class.
3. Creates an inner class from the ImplFactory interface, which creates the default Sample1Impl instance.
4. Returns the new test suite.
The returned test suite delays the creation of the individual test instances until necessary, so that you can add additional tests. When the suite creates all the tests, InterfaceTestSuite processes all standard JUnit TestCase classes as normal (one instance per test method). For inherited test classes, it creates one test instance per test method per factory. For the Sample1ImplTest example, three tests are created: Sample1TestI defines two test methods, Sample1ImplTest only specifies one factory for these (that's two tests), and Sample1ImplTest defines one test. If another factory was added, then five tests would be created: one set of each Sample1TestI test for each factory, plus the Sample1ImplTest test.
Abstract and Base Classes
You can test abstract and base classes the same way interfaces are tested. In these situations, more than just contracts can be testedyou can test real code again.
Sample2 (Sample2.java) is an abstract class that also implements Sample1. Its corresponding inherited test class (Sample2TestI.java) tests the newly defined method, and relies on the suite() method to perform contract tests for Sample1 (see Listing Three, also available electronically).
Using the factory, concrete test classes can initialize their classes in different ways to ensure that the integrity of the class is maintained with different initial states. But what if the tests themselves need to dictate the initial setup data?
In Sample3 (Sample3.java) an interface whose only method returns an array of strings is shown. The just mentioned use-case for the factory doesn't let inherited test classes test much more than if the method causes an exception. It would be more interesting if the inherited tests could define the input data and validate the return value.
Sample3TestI (Sample3TestI.java) declares an inner factory to create Sample3 instances with a specific initial setup. Instead of declaring that Sample3TestI tests Sample3 instances, it tells the superclass in the constructor that it tests these internal factories. Now the tests become much more interesting.
The class signature for Sample5Impl (Sample5Impl.java) poses a problem for most test tools, and at first glance, it poses several problems for GJE as well (see Listing Four, also available electronically). Sample5Impl implements Sample3, whose test class requires a special factory, while the other test classes only need the standard ImplFactory. The implementation of InterfaceTestSuite shares all added factory instances between all added InterfaceTestCase classes, so these two inherited factories cannot be added to the same InterfaceTestSuite.
Sample5ImplTest (Sample5ImplTest.java) changes the start of the suite() method by creating a new TestSuite() instance, initializing it with the concrete test class. This allows for adding as many inherited test suites as needed, so that each suite has its own independent set of factories, and there will still only be a single test suite.
The returned test suite has a test containment that looks something like Figure 3, for a total of 12 tests. Not too shabby for a group of woefully inadequate test classes.
This framework can be implemented for the other xUnit-like libraries (http://www.xprogramming.com/software.htm).
Languages that allow for multiple inheritance (C++ and Python) may be tempted to use proper language-supported inheritance on the inherited test classes. This, however, causes three unexpected results: inherited tests with the same name to be "hidden" and not executed unless explicitly called or logic is added to explicitly search all inherited classes; allowing for multiple types of factories becomes difficult to track properly and could potentially become a maintenance nightmare for complex inherited classes; and multiple contexts (initial states for factory-created objects) increase the complexity of creating a suite of tests from a concrete test class.
If a project doesn't have tests, then the adage "any tests are good tests" looks absolutely right, but as development code matures, so should its set of automated tests. In particular, maintainability should remain as much a priority as metrics such as code coverage. Without descent maintainability, the cut-and-paste approach can lead to holes in your test suites as the project matures, and those holes cannot be detected easily. Introduction of simple testing patterns, such as the inherited testing model presented here, can lead to better maintainability and lower test development costs.