Simplifying Contract Testing

The Interface Segregation Principle (ISP), which is the "I" in the list of SOLID coding principles, states that a client should not rely on interfaces it does not use. That is, large interfaces should be made small enough that clients actually will use them. ISP therefore implies that an application should have many small interfaces that are combined to create more-complex objects. Contract testing holds that the correctness of an application or component can be assured by producing unit tests, also called isolated object tests, that validate interface implementations to ensure that they do not violate the interface contract. There have been several articles discussing the advantage of this type of testing; however, I can find none that addresses how to test the implementation of the contract when several interfaces are combined in a single object.

Contract tests check the contract that is defined by a Java interface and associated documentation. The interface defines the method signatures, while the documentation often expands upon that definition to specify the behavior the interface is expected to perform. For example, the Map interface defines put(), get(), and remove(). But the human-readable documentation tells the developer that if you put() an object, you must be able to get() it unless the remove() method has been called. That last statement defines two tests in the contract.

Another example is the Apache Jena Graph interface, which has an add(Triple) method. From the interface, it is obvious that this method adds a triple to the graph. What is not clear is that when a triple is added, the graph must report that addition to all the registered listeners. The Graph contract test validates that this action occurs.

A more extreme case is java.io.Serializable, where there are no methods to test but the documentation tells us that all serializable objects must contain only serializable objects or implement three private methods with very specific signatures (only two methods prior to Java 1.4). In addition, all classes derived from Serializable classes are themselves serializable. See the Serializable javadoc for details. (An example contract test for the Serializable interface is provided in the examples for junit-contracts.)

The basic argument for the use of contract tests is that they can help prove code correctness. That is, if every object interface is defined as an interface, and every interface has a contract test that covers all methods and their expected operation, and all objects have tests that mock the objects they call as per the interface definition, then running the entire suite of tests demonstrates that the interconnection between each object works and is correct.

If we know that A calls B properly, and B calls C properly, then we can infer by transitivity that A calls C properly. Thus we can, with some work, prove that the code is correct.

Contract tests will not discover misconfiguration. For example, if class A uses a map and expects a map that can accept null keys, but the configuration specifies a map implementation that does not accept null keys, the contract test will not detect the error. But then, this error is a configuration error caused by a missing requirement for the configuration. Contract tests will also not uncover misuse of methods or classes. In the simple case, if A calls a power function on B instead of an intended multiplication function, the contract test will not discover it. However, the other side of the testing equation — collaboration testing — should catch it.

The Problem

At its most basic level, contract testing says that if you have an interface A there should be a test AT that tests the contracts that the interface prescribes. For example, in the Apache Jena project, there is an interface Model that has a method createResource(). In addition to returning that resource, the implementation must assure that if the getModel() is called on the returned resource, the model that created the resource is returned. The Model contract test would perform that test.

Because A is an interface, AT must be able to test any instance of A; thus, it must be either be a class with an abstract method that gets the implementation of A under test or a concrete class with a setter that sets the implementation of A under test.

For the simple solution, assume AImpl is the concrete implementation of A under test, and ATImpl is the concrete implementation of AT. ATImpl is a fairly simple class that extends AT and implements the abstract getter or setter. So far, things are clear. However, unlike classes, multiple interfaces may be implemented by a class or extended by another interface. This leads us to the case where multiple abstract tests must be combined to create a complete contract test.

Assume that interfaces A and B are defined and that interface C extends them. Each has abstract tests: AT, BT, and CT, respectively. As we have seen, AT and BT are simple and fairly straightforward to write. The problem is with CT. Because CT is an abstract class, it can only derive from one base class, not both AT and BT as is required for complete testing. In addition, in keeping with DRY principles, we don't want to reimplement AT or BT, particularly since a change in A or B would not necessarily be picked up by the embedded implementation of AT or BT. The problem becomes more complex when we consider that C may be an interface published as part of a library or utility project where integrators are expected to implement or extend that interface as represented by D in Figure 1. In this instance, DT should not have to be recoded when A or B or C change. The impact of changes to the interfaces higher up the tree should be limited to only the associated test class. The problem then becomes how to implement a test suite where, given a starting class, multiple abstract tests can be discovered and included.

Figure 1. The relationship between multiple interfaces to a project class.

The Solution

The solution I propose is to create a framework that allows the standard JUnit test engine to discover and aggregate multiple test implementations into a single test suite. This framework introduces three new annotations and a class: @Contract, @Contract.Inject, @ContractImpl, and the class ContractSuite:

• @Contract is applied to a test class and specifies that the class is a contract test for another class. So, in our example, the class AT would have the annotation @Contract(A.class). 
• The @Contract.Inject annotation denotes a getter and setter methods for the class under test.
• The @ContractImpl annotation is applied to a concrete test class implementation. This identifies the class under test. In our example, CTImpl would have the annotation @ContractImpl(CImpl.class).
• The @ContractTest annotation is replaces the standard JUnit @Test annotation to keep JUnit from running contract tests without proper configuration.
• The ContractSuite class is used with the standard JUnit @RunWith annotation to indicate that the class is defining a suite of contract tests to run. In our example, CTImpl would have the annotation @RunWith(ContractSuite.class).

Examples for @Contract and ContractSuite

The example code presented here is refactored slightly to achieve a couple of goals. I want to define an abstract test (CT) that is the contract test for the interface C. I want a concrete implementation of that class (CImplTest) that executes just the tests defined in CT, and a contract test CImpleContractTest that executes the CImplTest tests as well as the AT and BT tests.

C, extending A and B
public interface C extends A, B {}

// a single abstract CT test -- tests just the C interface (not A or B)
@Contract(C.class)
public abstract class CT<T extends C> {
...
}

// implements the single CT test – 
// Tests just the implementation of interface C
public class CImplTest extends CT<CImpl> {
...
}

// implements the suite of CT tests 
// Tests interface C
// as well as the A and B interfaces
@RunWith(ContractSuite.class)
@ContractImpl(CImpl.class)
public class CImpleContractTest{}