Channels ▼

A UML Testing Framework

April 1999: Design Center: A UML Testing Framework

One of the UML’s primary benefits is that its diagrams help you explain how any system works. In this article, I’ll show you how to use UML diagrams to explain a small testing framework. I’ll illustrate some of the more important UML diagrams and discuss how I chose which framework parts to illustrate.

The system I’ll cover is JUnit, a lightweight testing framework for Java. JUnit was written by well-known object-orientation megastars, Kent Beck and Erich Gamma. It is a simple, well-designed framework. You can download the source code from the World Wide Web via my homepage at You may find the source code useful for this article. If you program in Java, JUnit will improve your programming productivity by writing self-testing code. JUnit comes with documentation on how to use it.

The Packages

First, I’ll look at the packages that make up JUnit, illustrated in Figure 1. I refer to Figure 1 as a package diagram—but the UML does not actually define such a diagram. Strictly speaking, the diagram is a class diagram, but I’m using a set of class diagram constructs in a particular way. The class diagram style is what I call a package diagram.

The package diagram’s principal focus is the system’s packages and their dependencies. The diagram shows that the test package consists of three subsidiary packages: textui, ui, and framework. Each package contains a number of classes. Both textui and ui have a TestRunner class.

Java packages and dependencies map easily to the Java package and the import statement. The essential idea is more wide-ranging, however. A package is any grouping of model constructs; you can package use cases, deployment nodes, and so forth. I find a package of classes the most useful. Dependency, in general, is used to show that a change in one element may cause a change in another. Hence, the diagram indicates that if I change the framework package, then I may need to change the textui and ui packages. However, I can change the textui package and not worry about any other package changing.

In a large system, it is particularly important to understand and control these dependencies. A complex dependency structure means that changes have far-reaching ripple effects. Simplifying the dependency structure reduces these ripples. The package diagram does not do anything on its own, but by helping you visualize the dependencies, it helps you to see what to do.

Figure 1

You’ve probably noticed that I have left many things out of the diagram. I haven’t mentioned a few classes in the JUnit packages. I’ve shown a dependency to java.awt, but left out dependencies to things like and java.util. Is this sloppy diagramming? I don’t think so. You should use diagrams to communicate the most important things. I left out classes and dependencies to most of the Java base libraries that I don’t think are important. I’ve made an exception for awt because that line illustrates those classes that use a GUI. Choosing what is important is difficult, yet it is essential for any designer. All elements of a system are not equal. A good diagram reflects this.

Class Diagram of the Framework Package

Of the three packages in JUnit, the most interesting is the framework package. Figure 2 shows a class diagram of this package. I call this a specification perspective class diagram because I’m looking at the classes from their interface rather than their implementation. I know this because I drew the diagram based on the javadoc files, not on the underlying source code.

Figure 2

I’ll start with test case, which represents a single test. I haven’t shown all the methods, or the whole signature, for the operations on the test case; again I’ve selected the important ones. The run method, inherited from test will run the test case by executing setUp, runTest, and tearDown in sequence. RunTest is the abstract method implemented by a subclass with the actual testing code. Such code will do some things and then use assert and assertEquals to test expressions or values. SetUp and tearDown are defined as blank methods, and you can override them to setUp and tearDown a test fixture.

The diagram explains some of this. It doesn’t describe the role of setUp, tearDown, and runTest, which I’ll explain later in this article. This illustrates a key point in using the UML. Many people think a model is best done using diagrams and dictionary definitions. Case tools support this documentation trend. But I’ve found that understanding a model by dictionary definitions is like trying to learn about a new field by just reading a dictionary of its terms. Prose is essential to explain how things should work. So whenever you are writing documentation, use prose and illustrate it with diagrams. I rarely find the dictionary worth the trouble as a separate document. When you need lists of operations, you should generate them from the source code, like javadoc does.

When you run a test case, the results are stored in a test result. The test result keeps two collections of test failures; these are indicated by the two association lines to the test failure. Each line has a name and an asterisk to show it is a multi-valued association—each test result may have many failures. The failures collection contains details of where assertions failed, indicating a test failure. The errors collection contains details of any unhandled exceptions in the code that would otherwise lead to crashes. Each test failure records both the test and the exception that generated it. Again, it shows the link with an association; in this example, I used a “1” to show that there’s only one test and one throwable in each test failure. I’ve shown that throwable is in a different package by putting it inside a package symbol.

You can collect test cases into test suites. Figure 2 shows the characteristic shape of the Composite pattern described in Design Patterns: Elements of Reusable Object-Oriented Software (Addison Wesley, 1994) by Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides. A test suite can contain tests, which can be either test cases or other test suites. When you ask a test suite to run, it runs all its tests recursively.

Figure 2 explicitly shows that test is a Java interface by using the stereotype «interface» in the class. Statements in «guillemets» like this are called stereotypes. You can use them to show special information that is not defined in the core UML. The realization relationship, a dashed generalization, shows the implementation of the interface by test case and test suite. You could argue that in a specification model, it’s unnecessary to distinguish between subclassing and interface implementation. When you are looking at interfaces, generalization only means that the subtypes have interfaces that conform to the supertype. This way, you can write code to the supertype and any subtype can safely substitute for it. This property works for both subclassing and implementing interfaces in Java. But most Java programmers like to know the difference between interfaces and classes.

You can also show interfaces using the lollipop notation often used by Microsoft. Figure 3 illustrates this notation. It’s more compact if you have many interfaces, but then you can’t say much about the interface.

Figure 3

Using an Interaction Diagram

Now I will show you how a test runs. The best diagram for showing behavior is an interaction diagram. I will use the sequence diagram from Figure 4.

Figure 4 shows how running a test suite works. Each box at the top represents an object in the system. You name an object as “object name : class name.” The class name is optional, so I often use names like “a test runner.” Each object has a dashed lifeline running down the page. When the object is active on the call stack, a thin activation box appears. The activation box is optional, but I found it useful here.

Figure 4

You start a test by getting a test runner to run. I’ve shown this with an incoming message; it doesn’t matter which object sends the message. The test runner creates a new test result. Creating a new object is shown by a message coming into the object box rather than the lifeline. The test runner then tells the suite to run, passing the test result as an argument. The test result acts as a Collecting Parameter, described in Smalltalk Best Practice Patterns (Prentice Hall, 1996) by Kent Beck, for all the tests in the suite. The suite now calls run on all its tests. Figure 4 shows three test cases, one success, one error, and one failure. I’ve just chosen three for this example. As you know, a suite can contain any number of test cases or test suites.

In the successful test run, nothing untoward happens, and the test case returns. With an error, an exception is raised somewhere and the run method handles it. The handler calls add error in the test result. The UML defines a return of a result with a dashed line. It does not define a notation for exceptions. I show an exception with a return marked with a special statement «exception».

With the failure, the test calls the assert method to check a value. You show a call to the same object as an additional activation. If the value is not correct, the assert method throws an AssertionFailed exception that the run method catches. The handler then adds a failure to the test result.

Figure 4 illustrates both the strength and weakness of a sequence diagram. The strength of the interaction is that it illustrates well how a group of classes interact to get something done. However, the diagram does not do well at defining the logic that is used. Nothing in the diagram shows how the conditional logic is defined; the diagram merely shows one possible scenario. There are notations for showing conditional logic, but I find that using them spoils the clarity that is the sequence diagram’s strength.

Figure 4 is about as complex as I like an interaction diagram to get. If there are more objects than this, or more complexity, then I would split the interaction diagram into separate diagrams for each scenario. I could have taken each of the three cases as separate diagrams, but I felt one diagram worked well here.

One of the most difficult things about interaction diagrams is deciding what to show. You can’t show every method call, at least not if your code is well-factored. You can leave self calls out. Again, you have to choose the most important parts to show. The reader then uses the diagram as a guide before perusing the source code.

The great value of an interaction diagram is that it clarifies what is going on when many objects are collaborating. In these cases, reading the source code forces you to constantly jump from method to method. The interaction diagram makes the principal path clearer.

Using These Diagrams in Design

In this example, I’ve concentrated on using diagrams to help explain an existing testing framework that I know was not designed with the UML. You can also use the UML diagrams to help design something new. It is often easier to discuss a design with someone else by sketching a diagram on a whiteboard than by talking code.

I use package diagrams to discuss how you might lay out the packages and dependencies. This is a strong guideline for the whole team. Nobody should add a new inter-package dependency without discussing it with the architect and the other team members.

I use class diagrams to help design the key responsibilities of the classes. The attributes and associations indicate responsibilities for knowing about things. I use operations to suggest responsibilities for actions. Often, I like to add three or four sentences to summarize the responsibilities for a class. I treat the detailed interface and the data structure to be a matter for the developers working on that class. The class diagram represents the general knowledge that the team needs about those classes.

I don’t tend to use interaction diagrams to explore a design. With package and class diagrams, a team can design at the whiteboard, exploring alternatives with a pen and eraser. I find interaction diagrams too awkward for this. I’d rather explore class interactions with CRC cards. Once you’ve worked out a good interaction with CRC cards, you can capture the result with a sketch of an interaction diagram, or go straight to coding. I prefer to code right away, but many developers like to make a sketch to help them remember the result of the discussion.

In any case, I never treat such diagrams as statements of how things must be done. Instead, I treat them as statements of how I currently think things should be done. Developers should feel free to change things as they work on the code. As they get into the gory details, new insights are bound to emerge. If a big change occurs, it is a good idea to run the change past the rest of the team. I usually find a CRC session is the best way to do this quickly.

Keep It Simple

I’ve only showed a small amount of the UML in this article, concentrating on the key parts of the UML. For more details you can consult the UML documentation at, but beware—it is hard going. Try checking out one of the books published on the subject, like The Unified Modeling Language User Guide by Grady Booch, et al. (Addison-Wesley Technology Series, 1998).

However much you get into the UML, don’t forget that it is a communication device. Start with just the minimum notation like I’ve shown here. Keep your diagrams simple, yet expressive. Remember that any good design document should be short enough to read over a cup of coffee. I like to write a number of such documents for each major aspect of the system (and I’m being deliberately vague over what is a major aspect). Such a diagram should be descriptive prose supported by a diagram. The diagram should support the explanation, not the other way around.

Related Reading

More Insights

Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.