Simple Unit Tests in C++

Introduction

Of all the tools available to programmers, I think one of the most important is a good unit-test tool. This is a tool that quickly performs a series of tests on a unit (typically a function or class), to see if it works according to specifications. The concept of unit testing is based on the idea that it is usually best to test a new function or class in isolation before integrating it with other code.

To be truly useful to programmers, a unit test tool should have the following properties:

• It is easy to set up and use.
• It can apply an arbitrary number of test cases to the unit. Here, a test case is a list of input values, with an associated list of expected outputs. Each test case is a separate instance of the same test, not a new kind of test. After applying input values to the unit, the test tool compares the outputs of the unit against the list of expected outputs.
• It provides a simple success/fail indication for each test case. The programmer does not need to study the outputs to see if the unit passed a particular test case.
• It allows programmers to add, modify, or delete test cases without triggering a massive recompile.

I've developed a test framework which I believe meets the above criteria. The test framework consists of C++ template classes, which you must compile along with the test cases and the unit under test. This means of course that you must recompile every time you add or change a test case, so the framework would seem to violate the last criterion above. However, when you add a test case to a test, you need only recompile a single .cpp file and relink. You don't have to recompile the whole project. There are certain advantages to conducting tests such as this in native code, as opposed to the popular alternative of scripting. Given these advantages, which I discuss later, I think that compiling a single file is a small price to pay.

In this article I first show a sample use of the framework, then I explain how it works. This approach may seem backwards from what you're used to in technical articles. But since ease of use is critical in a testing tool, I want to show you what it's like to use it before dragging you through the implementation.

A Sample Test Problem

Suppose you have written a Triangle class to be used in a graphics program:

class Triangle
{
public:
   Triangle()
      : length1(1.0), length2(1.0), 
      length3(1.0) {}
   bool
   setLengths(float l1,float l2,float l3);
   ...
private:
   float length1; // lengths of sides 1, 2, and 3
   float length2; // clockwise around
   float length3; // the triangle
};

The default constructor creates an equilateral triangle with sides of length 1.0. The function setLengths inputs three lengths and sets the sides of the triangle in clockwise fashion. That ought to be simple enough — just copy the inputs to the members length1, length2, and length3. But not so fast. A devious or misguided caller could pass in lengths that were not valid for any triangle — for example, (1.0, 3.0, 1.0). If this happens, setLengths is supposed to return false and not touch Triangle's internal data members. If passed a set of valid lengths, setLengths is supposed to modify the members and return true [1]. To be sure that setLengths works correctly, you want to throw various sets of lengths at it to see if it properly discriminates valid input from invalid.

To conduct these tests using my framework, you would take the following steps:

Step 1

In a file named testtriangle.h, create a derived class as shown below:

class TestSetLengths :
   public BaseTest31<float, float, 
              float, bool>
{
public:
   TestSetLengths();
   void
   apply(const float &l1,
      const float &l2,
      const float &l3, bool &result)
   {
      Triangle triangle;
      result =
        triangle.setLengths(l1, l2, l3);
   }
   const char *getName() const
   {
      return "Test setLengths";
   }
};

Class TestSetLengths derives from a mysterious base class BaseTest31<float, float, float, bool>. I won't say much about this base class for now, except that it is a specialization of the template class BaseTest31, which is provided by the framework.

TestSetLengths contains three member functions. First, there is the default constructor. It must be declared, but not defined, in the body of TestSetLengths. For reasons that will become clear later, you must implement this constructor in testtriangle.cpp.

apply is a member function declared as pure virtual in BaseTest31; therefore, it must be implemented by class TestSetLengths, with the exact signature shown. Member function apply will be called by the test framework to apply a single test case against the unit under test. The test framework will provide the input arguments to l1, l2, and l3, and it will read the output argument result.

Note that the template arguments in BaseTest31 match the types in the signature of apply. That's not a coincidence, it's a requirement. Also in this example, the types in the signature of apply match the inputs and the output, respectively, of function setLengths. This is not a requirement of the framework. The types match in this example because apply acts as a simple forwarding function to setLengths. In more elaborate cases, apply might have to do more setup work — for example, to call a method with many parameters, or to call more than one method on an object. apply serves as an interface between the framework and the unit under test.

In this example, I implement apply in the body of class TestSetLengths for the sake of illustration. It could just as easily be implemented in the .cpp file.

getName is another function declared as pure virtual in BaseTest31; hence it must also be implemented by class TestSetLengths. It returns the name of the test, for reporting purposes. Again, I've implemented it within the body of TestSetLengths for the sake of illustration.

Step 2

In the file testtriangle.cpp, implement the default constructor for class TestSetLengths:

TestSetLengths::TestSetLengths() 
{
// oblique
   addCase(3.0, 2.0, 4.0, true); 
// right triangle
   addCase(3.0, 4.0, 5.0, true); 
// acute isosceles
   addCase(10.0, 100.0, 100.0, true); 
// big equilateral
   addCase(100.0, 100.0, 100.0, true); 
// another right triangle
   addCase(6.0, 10.0, 8.0, true); 
// another oblique
   addCase(3.0, 4.0, 2.0, true); 
   addCase(1.0, 2.0, 4.0, false); // bad
   addCase(1.0, 3.0, 1.0, false); // bad
// very thin triangle
   addCase(2.0, 1.0, 1.0, true); 
}

The function addCase used in the constructor is inherited from the base class BaseTest31. Each call to addCase adds a test case to the base class. The first three arguments are the input values to be passed to apply, and indirectly, to the function setLengths. The last argument is the output you expect apply to receive from the output of setLengths. Thus, you should pass true as the last argument if the first three arguments represent the sides of a valid triangle; pass false otherwise. These values are stored in STL vectors within the base class.

Step 3

In main.cpp, instantiate an object of class TestSetLengths, and call its run member function, which is also inherited from BaseTest31:

int main()
{

   // create a file for test report
   std::fstream fout;
   fout.open("testout.txt", 
      std::ios::out);

   // run the test
   TestSetLengths sltest;
   sltest.run(0, 100, fout);

   fout.close();

   return 0;
}

That's it. Running this program results in the output file shown in Figure 1.

Each line of the output file describes the results of a test case. The first part of the line shows the name of the test, followed by the number of the test case, in parentheses (zero-based). If the test case fails, the report shows the output value(s) that were expected and what the unit under test actually returned. In the last test case added, I had specified an infinitesimally thin triangle (2.0, 1.0, 1.0), and expected setLengths to call it valid. As the report shows, setLengths did not work as I expected.

In the code shown above, the first two arguments to run specify the range of test cases you want to run. If you specify a range that completely overlaps all the available cases, then the framework just runs all of them. This range feature is convenient for debugging. If you examine the report file and determine that a particular test case fails, you can call run for just that case, and step through the test with a single-step debugger.

The last argument to run is an already opened fstream object. run will write its report to this file. This arrangement enables you to "stack" any number of different unit tests. Each test appends its report to the open file. When you are done testing, you close the file.

Adding Test Cases

To add a test case to the test, just add a call to addCase in the body of the TestSetLengths constructor. Compile the .cpp file and relink. You're done.

How It Works

The test framework consists of a family of template classes named BaseTestXY, where X is a numeral indicating the number of input parameters, and Y is a numeral indicating number of outputs. Figure 2 shows the definition of template class BaseTest32. This template class is designed to run tests that take three input parameters and return two output parameters.

The BaseTestXY template classes serve as repositories of test case values. These values are stored in STL vectors, which appear as data members i1s through o2s in Figure 2. The template classes also supply a function run, which applies the test case values against the unit under test, compares the results with expected outputs, and writes a report.

The workhorse of the template classes is the member function run. It is shown in Figure 3. This function iterates through the range of requested test case values, and passes the input values as the first three parameters to the function apply. BaseTest32 does not implement function apply; it expects a derived class to do it. Thus, the run function calls apply polymorphically — it calls the implementation supplied by the derived class. The derived class apply calls the unit under test, and returns its output values through the last two parameters. Function run compares these two values against the expected values. If they match, it reports success; otherwise it reports failure and terminates execution.

To create a specific test, you derive a class from one of these template classes (more precisely, from a specialization of one of these classes). Pick the template that matches the number of input and output parameters you want to pass to apply. For example, if you were going to use two inputs and three outputs, you would derive from the template BaseTest23. When you derive from the template, you specialize it with the template arguments that match the input and output types you want to use with apply. You then implement the pure virtual functions apply and getName. You also implement the default constructor of the derived class to add test cases to the repository, by calling the base class function addCase. Since this constructor is implemented in a .cpp file, its definition is not visible to other translation units. It can be modified and recompiled without forcing a recompile of other files.

In a nutshell, the base class provides the functionality of running the test cases and writing reports; the derived class provides the functionality of calling the unit under test. This arrangement helps keep programmer effort to a minimum.

Conclusion

The framework presented here constitutes a fairly simple test tool. It consists of nine template classes: BaseTest11, BaseTest12, BaseTest13, ..., BaseTest33. I have tested this framework under both Visual C++ 6.0 and Borland C++Builder 4.0. The framework, and the source files for the sample test, are available from this month's online archives (www.cuj.com/code).

If want to test functions with more than three input or output parameters, you'll have to play some special tricks when you implement the function apply. Note that you could conceivably pass vector arguments to apply, and translate them to the parameters required by the unit under test. Alternatively, you can pass string arguments that combine the required parameters as a single string. This can get a little messy, of course.

As I hinted at earlier, an alternative way to add test cases without lengthy recompilation is to use scripting. In scripting, the programmer writes test cases in an interpreted language, such as TCL or Python; alternatively, she specifies test cases in a text file, which is more or less "interpreted" by the test program. The nice thing about scripting is that you can add or change test cases and just execute the test code on the fly.

This framework does not use scripting; you write your test cases directly in C++. This approach has two key advantages. First, you can step through any test, and the code being tested, with a single-step debugger. This may be difficult or impossible with scripting. Second, you don't have to maintain a separate file of script code or text. It's a lot easier, using modern IDEs and makefiles, to integrate test code with a project if it's written in the native language.

The framework presented here actually evolved as I wrote this article. As so often happens, in the course of describing my solution, I came up with a simpler one. I wouldn't be surprised if there's even a simpler way to conduct unit tests than I presented here. If you know of one, or just have an improvement on the technique presented here, I'd love to hear about it.

Note

[1] In a real implementation, I would probably have setLengths call a utility function isTriangle to check for valid inputs, and throw an exception if they were invalid.

Marc Briand is Editor-in-Chief of C/C++ Users Journal. He loves programming, writing, and too many other things for his own good. However, he hates to work, which is why he is an editor. He may be reached at [email protected].