Pex is a testing tool developed by Microsoft Research. It provides three very useful capabilities to support testing. First, it can explore your code and suggest which tests you should have. Second, if you have parameterized tests, Pex can figure out which combination of parameters needs be tested in order to give you full coverage of possible scenarios. And finally, if you use Code Contracts, then Pex will employ that information to fine-tune the unit tests it suggests or generates for you. I'll review the three aspects in more detail after a quick look at installation and set up.
You can download Pex from Microsoft. If you're curious about the underlying technology, you can also access a few PDF documents explaining the internal workings of the tool from the site. Pex is a standard add-in for Visual Studio and should be installed like any other.
Suggesting Unit Tests
Now let's look at what Pex can do. Suppose you are not a TDD proponent; at some point, you may happen to have a C# class and no unit tests for it. With Pex, all you need do is right-click on the class code in Visual Studio and click the Pex|Create Parameterized Unit Tests menu item. The effect is that a new test project is created and added to the selected solution; moreover, the test project can target MSTest the default Visual Studio testing framework as well as other frameworks, including NUnit. Open up the project and you'll find good parameterized tests are there for you to complete and run. However, auto-generated tests don't really do much except invoke a method with parameters. At the very minimum, you'll want to complete them with some assertions.
A parameterized unit test is simply a test that accepts parameters; parameters can be provided in a number of ways depending on the framework. Typically, you pass parameters from a database file. In Visual Studio MSTest, you need to write a wrapper parameter-less test configured to know about the data source. This wrapper test will be invoked iteratively; it reads values from the current row and passes those values to the parameterized test. The challenge is choosing meaningful input values for the tests. This is where the second key feature of Pex comes to the rescue.
Choosing Meaningful Input Values
Open the previously created unit test file and right-click on it. This time, click on the Run Pex Explorations menu item. At this point, Pex analyzes the code of the method under test and figures out which values it would make sense to test for. Pex uses an implementation of the dynamic symbolic execution technique that basically processes the code and calculates the results of the method execution. Every time a new possible result is found, Pex determines which combination of values leads to the result and adds the values to the list. In this way, it iteratively discovers all possible outcomes of a method and, at the same time, it also determines a way to produce them.
The result of a Pex exploration is a list of auto-generated classic unit tests, which call the method with any meaningful combination of inputs previously determined. You can just copy these tests into a test project and integrate them in the build process with the guarantee of a high coverage and, more importantly, strong insurance against corner cases. Generating parameterized unit tests is often a long and boring process Pex automates that process and reduces it to just a click or two.
Code Contracts and Pex
Fully integrated in .NET Framework 4.0, Code Contracts are Microsoft's implementation of the tried-and-true idea of software contracts most notably, preconditions, postconditions, and invariants. In particular, Code Contracts are used to annotate methods with conditions that must be preliminarily verified for the code to run. As an example of Code Contracts and Pex integration, consider the following scenario. Suppose you have a method that does some work on a string parameter. Suppose also that your implementation blindly assumes the received string is a non-null string. You are therefore potentially exposed to a null reference exception. A quick exploration by Pex reveals the issue, in the form of a notification, which allows you to add a test.
A warning about a possible defect is a great help, but Pex can do more. If Code Contracts are enabled for the project, then Pex will offer to add a proper precondition to your code so that you can catch any invalid data and take control of the situation. Pex will run an analysis of the code and build a base of knowledge about your code. Code Contracts provide additional information that contributes to an even more accurate analysis from Pex.
In summary, Pex is an innovative white-box testing tool that can be used both as an aid to generate nontrivial unit tests and as a peer reviewer to quickly look at your code and find holes and omissions in it. Pex is not dependent on Code Contracts, but it can use Code Contracts to enhance your tests if they are already employed in your code. One limitation I should mention is that it works only with managed code.
Pex is an innovative testing tool that represented a deeper commitment to unit testing than Microsoft had previously exhibited. Since it was released, Microsoft has continued bringing out tools that facilitate unit testing. In Visual Studio 2012, for example, it shipped a framework called Microsoft Fakes. Fakes facilitates the construction of unit tests with stubs and mocks, plus a runtime redirection of code in what the company calls "shims." These technologies will be discussed in upcoming articles. Meanwhile, if you need help generating humdrum unit tests that exercise code thoroughly, download Pex and have it do most of the heavy lifting.
[For a hands-on look at using Pex, see "Working with Microsoft PEX Framework" Ed.]
Dino Esposito is a frequent writer on Microsoft developer technologies.