Channels ▼


Change Code Without Fear

Source Code Accompanies This Article. Download It Now.

Establishing a Supporting Infrastructure

Running automated and regularly scheduled build and testing processes should involve minimal distraction if set up properly and if all required infrastructure is present. At minimum, that infrastructure should include a source-control system, a tool for automated test execution, and a reporting mechanism to track results of the automated build and testing.

Most development teams use a source-control system to store code they are working on. Some teams store their tests in the source-control system, and others leave testing to the discretion of the individual developers. In some cases, developers are testing the code on their own machines, but the tests never make it into the source-control system. In other cases, they do some testing if time permits and don't do any when in a crunch. In either case, the value of any tests that exist on an individual developer's machine is minimal and such tests are typically used only to verify that the code is functioning properly at the time it is written. Once stored into the source-control system, the test can be leveraged over and over to validate that the new code didn't break the existing functionality or introduce problems that could impact reliability or functionality.

The regression suite generation/execution should be scheduled to run regularly (for example, nightly, or several times a day in a continuous integration process) in whatever manner makes the most sense for your specific development environment and team. This could be a combination of an Ant script and Windows scheduler or CruiseControl, or a shell script and a crontab on UNIX. This process then regularly scans the project and generates more test cases for new and updated code, and executes all tests that comprise the regression suite, then reports all failures and cumulative coverage information. This report can be e-mailed to individual developers, or they can import it directly into their IDE—effectively triggering their review of the failures and the new/changed code. This cycle is then repeated every time the regression process is run.

Building a Behavioral Regression Test Suite

To demonstrate this process, I use Parasoft JTest to create a Behavioral Regression Test Suite for the JPetStore 4.0 project from iBatis ( group_id=60632). This is a simple web application that lets you purchase pets online.

First, automatically generate an initial regression suite for the JPetStore project. This one-time process scans the project and generates a test class for each Java file in the project. In the JPetStore example, 40 new test classes containing 702 test cases were generated. The tool proceeded to execute only these automatically generated test cases because the regression test suite did not include any manually written test cases. It reported coverage of 91 percent of the source code:

[exec] Executed Test Cases: 702
[exec] Runtime Exceptions: 0
[exec] Assertion Failures: 0/0 verified
[exec] Contract Violations: 0
[exec] Profiling Problems: 0
[exec] Unverified Outcomes: 0
[exec] Coverage:
[exec]    Line: 91% [994/1090 executable lines]

Next, set up the automated testing tool to run using Ant. The appropriate Ant properties are configured to point to the tool installation directory and the Eclipse workspace (which is where the JPetStore project is located); for example:

<target name="all" >
 <echo message="Testing 'JPetStore'" />
 <exec dir="." executable="${testtoolcli}" >
    <arg line="-data ${workspace} ${config} -resource 'JPetStore' 
   -report '${workspace}/JPetStore/Report.xml' -publish
   -localsettings '${workspace}/JPetStore/'"/>

Related Reading

More Insights

Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.