Channels ▼
RSS

JVM Languages

Continuous Integration and Performance Testing

Source Code Accompanies This Article. Download It Now.


Performance, Integration, And Load Tests

Now you can add performance tests into the continuous integration process. The most straightforward method is to create a new virtual project in CruiseControl, one that uses the same build script but executes a performance unit-testing target. The following shows the key changes you could make to a copy of Listing One's project definition:








 
<project name="ant-junit-performance-unit-tests">
 ...
  <schedule interval="600">
    <ant anthome="apache-ant-1.6.5"
       buildfile="projects/${project.name}/build.xml"
           target="performance-unit-tests.execute" />
  </schedule>
  <publishers>
   <onsuccess>
      <artifactspublisher dest="artifacts/${project.name}"
         dir="projects/ant-junit/profiling-reports"/>
    </onsuccess>
  ...

In this case, CruiseControl is configured to check for source updates every five minutes—because performance tests take longer to run, they cannot and should not be run as frequently. In a real-world application, checking hourly is more reasonable.

But most importantly, a new Ant task, performance-unit-tests.execute, is run, and the results are saved to a profiling-reports directory. The actual definition of this Ant task and the performance profiling process is entirely dependent on the analysis tool used for this. Besides commercial code analysis tools, some promising looking open-source alternatives are emerging, such as EMMA (emma.sourceforge.net).

Similarly, you can perform integration and load testing in this continuous integration environment. The freely available Apache JMeter is my choice for a simple load generator for our web-based application. It's simple enough to integrate JMeter with Ant using an Ant task created by Programmer Planet (programmerplanet.org/media/ant-jmeter/ant-jmeter.jar).

Once integrated, you define a JMeter Ant task by adding the following to the Ant script:


<taskdef name="jmeter"
classname="org.programmerplanet.ant.taskdefs.jmeter.JMeterTask"/>

JMeter test plans are written in XML and they specify the details of the load test. These test plans are called by the JMeter Ant task, which outputs the results to a log file. JMeter test plans can define such metrics as the number of virtual users (threads) and test iterations (loops). You can configure the JMeter Ant task to run different test plans to test basic integration and also to perform full load testing.

Although JMeter outputs a comma-separated values (CSV) file by default, you can change this to XML using the jmeter.save.saveservice.output_format=xml argument. This lets us programmatically determine success or failure, and produce a human-readable report using an XML style sheet (XSL). This report can be evaluated against the application's service-level agreements to give an idea, during development, of whether it meets its scalability requirements. It is possible to further integrate cross-JVM performance analysis tools to help diagnose scalability issues.

Conclusion

Continuous integration is becoming increasingly popular as teams see the benefits of automating more aspects of the development process. Extending continuous integration to different types of performance testing makes the investment in continuous integration even more worthwhile.


Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.
 

Video