Channels ▼
RSS

JVM Languages

Project of the Month: TestNG


Data Providers

Another common aspect of functional tests is that they often need to work on large amounts of data.

For example, let's assume you just implemented a geolocation service that accepts latitude and longitude, then returns a nice description of that location:

geoLocate(37.47, -122.12) -> "Palo Alto, CA"
geoLocate(48.85, 2.35) -> "Paris, France"
geoLocate(51.50, -0.12) -> "London, UK"

The code to test this logic will be very simple, but it will work on a lot of data. In this particular example, what we're testing is really the data, not the logic.

In TestNG, a data provider is identified by the @DataProvider annotation, and its function is merely to provide the data to a test method. Here is how we would implement a test for the aforementioned code:

@DataProvider
public Object[][] locations() {
  return new Object[][] {
    new Object[] { 37.47, -122.12, "Palo Alto, CA" },
    new Object[] { 48.85, 2.35, "Paris, France" },
    new Object[] { 51.50, -0.12, "London, UK" }
  }
}

@Test(dataProvider = "locations")
public void locationsShouldBeCorrect(float latitude, float longitude, String expected) {
  Assert.assertEquals(geoLocate(latitude, longitude), expected);
}

In this example, the @Test method tells TestNG which data provider should provide its parameters. TestNG then invokes the data provider to retrieve these values, and feeds them one by one to the test method.

Being able to specify parameters in test methods is a convenience that greatly simplifies readability and the ease with which you can set up data-provided tests. TestNG also allows you to specify as many of these data provider/test pairs in a class as you want. Several methods can even share data providers, and data providers can be inherited or even combined to form bigger data sets.

Note also that data providers don't need to specify their data in the source code, this data can come from anywhere: property file, XML, Excel spreadsheet, database, JMS publisher, or any kind of network source.

Parallelism

TestNG leverages parallelism in two different ways:

  1. Running your tests in parallel for speed.
  2. Testing that your code is multithread safe.

I will cover only the first way in this article. As mentioned earlier, functional tests can take a very long time to run because they not only have a lot of code to test, but they will test across all kinds of resources: CPU, network, file system, memory, and the like.

Many of these tests are typically independent of each other, and as such, they can be run in parallel. For example, there is very little reason why your servlet tests should wait for the database tests to finish before they can run. With the increasing popularity of multicore machines, it's not rare to see even laptops feature several cores, which means that two tasks running in parallel on these cores will run faster than if they were run sequentially.

TestNG makes it trivial to run your tests in parallel. Because nobody but you can know which tests are safe to run in parallel, TestNG gives you the flexibility to specify the kind of parallelism you want. The parallel modes that TestNG supports are: "methods," "classes," "tests," and "instances." I will only cover the first two to keep this article brief.

"methods" is the simplest mode of all: It just instructs TestNG to throw all your test methods in the same thread pool (which you can configure in many ways (as I'll discuss shortly), and run them as it sees fit.

This is the "maximal speed" mode in which you will probably get the best performance. However, it assumes that all your test methods can be safely run in parallel, which might not hold true for your entire code base.

You can also specify a parallel mode of "classes." In this mode, TestNG will be more conservative and it will run the test methods of each class in the same thread, but each class itself will be running in a separate thread. This is a safer approach that will only fail if some test methods in class A cannot be run concurrently with certain test methods from class B. The other two modes offer increasing granularity.

The real power of TestNG lies in the way you can mix and match these various modes. Without going too much into the details, TestNG uses an XML (or YAML) file to describe your test suites, and you can specify virtually any kind of parallel/non-parallel concurrent configuration with this file. This is also the place where you can configure the size of your thread pools and the time out value.


Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.
 

Video