Jolt Awards 2014: The Best Testing Tools, June 03, 2014 The best testing tools of the past 12 months
Jolt Productivity Award: StresStimulus v3.5
StresStimulus v3.5 is the kind of product we love to highlight in the Jolt Awards. It's developed by a small company with one product that it focuses on exclusively. In this case, it's a load testing tool, which is intended to deliberately stress systems to surface defects and bottlenecks. Typically, this is done by simulating many concurrent users who appear to be using the system simultaneously and thereby simulating peak load.
StresStimulus makes creating and running such a load stress test a breeze. The tool can be thought of having three parts: test design (you record a test), add load configuration to the test, and obtain results as reports or graphs.
Recording a test using your favorite browser is easy. Once recorded, you can refine the test further by filtering out hosts that are not part of the test. A very interesting feature is the ability to parameterize requests. You can define variables and include these as part of the request and the values can come from a data set, a data generator, or a function. This is a very powerful feature. For a quick run, you can use the built-in random number generator. In cases where you need to drive the test with a carefully crafted data set, you can define a data set and populate the test by importing a CSV file.
Browser caches usually interfere with test runs. And you should start the recording with the cache cleared. However, clearing the global cache may take a long time, and you may not want to clear all cache data. The handy StresStimulus recorder can clear only those resources that are related to the tested websites.
Load configuration involves specifying how many virtual users the run needs to simulate. You can also specify whether all these users run at the same time ("Steady load") or whether their numbers increase in steps ("Step load"). These options give you a pretty good simulation of a real world scenario.
After the setup is complete, run the test. Once the run completes, you have access to multiple perspectives to analyze the data effectively.
You can look at graphs to see how various important metrics (such as number of users) correlate with the pending requests. For example, pending requests going up clearly indicates that the system slowing down. You can choose what metrics to view for easy comparison. A test summary page shows the fastest and the slowest pages, and you can also check out the previous runs.
Modern browsers issue requests in parallel to download pages faster. StresStimulus has good support for simulating this concurrency, so a test run can emulate a real world scenario effectively. You can fine-tune the requests to fire concurrently via the Page Breakdown feature.
Overall, StresStimulus is a very nice tool that makes load testing so easy you will end up doing it more often.