When choosing the server architecture, make simple test cases for your performance requirements before you commit to a platform. Make time for these test cases in any project plan you have for evaluating vendors. With adequate documentation available, it should take approximately one day for a developer to evaluate each of these requirements. These tests have several important results, including:

• Scalability of platform.
• Sizing of production hardware.

• Familiarity with the platform.

An important detail that often gets overlooked is the amount of data in the system being tested. On one web application project I helped develop, the administrator logged into a homepage that retrieved the entire set of users in the system. Each user was faulted into memory using an object-to-relational mapping layer, and several properties were used off each object. This approach worked on developer machines with limited test data. Unfortunately, the client intended to scale the number of users by 2 or 3 orders of magnitude from the test data, and this led to severe application bottlenecks. This required a significant amount of developer effort to fix, and a redesign for this module was necessary. On a later project, the application was developed with a full dataset (tens of thousands of objects) from the beginning. This application had a global maximum load time for each page as a key business requirement.

JMeter Concepts

JMeter is organized with a tree metaphor. This tree is visible on the left side of the screen and property sheets for each tree element appear on the right. JMeter is configured with a Test Plan, which is the root element of the tree. Test Plans are containers that store everything else that describes your test. JMeter also contains a WorkBench, which is for temporary storage of test elements and isn't needed for operation. Test Plans can contain multiple Thread Groups, and Thread Groups can contain:

• Timers.
• Listeners.

• Controllers.

• Configuration Elements.

JMeter can also assign multiple threads to your tests. The default number of threads for a Thread Group is 1, but this is easy to change. Use multiple Thread Groups in your test plan to simulate multiple users.

Timers are used to determine how much load to put on your application. They control how often JMeter sends a request to your application. If you want to emulate 10 users that each click a new page every three seconds, set the delay on the timer to 300 milliseconds. JMeter includes one constant delay timer and two timers that offset the delay by a random deviation.

Listeners are the output handlers for JMeter. JMeter outputs the amount of time, in milliseconds, that it takes for a web server to respond to the JMeter request. JMeter should have at least one listener defined, possibly more. There are four listeners in JMeter:

• File Reporter: logs output to a user-defined file. Be sure to open the file from the configuration panel before running your test and close the file when you are finished.
• View Results: displays the web page requested from the server as HTML source.

• Graph Results: displays each response time as a dot on an x-y graph. JMeter also tracks the average and the deviation.

• Spline Visualizer: generates a smooth curve of the results over time.

Controllers can be split into two groups — protocols and logic. Protocol controllers are Web Testing, Database Testing, and FTP Testing. These controllers are configured with user names, passwords, URLs, and so on. The HTTP and JDBC controllers are the most useful. However, the FTP controller is probably most useful if you are writing an FTP server.

Logic controllers are the Once Only controller and the Interleave controller. The Once Only controller can be used if a given page needs to be initialized, but not afterwards. The Interleave controller is for simulating multiple user names and passwords with one thread group.

Configuration elements let you set up controllers with additional information. Each controller can be set up with multiple test cases, which are called "samples" in JMeter. These samples include the FTP Sample, the Url Sample, and the Sql Query. If you are running just one test case in a test plan, it's easier to configure the controllers themselves and not use configuration elements.

The Web Testing controller also makes use of the Cookie Manager and HTTP Authorization configuration elements. The Cookie Manager stores cookies that you can set up. This mimics the cookie functionality of a web browser. HTTP Authorization stores user names and passwords for base URLs that require a login. This authentication is done at the HTTP protocol level, which is why there is a separate configuration element.

The Database Testing controller uses the Database Login and Database Connection Pool configuration elements. The Database Login can also be configured on the controller. Database Connection Pool is used for sharing connections between tests. This could be useful if you have limited client licenses for your database, for example.

Using JMeter

JMeter is open-source software and can be freely downloaded at http://jakarta .apache.org/jmeter/. At this writing, the most recent release of JMeter was Version 1.6. JMeter can also use three optional libraries — a SAX XML parser, the JavaMail API, and JSSE — which enables SSL support for testing web applications that use HTTPS.

JMeter includes batch and shell scripts that make running the application simple. Install a Java 1.2 or 1.3 run-time environment, if you haven't already done so. Run the jmeter.bat (for Windows) or jmeter shell script (for UNIX) in the /bin subdirectory.

To demonstrate how you can use JMeter to test and improve performance, I will use two Java servlets that run under Tomcat (another open-source program from the Jakarta Apache project; http://jakarta.apache.org/site/binindex.html). The first servlet prints all the numbers from 0 to 511 on an HTML page, while the second servlet does exactly the same thing — only faster.

Tomcat needs two environment values set before it can be run. Set JAVA_HOME to the base directory of your JDK (for example, c:\jdk1.3.0). Set TOMCAT_HOME to the base directory of Tomcat (for example, c:\tomcat3.2.1). Then create a subdirectory called "jmeter" under the /webapps subdirectory of Tomcat. Here, I will install the servlet as its own web application. Create a subdirectory called "WEB-INF" under jmeter, then another subdirectory called "classes" under WEB-INF. Copy SlowJMeterExample.java and FastJMeterExample.java into the classes subdirectory, and compile them. Make sure your servlet.jar Java library is in your class path. Copy web.xml into the WEB-INF subdirectory; see Listings One and Two. (An alternative to this is to use the JMeterExample.war archive, which is available electronically; see "Resource Center," page 5.)

JMeter releases before 1.6 final are unable to change the port number that JMeter uses for HTTP testing. To remedy this, you will need to run Tomcat on port 80, the normal HTTP server port. By default, Tomcat uses port 8080, and this is configured in the <tomcat-dir>/conf/server.xml file. Do a text search for "8080." On my release of Tomcat, this was line 225, and was under an XML comment that said "Normal HTTP." Change the "8080" to "80." For UNIX users, this means you will have to run Tomcat as root, as normal users on UNIX can have daemons that listen to port numbers less than 1024.

To launch Tomcat with the servlet, run startup.bat (for Windows) or startup.sh (for UNIX) in the /bin subdirectory of Tomcat. Load the servlet in your web browser with the URL http://localhost/ jmeter/SlowExample to be sure it installed properly.

Performance Problems

The easiest way to see how JMeter works is to use a real web application that has been coded in two different ways. I call these "Slow" and "Fast." They both do the same thing, but the implementation is different. I start by using JMeter to test the slow example.

1. Start JMeter from the command line. JMeter displays in a new window; see Figure 1.

2. Right-click the Test Plan element in the tree on the left side and select the Add submenu from the menu that appears. Select Thread Group from that submenu.

3. The Thread Group is nested under the Test Plan, but won't be visible until the expansion icon to the left of Test Plan is clicked.

4. Select the Thread Group element and a Panel appears on the right side of the display. Change the Name to SlowExample. The number of threads doesn't need to be changed.

5. Add a Timer to the SlowExample thread group. Right-click on SlowExample, select the Add submenu, then the Timer submenu, and finally, the Constant Timer.

6. Expand the SlowExample element and click on the Constant Timer. The constant delay should be 300, which is in milliseconds. Change this number to 100, to put more of a load on the application being tested; see Figure 2.

7. Right-click on SlowExample again and add two Listeners. You are going to use a View Results Listener and a Graph Results Listener.

8. The View Results Listener does not need to be configured. The Graph Results Listener has a name, which you can change to SlowGraph. These two panels will contain results data once the test plan is run.

9. Add a Web Testing Controller to SlowExample. Select the Web Testing Controller and change the domain to localhost (or whichever machine is running Tomcat). Set the path to /jmeter/SlowExample, and the method to Get. If these are not set properly, JMeter may complain that the start function is not implemented.

10. To run the first test, select the SlowGraph Graph Results Listener from the tree on the left-hand side, and start the test script with the Start command in the Run menu.

11. The first result you get on the Graph Results Listener will probably be slow. This is the time it takes for JMeter to compile the script. It is best to start JMeter, run the script for a few seconds, and then stop JMeter with the Stop command in the Run menu. Restart the test script after clearing the results with the Clear command, also in the Run menu.

12. The View Results Listener will show you the HTML code returned from the server for the request. This is mostly useful for debugging purposes.

13. Stop the test with the Stop command on the Run menu. On my P2-300 machine, the servlet took an average of 31 milliseconds to execute; see Figure 3.

Now improve the servlet's performance. This servlet uses some poor Java programming techniques. The use of the concat() method on String classes should be avoided in Java because the String class is immutable, which means a new string has to be constructed in memory. Instead, use StringBuffer classes, with the append() method. This is the recommended way of adding two strings together. If you're not a Java programmer, the details of this performance improvement isn't actually important; what is important is that I will increase the speed of the servlet in the loop. I'll also append the loop counter, ctr, as an integer, without turning it into a string as an intermediate step; see Listing Three.

So that you can get a text file of the data, add a File Reporter to the Thread Group:

14. Clear the testing results for SlowExample (Run->Clear).

15. Select SlowExample and change the name to FastExample.

16. Select SlowGraph and change the name to FastGraph.

17. Select Web Testing and change the path to /jmeter/FastExample.

18. Add a File Reporter to FastExample. Right-click FastExample, select the Add submenu, and under the submenu Listeners, choose File Reporter.

19. Select the File Reporter and set the output file to fast.log. Make sure the Append and Verbose Output checkboxes are checked. Click the Open button; see Figure 4.

20. Select FastGraph, and then run the test case. Be sure to clear the results out once the first few hits have occurred.

21. When you're finished with the test, select the File Reporter and close the file, so it gets written to disk and the logging stops; see Listing Four.

On my machine, the new Fast servlet took an average of four milliseconds to run — quite an improvement.

Interpreting Results

Keep a spreadsheet with response times, dates, machine names, and pages tested, along with previous logs of data from the File Reporter. This lets you analyze historical data; if the response time for a page jumped a month ago, check the source control system to see what changes were made to the code.

If your application is backed by a database, graph response times versus the amount of data in the system. This is easy to do with a spreadsheet. If your response times go up in a linear trend, your database doesn't have the proper indexes or your algorithm may need to be rewritten.

Other Uses of JMeter

JMeter can also be configured for FTP testing and JDBC testing. JDBC testing can be extremely useful when determining how long complex queries take to execute. The JDBC-related elements are the Database Testing controller, SQL Query configuration element, Database Login configuration element, and Database Connection Pool configuration element. The Database Testing and SQL Query elements need to be configured for any database tests. The results appear in the same listeners as HTTP testing.

JMeter can also use custom classes for listeners, controllers, timers, and configuration elements. Generally, the easiest way to get your functionality is to extend existing classes with your new code — that way you start from a code base that works. If you want to create your classes from scratch, it's as easy as implementing the Java interfaces provided with JMeter. There is an interface provided for each element type, an interface for the GUI, and an interface for saving configuration information to an XML file. To make sure JMeter can see your classes and load them, add your classes to the search_paths class path in the jmeter.properties file.

JMeter can also be used to test multiple pages on a site at once. Add another Web Testing controller to your existing Test Plan, and set the domain and path to your second page. JMeter will alternate between the two pages. If you have a login page, it is possible to tell JMeter to only hit that page once by adding a Once Only controller, and then adding the Web Testing controller for the login page to the Once Only element. Unfortunately, JMeter does not provide any kind of real automation or scripting for testing complex web pages as of JMeter 1.6.

DDJ

Listing One

import java.io.IOException;
import java.io.PrintWriter;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

/** A slow servlet for the JMeter test article
 * Uses String concatenation to get a slow servlet.
 * @author Jeff Linwood
 */

public class SlowJMeterExample extends HttpServlet {
  public void doGet(HttpServletRequest request, HttpServletResponse response)
        throws IOException, ServletException
  {
                //Tell the browser we're sending it html
        response.setContentType("text/html");
                String myhtml = new String("");
                String addon = new String("<html>");
                myhtml = myhtml.concat(addon);
                addon = "<head>";
                myhtml = myhtml.concat(addon);
                addon = "<title>";
                myhtml = myhtml.concat(addon);
                addon = "Slow Example for JMeter";
                myhtml = myhtml.concat(addon);
                addon = "</title>";
                myhtml = myhtml.concat(addon);
                addon = "</head>";
                myhtml = myhtml.concat(addon);
                addon = "<body><h1>";
                myhtml = myhtml.concat(addon);
                myhtml = myhtml.concat("JMeter Slow Example Test");
                myhtml = myhtml.concat("</H1>");
                for (int ctr = 0; ctr < 512; ctr++) {
            myhtml = myhtml.concat(String.valueOf(ctr));
            myhtml = myhtml.concat(" ");
                }
        myhtml = myhtml.concat("</body>");
        myhtml = myhtml.concat("</html>");
        PrintWriter out = response.getWriter();
        out.println(myhtml);
    }
}

Back to Article

Listing Two

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE web-app
    PUBLIC "-//Sun Microsystems, Inc.//DTD Web Application 2.2//EN"
    "http://java.sun.com/j2ee/dtds/web-app_2_2.dtd">
<web-app>
    <servlet>
      <servlet-name>SlowExample</servlet-name>
      <servlet-class>SlowJMeterExample</servlet-class>
    </servlet>

    <servlet>
      <servlet-name>FastExample</servlet-name>
      <servlet-class>FastJMeterExample</servlet-class>
    </servlet>
    
    <servlet-mapping>
      <servlet-name>SlowExample</servlet-name>
      <url-pattern>/SlowExample</url-pattern>
    </servlet-mapping>

    <servlet-mapping>
      <servlet-name>FastExample</servlet-name>
      <url-pattern>/FastExample</url-pattern>
    </servlet-mapping>
</web-app>

Back to Article

Listing Three

import java.io.IOException;
import java.io.PrintWriter;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

/** A fast servlet for the JMeter test article
 * Uses StringBuffer appends to be faster than the Slow servlet.
 * @author Jeff Linwood
 */

public class FastJMeterExample extends HttpServlet {
   public void doGet(HttpServletRequest request, HttpServletResponse response)
        throws IOException, ServletException
    {
                //Tell the browser we're sending it html
        response.setContentType("text/html");
                StringBuffer myhtml = new StringBuffer();
                String addon = new String("<html>");
                myhtml.append(addon);
                addon = "<head>";
                myhtml.append(addon);
                addon = "<title>";
                myhtml.append(addon);
                addon = "Fast Example for JMeter";
                myhtml.append(addon);
                addon = "</title>";
                myhtml.append(addon);
                addon = "</head>";
                myhtml.append(addon);
                addon = "<body><h1>";
                myhtml.append(addon);
                myhtml.append("JMeter Fast Example Test");
                myhtml.append("</H1>");
                for (int ctr = 0; ctr < 512; ctr++) {
            myhtml.append(ctr);
            myhtml.append(" ");
                }
        myhtml.append("</body>");
        myhtml.append("</html>");
        PrintWriter out = response.getWriter();
        out.println(myhtml.toString());
    }
}

Back to Article

Listing Four

# Sample data created by null
#       URL                    Milliseconds
http://localhost:80/jmeter/FastExample  0
http://localhost:80/jmeter/FastExample  10
http://localhost:80/jmeter/FastExample  0
http://localhost:80/jmeter/FastExample  10
http://localhost:80/jmeter/FastExample  0
http://localhost:80/jmeter/FastExample  0
http://localhost:80/jmeter/FastExample  10

Back to Article

Figure 1: JMeter on startup.

Figure 2: Slow example: changing the Constant Timer.

Figure 3: Slow example: Graph results.

Figure 4: Fast example: File Reporter after open.

Terms of Service | Privacy Statement | Copyright © 2024 UBM Tech, All rights reserved.