Automated Testing for Web Applications

The author is an engineer for Texas Instruments India Ltd. He can be contacted at [email protected].

Web-user interfaces (WUIs) are now as common as the familiar GUI and command-line interfaces. WUIs can be constructed in many ways, including HTML, Java applets, and plug-ins. Of these, HTML is the most widely adopted approach. Furthermore, HTML can often be combined with Javascript to provide additional functionality. Web search engines such as HotBot, AltaVista, and Infoseek are examples of WUIs based on HTML.

As with GUIs, WUI testing is a critical part of the development process. Consequently, the need to formalize and automate WUI testing is paramount. In this article, I'll present an approach for automated WUI testing that I implemented and deployed for a WUI developed on a commercially available data-management application. This technique is based on HTML, Javascript, and CGI. The implementation environment for this technique was based on Netscape Communicator 4.04 (browser) and Apache 1.2 (server).

Existing Testing Techniques

There are three widely used techniques for testing CGI-based applications:

• Simulating a browser, to test the application logic performed by CGI programs. This is done by developing a tester program that generates an HTTP request (as if it came from a browser) and sends it to the web server. The server in turn invokes the respective CGI program and passes the result back to the tester program. It then checks and compares the output received. The disadvantages to this approach are: The browser's rendering of output HTML documents is not tested; client-side processing (done using Javascript) is not tested; and the test program and application have to communicate with each other using the HTTP protocol.
• Adding test structures to the application code, which involves two interfaces -- one for interacting with the web server, and another for interacting with the tester. This lets the tester program interact with the application using the simpler, nonHTTP protocol. To test a login feature, for example, the tester program sends the values UserId and Password as command-line arguments to the program instead of generating an HTTP request. Hence, developing a tester program becomes easier. But the other disadvantages -- no testing of browser rendering and client-side processing -- remain. Also, the simulation approach introduces the disadvantage of embedding structures in the application to handle test-mode interaction.

• Manual testing, which checks all components of the system, but the disadvantages are obvious -- it is cumbersome, requires lots of time, and introduces the possibility of human errors.

Browser Representation of HTML Documents

When HTML documents are loaded into browsers, they sometimes create a number of Javascript objects for different components of the document; for example, one image object is created for each <img> tag in the HTML source. The created objects have attributes and methods. Attributes denote the properties of an HTML component, while methods, when invoked, perform relevant operations on the object, such as a Text object. One attribute of a Text object is value, denoting the current value entered in the form. focus is a method that, when invoked, sets focus to the Text field in the browser screen.

Similarly, browsers create objects, then initialize and arrange them in a hierarchy that reflects the structure of an HTML page itself. In Figure 1, the browser represents a given HTML document. The sample HTML document contains one image, one form, two Text objects, and one submit component. All these components get mapped to Javascript objects inside the browser and are arranged in a hierarchy. All HTML documents loaded in the browser are internally represented in the same way.

Accessing Browser's HTML Objects

HTML documents that originate from the same server can access objects of other documents once they're loaded in the browser. This can be done using Javascript.

For example, in Figure 2, Pages A and B are HTML documents originating from the same server. Page A is a simple HTML document with a form that contains a component named "Industry," which is a text object. Page B contains Javascript functions to access the Page A objects. It contains three buttons attached to three different Javascript functions. Clicking on the first button (Open Page A) loads Page A on a new browser window and receives a handle to the newly created window. Clicking on the second button (Set a Form Variable [Industry]) accesses the Page A object hierarchy and sets a value (Software) to its Text object. Finally, clicking on the third button (Close Page A) closes the browser window that held Page A. Thus, Page B simulates some user events on Page A by directly manipulating Page A's objects. For details, see Listings One and Two implement Pages A and B, respectively.

Automating WUI Testing

A WUI to an application can sequentially generate HTML documents. In a typical database search scenario, for instance:

• 1. Users click on a Search button.
• 2. Get back a keywords form.

• 3. Users fill in and submit the form.

• 4. Get results.

In this case, two HTML documents (keywords and output) are generated and two input events (click on Search and enter keywords) take place. Additional features might result in the generation of many more HTML documents.

Manually testing this feature might involve many iterations of the database search steps each time with different test cases (keyword values). However, testing can be automated by writing a Javascript function that simulates user events. The steps to this might be:

• 1. Simulate pressing the Search button.
• 2. Wait until the HTML-document-with-keywords form is received.

• 3. Supply keyword values and submit the form.

• 4. Wait until the HTML-document-with-search-results is received.

• 5. Check for the success of the search operation.

Once testing is automated, it can be deployed many times with different test cases. The search tester function takes keyword values as parameters and tests using those values. Figure 3 shows one way to automate WUI testing. The test engine is the core component that automates the testing. As Listing Four shows, it is simply an HTML page containing tester functions, a main function, and test cases.

Tester functions are used for testing individual features of the application. Every feature that needs to be tested is associated with a specific tester function. All tester functions are written in Javascript. Each tester function takes all the input values required for a feature, applies them to the application screen by manipulating its objects, and submits the request. It then waits until the response for the given request is received and rendered by the browser. After output is received, it checks for the success of the operation by accessing the value of a fixed hidden variable in the application page. In case of unexpected results, it takes appropriate actions, such as suspending testing and informing users.

If the feature involves multiple screens, it takes care of the synchronization by waiting until a response is received before proceeding with the next HTML document.

The main tester function manages the testing and determines the sequence in which features should be invoked and selects different test cases for each feature.

The test engine maintains the test cases to be applied in Javascript data structures. The test engine is generated by test engine generators, which take the tester functions, read test cases from files, and embed them in Javascript data structures. Example 1 shows the format of test cases files. ID identifies the test case, FEATURE denotes which feature to invoke, and KEY and VALUE pairs denote the expected input variables and values for them. For a database search scenario, the test case file might look like Example 2.

After updating test case files, the test engine generator needs to be run again so that the new test cases get loaded in the test engine.

Testing a Sample Application

For illustration, consider a hypothetical company called "ABC Audio" that sells audio products. ABC has enhanced its homepage to take online orders from customers. The WUI lists the available brands and price details. Customers select brands, specify quantity, method of payment, and order. As Figure 4 illustrates, the WUI calculates total prices and informs the customer. This processing is straightforward, but real-life applications involve much complex processing.

The WUI can be tested using a tester that implements the technique described here. The tester generates orders by getting realistic random numbers for each of the parameters (Quantity, Payment method, and so on), applies it to the online shop, and waits for the order to be processed. Once done, it goes back to the online shop page and proceeds with the next order. It repeats the testing to the specified number of times. Listings Three and Four present the HTML source for the online shop and online shop tester implementations, respectively.

Extensions

There are several ways in which you can enhance or otherwise extend the test engine I present here. These enhancements include:

• Regression testing. The test engine can be extended to perform regression testing. It can access the output from the application for the given test case, send it to another CGI program, which will compare the new output with old output, and store the results.
• Report generation. The test engine can be made to check for the success of the operation for each test case applied and generate a test report. The application can inform the test engine whether the operation was successful by setting a status variable in the output screen. The test engine can access the status variable to check the result.

• Record and play user events. This technique still requires writing tester functions for each of the features and generating test cases. This overhead can be eliminated if user events on an HTML page can be recorded to generate tester functions and test cases automatically.

Conclusion

The automated testing technique presented here addresses most of the disadvantages of simulation and manual testing. Still, it does have an overhead cost of writing tester functions for each application feature. But it is a one-time effort, and the benefit achieved by this technique will be considerable for web-based applications that are released frequently.

DDJ

Listing One

<html>
  <!-- ------------------------------------------------------------  
    Subject: pageA.html
    To     : Doctor Dobb's Journal
    Section: Internet Programming
    From   : M Selvakumar ([email protected])
    Copyright, 1998, M Selvakumar, Texas Instruments Inc.
  ------------------------------------------------------------- -->
<body >
  <center>
  <font size=5> Page A </font><br><hr>
  </center>
  <form method=post>
     Industry : <input type=text name="industry"> 
  </form>
  <hr>
</body>
</html>

Back to Article

Listing Two

<html>
  <!-- ------------------------------------------------------------  
    Subject: pageB.html
    To     : Doctor Dobb's Journal
    Section: Internet Programming
    From   : M Selvakumar ([email protected])
    Copyright, 1998, M Selvakumar, Texas Instruments Inc.
  ------------------------------------------------------------- -->
<head>
   <script language=Javascript>
      var Win;
      // ---------------------------------------- 
      // Open 'Page A'
      // ---------------------------------------- 
      function init () {
           Win = window.open("http://<your domain>/pageA.html");
      }
      // ---------------------------------------- 
      // Modify 'Page A'
      // ---------------------------------------- 
      function modifyFormA () {
        // Set a value to form variable - Industry
        Win.document.forms[0].industry.value = "Software";
      } 
      // ---------------------------------------- 
      // Close 'Page A'
      // ---------------------------------------- 
      function term () { Win.close(); }
   </script>
</head>
<body >
  <center>
  <font size=5> Page B </font><br><hr>
  </center>
  <form name=formB action="">
    <input type=button value="Open PageA"             onClick="init();" ><br>
    <input type=button value="Set a Form Variable (Industry)"  onClick="modifyFormA();" ><br>
    <input type=button value="Close PageA"            onClick="term();" ><br>
  </form>
  <hr>
</body>
</html>

Back to Article

Listing Three

<html>
  <!-- ------------------------------------------------------------
    Subject: Online Shop 
    To     : Doctor Dobb's Journal
    Section: Internet Programming
    From   : M Selvakumar ([email protected])
    Copyright, 1998, M Selvakumar, Texas Instruments Inc.
  ------------------------------------------------------------- -->
  <head>
    <title> ABC Audio</title>
  </head>
  <body bgcolor="#ffffff">
  <center>
  <font size=6>ABC Audio</font><br>
  <font size=5>On-line Shopping!</font><br>
  </center>
  <br><hr>
  <font size=5>Personal Audio<font>
  <br><br>
  <font size=4>Please select the items & quantities</font>
  <form method=post action="http://<ABC Audio Domain Name>/cgi-bin/shop/onlineShop.pl">
  <blockquote>
  <!-- ---------------------------------------------
   // Display Brands and their prices
   --------------------------------------------- -->
  <table border=1>
    <tr bgcolor=yellow><th><th>Brand<th>Model Name<th>Price<th>Quantity
    <tr><td><input type=checkbox name=brands value="B1">
        <td> Sony
        <td> SA-1670
        <td> 20$
        <td><input type=text size=5 name="Q1">
    <tr><td><input type=checkbox name=brands value="B2">
        <td> Aiwa
        <td> AI-W660
        <td> 22$
        <td><input type=text size=5 name="Q2">
    <tr><td><input type=checkbox name=brands value="B3">
        <td> Panasonic
        <td> PA-X1250
        <td> 21$
        <td><input type=text size=5 name="Q3">
  </table>
  </blockquote>
  <br>
  <!-- ---------------------------------------------
   // Display Payment Methods
   --------------------------------------------- -->
  <font size=5> Payment Method<font>
  <blockquote>
     <input type=radio name=payment value="P1">Visa  
     <input type=radio name=payment value="P2">Master Card  
     <input type=radio name=payment value="P3">American Express  
  </blockquote>
  <hr>
  <input type=submit value="Order Now!">
  <input type=reset value="Clear">
  
  </form>
  </body>
</html>

Back to Article

Listing Four

<html>
  <!-- ------------------------------------------------------------  
    Subject: Online Shop Tester Program
    To     : Doctor Dobb's Journal
    Section: Internet Programming
    From   : M Selvakumar ([email protected])
    Copyright, 1998, M Selvakumar, Texas Instruments Inc.
  ------------------------------------------------------------- -->
   <head>
      <title> ABC Audio On-line Shop Tester </title>
      <script>
         var Win, RepWin;
         var count=0, noOfTests;
         // -------------------------------------------------- 
         // TEST ENGINE -- Contains  Main Function, Tester Functions and
         //   randomly generated Testcases
         // -------------------------------------------------- 
         // -------------------------------------------------- 
         // Main Function - Test Shop
         //   1. Generate and apply a random order
         //   2. Wait for the response
         //   3. Getback the Shop for next test
         // -------------------------------------------------- 
         function test( task ) {
            if ( task == 1 ){
               submitAShoppingReq();
               task = 2;
            }
            if (task == 2) {
               var ret = goBack();
               if ( ret == false ) {
                  setTimeout("test(2)", 2000);
                  return false;
               }
            }
            count++;
            // Move to next order
            if ( count <= document.forms[0].testCount.value) {
              setTimeout("test(1)", 500);
            }
            return true;
         }
         // -------------------------------------------------- 
         // Tester Function 1 - Open Shop
         // -------------------------------------------------- 
         function init() {
           Win = window.open("http://<ABC Audio Domain Name>/shop.html",'',
                             'width=500,height=600');
         }
         // ----------------------------------------------------------------- 
         // Tester Function 2 - Generate a random order and apply to the Shop.
         // ----------------------------------------------------------------- 
         function submitAShoppingReq() {
            var tmp = Math.round(Math.random()*10);
            var payment = tmp%3;
            Win.document.forms[0].reset();
            Win.document.forms[0].brands[0].checked=true;
            Win.document.forms[0].brands[1].checked=true;
            Win.document.forms[0].brands[2].checked=true;
            Win.document.forms[0].Q1.value= Math.round(Math.random()*300);
            Win.document.forms[0].Q2.value= Math.round(Math.random()*200);
            Win.document.forms[0].Q3.value= Math.round(Math.random()*100);
            Win.document.forms[0].payment[payment].checked = true;
            Win.document.forms[0].submit();
         }
         // --------------------------------------------------
         // Tester Function 3 - Close Shop
         // --------------------------------------------------
         function term() { Win.close(); }
         function goBack() {
            // Check if the response is received
            if ( typeof(Win.opOver) == 'undefined' ) {
               return false;
            }
            // Check for success of the operation
            if ( Win.document.forms[0].result.value != "Success" ) {
               alert("Operation not Successful");
            }
            Win.history.go(-1);
            return true;
         }
      </script>
    </head>
    <body bgcolor="#ffffff">
         <center>
         <font size=5>ABC Audio On-line Shop </font><br>
         <font size=4>Web Interface Tester</font><br>
         </center>
         <br><hr>
         <form name=Tester onSubmit="parent.test()">
            <input type=button value="Open the Shop!" onClick="init();"><br>
            No. of Tests
            <input type=text name=testCount size=5 value=1>
            <input type=button value="Test!" onClick="count=1; test(1);"><br>
            <input type=button value="Close the Shop!" onClick="term();"><br>
            <br><hr>
         </form>
      </center>
    </body>
</html>

Back to Article