Simulating user interaction via interclient communication
Birger is a software consultant based in Tnsberg, Norway. He can be reached by phone at +47 94 26 27 52 or +47 33 31 97 64 or by fax at +47 22 55 33 77.
According to the Quality Assurance Institute, as much as 80 percent of all software is tested manually. Generally, this is expensive, boring, and inefficient. Consequently, testing is usually kept to a minimum and often squeezed in at the end of the development process. Yet, to be competitive, all parts of software products must be thoroughly tested and verified. Furthermore, quality-assurance programs are necessary to meet emerging standards like the ISO 9000-3. (For more on this subject, see the accompanying text box entitled "ISO 9000-3 and Software Quality Assurance.")
Test drivers, most often written to test individual routines, can be applied to the complete application. Especially with modular designs (such as with X Windows), it's possible to write a computer-aided software-testing system that simulates user interactions. The system can perform a sequence of operations predefined in script files.
The advantages of even a simple automatic test system are obvious: Long-lasting testing is enabled by automatically repeating the script, control of memory consumption and other errors (which appear after long and heavy use) is made easier, and regression testing is facilitated by rerunning the scripts.
The X Windows Approach
In X Windows, the X server is software which manages the keyboard, screen, and mouse. When the user presses a button or key, the X server sends events to the right window of the client (application). The server is responsible for managing the hierarchy of overlapping windows on the screen and gives notice about changes to the client's windows via events. The events are true messages sent over an asynchronous communication channel.
The test system I'll describe here simulates user interaction by producing events and sending them to the applications. The applications' event-dispatching mechanism doesn't distinguish between events generated by the server and those sent by another client; it reacts similarly to both. Most of the code is placed in the test driver, a separate process that sends events to all applications initialized for the test system, based on button or key commands from a script file.
Example 1 shows the format of a typical script file. The process_name token is the name of the application program. The widget_name is the name of the widget that will eventually receive and act on the events. The input file is parsed using strtok. (The process is straightforward and is therefore not detailed in this article.) A test script for opening a file is shown in Example 2.
The Application
External test drivers keep the extra test code linked into the applications to a minimum. The production system can then be released with that code still in place exactly as it was tested.
Some test-related code is needed in the application because the test driver doesn't know the IDs or internal addresses of all the application's widgets and gadgets. The test driver sends the events to one known widget in each application; code associated with that widget reroutes all test events to the individual widgets within the same process.
First, let's look at initialization. During this process, the test driver is able to pick up the IDs of the application's main widgets via two interclient communication facilities: properties and atoms.
The main widgets of each application (in Motif, the XmMainWindow) receive all the events from the test driver. The window IDs of these MainWindows are stored as properties--data maintained by the X server and available to all other processes running on the display that know the properties' names.
The window IDs of the MainWindows are obtained by calling XtWindow with a widget structure as input. To identify properties, X uses unique resource IDs, called "atoms" which are created by the Xlib function XInternAtom. The input to XInternAtom is a string (in this case the name of the receiving application) taken directly from the test scripts. The routine XChangeProperty creates the properties, which are also associated with windows. We use the root window because the data is shared by multiple windows and processes; see Listing One (page 78). Listing Two (page 78) is the general routine for sending an event from the test process to the applications.
In each application, a table is created to keep track of all of its widgets. In this system, this array is called the WidgetTable. Symbolic widget names are #defined (Listing Five, page 80) and used as indexes into WidgetTable. When the widgets are created, the returned addresses are put into the array. Apart from being necessary for this test system, WidgetTable simplifies all later operations on widgets and gadgets and also follows the example set by the demo programs shipped along with Motif.
The Test Driver
The test driver declares and sends ClientMessage events to the MainWindow widget of the application processes. ClientMessage events carry a data area which is a union structure of 20 bytes, 10 shorts, or 5 long values. By selecting one of these data types, the events can be used to send an array between clients on the display. In this case, we use them to give information to the applications about the commands in the test script.
The test driver has to translate a widget name found in the test script to the number used as an index into the WidgetTable of the relevant application. This is done most easily by scanning a table, as shown in Listings Six and Seven (page 80). The test driver puts the widget index into one of the locations of the ClientMessage event's data field. The type of operation is put into the next data-field location, and in case of key events, the keycode is put into the third.
Then the test driver sends the XClientMessage events to the application via the XSendEvent function call to the window ID (found as a property in the X server; see Listing Three, page 78).
To enable a MainWidget to receive events, XtAddEventHandler must be called for it. ClientMessage events should not be specifically selected by XSelectInput; they are always received by the window they are sent to.
When the MainWidget receives a ClientMessage, the event handler uses the destination widget's index placed in the data field to look up the widget structure in the WidgetTable. The receiving-window ID is obtained by calling XtWindow on this widget structure. The desired event type (XButtonEvent, for example) is declared, initialized, and passed to the intended window by XSendEvent and is therefore handled by Xlib's ordinary event-dispatch mechanism. The receiving widgets react by calling their callback routines as if the events were coming from the server; see Listing Four, page 78. The application is also able to process events sent by the X server. Manual user interaction is therefore possible, whether automatic tests are running or not.
This automatic test method gives a true simulation of user action because the exact same code is executed in the application in both situations. A system like this won't replace manual test methods, but it can certainly improve their performance.
ISO 9000-3 and Software Quality Assurance
ISO 9000 was established to facilitate the international exchange of goods and services. ISO 9000-3 (officially titled "Guidelines for the Application of ISO 9001 to the Development, Supply, and Maintenance of Software") is a subset of the original standard established to set international guidelines for software quality assurance. Although ISO 9000-3 addresses several issues, standardized software testing and validation procedures are central to the proposal. It also covers related areas, including internal audits, purchasing specifications, training, and the like.
Generally speaking, ISO 9000-3 describes uniform software-development methods that meet client requirements, as defined by a specification. To be ISO 9000-3 compliant, the development process must adhere to a standard set of procedures by documenting the use of a formal life-cycle model governing the design, development, and maintenance processes. These formal procedures are referred to in terms of the development framework (in-house quality-assurance programs), life-cycle activities (the overall software-development process), and supporting activities (those necessary to qualify, conform, and confirm that the software product was developed properly). Conformance to ISO 9000 is certified by third-party auditors, and certification is valid for three years.
If you're interested in implementing ISO 9000 guidelines, a video course entitled, "Understanding and Implementing ISO 9000" developed by The Media Group (Williston, VT) is one place to start. The training video, which is one in a series of ISO 9000-related courses, details the 20 elements of ISO 9000-1, including subsets ISO 9000-2 and 9000-3, and includes practical suggestions from companies such as Polaroid and Bachman, which comply with the standard. For more information, call The Media Group at 800-678-1003.
As with many ISO standards, 9000-3 has gained more momentum in Europe than North America. However, U.S. software vendors with plans for cultivating a global client base may find it necessary to seek ISO 9000 certification. Likewise, implementing a certification software quality-assurance program may also provide a competitive edge when cracking non-U.S. markets. In any event, establishing well-defined software testing and quality-assurance programs makes good sense for all software developers.
--editors
Example 1: Typical test script.
Example 2: Sample test-script syntax.
Copyright © 1994, Dr. Dobb's Journal
B[uttonPress]
<process_name>
<widget_name>
K[eyPress]
<process_name>
<widget_name>
<key>
ButtonPress myapp file-cascade
ButtonPress myapp open-button
KeyPress myapp open-file-field n
KeyPress myapp open-file-field e
KeyPress myapp open-file-field w
KeyPress myapp open-file-field .
KeyPress myapp open-file-field t
KeyPress myapp open-file-field x
KeyPress myapp open-file-field t
ButtonPress myapp open-ok-button
[LISTING ONE]
void APPstoreMainID(Widget W, /* I: Widget of MainWindow */
char *ProcessName) /* I: Name of the property */
{
/* Code to initialization the test system within the application. Stores
the Main Widget window ID as a property in the X server */
Atom atom;
Display *display;
Window window;
display = XtDisplay(W);
window = XtWindow(W);
/* Atoms are the addressing mechanism for properties */
atom = XInternAtom(display, ProcessName, 0);
/* Adding property for ProcessName. The property is associated
with the root window */
XChangeProperty(display, DefaultRootWindow(display), atom,
XA_WINDOW, 32, PropModeReplace, (unsigned char *)&window, 1);
}
[LISTING TWO]
int GENsendEvent(Widget FromW, /* Widget the event is sent from,
used to get the display ID. */
XEvent *Event, /* Event to be sent */
char *ProcessName) /* Destination process name */
/* Description: General routine to send an event, here used to send events
from the test process to the applications. */
{
int ret, format;
unsigned long nitems, left;
Display *display=NULL;
Window window, *retData=NULL;
Atom atom, type;
long eventMask = 0;
display = XtDisplay(FromW);
/* Get propery identifier. */
atom = XInternAtom(display, ProcessName, 1);
/* Get the receiving window ID, stored as a property in the server. */
ret = XGetWindowProperty(display, DefaultRootWindow(display), atom,
0, 4, False, XA_WINDOW, &type, &format, &nitems,
&left, (unsigned char **)&retData);
/* Check ret: Success value is special for XGetWindowProperty, see Ref Man.*/
if (retData != NULL) {
window = *retData;
eventMask |= ButtonPressMask;
eventMask |= ButtonReleaseMask;
eventMask |= NoEventMask;
Event->xany.window = window;
Event->xany.display = display;
ret = XSendEvent(display, window, TRUE, eventMask, Event);
}
XFree((caddr_t)retData);
}
[LISTING THREE]
int TESTsendButtonEvent(int Down, /* I: TRUE if ButtonPress */
int WidgetIndex, /* I: Destination widget number
converted from the name read
from the test script via a table */
char *ProcName, /* I: Name of application read
from test script*/
Widget FromWidget) /* I: Main widget of test driver */
/* Declares and sends events from the Test Driver to the applications */
{
int eventType;
XClientMessageEvent event;
if (Down)
eventType = ButtonPress;
else
eventType = ButtonRelease;
event.type = ClientMessage;
event.format = 32;
event.data.l[0] = eventType;
event.data.l[1] = WidgetIndex;
GENsendEvent(FromWidget, (XEvent *)&event, ProcName);
/* GENsendEvent is shown in Listing Two */
}
[LISTING FOUR]
#include <Xm/PushBGP.h>
void APPdistributeEvent(Widget W, XtPointer ClientData,
XClientMessageEvent *Event,
Boolean *ContToDispatch)
/* Receives events from the test driver and passes the events to individual
widgets based on the message received by the test driver. This is a standard
callback routine. Therefore the parameters are not explained. */
{
int ret, format, widgetIndex, eventType, keyCode;
Display *display;
Window window;
long eventMask = 0;
XButtonEvent newEvent; /* Used both for button and key events. The structures
are identical, except for the button/code field. */
eventType = (int)Event->data.l[0];
widgetIndex = (int)Event->data.l[1];
if ((eventType == KeyPress) || (eventType == KeyRelease)) {
keyCode = (int)Event->data.l[2];
}
/* Checks should be performed to ensure that widget index is within legal
range, skipped in this listing */
if (XtIsWidget(WidgetTable[widgetIndex])) {
display = XtDisplay(WidgetTable[widgetIndex]);
window = XtWindow(WidgetTable[widgetIndex]);
/* Position is set inside the window */
newEvent.x = 4;
newEvent.y = 4;
}
else { /* gadgets */
display = XtDisplay(((XmGadget)WidgetTable[widgetIndex])->object.parent);
window = XtWindow(((XmGadget)WidgetTable[widgetIndex])->object.parent);
/* Gadgets are part of their parent widget's window. Below is code to find
where gadget is positioned inside parent. Event is sent to parent
in the case of gadgets. */
newEvent.x = ((XmGadget)WidgetTable[widgetIndex])->rectangle.x + 4;
newEvent.y = ((XmGadget)WidgetTable[widgetIndex])->rectangle.y + 4;
}
newEvent.window = window;
newEvent.display = display;
newEvent.subwindow = 0;
newEvent.root = XRootWindowOfScreen(XtScreenOfObject(W));
newEvent.type = eventType;
newEvent.time = CurrentTime;
newEvent.state = 0;
if ((eventType == KeyPress) || (eventType == KeyRelease)) {
/* The field below is the only difference between the event types used. */
newEvent.button = keyCode;
}
else {
newEvent.button = Button1;
}
ret = XSendEvent(display, window, TRUE, eventMask, (XEvent *)&newEvent);
/* Error checks should be done on ret */
}
[LISTING FIVE]
/* Application include file which defines widget names used as
index into WidgetTable */
#define file-cascade 1
#define open-button 2
#define open-file-field 3
#define open-ok-button 4
[LISTING SIX]
/* Table used to find widget indexes of application App1 in test driver */
TEST_ITEM App1TestTable[] = {
{"file-cascade", 1},
{"open-button", 2},
{"open-file-field", 3},
{"open-ok-button", 4}
};
int App1TableSize = (sizeof App1TestTable / sizeof App1TestTable[0]);
[LISTING SEVEN]
/* Definition of the TEST_ITEM structure */
typedef struct {
char *WidgetName;
int WidgetIndex;
} TEST_ITEM;
End Listings