VToolsD for VxD Development

Charles works in Silicon Valley, specializing in communications, multimedia, and games. He can be reached at [email protected].

QuickVxD Wizard

Figure 1 is the main dialog. The collection of controls in the upper left is called "Device" and defines information in the VxD device-descriptor block. This information includes the name of the VxD (which also becomes the VxD filename, header filename, and make filename), the VxD ID value (if the VxD exports services), the initialization order when VMM loads the VxD at Windows launch time, and the major and minor version numbers of the VxD. The result of completing this section is the line Declare_Virtual_Device(SPELLX) in the VxD skeleton; see Listing One. Listing Two is the equivalent skeleton in C++.

The name SPELLX appears in the declaration. QuickVxD creates a header file for the VxD containing declarations for the major and minor version numbers, the device id, and the initialization order. These definitions are used to expand the Declare_Virtual_Device macro. For Spellx, the major and minor versions are 1 and 0, the device id is undefined, and the initialization order is VKD_INIT_ORDER+1. This produces the header file in Listing Three. These definitions are used to expand the declaration of the device-descriptor block as in Example 1. (The complete source code for the SPELLX.386 VxD is provided electronically; see "Availability," page 3.)

The main dialog includes options for creating the skeleton in C or C++ and including or excluding debug information in the build. The debug symbols supported are for Soft-ICE and WinDeb386 (the same as for the Microsoft C8 and C9 compilers). The source code for the VxD did not exactly match up with the executable instructions, but this is likely a problem with Soft-ICE, and the discrepancies were minor. The dialog includes options for which entry points the VxD should have: vendor-specific V86 mode, vendor-specified protected mode, and the V86 and PM entry points called by VMM when the VxD registers a device id with VMM. Checking off these boxes saves work because QuickVxD will create empty entry points in the skeleton for you to fill in. If the VxD exports any services, QuickVxD can list the prototypes for these exported services in the skeleton.

Clicking on the Control Messages button causes the QuickVxD Control Messages dialog to be displayed. Each check box corresponds to a VMM message for the control dispatch routine to trap and process. For each box checked, a trap is inserted into the control dispatch routine in the VxD skeleton. The trap will call an (initially) empty handler function to process the event message. It's your job to fill in these handlers with useful code.

Next, clicking the Generate Now button creates the C or C++ skeleton, header file, and make file for the VxD. QuickVxD will ask you for the pathnames under which to save these files. To build the VxD, open a DOS box and run nmake -f <name of make file>. When I was writing Spellx, the example skeleton built on the first try without warnings or errors.

The Spellx VxD

Listing One is the skeleton generated by QuickVxD as a result of these settings. After the device declaration, macros contained in the skeleton declare control handlers for the five VMM control messages that the VxD processes. The DefineControlHandler(MessageType, HandlerName); macro expands to extern MessageType_type HandlerName;. For example, DefineControlHandler(VM_INIT, OnVmInit); expands to extern VM_INIT_type OnVmInit;. Parameter checking is not done by the prototype. This isn't a problem because the call to the control handler is also generated by the Wizard. Next comes the VxD control dispatcher, whose entry point has the declaration BOOL ControlDispatcher (DWORD dwControlMessage, DWORD EBX, DWORD EDX, DWORD ESI, DWORD EDI).

The actual entry point to the control dispatch routine is in the VToolsD prologue code, which is linked to the skeleton automatically by the make file. The prologue code reformats the register information passed from VMM to the control dispatcher into C parameters and then calls the skeleton entry point ControlDispatcher.

Within the body of the control dispatcher, calls to the various event handlers are implemented with message-cracker macros. Example 2 lists the five VMM message traps Spellx requires. The START_CONTROL_DISPATCH and END_CONTROL_DISPATCH macros expand into a switch statement on dwControlMessage. Each ON_xxx macro expands into a case statement and call to the appropriate handler for the message. For instance, the statement in Example 3(a) expands to Example 3(b).

Of course, the wizard only creates skeleton code--it can't write the code that makes a VxD do something useful. However, the toolkit contains two libraries that facilitate writing useful code. The first is a C run-time library full of the functions that made C popular in the first place: calloc, sprintf, itoa, strcpy, and so on. The second is a library of wrapper functions for calling VMM and VxD services from C or C++.

C Run-Time Library

The function to add a node to the list (AddVMContext) is called when the system VM is created and whenever a new VM is created. Thus, AddVMContext is called to create a new context whenever the control dispatcher receives a VM_INIT or SYS_VM_INIT message from VMM. AddVMContext calls calloc to allocate a new context node; see Example 4.

Each VM context contains the VM handle, key history buffer, buffer index, and link to the next context. Each context also contains a field to record the global shift state when each character N, D, and Space is pressed. The key history buffer contains a record of the last MAXVMKEYS pressed in the VM. MAXVMKEYS is defined as 5, which is just enough space to record the five-key sequence of <space>adn<space> that the VxD wants to detect. The buffer index wNextKey indicates the next space in the buffer to place a key. The shift state fields are dwDShift, dwNShift, and dwSShift. When the keys D, N, or Space are pressed, the VxD records the global shift-state of the keyboard in these fields.

AddVMContext takes a VM handle, allocates a new context using calloc, and puts the VM handle into the context. It then links the context into the linked list and returns a pointer to the new context. The complementary function DeleteVMContext is called whenever any VM is destroyed. DeleteVMContext is called to delete a VM context whenever the control dispatcher receives a VM_TERMINATE or SYS_VM_TERMINATE message from VMM. DeleteVMContext frees the context node associated with a VM; see Example 5. DeleteVMContext calls the local function ThisVMContext to get a pointer to the node associated with the given VM handle.

The C run-time library allowed me to replace the VMM linked-list services with my own using calloc and free. Of course, most VMM services cannot be easily replaced, and so an easy method of calling VMM and VxD services in C is necessary. That's where the wrapper functions come in.

Wrapper-Function Libraries

VToolsD comes with wrapper functions for most of the VMM and standard VxD services--no more pushing arguments into high and low registers. VMM and other VxD services are available through C function calls. These wrapper functions are written in assembler, and the ones I examined take the C parameters off the stack, put them in registers, call the VMM services, and place the return value in EAX. Using the VKD virtual keyboard driver, I trapped the A, N, D, Space, and Backspace keys with the wrapper functions. The wrapper function VKD_Define_Hot_Key tells VKD to notify the VxD when a particular key is pressed. Like most wrapper functions, VKD_Define_Hot_Key corresponds to the service of the same name. Example 6(a) illustrates its syntax. Example 6(b) will trap the A key.

The first parameter is the scan code of the key to trap, and the second is the type of scan code to trap. Some keyboards have keys with extended scan codes. I set the value of the second parameter to SCAN_EITHER, which traps both normal and extended scan codes. The third parameter is the shift state of the key to trap. The shift state defines, with great precision, the combination of a regular key and Shift key (left shift, right shift, left or right control key, left or right Alt key, caps lock, scroll lock, and other keys that shift the value of a key). The shift state works like this: When a key is pressed, the states of all shift keys on the keyboard are collected into a 16-bit word called the "global shift state." This global shift state is ANDed with the high word of the shiftstate parameter, and the result of the AND is compared with the low-order 16 bits of the shiftstate parameter. If the result of the AND matches the lower 16 bits, VKD calls the VxD's hot-key handler (defined in the fifth parameter); otherwise, the handler is not called. I set shiftstate (for both high and low words) to 0. That way, the global shift state is always ANDed with 0, resulting in 0. This value is then compared with the low word, which is also 0. In other words, the hot-key handler will always be called, regardless of the keyboard's shift state.

After the shiftstate parameter, the flags further define which keyboard events will result in a callback to the hot-key handler. The flags specify whether VKD will call the handler when a key is pressed, released, or autorepeated, and when a shiftstate ends. I had problems with this parameter. I specified CallOnPress|CallOnRepeat which, according to the documentation, should have resulted in calls to the hot-key handler when a key was pressed or autorepeated. However, the hot-key handler is called only when the key is pressed. I'm not an expert on the VKD services, so I don't know if this is a VKD bug, a bug in the VToolsD documentation or wrapper function, or my own ignorance. Whatever the case, the VxD won't detect and reflect autorepeat presses of the trapped keys.

The parameters RefData and MaxDelay define, respectively, data that can be passed transparently during the callback (for example, a pointer to a unique context for each hot key), and the maximum delay time allowed between occurrence of the event and notification of the event handler. I didn't use either parameter, so both are set to 0. VKD_Define_Hot_Key returns a handle which uniquely identifies the hot key for calls to other VKD services.

The fifth parameter specifies a pointer to the hot-key handler that VKD will call when the trapped keys are pressed. The wrapper libraries implement event handlers using thunks, which are nothing more than chunks of a locked data segment used as the entry point for the event handler. When VMM or a VxD calls an event handler in another VxD, the arguments to the event handlers are passed in registers. When the event handler is a C function, the register arguments must be translated into stack-based arguments compatible with C. To accomplish this, VToolsD uses thunks. To trap hot-key events, I passed the address of my C event handler to VKD_Define_Hot_Key, and I also passed the address of a thunk (in the last parameter). All wrapper functions that define event handlers take a thunk argument, which is a pointer to an area of locked memory. The wrapper function modifies this locked memory area with the code necessary to translate the register arguments in C stack parameters. It also inserts into the thunk a call to the entry point of the event handler in the VxD. The wrapper function then passes the address of the thunk to VMM or the other VxD, which detects the event to trap and calls the thunk. The thunk reformats the register arguments into C stack arguments and calls the VxD event handler. After the call, the VxD event handler returns to the next instruction in the thunk. The thunk reformats any return values into the appropriate registers and returns to VMM or the calling VxD.

I declared the thunk for VKD_Define_Hot_Key in the locked data segment. Because different event handlers require different arguments, the wrapper library contains a unique template for each type of event handler available from VMM and other VxDs. Unfortunately, the template for VKD_Define_Hot_Key did not match the function prototype in the VToolsD documentation: Two of the parameters were switched.

In particular, the documentation defines the event handler for VKD_Define_Hot_Key as VOID __stdcall HotKeyHandler(DWORD scancode, DWORD hotkeyhandle, DWORD shiftstate, DWORD data, DWORD delaytime). The first parameter is the scan code of the key that caused the event, the second is the hot-key handle returned from VKD_Define_Hot_Key, the third is the global shift state, the fourth is the reference data passed to VKD_Define_Hot_Key, and the fifth is the delay between the event and the handler being called (this is normally 0 unless you specifically request delay notification). However, the thunk was reversing the order of the first two parameters, scancode and hotkeyhandle, on the C stack, which created a lot of confusion. This bug was especially frustrating because the thunk comprises code executing in the data segment, so I could not set a break point on it when debugging. I talked to the support staff at Vireo, who were very helpful and claimed this would be fixed in the next release.

Processing Hot Keys

Conclusion

For More Information

[email protected].

Figure 1: The main dialog of QuickVxD.

Example 1: Expanding the declaration of the device-descriptor block.

DDB The_DDB = {0,           /* used by VMM */
        DDK_VERSION,        /* version of the DDK used */
        SPELLX_DeviceID,    /* device id entered by QuickVxD */
        SPELLX_Major,       /* major version number */
        SPELLX_Minor,       /* minor version number */
        0,                  /* flags */
        {' ', ' ', ' ', ' ',' ', ' ', ' ', ' '},   
          /* eight spaces where VToolsD prologue code fills in VxD name */
        SPELLX_Init_Order,             /* in the order device should be 
                                         loaded at Windows launch time */
        (DWORD)LocalControlDispatcher,  /* points to entry point in VToolsD 
               prologue code linked to VxD. This is what VMM calls with 
               control events. Prologue calls VxD control dispatcher */
        (DWORD) LocalV86handler,        /* entry point in VToolsD prologue 
               code which VMM calls to implement V86 services. 
               Prologue calls VxD V86 service entry point */
        (DWORD) LocalPMhandler,        /* entry point in VToolsD prologue 
               code which VMM calls to implement protected mode services. 
               Prologue calls the VxD protected mode service entry point */
        0,                            /* used by VMM */
        0,                            /* used by VMM */
        0,                            /* used by VMM */
        (DWORD) VXD_SERVICE_TABLE,      /* points to VxD service table */
        0}                            /* size of service table */

START_CONTROL_DISPATCH
   ON_DEVICE_INIT(OnDeviceInit);
   ON_VM_INIT(OnVmInit);
   ON_VM_TERMINATE(OnVmTerminate);
   ON_SYS_VM_INIT(OnSysVmInit);

ON_SYS_VM_TERMINATE(OnSysVmTerminate);
END_CONTROL_DISPATCH

return OnDeviceInit((VMHANDLE)EBX, (PCHAR) ESI); Example 4: AddVMContext calls calloc to allocate a new context node.

#define MAXVMKEYS   5
typedef struct tag_VMContext {
    VMHANDLE hVM;
    BYTE btKeyHistory[MAXVMKEYS];
    DWORD dwDShift;
    DWORD dwNShift;
    DWORD dwSShift;
    WORD wNextKey;
    struct tag_VMContext *pNext;
} VMContext;
VMContext *AddVMContext (VMHANDLE hVM)
{
    VMContext *NewVM=calloc
(1, sizeof(VMContext));
    if (!NewVM)
        return NULL;
    NewVM->hVM = hVM;
    NewVM->pNext = VMListHead;
    VMListHead = NewVM;
    return NewVM;
}

VOID DeleteVMContext (VMHANDLE hVM)
{
    VMContext **PrevVM = NULL;
    VMContext *ThisVM =
ThisVMContext (hVM, PrevVM);
    if (ThisVM)
    {
        *PrevVM= ThisVM->pNext;
        free (ThisVM);
    }
}

(a)
HOTKEYHANDLE VKD_Define_Hot_Key (BYTE ScanCode, BYTE ScanType, DWORD ShiftState,
  DWORD Flags, PVXDHotkey_HANDLER Callback, CONST VOID *Refdata, DWROD MaxDelay, 
                                                      PVKD_Hotkey_THUNK pThunk).

(b)
#define CHAR_A       0x1E

VKDHotkey_THUNK      HKThunk

..

..

hCHAR_A=VKD_Define_Hot_Key (CHAR_A, SCAN_NORMAL, 0, CallOnPress|CallOnRepeat,
                              HotKeyHandler, 0, &HKThunk);

Listing One

// SPELLX.C - main module for VxD SPELLX
#define   DEVICE_MAIN
#include  "spellx.h"
#undef    DEVICE_MAIN
Declare_Virtual_Device(SPELLX)
DefineControlHandler(DEVICE_INIT, OnDeviceInit);
DefineControlHandler(SYS_VM_INIT, OnSysVmInit);
DefineControlHandler(SYS_VM_TERMINATE, OnSysVmTerminate);
DefineControlHandler(VM_INIT, OnVmInit);
DefineControlHandler(VM_TERMINATE, OnVmTerminate);
BOOL ControlDispatcher(
    DWORD dwControlMessage, DWORD EBX, DWORD EDX, DWORD ESI, DWORD EDI)
{
    START_CONTROL_DISPATCH
        ON_DEVICE_INIT(OnDeviceInit);
        ON_SYS_VM_INIT(OnSysVmInit);
        ON_SYS_VM_TERMINATE(OnSysVmTerminate);
        ON_VM_INIT(OnVmInit);
        ON_VM_TERMINATE(OnVmTerminate);
    END_CONTROL_DISPATCH
    return TRUE;
}
BOOL OnDeviceInit(VMHANDLE hVM, PCHAR CommandTail)
{
    return TRUE;
}
BOOL OnSysVmInit(VMHANDLE hVM)
{
    return TRUE;
}
VOID OnSysVmTerminate(VMHANDLE hVM)
{
}
BOOL OnVmInit(VMHANDLE hVM)
{
    return TRUE;
}
VOID OnVmTerminate(VMHANDLE hVM)
{
}

Listing Two

// SPELLXP.CPP - main module for VxD SPELLXP
#define DEVICE_MAIN
#include "spellxp.h"
Declare_Virtual_Device(SPELLXP)
#undef DEVICE_MAIN
SpellxVM::SpellxVM(VMHANDLE hVM) : VVirtualMachine(hVM) {}
BOOL SpellxDevice::OnDeviceInit(VMHANDLE hSysVM, PCHAR pszCmdTail)
{
    return TRUE;
}
BOOL SpellxVM::OnSysVMInit()
{
    return TRUE;
}
VOID SpellxVM::OnSysVMTerminate()
{
}
BOOL SpellxVM::OnVMInit()
{
    return TRUE;
}
VOID SpellxVM::OnVMTerminate()
{
}

Listing Three

// SPELLX.H - include file for VxD SPELLX
#include <vtoolsc.h>
#define SPELLX_Major        1
#define SPELLX_Minor        0
#define SPELLX_DeviceID     UNDEFINED_DEVICE_ID
#define SPELLX_In
it_Order   VKD_INIT_ORDER+1