UNDOCUMENTED CORNER
Undocumented OS/2: DosQProcStatus
Troy Folger
Troy is an OS/2 developer for a large company in the retail industry. He can be reached on CompuServe at 72360,427.
Applications developed for OS/2 2.x may employ multiple threads or processes, and OS/2 developers have a variety of interprocess communication (IPC) mechanisms to manage them. It would be helpful if OS/2 apps could examine external processes, IPC resources, and DLLs in this complex environment, but IBM does not provide the necessary functionality in OS/2's documented API.
Of course, you can observe external processes or system resources with the OS/2 command PSTAT. In this article, I'll examine DosQProcStatus (Dos Query Process Status, or DQPS), the undocumented API that PSTAT uses to obtain low-level OS/2 system information.
DQPS is a 16-bit API that has existed since OS/2 1.1. As undocumented OS/2 functions go, it is the most well known. A quick search of CompuServe's OS/2 forums reveals a number of DQPS explorers who have made relevant code available, among them George Brickner, Rick Fishman, Franz Krainer, and Chris Laforet.
In July 1993, an IBM employee released DOSQPS.ZIP on IBM's CompuServe OS2DF1 Forum (Lib 1). DOSQPS.ZIP includes DOSQPROC.TXT, a description of DQPS, and DOSQPROC.INF, the same information formatted for the OS/2 VIEW facility. Although these documents don't focus on PSTAT, they do mention that PSTAT uses information returned by DQPS. IBM has provided these documents and allowed them to be circulated freely, but declares that:
...some or all of the interfaces described in this document are unpublished. IBM reserves the right to change or delete them in future versions of OS/2 at IBM's sole discretion, without notice to you. IBM does not guarantee that compatibility of your applications will be maintained with future versions of OS/2.
This is not an idle threat: DQPS changed significantly between OS/2 1.3 and 2.0. DQPS currently remains "officially undocumented" and is not discussed in any IBM-provided toolkit, online reference, or sample code, apart from the file on CompuServe.
Unfortunately, the information in DOSQPROC.TXT is incomplete and inaccurate. After examining the DQPS API to learn more about the areas left unexplained (and to correct the inaccuracies), I wrote the C header file DOSQPROC.H (available electronically; see "Availability," page 3), which allows anyone armed with a 32-bit OS/2 compiler capable of calling a 16-bit API and passing 16-bit pointer parameters to fully use DQPS on OS/2 2.x systems.
I've also written a simple OS/2 text-mode program, PROCINFO.C (also available electronically), which uses the header file to demonstrate most of the data supplied by DQPS. PROCINFO shows the process IDs of arbitrary system processes or the names of their corresponding executables, and allows you to view the resources in use by those processes.
The data returned by DQPS can be divided into four major classes: process and thread records, executable module records, 16-bit system semaphore records, and named shared-memory segment records. Not every type of system resource is represented in the information maintained by DQPS; for instance, data regarding OS/2 pipes, queues, and 32-bit semaphores is absent.
Accessing DQPS
OS/2's DOSCALL1 library provides access to DQPS. DOSQPROC.TXT indicates that the DQPS entry point is DOSCALL1 ordinal 154, so applications should link with a module-definition file (.DEF) having the entry DOSQPROCSTATUS=DOSCALL1.154 in the IMPORTS section.
Running IBM's EXEHDR utility (using the /VERBOSE option) on OS/2 2.1's PSTAT.EXE confirms that PSTAT uses DQPS: PSTAT has a relocation record for DOSCALLS.154. (In the context of this discussion, the DOSCALL1 and DOSCALLS libraries can be considered equivalent.)
IBM LAN Server and LAN Requester reveal that FFST/2 (IBM's logging and diagnostic facility for network users) also uses DQPS: One of the FFST/2 DLLs (EPWPSI16.DLL) imports DOSCALLS.154. The serviceability and diagnostic aids to LAN administrators FFST/2 provides give an idea of the types of applications that can benefit from DQPS. If nothing else, IBM's own use of DQPS provides some assurance that the API will be around for a while.
The 16-bit DQPS API takes two parameters: a 16-bit far pointer to a user-allocated memory buffer and a USHORT containing the length of the buffer. It returns a 16-bit unsigned value indicating the success or failure of the call. Figure 1 presents a C-function prototype. (IBM's CSet++ compiler will automatically thunk the pointer parameters on functions declared with the APIENTRY16 modifier from 32-bit to 16-bit. However, the current version of Borland's C++ for OS/2 compiler requires the _far16 keyword in the parameter declarations.)
DOSQPROC.TXT doesn't enumerate the possible error returns, only mentioning that a return of zero indicates correct operation, and nonzero indicates an error. DOSQPROC.TXT also suggests that 64K is the preferred buffer-allocation size because you cannot accurately predict how much information a DQPS call will return.
The first parameter to DQPS, p16Buf, is declared as a PVOID (VOID _far16* in Borland) because the information copied to the passed buffer by the API is not a single structure, but a collection of several different data types relating to global data, processes and threads, 16-bit system semaphores, executable modules, and shared-memory segments (see Figure 2). The buffer is an unwieldy collection of pointers to arrays of structures and linked lists; to use the information, you must match the handles found in one data structure to corresponding handles in other, separate data structures.
The layout of the data holds a clue as to why this API remains undocumented--it exposes a broad assortment of information that should be provided in a more accessible format. Perhaps the complexity of DQPS is also why PSTAT has been one of the least reliable and most visually crude OS/2 utilities (for example, compare the output from PSTAT /L to the output from PSTAT /P:xx, where xx is the PID of the first instance of the process PMSHELL.EXE loaded in the system: PSTAT /L indicates fewer DLLs loaded for PMSHELL than does PSTAT /P).
Using the DQPS API: PROCINFO.EXE
My PROCINFO.EXE utility shows how to use the contents of the DQPS buffer. PROCINFO accepts either the ID or the name of an OS/2 process on the command line and displays information regarding any matching process's usage of system resources. PROCINFO was developed at the suggestion of an OS/2 developer who was trying to see if an instance of a given program was already running on an OS/2 system.
The PROCINFO utility starts by obtaining command-line information, allocating a buffer of size DQPS_BUFSIZE, and then calling DosQProcStatus. Upon successful return from the API, p16Buf fills with system-process and resource information. The first few bytes of the buffer are cast to a pointer to a DQPS pointer record, a structure that in turn contains pointers to the various lists of information represented in the buffer. The structure type is named qsPtrRec_t, and it defines five important members (see the excerpt from DOSQPROC.H in Listing One), which are 32-bit pointers to the beginning of the other sections of the buffer.
The first member of qsPtrRec_t, pGlobalRec, points to a "global" data record containing system-wide information. The next member, pProcRec, points to the process-data section, the heart of the DQPS API. The pointers p16SemRec, pShrMemRec, and pLibRec each point to lists of system resources--the 16-bit system semaphores list, the named shared-memory-segment list, and the executable-module list.
DOSQPROC.TXT contains a serious error in its definition of qsPtrRec_t. It defines only the five significant structure members, omitting the 4-byte value occurring between the p16SemRec and pShrMemRec members of the structure. This is a source of confusion for first-time users of DOSQPROC.TXT; an attempt to use this erroneous type definition will result in an improper initialization of the pShrMemRec and pLibRec structure members. The type definition of qsPtrRec_t in Listing One modifies IBM's definition by adding the necessary new structure member, VOID*Reserved.
DQPS Process Data
The process data records referenced by the pProcRec member of qsPtrRec_t are part of a block of information regarding the system processes and threads active at the time of the call to DQPS. Each process record provides detailed information about the process's status and lists some types of resources that it is using. These lists are actually arrays of indexes or handles into the resource lists found in the other sections of the buffer, allowing the developer to examine system-resource usage on a per-process basis. Figure 3 shows a sample process that includes two threads, a 16-bit semaphore, and three loaded DLLs. The process has mapped one shared-memory segment.
Information on threads owned by the process is available via a pointer to an array of thread data records. The number of thread data records in the array is given by the thread control-block count member of the process data structure.
One of the key uses of DQPS is to determine the name of a process given a PID or, conversely, the ID of a process given an executable name. Each process has an associated module table entry in the executable module data-record list, and the handle (HMTE) of the entry in that list is found in the process data record. The name of the executable that corresponds to the process data record can be determined by cross-referencing the process record's module-table handle with the entries in the executable module data-record list.
PROCINFO obtains the pointer to the block of process data records from the DQPS pointer record and performs a quick sanity check to make sure that it doesn't point to NULL. Depending on the type of request made on the command-line, PROCINFO will traverse the list of process records looking for either a particular process ID or a process having a certain name.
When it finds a process record indicating that a process matches the command-line specification, PROCINFO calls DisplayProcessInfo to display system-resource information regarding that process. This information includes: the full path specification of the executable that the process is running; the process ID and parent PID; the type and status of the process; and the number and characteristics of the threads, semaphores, DLLs, and named shared-memory segments in use by the process.
In addition to process information, you can also examine the individual resources a process is using. Sandwiched between the last member of the process data structure and the first member of the initial thread data structure may be one or more arrays of indexes or handles into the DQPS buffer's semaphore, library module, or shared-memory record lists. To determine the resource associated with a particular process-resource index or handle, traverse the corresponding resource list and examine successive nodes until the matching resource is found. PROCINFO performs these operations in its Display-Semaphores, DisplayLibraryRecords, and DisplaySharedMemory functions.
DQPS can also be used to determine the type or status of external processes. For instance, the OS/2 DosGetInfoBlocks API reports process type and status only for the process calling the API. DQPS reports the status and type of all system processes.
The type member of qsPrec_t will have the same value that the pib_ultype member of the PIB struct returned by DosGetInfoBlocks would have if the corresponding process called DosGetInfoBlocks. I have followed the qsPrec_t type definition in DOSQPROC.H with macros defining the known process-type values (see Listing Two).
IBM's DOSQPROC.TXT lists the possible values for the stat member (the process status flag) of the qsPrec_t struct. This process status flag will have the same value that the pib_flstatus member of the PIB struct returned by DosGetInfoBlocks would have if the corresponding process called DosGetInfoBlocks. I have followed the qsPrec_t type definition in DOSQPROC.H with macros defining the known process status-flag values (see Listing Two). Any application using the process-record status flag should treat the value in the field as a collection of bit flags. PROCINFO utilizes the type and stat members of qsPrec_t to display additional detail concerning the designated process.
DQPS Thread Data
If you've written multithreaded or time-critical applications, you know the importance of managing thread priority in an OS/2 system. Even established software from large companies is vulnerable to the pitfalls of poor thread management. Borland's Brief, for example, controls mouse operations through a low-priority thread. Borland's implementation does not handle thread starvation, and the result is Brief's erratic mouse performance and its failure to terminate completely in certain situations.
DQPS allows developers to easily recognize these situations and respond to them. With this power comes responsibility--altering the behavior of OS/2's task scheduler by modifying application thread priorities can have a negative impact on overall system performance. Of course, DQPS can also be used to observe system thread priorities, and this is often sufficient for debugging multithreaded applications. PROCINFO is useful for this purpose; it takes the information found in DQPS thread data records and displays the characteristics of a given process's threads.
The pThrdRec member of a process record (qsPrec_t) points to a structure of type qsTrec_t, the thread data record. The thread data records maintain information specific to individual threads, including their priorities. Each process has one or more threads associated with it, and the process's associated thread records follow the process record in the DQPS buffer. Accessing thread records is therefore process dependent; you must traverse the DQPS buffer process by process to access each of the system's thread records. PROCINFO's DisplayThreads function displays each of the threads associated with the passed process record, demonstrating how you can dynamically determine the status and priority of any thread in an OS/2 system.
The thread records following a process record can be referenced as elements in an array. The first member of a thread record, RecType, is always 0x100, and is not really useful. The final three bytes of the structure are pad bytes, probably to provide DWORD alignment.
The slot member of qsTrec_t has the same value that the tib_ordinal member of the TIB struct returned by DosGetInfoBlocks would have if the thread described by the qsTrec_t struct were to call DosGetInfoBlocks. The same relationship exists between the priority member of qsTrec_t and DosGetInfoBlocks' TIB2 member, tib2_ulpri. IBM's DOSQPROC.TXT lists three possible values for the state member of qsTrec_t; see Listing Two.
DQPS Semaphore and Named Shared-Memory Segment Data
Many OS/2 applications require semaphores or shared memory. PROCINFO reveals how you can use DQPS to track the names of, and references to, 16-bit system semaphores and named shared-memory segments.
The 16-bit system semaphore-detail data records (qsS16rec_t) and named shared-memory-segment data records (qsMrec_t) are both arranged as linked lists in the buffer returned by DQPS. Each node in the list of semaphore records carries the specifics of an associated semaphore, and the nodes in the shared-memory list likewise detail their associated, named shared-memory segments. The pointer record's pShrMemRec member points directly to the list of shared-memory records, but the p16SemRec member of the pointer record points not to the list of semaphore-detail records, but rather to the 16-bit system semaphore-header record that immediately precedes it. A pointer to the list of semaphore-detail records is constructed using pointer arithmetic to increment the pointer record's pointer to the semaphore-header record by one and by casting that result to type qsS16rec_t (see Listing Three).
PROCINFO.C shows the relationship between the SEMINDEX indexes and HMEM handles that follow a process data record and individual semaphores and named shared-memory segments. The ShowSemaphore function runs through the linked list of semaphore records, stopping at the record whose ordinal position matches the passed SEMINDEX parameter. ShowSharedMemorySegment similarly finds desired shared-memory information by looking at the shared-memory segment records in the list and comparing each record's HMEM member to the one requested.
DQPS Executable-Module Data
The executable-module data structures (also called "library records" or "module-table entries"--MTEs) in the DQPS buffer contain system-resource summary information describing OS/2 executable modules. Examples of OS/2 executable modules are executables, DLLs, device drivers, and font files. Each executable-module data structure contains a pointer to the full pathname of the corresponding module, the module's module-table entry handle (HMTE), and a count of the number of external modules directly referenced by this module. An array of MTE handles enumerating the directly referenced modules immediately follows the library record. The module's pathname pointed to by the pName structure member follows the HMTE array. A library record has, as its first member, a pointer to the next executable-module data record in the list.
The executable-module data records maintain many useful bits of information. The most valuable use of the linked list is to cross-reference its HMTE member with the MTE handle found in a process data record. This determines the full pathname of a given process and the names of each of the executable modules loaded by the process. An application can also traverse the chain of modules statically referenced by a process by examining the array of module handles that follows each relevant library record, determining which DLLs or fonts the process loads.
Executable-module data records have other valuable properties. Rick Fishman's PROCSTAT.H file (from the KILLEM.ZIP archive in Library 1 of CompuServe's OS2DF1 Forum) indicates that the first reserved field of the qsLrec_t structure equals 0 when 16-bit modules are referenced by a given executable-module record, and 1 when 32-bit modules are referenced. I've adopted Rick's name for this member of the structure: usModType. PROCSTAT.H also informs you that the second reserved field is a count of the number of segments in the executable module associated with the structure (as can be verified with EXEHDR). I call this member of the qsLrec_t structure ctSegments.
PROCINFO's ShowLibraryRecord and GetModuleName use the relationship between the HMTE field in a process record and the handles of records in the executable-module list. PROCINFO uses GetModuleName to find the name of a given process by taking the associated process record's MTE handle and traversing the linked list of library records until a record with the matching HMTE is encountered. This returns the name of the executable loaded by the process, pointed to by the pName field of the library record.
Using DQPS in Your Applications
Although IBM prefers that you not use DQPS in OS/2 applications, you sometimes don't have a choice. Some environments demand supervisory control of processes and system resources, and the documented OS/2 API does not address this requirement. IBM's own use of DQPS underscores this point. It is true that if you use undocumented functions, you risk having to scramble to fix broken applications when new versions of operating systems are released, but this is often a trivial concern. Ask someone porting an application from OS/2 1.x to OS/2 2.x, or someone moving a Win16 app to Win32, which set of functions underwent the greatest degree of change: the undocumented functions or the documented? In any case, remember that you, not the operating-system vendor, are in the best position to decide what is best for your particular application.
Acknowledgments
I would like to thank Jon Wright, Howard Kapustein, Rick Fishman, Wayne Kovsky, Dwayne Nebeker, and Jim Masse for their assistance with this article.
Other Undocumented OS/2 Areas
There are other undocumented aspects of OS/2 besides the DosQProcStatus API. What follows is a list of several of the more prominent undocumented functions or features.
- DosGetSTDA. OS/2's system-trace facility API. Potentially useful for debugging and profiling tools.
- DCF (Data Collection Facility). DCF is somehow connected with APIs that allow performance-monitoring applications to get system statistics and measurements directly from the OS/2 kernel. IBM's SPM/2 (System Performance Monitor/2) and a number of third-party applications use these APIs.
- DosReplaceModule. When an OS/2 executable module is in use, OS/2 locks the file, preventing the file from being replaced or deleted. The DosReplaceModule API allows the on-disk replacement of an existing module with a new module, while the system continues to run with the old one. The function is located at DOSCALLS.417, and the prototype is: APIRET APIENTRY DosReplaceModule(PSZ pszOldModule, PSZ pszNew-Module, PSZ pszBackupModule). The contents of pszOldModule are cached by the system, the file is closed, and a backup copy of the file, pszBackupModule, is created for recovery purposes, should the routine performing the module replacement fail. The new module pszNewModule then takes the place of the original module on the disk. Calling the function and specifying only the first parameter makes it possible to delete or copy the indicated module. Apparently, DosReplaceModule is the routine that IBM's display-driver installation program (DSPINSTL) uses to replace or upgrade OS/2 display drivers.
- IFS/HPFS. Installable file systems, an important part of OS/2, are largely undocumented.
- INF/HLP file format. It is widely known that OS/2 .HLP and .INF files differ by only one byte. Why is it that the OS/2 view facility works with .INF files but not with .HLP files? The format of these files is not documented.
- DosQueryTmr. This is supposedly an API that allows higher-resolution timing than that provided by documented OS/2 APIs.
- Presentation Manager window messages. Undocumented window messages abound among PM applications. For instance, WM_CANCELMODE (0x005A) is sent to a window when it is "usurped" by the Window List or by other system pop-ups. Responding to this message with an (MRESULT)TRUE will disable the Ctrl-Esc and Alt-Esc hot-key combinations for that window, and responding to a subsequent WM_CANCELMODE message with (MRESULT)FALSE will reenable the hot keys.
- WinSetErrorInfo. The online programming reference for the OS/2 WorkPlace Shell describes wpclsSetError, a method "analogous to the WinSetErrorInfo function that is used by Presentation Manager functions to log their error return codes." No mention is made of the WinSetErrorInfo API in IBM's Presentation Manager documentation.
- Calling DosSleep from a DOS application. Example 1(a) shows an undocumented way for DOS applications running in an OS/2 VDM to yield a time slice or sleep for a specified number of milliseconds.
- WinStretchPointer. This function is intended to allow 32x32 or smaller pointer bitmaps to be stretched or shrunk to new sizes. The function's entry point is at PMWIN.968, and the function prototype is similar to the prototype for the documented API WinDrawPointer, with the addition of parameters specifying the destination bitmap height and width; see Example 1(b). Rick Fishman adds that his tests indicate that "if you have a mini-icon as one of your formats [passed in the HPOINTER parameter], WinStretchPointer will always draw the mini-icon. If not, it will stretch or compress the pointer." Rick also points out that this function is not entirely undocumented; it appears in the OS2386.LIB import library that comes with the IBM OS/2 2.1 Toolkit.
Example 1: (a) Calling DosSleep; (b) function prototype for WinStretchPointer.
(a)
xor dx,dx
mov ax,woMilliSecs ; Number of milliseconds in DX:AX.
; A value of 0 means that the current
; timeslice is released.
hlt ; Trigger OS/2's exception manager.
db 35h,0CAh ; Signature to differentiate between a
; normal HLT instruction and the call
; to DosSleep().
(b)
BOOL APIENTRY WinStretchPointer(
HPS hps,
LONG x,
LONG y,
LONG cx,
LONG cy,
HPOINTER hptr,
ULONG fs);
Figure 1: DosQProcStatus prototype.
#ifdef __BORLANDC__ /* OS/2 2.x prototype with Borland C++ 1.0, 1.01 _far16 * semantics */ APIRET16 APIENTRY16 DosQProcStatus(VOID _far16 * p16Buf, USHORT cbBuf); #else /* typical OS/2 2.x prototype */ APIRET16 APIENTRY16 DosQProcStatus(PVOID p16Buf, USHORT cbBuf); #endif /* suggested buffer size for DosQProcStatus */ #define DQPS_BUFSIZE 0xFFFF
Figure 2 Contents of buffer returned by DosQProcStatus. Figure 3 Sample process record and associated data structures.
Listing One
/* Excerpt from DOSQPROC.H. Complete header file available electronically. */
/* VOID * p16Buf = NULL;
* USHORT cbBuf = DQPS_BUFSIZE;
* qsPtrRec_t * pPtrRec;
* DosQProcStatus(p16Buf,cbBuf);
* pPtrRec = (qsPtrRec_t *)p16Buf;
* // ...
*/
typedef struct qsPtrRec_s
{
qsGrec_t * pGlobalRec; /* ptr to the global data structure */
qsPrec_t * pProcRec; /* ptr to process data list */
qsS16Headrec_t * p16SemRec; /* ptr to 16 bit system sem list */
VOID * Reserved; /* always NULL - see comments above */
qsMrec_t * pShrMemRec; /* ptr to shared memory seg list */
qsLrec_t * pLibRec; /* ptr to module table entry list */
} qsPtrRec_t;
Listing Two
/* Excerpt from DOSQPROC.H. Complete header file available electronically. */ /* process 'type' definitions */ #define PT_FULL_SCREEN 0 #define PT_DOS_OR_WINOS2 1 /* kernel (SYSINIT) process */ #define PT_WINDOWED 2 /* OS/2 windowed session */ #define PT_PM 3 /* Presentation Manager */ #define PT_DETACHED 4 /* process status definitions */ #define PS_IN_EXITLIST 0x01 #define PS_EXITING_THREAD_1 0x02 #define PS_PROCESS_EXITING 0x04 #define PS_TERMINATION_AWARE 0x10 #define PS_PARENT_EXEC_WAIT 0x20 #define PS_DYING 0x40 #define PS_EMBRYONIC 0x80 /* thread status definitions */ #define TS_READY 1 #define TS_BLOCKED 2 #define TS_RUNNING 5
Listing Three
/* Constructing a pointer to the list of semaphore detail records. Excerpt from DOSQPROC.H */ qsPtrRec_t * pPtrRec; qsS16rec_t * pSemRec; // ... pSemRec = (qsS16rec_t *)(pPtrRec->p16SemRec + 1);
Copyright © 1994, Dr. Dobb's Journal
