Extending HTTP Servers with DGI and TGI
Avoiding CGI's overhead
Andrew Montefusco
Andrew is a C/C++ programmer and analyst currently working for an IBM Semea subsidiary in IBM's Rome, Italy, Network Labs. He can be contacted via http:// www.montefusco.com.
Most HTTP servers allow their functionality to be extended with new services. The most widespread standard for this type of extension is the Common Gateway Interface (CGI), which runs external programs to process certain HTTP requests. However, starting a new task for every request requires a significant amount of overhead, especially for sophisticated extensions. For example, some extensions access relational databases, and CGI requires the database connection to be opened and closed for each new request.
To reduce this overhead, I've implemented two alternative approaches in my HTTPD2 server for OS/2. The Task Gateway Interface (TGI) starts the extension as a separate program but leaves it running continuously rather than restarting it for each new request. The Dynamic Gateway Interface (DGI) loads the extension as a DLL (if it is not already loaded) and calls callback functions to handle client requests.
The only constraint of the current OS/2 implementation is that TGI extensions must be started after the HTTP server task. Other implementations may not require this (TGI for Apache, for example, is insensitive to the starting order of the server and extensions). Of course, if the server receives a request for a URL whose extension task is not running, the client receives an error 11 (TGI_MOD_NOT_FOUND). Both approaches avoid the overhead of starting a new process for each separate request.
Task Gateway Interface (TGI)
Like CGI, a TGI extension module is a process external to the server program. Unlike CGI, a separate process is not started for each new request. Rather, the server uses interprocess-communication methods to communicate with the extension process. Figure 1 shows how this arrangement works. Although my OS/2 server uses shared memory, the details are hidden from programmers by a C/C++ API. This API, which can be implemented differently for different platforms, makes the program portable to any platform that supports a C/C++ programming environment and an HTTP TGI server.
Each TGI extension handles requests for a particular URL. Normally, each TGI program can handle only one request at a time. However, any number of different TGI programs can be running simultaneously, each responding to a different URL. If necessary, multiple programs or a single program with multiple threads can be used to handle multiple, simultaneous requests for a single URL.
Listing One defines the TGI API. A TGI program registers itself with the HTTP server when it starts, including the address of a suitable callback procedure. The server calls this procedure when it receives a request for the corresponding URL.
For instance, if the HTTP server receives a request for http://www.roma.ccr .it/ tgi/es1, it would call a callback function registered under the name "es1." Usually, the function will then feed a data stream to the server to satisfy the request.
The extension program registers with the server by calling the TgiRegisterModule function in the TGI library. This function takes the name of the service (es1 in the previous example), a pointer to the callback function, and a time-out interval specifying how long to wait for the server to respond. The time out is useful when a large number of TGI extension programs are being started simultaneously. TgiRegisterModule returns a handle to be used by later TGI requests. If a single program provides multiple extensions, TgiRegisterModule will need to be invoked once for each separate extension.
After the registration is complete, the extension program calls TgiProcessModule to handle requests for that service. TgiProcessModule requires the handle returned by the registration process and a pointer to callback data. Each call to TgiProcessModule sets up a single service for the corresponding URL. To allow multiple, simultaneous requests for that URL, TgiProcessModule will need to be called several times (from different threads or different programs).
TgiProcessModule is synchronous; it receives and dispatches requests from the server until the server terminates or the callback function calls TgiDeregisterModule. When TgiProcessModule does return, the TGI extension program should clean up and terminate.
TGI Callback Function
The callback function is called by the server whenever the corresponding URL is requested. Generally, the callback function synthesizes an appropriate document to satisfy the request. This document must be a complete HTTP response, including appropriate headers.
The callback function has the type PFNACTION, defined in Listing One. The first parameter is a handle identifying this particular request, the second parameter is the callback data pointer provided to TgiProcessModule.
To satisfy the request, the callback function may need to obtain additional information from the server, send the HTTP response back to the server, and interact with the server in other ways.
To get additional information about the request, the extension program can call TgiGetPathInfo, TgiGetQueryInfo, or TgiGetFormData with the corresponding request handle. These functions return the URL path, the URL query information, or the form data contained in an HTTP POST message, respectively.
To respond to the request, the extension program calls TgiPutHtml with a request handle, a pointer to data to be returned, and the length of the data. This function can be used equally well for binary or text data, but it will usually be more convenient to use the wrapper function shown in Example 1 to handle text data. This function is called like a normal fprintf(), with the request handle instead of a FILE handle.
TGI Example
Listing Two is tgiex1.c, a complete TGI program example. When called by the server, it synthesizes a short page containing the input information, the time the program started, and the current time.
Because the program registers only one instance of the "example_1" extension, only one request can be handled at a time. To verify this, you can uncomment the sleep() function call on line 102, then open two browser windows on the same URL (/tgibin/example_1). The second requester will receive the message "Error 17 (no free instance found) in runtgi.cpp at 74."
The file tgiex2.c (available electronically) is an example TGI program that registers multiple instances, so that it can handle multiple requests at once. This is essential for time-consuming TGI callback functions such as database searches.
The program reads the number of instances to activate from the command line and registers the extension module with the name "example_1". It allocates an array of THREAD_PARAM structures and registers one instance in each thread. Notice that the call back function code is shared by all instances. This function must be reentrant; updates to static or global variables will need to be carefully managed.
TGI Implementation
The OS/2 version of TGI is implemented as a DLL. The source is also available electronically. The library exports both TGI API functions and the functions the server uses to manage extension modules. The server functions have names beginning with an underscore.
The tgiglob.c module has a shared data segment. This means that only one instance of this segment is created in memory, and all the tasks using the DLL share this area. Access to this area is established through the hGlobMutex semaphore, which is managed by the _LockData and _UnLockData functions.
The tgi.c module has a separate data segment for every task that uses it. Among the other variables is the
hLocMutex handle used to access the hGlobMutex semaphore. The global semaphore is created in the DLL initialization phase by the HTTP server task. Other tasks must call DosOpenMutexSem before trying to use this semaphore. Because the server task created the semaphore, it's not strictly necessary for it to reopen the semaphore this way, but it does keep things simple.
The shared data segment uses two tables to store registration information. Every time a module registers itself via the TgiRegisterModule function, the DLL stores this information in a free position in the modTab table. Notice that the callback function pointer stored in this table is only valid in the registering task. (The module registration is also stored in a local table to simplify deregistering all modules in a particular task.) When the TGI task calls the TgiProcessModule function, the DLL allocates a _PROC_INSTANCE structure in the instTab table and adds it to a linked list of instances of each module. The function then enters a loop waiting for activation.
Figure 2 shows how these data structures work. This figure shows an extension "example_1" with only one instance and another extension "example_2" with two instances. Notice that the callback function itself is actually shared by different instances.
When the HTTP server receives a request for a TGI extension, it calls UnLockModuleInstance with the extension module name and a pointer to a TGI_DATA structure containing data about the specific request. The data is communicated to the separate thread through a shared memory segment. Note that the separate thread is the one that called the TgiProcessModule in the extension task.
The _UnLockModuleInstance function finds the module entry with the correct name and searches the instance list to find one not already in use. The instance thread is unlocked with a post to the semaphore. Now, in the context of the task that registered the instance, the _LockOnRequest function returns a TGI_OK value, which prompts _DoAction to call the callback function. The task server thread waits in MbxGetMsg for data from the extension task. After the callback returns, the _DoAction function sends a zero-length message, which indicatesthe end of the data and guarantees the server thread will be unlocked.
Dynamic Gateway Interface (DGI)
DGI extensions are DLLs loaded directly into the server. The advantage of this approach is that it is faster to load a DLL than a new program. It's also possible to cache commonly used extensions in memory.
For example, if a client requests the URL http://www.roma.ccr.it/dgibin/dgiex1.dll, the server loads the DLL dgiex1.dll. It then calls DgiStart to initialize the DLL, DgiDoAction to handle this request, then DgiEnd to deinitialize the DLL prior to unloading.
Listing Three is the DGI API. DgiStart is called each time the DLL is loaded. It is called with pointers to several utility functions that must be used to call utility routines in the server. It is responsible for any special initialization needed by this DLL.
DgiDoAction is called once for each request for this URL. It is called with pointers to the various data associated with the request (for example, the URL and any associated form information). DgiDoAction parses the request and calls putHtml (using the pointers given to DgiStart) to emit the response.
DgiEnd is called once just before the DLL is removed from memory. In my initial implementation, DLLs are loaded and unloaded for each request; so, in fact,DgiStart and DgiEnd are called once per request. However, servers may cache DLLs, leaving them in memory to handle multiple requests. In that case, it is very important to properly separate the initialization, request processing, and deinitialization.
As you can see in Figure 3, DGI handles multiple requests more easily than does TGI. The server can have calls to several DgiDoAction functions pending at the same time in different server threads.
DGI Examples
To be recognized as DGI, the DLL must have certain entry points. The prototypes for these entry points are declared in the DGI.H file.
Listing Four is dgitest.cpp, an example DGI extension. When the client requests the URLhttp://localhost/ dgibin/dgitest1.dll, the module dgitest1.dll is loaded. The DgiDoAction function synthesizes a document containing the current date and time.
The dgisrc.cpp DLL (also available electronically) is a more-complex example. This DLL is associated with a document containing an ISINDEX tag. If a user fills the entry field with *.cpp, the client requests the URL http://localhost/dgibin/dgisrc.dll?*.cpp. The server then invokes dgisrc.dll, which searches a particular directory for all matching files and synthesizes a simple HTML stream containing the file list.
DGI Implementation
My HTTPD2 server for OS/2 has a simple DGI implementation that doesn't attempt to cache DLLs. Listing Five is part of loadext.cpp, the server code that supports DGI. This code is very OS/2 specific; porting requires adapting this code to the run-time linking services provided by the system.
The first operation in ClientAgent::LoadExt is loading the correct DLL. It then finds the address of the functions exported by the DLL and calls each one accordingly.
The server cannot check for errors in the extension (except for the presence of the correct function names). As a result, the module developer is responsible for ensuring the extension won't crash the server. My implementation takes one safeguard: It registers an exception handler to intercept possible fatal exceptions. This limits the effect of an error in a DGI DLL to termination of the thread, not of the entire server.
Example 1: Wrapper for simple text output.
#include <stdio.h>
#include <stdlib.h>
#include <stdarg.h>
int htPrintf (HREQUEST hReq, const char *fmt, ...)
{
char szBuf[1024];
va_list argptr;
int cnt;
va_start(argptr, fmt);
cnt = vsprintf(szBuf, fmt, argptr);
va_end(argptr);
TgiPutHtml (hReq, szBuf, (long)cnt);
return(cnt);
}
Figure 1: Tasks involved in TGI scheme.
Figure 2: Data structure maintained by tgi.dll.
Figure 3: Tasks involved in DGI scheme.
Listing One
/* Task Gateway Interface v1.00 -- (C) A.Montefusco 1995 */
#if !defined(__TGI_H__)
#define __TGI_H__
#if defined(__cplusplus)
extern "C" {
#endif
typedef void * HTGI;
typedef void * HREQUEST;
typedef long (*PFNACTION) (HREQUEST, void *) ;
long TgiRegisterModule (char *name,PFNACTION pFn, HTGI *pHTgi, long timeout);
long TgiProcessModule (HTGI hTgi, void *pCbData);
long TgiDeregisterModule (HTGI hTgi );
long TgiGetPathInfo ( HREQUEST hReq, char **pszPi );
long TgiGetQueryInfo ( HREQUEST hReq, char **pszQi );
long TgiGetFormData ( HREQUEST hReq, char **pszFd );
long TgiSetRedirFile ( HREQUEST hReq, char *pszRf );
long TgiPutHtml ( HREQUEST hReq, char *buf, long len );
char *TgiStrError ( long tgiErr );
#define TGI_OK 0
#define TGI_EXIT 1
#define TGI_UNLOCK 2
#define TGI_NOT_RUN 3
#define TGI_LOC_OPEN_SEM 4
#define TGI_REQUEST 5
#define TGI_ALREADY_INIT 6
#define TGI_LOC_RELEASE_SEM 7
#define TGI_TOO_MANY_MOD 8
#define TGI_BAD_PARAM 9
#define TGI_UNLOCK_OK 10
#define TGI_MOD_NOT_FOUND 11
#define TGI_BAD_INSTANCE 12
#define TGI_SEM_CREATE 13
#define TGI_UNLOCK_ERROR 14
#define TGI_MOD_ALREADY_REG 15
#define TGI_INST_NOT_FOUND 16
#define TGI_MODULE_BUSY 17
#define TGI_NOT_INIT 18
#define TGI_NO_MEM 19
#define TGI_MLBX_OPEN 20
#define TGI_MLBX_SEND 21
#define TGI_FATAL(rc) { if (rc != TGI_OK) { \
fprintf (stderr, "Error %d (%s) in %s at %d\n", \
rc, TgiStrError(rc), __FILE__, __LINE__ );\
exit (rc); \
} \
}
#if defined(__cplusplus)
};
#endif
#endif
Listing Two
/* Task Gateway Interface -- A. Montefusco
* TGI program with only one instance.
*/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stdarg.h>
#include <time.h>
#include <dos.h>
#include "tgi.h"
/* htPrintf printf-like function emits HTML output stream from TGI callback */
int htPrintf (HREQUEST hReq, const char *fmt, ...)
{
char szBuf[1024];
va_list argptr;
int cnt;
va_start(argptr, fmt);
cnt = vsprintf(szBuf, fmt, argptr);
va_end(argptr);
TgiPutHtml (hReq, szBuf, (long)cnt);
return(cnt);
}
/* HTML 2.0 compatible header... */
static char szHtmlHeader [] =
"Content-type: text/html\r\n\r\n"
"<html>"
"<head><title>Returned from TGI executable !!! </title></head>"
"<body>"
"<h1>TGI TEST</h1>"
"<p>Returned from Task Gateway Interface executable." ;
/* ... and trailer */
static char szHtmlTrailer [] = "</body></html>" ;
/* TGI callback function. hReq parameter must be used in all following calls
* to TGI API: pCbData points to optional extension's data. This callback sends
* to client a simple screen containing the request's data */
long CbExample1 (HREQUEST hReq, void *pCbData)
{
printf ("Callback: recalled with: %p *p\n", (void *)hReq, pCbData);
TgiPutHtml (hReq, szHtmlHeader, strlen(szHtmlHeader));
{
char *pszPi;
char *pszQi;
char *pszFd;
htPrintf (hReq, "<p><p><pre>");
TgiGetPathInfo ( hReq, &pszPi ) ;
TgiGetQueryInfo ( hReq, &pszQi ) ;
TgiGetFormData ( hReq, &pszFd ) ;
htPrintf (hReq, "PathInfo : %s<p>", pszPi);
htPrintf (hReq, "QueryInfo : %s<p>", pszQi);
htPrintf (hReq, "FormData : %s<p>", pszFd);
htPrintf (hReq, "<p></pre>");
}
/* Remove remark tokens from the following to simulate a lengthy operation*/
/* sleep (10); */
{
time_t timeNow = time(0); // request start timestamp
time_t timeStart = *((time_t *)pCbData); // program start timestamp
htPrintf (hReq, "<p>Run @ %s<p>", asctime (localtime(&timeNow)));
htPrintf (hReq, "<p>Tgi module start @ %s<p>",
asctime (localtime(&timeStart)));
}
TgiPutHtml (hReq, szHtmlTrailer, strlen(szHtmlTrailer));
return TGI_OK;
}
int main (void)
{
long rc;
HTGI hTgi; // handle of the TGI extension
char *pszModName = "example_1"; // TGI extension name
time_t timeStart; // program start timestamp
timeStart = time(0);
rc = TgiRegisterModule ( pszModName, CbExample1, &hTgi, 0 );
TGI_FATAL(rc);
rc = TgiProcessModule ( hTgi, &timeStart );
TGI_FATAL(rc);
rc = TgiDeregisterModule ( hTgi );
TGI_FATAL(rc);
return 0;
}
Listing Three
/* Dynamic Gateway Interface v1.00 -- (C) A.Montefusco 1995 */
#ifndef __DGI_H__
#define __DGI_H__
#ifdef __cplusplus
extern "C" {
#endif
void DgiPutHtml (char *pszBuf, long len, void *pPrm);
const char *DgiGetData (char *pszParam, void *pPrm);
typedef void (* PDGI_FN_OUTMSG)(char *, long, void *);
typedef const char *(* PDGI_FN_GETDATA)(char *, void *);
long DgiStart (PDGI_FN_OUTMSG pFnOutMsg, PDGI_FN_GETDATA pFnGetData,
void *pCbData);
typedef long (* PDGI_FN_START)(PDGI_FN_OUTMSG, PDGI_FN_GETDATA, void *);
long DgiDoAction (const char *pszQueryInfo, const char *pszPathInfo,
const char *pszFormData, char *pszRedir);
typedef long (* PDGI_FN_DOACTION)(const char*,const char*,const char*,char*);
long DgiEnd (void);
typedef long (* PDGI_FN_END)(void);
extern PDGI_FN_OUTMSG pFnOut;
extern PDGI_FN_GETDATA pFnGetData;
extern void *pCbPrm;
#ifdef __cplusplus
};
#endif
#endif
Listing Four
/* Dynamic Gateway Interface -- A. Montefusco */
#include <time.h>
#include <string.h>
#include <iostream.h>
#include "dgi.h"
#include "dgihelp.h"
#if defined(__OS2__) && defined(__BORLANDC__)
#define EXPORT _export
#else
#define EXPORT
#endif
/* HTML 2.0 compatible header... */
#define HTML "Content-type: text/html\r\n\r\n" \
"<html>" \
"<head><title>Returned from DGI executable !!!</title></head>" \
"<body>" \
"<h1> " \
"DGI TEST " \
"</h1> " \
"<p>Returned from Dynamic Gateway Interface executable. "
/* ...and trailer */
#define HTML_TRAILER "</body>" \
"</html>"
PDGI_FN_OUTMSG pFnOut;
PDGI_FN_GETDATA pFnGetData;
void *pCbPrm;
long EXPORT DgiStart (PDGI_FN_OUTMSG pFnOm, PDGI_FN_GETDATA pFnGd, void *pPrm)
{
cerr << "DgiStart" << endl;
pFnOut = pFnOm;
pFnGetData = pFnGd;
pCbPrm = pPrm;
return 1;
}
long EXPORT DgiDoAction (const char *pszQueryInfo, const char *pszPathInfo,
const char *pszFormData, char *)
{
cerr << "DgiDoAction:" << endl
<< "Query info:\t[" << pszQueryInfo << "]"
<< endl
<< "Path info:\t[" << pszPathInfo << "]"
<< endl
<< "Form data:\t[" << pszFormData << "]"
<< endl;
htPrintf ("%s", HTML);
time_t timeNow = time(0);
htPrintf ("<p>Run @ %s GMT+1<p>", asctime (localtime(&timeNow)));
htPrintf ("Remote host: <b>%s</b><p>", pFnGetData ("REMOTE_HOST", pCbPrm));
htPrintf ("%s", HTML_TRAILER);
return 1;
}
long EXPORT DgiEnd (void)
{
cerr << "DgiEnd" << endl;
return 1;
}
Listing Five
int ClientAgent :: LoadExt (const string &sModName, const string &sQuery,
const string &sPathInfo, const string &sFormData)
{
APIRET rc;
char szObjNameBuf [128];
HMODULE hModHandle;
rc = DosLoadModule(szObjNameBuf, sizeof szObjNameBuf, sModName.c_str(),
&hModHandle);
if (rc != 0) { // load the module in memory
TRACE(cerr<<"Unable to load "<<sModName<<" module [" <<rc<< "]"<< endl;);
return rc;
}
/* module loaded, resolve functions address */
PDGI_FN_START fnStart;
PDGI_FN_DOACTION fnDoAction;
PDGI_FN_END fnEnd;
PFN pProcAddr;
rc = DosQueryProcAddr(hModHandle, 1, 0, &pProcAddr);
if (rc != 0) {
TRACE (cerr<< "Error [" << rc << "] addressing Start function" << endl;);
fnStart = 0;
} else fnStart = (PDGI_FN_START) pProcAddr;
rc = DosQueryProcAddr(hModHandle, 2, 0, &pProcAddr);
if (rc != 0) {
TRACE (cerr<<"Error [" << rc << "] addressing Action function" << endl;);
fnDoAction = 0;
} else fnDoAction = (PDGI_FN_DOACTION) pProcAddr;
rc = DosQueryProcAddr(hModHandle, 3, 0, &pProcAddr);
if (rc != 0) {
TRACE (cerr << "Error [" << rc << "] addressing End function" << endl;);
fnEnd = 0;
} else fnEnd = (PDGI_FN_END) pProcAddr;
TRACE(printf ("Start: %p\n", fnStart););
TRACE(printf ("Do: %p\n", fnDoAction););
TRACE(printf ("End: %p\n", fnEnd););
/* Invoke start function */
long retCode;
if ((retCode = fnStart(putHtml, getData, (void *)this)) != 0) {
TRACE (cerr << "Start OK" << endl;); // OK
} else {TRACE (cerr << "Start in error" << endl;);}
char szUrl [2560];
szUrl[0] = '\0';
if ( (retCode = fnDoAction (sQuery.c_str(), sPathInfo.c_str(),
sFormData.c_str(), szUrl ) ) != 0) { // OK
TRACE (cerr << "DoAction OK" << endl;);
} else {
TRACE (cerr << "DoAction in error" << endl;);
}
if ((retCode = fnEnd()) != 0) { // OK
TRACE (cerr << "End OK" << endl;);
} else {
TRACE (cerr << "End in error" << endl;);
}
DosFreeModule (hModHandle);
// check if external module has rquested redirection
if (strlen(szUrl)) {
if (strstr(szUrl,"http:")) {
char msg [512];
fprintf (stderr, "Returning 301 ERROR\n");
sprintf (msg, "HTTP/1.0 301 \"HTTPD/2 Server\"\r\n"
"Server: A.Montefusco 3.0 -- on OS/2 2.x+\r\n"
"Location: %s\r\n\r\n",
szUrl
);
pCn->Send (msg, strlen(msg));
} else
print (string(szUrl), TYPE_HTML);
}
return 0;
}
