Ann, a long-time Paradox developer, is cofounder of the Delphi Northbay SIG in Petaluma, CA. She can be contacted at [email protected].
When it comes to measuring traffic on the Internet, the numbers are sometimes mind boggling. Last year, traffic on the World Wide Web reportedly increased 1800 percent. By 1998, some forecasters are predicting 11.8 million Web users, while others estimate the Internet market will grow tenfold between 1994 and 1998. Clearly, the Internet can't be ignored.
But after you've come up with a business plan, studied books on HTML, rented disk space on an Internet-access provider's system, and launched your own World Wide Web home page, how do you determine if the system is effective? Currently, the predominant means of measuring home-page traffic is the number of "hits" you get over a given period of time. The graphics-intensive Web site for Playboy magazine, for instance, is racking up about 800,000 hits per day. Likewise, Wired magazine's HotWired site reportedly gets nearly half a million hits per day. Hits, however, can be a misleading metric because they depend on the graphical content of the page--each graphic counts as a hit. To get the number of actual users visiting a site, you need to divide the hits by a factor, say between 3 and 6, that depends on the characteristics of the site.
Furthermore, the average Internet provider is too busy to give you customized statistical information. You might be able to find out how the whole server is doing, or once a week, you might find out how your page is doing. Or, you could learn to analyze the server-log files to get an in-depth understanding of your traffic.
But how about a straightforward meth-od that lets you (and surfers) see how popular your page is? You've probably seen pages that say "You are guest #nnn on this page." A more accurate report, like that in Figure 1, might tell you that "there have been nnn requests for this page since mm/dd/yy hh:mm." Webmasters at these pages have installed a "traffic-counter" program that logs the number of individual accesses to a particular page. In this article, I'll present a minimal traffic counter that tracks and reports on user access; see Figure 2. This counter was built using components for Borland's Delphi. The resulting traffic.exe program runs on either of Bob Denny's web servers: WebSite (32-bit, for Win '95 or NT 3.5x, published by O'Reilly & Associates), or Win-HTTPD (16-bit, for Win 3.1x, shareware). Since it also uses the Borland Database Engine, IDAPI must be installed on the server. (If you are working on a UNIX server or have a different Web server, you might take a look at http://www.stars.com for other approaches in the public domain.)
A CGI Backgrounder
Before launching into specifics about my traffic counter, an introduction to CGI is in order. In this discussion, I'll refer to Robert Denny's win-cgi implementation (for WebSite) for illustrative purposes.
When a Web browser requests a static page from a WebSite server, that request is sent to the server using HTTP protocol, and the server responds with a document from its disk. To request a dynamic page, however, the browser sends a request and, based on the URL, the WebSite server realizes that it is a request for a CGI program. The server then runs that CGI program (WinExec() of an .EXE) and waits for it to finish. (Note that WebSite is multithreaded; this has obvious benefits.) The CGI program executes its instructions and feeds a response document back to the server. When the CGI program exits, WebSite sends the document back to the browser.
More specifically, under win-cgi, all the CGI environment data (browser name, surfer's IP address, referring page, authentication data, and the like) plus any HTML-form data (data requested on the referring page) is parsed by the server and stored in an .INI file in a temporary directory. When WebSite calls your CGI application, it passes the name of that temporary .ini file as the first command-line argument. Thus, your CGI application can orient itself to the current session by reading values from the .INI file.
In addition to all the standard CGI information, the WebSite server also adds a key=value statement in the .INI file, defining the name of the output file that your program should build. This will also be in the temporary directory. WebSite takes care of unique naming issues and file cleanup after the file is sent to the browser.
In short, the CGI program executes, sending its output to a prenamed, temporary output file. Generally, the output is a standard HTTP prologue followed by a blank line and some valid HTML syntax; see Example 1. The dynamic HTML page at my online classified-ads site (http://super.sonic.net/ann/forsale/) illustrates this. When you look up "Bicycles," my CGI application searches a Paradox table and displays an HTML 3.0 table displaying the "answer" set.
Returning to the Traffic Counter
When building an application like a traffic counter, you're faced with a number of challenges. For instance, you need to:
- Launch a CGI program in the middle of an otherwise static page.
- Output a graphical image instead of plain text/html.
- Track the hits.
- Make the graphical version of the longInt counter value.
- Do file I/O that works with Windows NT and Windows 3.x.
Outputting a graphical image instead of plain text/html was simply a matter of changing the "prologue" portion of the output file, to use Content-Type: image/gif and to send the actual .GIF data in the place of plain text. (Remember the blank line between prologue and content!)
How do you know where the hits are coming from? The CGI specification provides the environment variable Referer, which refers to the name of the calling page (unless the surfer typed in the URL to your CGI program directly, using no form at all). The only difficulty with tracking hits by Referer is that there can be many ways to address the same page. My server, for example, is named both "super.sonic.net" and "www2.sonic.net." My Web site is running at port 80, so a Referer could say "super.sonic.net:80/." Also, there are capitalization differences, and it's not necessary to enter the home-page name of index.html in order to land on that page (for example, super.sonic.net/ann/ is equivalent to super.sonic.net/ann/index.html). To deal with these issues, I decided to strip off the server name, append index.html if the Referer ends in "/", and change everything to lower case.
With Delphi, making the graphical version of the longInt counter value is relatively straightforward once you know the rules. Delphi provides a method, TextOut, which can "draw" text into a bitmap at any coordinate. It cannot "draw" a numeric value, so we convert the longInt count to a string. See Listing One (beginning on page 30) to see exactly how a counter bitmap is created.
Since Delphi doesn't have complete support for .GIF files, I converted .BMP data to .GIF format using Dan Dumbrill's BmpToGifStream shareware program (available at my site), which does graphical-file-format conversion, in memory, within Delphi. Once the .GIF image is in memory, it is fairly easy to append that data to the prologue in the output file.
Finding a way to do file I/O that works with Windows NT as well as Windows 3.x was the most time-consuming portion of building this application. Under Windows NT (not Windows 3.1), it is necessary to open the stdout file via stdout =TFileStream.create( stdoutname, fmOpenWrite );. (Under Windows 3.1, I used the fmCreate flag instead of fmOpenWrite.)
The complete source code to the traffic-counter application is available electronically from DDJ (see "Availability," page 3) and at my home page.
Figure 1: Page with a sophisticated traffic counter.
Figure 2: Low-volume traffic counter.
Example 1: Standard prologue and HTML code
HTTP/1.0 200 OK SERVER: _server name here_____LINEEND____ DATE: _date/time in GMT format_____LINEEND____ CONTENT-TYPE: text/html <HTML><HEAD><TITLE>A Sample Page</TITLE></HEAD><BODY> <h1>Welcome</h1> <hr><address>Goodbye</address> </BODY></HTML>
Listing One
unit Trafform;
{ TRAFFIC.EXE : Track web traffic counts on a page-by-page basis. This is a
16-bit application that will run under Windows 3.1x, Win 95 and Windows NT.
It has been tested extensively under Windows NT AS 3.5 and 3.51.
The program requires three components which can be downloaded at
http://super.sonic.net/ann/delphi/cgicomp/detail.html
TCGIEnvData (free, written by Ann Lynnworth )
TCGIDB ($39 shareware from Ann )
BMPGIF.dcu ($15, shareware written by Dan Dumbrill )
Usage: <img src="/cgi-win/traffic.exe">
See cgicomp/index.html for sample usage.
traffic.exe requires one DLL on the PATH of the server, to make the .gif
image: BIVBX11.DLL which ships with Delphi. It requires IDAPI installed
on the server (on the Delphi CD). It also requires a BDE Alias named
WebTrafficCounter, pointing to a directory which contains hit.db and hit.px.
This Paradox table (TableHit object) should be included in the .zip file you
found this source code in!
This program is Copyright c 1995 Ann Lynnworth. Permission is hereby granted
for any registered user of TCGIDB and BMPGIF to freely copy and/or modify
this program provided that these original credits are kept intact.
Suggestions should be mailed to [email protected] -- thank you.
}
interface
uses
SysUtils, WinTypes, WinProcs, Messages, Classes, Graphics, Controls,
Forms, StdCtrls, ExtCtrls, Cgidb, Cgi, DB, DBTables,
gifbmp; {written by Dan Dumbrill}
{ use TDebugControl from TPack if you want to trace the code }
const
wm_Traffic = wm_User;
type
TForm1 = class(TForm)
DataSource1: TDataSource;
TableHit: TTable;
CGIEnvData1: TCGIEnvData;
CGIDB: TCGIDB;
Image: TImage;
procedure FormCreate(Sender: TObject);
function makegif : TMemoryStream;
function getCount : string;
private
{ Private declarations }
procedure wmTraffic(var Msg: TMessage);
message wm_Traffic;
public
{ Public declarations }
end;
var
Form1: TForm1;
implementation
{$R *.DFM}
procedure TForm1.FormCreate(Sender: TObject);
begin
with CGIEnvData1 do
begin
websiteINIfilename := paramstr(1);
application.onException := cgiErrorHandler;
application.processMessages;
end;
PostMessage(Handle, wm_Traffic, 0, 0); { this takes us to wmTraffic below }
{ postMessage of a custom message only works if
application.run; is left in the .dpr file
}
end;
function getFileSize( filename : string ) : longint;
var
tmpFile : file of byte;
begin
try
assignfile( tmpFile, filename );
except
raise exception.create( 'error assigning FILE ' + filename );
end;
{ these 2 lines might not be necessary }
filemode := 0;
reset( tmpFile );
result := filesize( tmpFile );
closeFile( tmpFile );
end;
procedure TForm1.wmTraffic(var Msg: TMessage);
var
gifFile : integer;
stdoutname : string;
stdout : TFileStream;
gifFilename : string;
bufSize : word;
buf : pchar;
gifbuf : TMemoryStream; { this is created & filled in makegif }
count : longint;
begin
buf := nil;
stdoutname := CGIEnvData1.SystemOutputFile^;
{ This is where we actually create the .gif image based on the count... }
gifBuf := makegif; { makegif looks for CGIReferer, which could be
cginotfound and cause writing to stdout! Therefore
this call must be before any other use of stdout. }
if gifBuf = nil then
begin
CGIEnvData1.sendNoOp;
closeApp( application );
end;
{ A Note about File I/O techniques ...
* The fileOpen, fileWrite series of commands worked ok under Win HTTPD, but
not WebSite with Win NT.
* TFileStream.open with fmOpen parameter worked in the same situations.
* Only TFileStream.open with fmCreate OR fmOpen worked with WebSite under NT.
* If anyone would like to tell me *why*, please do. I'm just glad
I got it working. -ann
}
try
{ adding fmCreate seems to have done the trick for NT }
stdout := TFileStream.create( stdoutname, fmCreate OR fmOpenWrite );
except
raise exception.create( 'failed to open stdout: ' + stdoutname );
exit;
end;
try
getmem( buf, bufsize ); { buf is used to hold header below }
strpcopy( buf, 'HTTP/1.0 200 OK' + #13#10 + 'Server: ' +
CGIEnvData1.CGIServerSoftware^ + #13#10 +
'Date: ' + CGIEnvData1.webDate(now ) + #13#10 +
'Expires: ' + CGIEnvData1.webDate( now +
(1/(24*120)) ) + #13#10 + {in 30 seconds}
'Content-type: image/gif' + #13#10 +
#13#10 ); { blank line after prologue }
try
{ send header info defined above }
stdout.write( buf[0], strlen(buf) ); { from CWG.HLP }
except
freemem( buf, bufsize );
raise exception.create( 'write of buf failed' );
end;
gifBuf.saveToStream( stdout );
finally
gifbuf.free;
if buf <> nil then
freeMem( buf, bufsize );
stdout.free;
end;
application.processMessages;
closeApp( application ); { see cgihelp.hlp file }
end;
{ getCount figures out the count and returns it as a string }
function TForm1.getCount : string;
var
n : double;
refer : string;
x : byte;
const
URLFld = 0;
countFld = 1;
begin
result := '???'; { hopefully we'll have something better to say shortly }
refer := CGIEnvData1.CGIReferer^; { get URL of page that launched us }
if refer = cginotfound then
begin
result := 'N/A';
CGIEnvData1.closeStdout; { don't want error message keeping file open! }
exit;
end;
refer := lowercase( refer );
x := pos( '//', refer );
if x > 0 then
begin
{ strip off http://super.sonic.net portion of referer }
refer := copy( refer, x + 2, 60 );
x := pos( '/', refer );
if x > 0 then
refer := copy( refer, x, 60 );
end;
{ if URL ends in /, append index.html as document name }
if refer[ length(refer) ] = '/' then
refer := refer + 'index.html';
with tableHit do
begin
open;
edit;
if NOT findKey( [ refer ] ) then
begin
insert;
fields[ URLFld ].asString := refer; { primary key field }
fields[ CountFld ].asFloat := 2.0; { # for next surfer }
n := 1;
end
else
begin
edit;
n := fields[ CountFld ].asFloat;
fields[ CountFld ].asFloat := n + 1;
end;
post;
close;
end;
result := floatToStr( n );
end;
{ this function generates a .bmp first, and then converts that to a .gif }
function TForm1.makegif : TMemoryStream;
var
pict : TPicture;
Bitmap: TBitmap;
theGifImage : TMemoryStream;
begin
theGIFImage := nil;
try
pict := TPicture.create;
image.picture := pict; { image is a TImage on the form }
bitmap := TBitmap.create;
bitmap.height := 20;
bitmap.width := 80;
bitmap.monochrome := true; { added to fix >256 color problem on
Ann's server }
image.picture.bitmap := bitmap;
{ here's the magic -- use textOut to create a bitmap with count value !! }
image.picture.bitmap.canvas.textout( 2, 2, getCount );
theGIFImage := TMemoryStream.create;
if BMPToGifStream( image.picture.bitmap, theGIFImage ) <> CVROK then begin
{ error during conversion bmp to gif }
theGifImage.free;
theGifImage := nil;
end;
finally
bitmap.free;
pict.free;
end;
result := theGIFImage;
end;
