Michael is senior systems analyst for Micro Endeavors, where he develops applications for Fortune 500 companies and conducts FoxPro training seminars. He's published articles in Data Based Advisor, FoxTalk, Science, Brain Research, Journal of the Acoustical Society of America, and other publications. Michael can be contacted at 215-449-4680 or on CompuServe at 71045,1217.
After developing a database application in FoxPro, you do a quick benchmark and discover to your horror that performance is going to be too slow by a factor of five. Luckily, FoxPro's application programming interface (API) lets you add functions and features that can, among other things, improve performance anywhere from three- to fifty-fold, depending upon what you are trying to do.
Let's say that your application required counting the number of occurrences of a string in a number of files. This could be the forerunner to a global search-and-replace utility for a document index generator, for example.
Several string-counting strategies come to mind. One might be to place every word or every line of a document into a database, but we'll assume this isn't practical. Another strategy might be to use low-level file I/O to read in the document and check the incoming stream against the comparison string and count the number of matches. Listing One (page 37) shows FoxPro code for this method. This may not be the best method of performing this function; it's simply presented as a slightly artificial demonstration of how to use the API. You would call it from FoxPro with ?search1("testfile.prg", "cor") and it would count the occurrences of cor in the file TESTFILE.PRG. This may not yield the desired count because the string cor is also part of the words corn, rancor, and incorrect. To account for this, the program accepts the % symbol as a wildcard. Without the % sign, the routine only counts the occurrences of the desired string delimited with nonalphabetic characters. Our example would reject corn, rancor, and incorrect as invalid occurrences. If the % sign is the first character, it relaxes the rule at the beginning and will include rancor as an occurrence. If the % sign is the last character, the routine relaxes the rule at the end and will include corn as a valid occurrence. If the comparison string includes a % sign at the beginning and end, any occurrence of cor will be included. This is the same as the $ operator in FoxPro.
Now for the critical test. Assume that you implemented this routine and tested it on a particular file on a particular machine--sort of a benchmark. Say that the response took about 2.5 seconds, which is not unreasonable. The only problem is, your application hinges on this taking under one second for a file this size. Your choices are simple: Get a faster machine or do it a different way. We'll assume that you don't have the option of getting a faster machine since your client has already made a substantial investment in hardware and is unwilling to upgrade 400 machines just to make one routine run faster. The next option would be to not use FoxPro and rewrite the whole application in another language. Because FoxPro offers a rich application-development environment and a powerful database manager, abandoning it is the last thing you want to do. There must be another way.
FoxPro Backgrounder
There are two kinds of computer languages in the world: compiled and interpreted. An interpreted language reads in each statement and interprets it into machine language every time the statement is encountered. This is somewhat inefficient, but makes application development particularly easy. Compiled languages introduce a preprocessor, which compiles the program down to machine language one time. This boosts performance tremendously, but it takes much longer to develop programs because every time a change is made, the program must be recompiled.
FoxPro's procedural language is a superset or dialect of Xbase. Traditionally, Xbase has been an interpreted language, making application development easy but limiting performance. FoxPro is a hybrid. At its heart, it is still an interpreter but it does not interpret your original source code. It precompiles each program module the first time it encounters it into a compact code called "p-code" and thereafter interprets the p-code. Performance is much better than an ordinary interpreted language, but there is still some overhead involved with interpreting each line of p-code. Do not confuse this with performance of an individual command. Each FoxPro high-level command launches a procedure written in C or assembler. Performance here is the best it can be. What we're talking about are looping constructs and the overhead associated with the loop itself. In the string-search example, we use a FOR/ENDFOR loop to work our way through the test code.
Accessing the API
So the solution becomes clear. Why not rewrite the specific section in C and call it from FoxPro? Listing Two (page 37) and Listing Three (page 38) provide a means to do just that. This time we take advantage of the knowledge that none of our source files will be all that large, and we read the file into memory first. We use malloc() to allocate as much memory as we need to hold the entire file. The search now proceeds memory to memory. This code was compiled using the Watcom C compiler. The reason for that will be explained later. Once compiled and linked, the program called SEARCH2.EXE was tested from DOS using the command line search2 testfile.prg cor.
A quick test with a stopwatch reveals that this program executes so fast, it can't even be timed properly. That means an execution time of under 0.1 seconds. Things are looking good. The program stores the answer in a file called SEARCH2.ANS, which reveals the correct answer when typed to the screen. All we need to do now is go back to FoxPro and issue the command RUN search2 testfile.prg cor. Disaster! This one line of code takes over four seconds to execute, not counting the time it takes to open and read in the answer. What happened? The mechanics of the RUN command dictate that FoxPro open a new copy of COMMAND.COM. This takes time. FoxPro must tuck away certain registers and memory locations before launching COMMAND.COM, and this takes time also. We assume that the program executes as quickly as it did before. Returning from DOS, FoxPro must purge the second copy of COMMAND.COM and restore all the memory locations and registers. The overhead for all of this is about four seconds on the particular machine tested.
You can reduce the time it takes to swap out to DOS by placing a copy of COMMAND.COM on a RAMDISK and changing COMSPEC=. You can also reduce the time a small amount by creating SEARCH2.ANS on the RAMDISK. There are various other tricks you can perform within FoxPro, like turning off the resource file, forcing FoxPro's swap file to the RAMDISK, and so on. But even after all these tricks, the total execution time is still around three seconds. So what have we gained?
The solution, of course, is to take the string-search routine and attach it to FoxPro using the API, bypassing the need to swap out to DOS. This isn't just a matter of recompiling and relinking, but it is very nearly so. There are certain things you cannot do from within the API, but in every case a substitute is available. Listing Four (page 38) is the port of SEARCH2.C, called SEARCH3.C, which is compatible with FoxPro's API. Note that a new header file has been added: PRO_EXT.H. This file contains all the structures and function predefinitions you will need to attach routines to FoxPro. It is available in the Library Construction Kit (LCK) described further on.
Several general rules must be observed. First, FoxPro controls all of memory. You cannot use malloc(). Instead, you ask FoxPro for a block of memory, which is then assigned a memory handle using the function_AllocHand(). The memory handle can be converted to a pointer using the function_HandToPtr(). Next, because you cannot use a file pointer, you instead use a file channel, which again is assigned by FoxPro. You cannot use low-level file routines such as fopen() and fread(). Instead, you use the equivalent routines _FOpen() and _FRead(). Calls to string routines such as strlen() and memmove() must be replaced by calls to _StrLen() and _MemMove(). You cannot use printf(), but there are equivalent routines for that also.
Parameters are passed via a special structure called a ParamBlk, which is a union of two "faces." The first face, called the Value structure, is used when parameters are passed using call by value. The other face is called a Locator structure and is used for call by reference. The ParamBlk structure is a subscripted structure that permits multiple arguments to be passed. At the bottom of Listing Four, you will see a structure labeled FoxInfo. This is the table used by FoxPro when you attach a library file. The first argument (always in upper case!) is the name of the function FoxPro will use to call your routine. The second argument is the function to be called. The third item is the number of arguments that will be passed. The last table entry tells FoxPro which types of arguments should be passed. In this particular case, both arguments must be of type Character.
Converting a FoxPro-style string to a null-terminated string is demonstrated in Section 2 of Listing Four. You must retrieve the string length from an element of the Value face using the notation shown in Example 1.
Example 1: Retrieving the string length from an element of the Value face.
parm->p[0].val.ev_length parm-> pointer to the ParamBlk structure p[0]. first argument val. Value face ev_length integer length element
The actual string is not contained in the Value structure, but rather in its memory handle, which is found in parm->p[O].val.ev_handle. You convert this to a real pointer using _HandToPtr(), then copy the bytes to a local array. You cannot count on FoxPro to null terminate strings; you must supply the null termination explicitly.
The rest of the code parallels the stand-alone DOS function. Once things have been set up properly, the function contained in SENGINE.C is called, which is identical to the function called from DOS. This is not an accident. I purposely set up an interface which would be compatible under both circumstances to reduce my debugging time. Not all situations lend themselves to this organization, but you should try to do this if you can. Create a routine and test it from DOS, then replace the front end with one written for FoxPro, if possible. If you go back and check SEARCH2.C, you'll see that I created definitions for _StrLen and _MemMove so that I wouldn't have to change SENGINE.C when I switched to FoxPro.
Once the function has calculated the number of matches, you will want to return the value back to FoxPro. Unfortunately, you cannot use a standard return because all functions invoked from FoxPro must be of type Void. Instead, you use special _RetXXXX() functions that format the data properly for FoxPro. In this particular example, we use _RetInt() to return an integer. The first argument of _RetInt() is the value itself; the second is the width the argument would take on screen.
After all this, you compile the function using the Watcom compiler with the command line wcc /s /zu /zW/ml /fpc search3. This invokes the WATCOM compiler WCC.EXE. The command line options are: /s, remove stack over-flow checks; /zu, decouple the stack segment from the data segment; /zW, use Microsoft Windows naming conventions; /ml, use the large model; and /fpc, use only the floating-point library, not the coprocessor. You then link the output of the compiler (SEARCH3.OBJ) with the command line wlink file api_l, search3 lib proapi_l, clibl.
API_L.OBJ is a Microsoft-supplied file that contains the interface to FoxPro. PROAPI_L.LIB contains all the functions used by the API. Both files are available for purchase in the LCK sold by Microsoft. Also contained in the FoxPro 2.0 version of the LCK is a copy of the Watcom compiler, the only compiler that works at present, due to its method of passing parameters. Other compilers such as Microsoft C 7.0 will likely also be supported when FoxPro 2.5 is released, but this was not certain at the time this article was written.
After linking, you will need to copy or rename the file API_L.EXE to SEARCH3.PLB. PLB is the default extension for API add-on routines. You attach the function within FoxPro using the command: SET LIBRARY TO search3. If you do a LIST STATUS, you will see that the function SSEARCH has been added. You would invoke with the command: ?ssearch("testfile.prg", "cor"). A quick benchmark reveals an average execution time of 0.066 seconds, a performance improvement of about 40-fold! Other examples may not be quite as dramatic, but the fact remains: The API can be used to boost the performance of I/O-bound routines or computationally intensive loops by at least a factor of three up to as much as 50-fold.
The API can be used to add graphics, serial communications, encryption, compression, network, and other functions to FoxPro. Many of these functions have already been written by third parties and are available for purchase, or you can write your own routines.
Conclusion
In summary, FoxPro provides an exceptional application-development environment and superior database-management tools. Under all circumstances, you should write your application in FoxPro and only consider using the API if you need a function it cannot provide, such as access to new hardware or when a particular section of code (most likely a looping construct) needs to run faster. Working in C is like walking a tightrope without a net. No one is going to be able to help you if you crash. However, with a little bit of care, you can achieve great results. Good luck!
_EXTENDING FOXPRO_ by Michael L. Brachman[LISTING ONE]
<a name="03ba_0009">
* PROCEDURE search1
* single file string search
* notes: case-sensitive. word must match exactly unless '%' relaxes rule on
* that end. e.g. "%int" means char preceding can be alpha
* searches default directory only
* Copyright 1993 by Micro Endeavors, Inc. 3150 Township Line Road,
* Drexel Hill, PA 19026, (215) 449 - 4680
PARAMETER l_infile,l_srchstrg
* l_infile = file specification
* l_srchstrg = string to search for
* returns number of occurrences
*
PRIVATE ALL LIKE l_* && protect other variables
l_count = 0
l_ifh = FOPEN(l_infile) && open the file
IF l_ifh > 0 && meaning valid file
DO CASE
CASE l_srchstrg = '%' AND RIGHT(l_srchstrg,1) = '%'
STORE .T. TO l_relaxr
STORE .T. TO l_relaxl
l_srchstrg = SUBSTR(l_srchstrg,2,LEN(l_srchstrg)-2)
CASE l_srchstrg = '%' && can begin with any character
STORE .F. TO l_relaxr
STORE .T. TO l_relaxl
l_srchstrg = SUBSTR(l_srchstrg,2)
CASE RIGHT(l_srchstrg,1) = '%' && can end with any character
STORE .T. TO l_relaxr
STORE .F. TO l_relaxl
l_srchstrg = LEFT(l_srchstrg,LEN(l_srchstrg)-1)
OTHERWISE && strict checking
STORE .F. TO l_relaxr
STORE .F. TO l_relaxl
ENDCASE
l_srchleng = LEN(l_srchstrg)
l_fileng = FSEEK(l_ifh,0,2) && determine the file length
= FSEEK(l_ifh,0) && put file pointer back to the beginning
l_prevchar = " " && will represent character just to the left
FOR l_char = 0 TO l_fileng
l_curstrg = FREAD(l_ifh,l_srchleng+1) && read in length + 1
IF l_curstrg = l_srchstrg AND ;
(l_relaxr OR NOT ISALPHA(RIGHT(l_curstrg,1))) AND ;
(l_relaxl OR NOT ISALPHA(l_prevchar))
** A match!
l_count = l_count + 1
l_char = l_char + l_srchleng && bump loop index by this amount
l_prevchar = RIGHT(l_curstrg,1) && take snapshot of last char
ELSE
l_prevchar = LEFT(l_curstrg,1) && take snapshot of next char
= FSEEK(l_ifh,l_char+1) && put fp back to where it belongs
ENDIF
ENDFOR
= FCLOSE(l_ifh)
ELSE
l_count = -1 && meaning couldn't open file
ENDIF
RETURN l_count
<a name="03ba_000a">
<a name="03ba_000b">[LISTING TWO]<a name="03ba_000b">
/* S E A R C H 2 . C
* single file search
* notes: case-sensitive. word must match exactly unless '%' relaxes rule on
* that end. e.g. "%int" means char preceding can be alpha
* searches default directory only
* Copyright 1993 by Micro Endeavors, Inc., 3150 Township Line Road,
* Drexel Hill, PA 19026, (215) 449 - 4680 */
#include <stdio.h>
#include <stdlib.h>
#include <dos.h>
#define _StrLen strlen
#define _MemMove memmove
#define _MemCmp memcmp
extern int
main(int argc,char *argv[])
{
/* SECTION 1 - DEFINE VARIABLES */
FILE *fp; /* File Pointer (structure) */
char *buff,fname[40];
char ans[20];
long filen;
int retval=0,icount;
struct find_t finfo;
/* SECTION 2 - SETUP NAME OF FILE */
/* not needed because argv[1] points to name already */
/* SECTION 3 - GET SIZE OF FILE */
_dos_findfirst(argv[1],_A_NORMAL,&finfo);
/* SECTION 4 - SEE IF FILE EXISTS */
fp = fopen(argv[1], "r" );
if (fp == NULL) { /* illegal file name */
if (argv[1][0] < 32)
icount = -1;
else
icount = -2;
}
else { /* file is legal */
/* SECTION 5 - CREATE BUFFER FROM MEMORY POOL */
buff = malloc(finfo.size);
if (buff == NULL)
icount = -3;
else {
/* SECTION 6 - LOAD BUFFER FROM DISK FILE */
fread(buff,1,finfo.size,fp);
/* SECTION 7 - CALL THE "ENGINE" */
icount = nomatches((char far *)buff,finfo.size-1,argv[2]);
/* SECTION 8 - CLEAN UP */
free(buff);
fclose(fp);
}
}
/* SECTION 9 - RETURN ANSWER */
fp = fopen("search2.ans","w");
if (fp != NULL) {
sprintf(ans,"%d",icount);
fputs(ans,fp);
fclose(fp);
}
return 0;
}
#include "sengine.c"
<a name="03ba_000c">
<a name="03ba_000d">[LISTING THREE]<a name="03ba_000d">
/* THE STRING SEARCH "ENGINE" */
extern void strncopy(char *,char *,int);
extern int
nomatches(char far *bf, long filelength, char far *sstring)
{
char l_srchstrg[500],l_prevchar;
char l_curstrg[500];
int l_count = 0;
int l_relaxr,l_relaxl;
int l_srchleng;
int l_fileng = filelength;
int l_char;
int sstrlen = _StrLen(sstring);
switch ( 2 * (sstring[0] == '%') + (sstring[sstrlen-1] == '%') ) {
case 3: /* both left and right are % */
l_relaxr = 1;
l_relaxl = 1;
strncopy(l_srchstrg,sstring+1,sstrlen-2);
break;
case 2: /* only left is % */
l_relaxr = 0;
l_relaxl = 1;
strncopy(l_srchstrg,sstring+1,sstrlen-1);
break;
case 1: /* only right is % */
l_relaxr = 1;
l_relaxl = 0;
strncopy(l_srchstrg,sstring,sstrlen-1);
break;
default: /* neither are % */
l_relaxr = 0;
l_relaxl = 0;
strncopy(l_srchstrg,sstring,sstrlen);
break;
} /* end of switch */
l_srchleng = _StrLen(l_srchstrg);
for (l_char=0; l_char<(l_fileng-l_srchleng); l_char++) {
strncopy(l_curstrg, (char *)(bf + l_char), l_srchleng+1);
if ( ( _MemCmp(l_curstrg,l_srchstrg,l_srchleng) == 0 ) &&
( l_relaxr || !isalpha(l_curstrg[l_srchleng-1]) ) &&
( l_relaxl || !isalpha(l_prevchar) ) ) {
l_count++;
l_char += l_srchleng;
l_prevchar = l_curstrg[l_srchleng-1];
}
else
l_prevchar = l_curstrg[0];
}
return l_count;
}
extern void
strncopy(char *dst, char *src, int count)
/* same as strncpy but guarantees null-termination */
{
_MemMove(dst,src,count);
dst[count] = 0;
}
<a name="03ba_000e">
<a name="03ba_000f">[LISTING FOUR]<a name="03ba_000f">
/* S E A R C H 3 . C
* single file search
* notes: case-sensitive. word must match exactly unless. '%' relaxes rule on
* that end. e.g. "%int" means char preceding can be alpha
* searches default directory only
* Copyright 1993 by Micro Endeavors, Inc., 3150 Township Line Road,
* Drexel Hill, PA 19026, (215) 449 - 4680 */
/* FoxPro 2.0 PLB-version--adds function ssearch(filevar,sstring)--
returns # of words */
#include <stdlib.h>
#include <dos.h>
#include <pro_ext.h>
/* FUNCTION SSEARCH
called from FoxPro as:
ssearch(filevar,sstring) filevar can be a constant or char. variable
returns: integer number of matches; -1 means "File Not Found";
-2 means "Insufficient Memory" */
void far
ssearch(ParamBlk FAR *parm)
{
/* SECTION 1 - DEFINE VARIABLES */
FCHAN fh; /* File Handle (int) */
MHANDLE bmh;
char far *buff,fname[40];
char far *srchstrg;
int retval,filen;
struct find_t finfo;
/* SECTION 2 - SETUP NAME OF FILE */
filen = parm->p[0].val.ev_length;
_MemMove(fname,_HandToPtr(parm->p[0].val.ev_handle),filen);
fname[filen] = 0; /* add null-terminator */
/* SECTION 3 - GET SIZE OF FILE */
_dos_findfirst(fname,_A_NORMAL,&finfo);
/* SECTION 4 - SEE IF FILE EXISTS */
fh = _FOpen(fname,FO_READONLY);
if (fh < 1) { /* illegal file name */
retval = -1; /* return of -1 means can't find it */
}
else { /* file is legal */
/* SECTION 5 - CREATE BUFFER FROM MEMORY POOL */
bmh = _AllocHand(finfo.size);
buff = (char far *)_HandToPtr(bmh);
if (bmh < 1)
retval = -2; /* return of -2 means not enough memory */
else {
/* SECTION 6 - LOAD BUFFER FROM DISK FILE */
_FRead(fh,buff,finfo.size);
/* SECTION 7 - CALL THE "ENGINE" */
srchstrg = _HandToPtr(parm->p[1].val.ev_handle);
srchstrg[parm->p[1].val.ev_length] = 0; /* a little dangerous */
retval = nomatches(buff,finfo.size-1,srchstrg);
/* SECTION 8 - CLEAN UP */
_FreeHand(bmh);
_FClose(fh);
}
}
/* SECTION 9 - RETURN THE ANSWER */
_RetInt(retval,10);
}
#include "sengine.c"
FoxInfo myFoxInfo[] = {
{"SSEARCH", ssearch, 2, "C,C"}
};
FoxTable _FoxTable = {
(FoxTable FAR *)0, sizeof(myFoxInfo) / sizeof(FoxInfo), myFoxInfo
};
Copyright © 1993, Dr. Dobb's Journal
