Macro languages are user-programmable interfaces to applications. Brett adds to David Betz's original Bob an API that turns a stand-alone Bob interpreter into a macro processor library. David Betz then presents a new version of Bob that runs as a Windows DLL.

May 01, 1995
URL:http://www.drdobbs.com/tools/bob-as-a-macro-processor-library/184409554

Bob as a Macro Processor Library

Turning a tiny OO language into a macro language

Brett recently completed his studies at Queensland University of Technology in Brisbane, Australia. He now heads RAX Information Systems, a consulting firm specializing in application software, and can be contacted at [email protected].

Changes to Bob

To extend internal functionality, I extended the number of internal functions to include a useful subset of C functions; see Table 1.

I also added the ability to initialize on load-up by using a function of the same name as the file being loaded up. Finally, I maintained backward compatibility so that saying "make Bob" will build an executable that works as originally designed.

Figure 1 represents a model of how the Bob macro processor library works. Although complicated, it is not necessary for you to fully understand it unless you would like to modify the Bob library. Luckily, my API hides all the complexity within a few functions.

Bob API

Listing One is a sample application that demonstrates use of the Bob API. (The API itself is fully documented in the bob.h source file.) Although this particular example was written with the X Toolkit Intrinsics (Xt), you do not need an X Window-based system to use it. All of Bob's processing is done through the function BobCheckClient, which polls the Bob communications socket for traffic and processes any packets that have come through. It returns True if there is a packet to process, and False if not. In the Xt library, the function XtAppAddInput "listens" on a socket. Bob has a similar function, BobBlockWait, that can be passed a socket to listen on. BobBlockWait returns when traffic arrives on either the Bob comms socket or the passed socket.

The power of Bob is in the external functions defined by the application. This application defines two functions (message and error), but unlimited external functions can be added. External function arguments are limited to three data types: number, string, and NIL. This limitation arises because the data-packet has only been made to transfer these data types across the socket.

Because Bob is a typeless language, a variable is defined as a VALUE. It is up to the function to test the type of the values being passed to it and either coerce the value or reject it as an error.

In examining Listing One, note that:

• int BobInitialize (void) initializes the Bob macro processor and returns either the socket on which to listen or --1 on failure.
• int BobLoadFile (char *fname) sends a request for Bob to load and compile a file. The filename is passed in fname and needs neither a suffix of .bob nor a path if it is in either the home directory or the path pointed to by the environment variable BOBLIB. Errors in the compilation are reported to stderr, and the processing stops. If the file loads successfully, Bob looks for the function of the same name as the file (illegal characters are replaced by underscores). If this function exists, it is executed and deleted upon completion. This function is considered the initialization function.
• int BobAddFunction (char *fname,BobExternFunc func) adds an external function to Bob. When that function is called by Bob or the application, the function will execute. The definition of BobExternFunc is typedef int (*BobExternFunc)(int, VALUE *args, VALUE *etval);.
• int BobExecute (char *name,int cnt,_) executes the passed function name, passing the arguments to the function. The argument list is in pairs, indicating the type of the argument and the value. The exception to this case is when NIL is passed. It has type only, no value. An example of running the print function with a number of arguments would be BobExecute ("print", 4, DT_ STRING, "Hi", DT_INTEGER, 20, DT_NIL, DT_STRING, " World\n" ); and the result would be Hi 20nil World.
• char *BobGetString (char *buf, int len, VALUE *val) is a utility function to extract a string from a value. It is used in remote functions.
• int BobGetInteger(VALUE *val) is a utility function to extract the integer part of a VALUE.
• int BobCheckClient(void) polls the socket, checking if there is anything to read, and processes one waiting packet. If there are more packets, they will be ignored until the next call to BobCheckClient.
• int BobExtArgs(int argc, int mn, int mx,VALUE *retval) checks the number of passed arguments. mn is the minimum number of arguments that this function needs, while mx is the maximum. If there is no minimum number of arguments for the function, set mn to --1. If there is no maximum, set mx to --1. If mn or mx is set to --1, the external function must verify argument counts.
• int BobExtCheckType(VALUE *arg, int type,VALUE*retval) checks the type of the passed VALUE and handles or checks the type of arguments.
• int BobReturnValue(VALUE *retval, int type,_) returns a value to the function that invoked the Bob function call. The use of this function is similar to BobExecute.
• int BobTalkTerm(void) terminates the Bob macro processor. It's sufficient to simply exit, but using this function is a little cleaner.
• int BobBreak(char *fmt,_) sends a break signal to Bob. If Bob encounters a break, it stops any processing and returns to the EventLoop. This function is useful to break large, slow functions. Its arguments are the same as those for sprintf-type functions.
• void BobBlockWait(int fd) waits for activity on either the Bob sockets or on the passed socket and returns when activity is detected. None of the packets are processed. This function was originally used when the application was written using Xlib, and it was difficult to add a socket to the application. It was easier to have Bob keep an eye on the X socket and return if there was activity on the passed socket or on the Bob socket.

When you start the example application, it will give you a list of functions to execute and files to load. Try these by typing them into the appropriate dialog box, then selecting OK (but don't press Return; the application isn't that smart).

Limitations and Future Enhancements

Bob currently does not support floating-point arithmetic, although this could be implemented via classes. The internal functions are only a subset of C-type functions. It is possible to extend the number of internal functions, but it would be better if the Bob macro processor library stayed free from application-dependent code. The file bobfcn.c contains all the internal functions; see Table 1. It may also be necessary in the future to extend the data types that external functions can handle.

You can now link Bob into your applications and get all the benefits that a macro language provides.

Callable Bob

David Betz

David is a DDJ contributing editor and can be contacted through the DDJ offices.

Last fall I presented a version of my Bob programming language that was extended to support an object store designed for use as the basis for building an online conferencing system (see "An Online Conferencing System Construction Kit," Dr. Dobb's Information Highway Sourcebook, Winter 1994). While that Bob interpreter was easy to extend with additional built-in functions, it was essentially a stand-alone interpreter. Bob was the main program, and extensions were called as subroutines. This works well in many applications but falls short when trying to use Bob as an extension language for an existing application.

To solve this problem, I recently designed a version of Bob that runs as a Windows DLL. Along the way, I separated Bob into several modules so that each could be used independently. The memory manager is now an independent module that can be used as the basis for other languages that need a heap with automatic garbage collection. I've also separated the Bob interpreter from the Bob compiler, since some applications only need to run already-compiled code. In fact, I've separated the run-time library from the rest of the interpreter so it is possible to run programs that only need the intrinsic functions without any library at all. To make things simpler, I've included all of these modules in the Bob DLL even though they are logically separate. The complete source code to this version of Bob is available electronically; see "Availability," page 3.

Windows DLLs have only a single data segment, even though several applications may be linked to them at the same time. This was a problem, since Bob had many global variables, most having to do with the memory manager. My first step in turning Bob into a callable library was to move all of the globals into context structures and add a parameter to every function to explicitly pass in the appropriate context. I created two context structures: the interpreter context, which contains the bytecode interpreter variables as well as the memory manager variables; and the compiler context, which contains the compiler and scanner variables. The compiler context also points to an interpreter context so that the compiler has access to the memory manager for creating objects.

Passing the compiler and interpreter contexts into each function explicitly makes it possible to create more than one context at a time. This allows a multithreaded program to have multiple threads, all executing Bob programs independently. It also means several programs linked to the same Bob DLL can operate without interfering with each other.

Now I'll show how to invoke the Bob DLL to create a simple read/eval/print loop for Bob expressions. First, it is necessary to create an interpreter context, as in Example 1(a). The first parameter is the size of the heap, and the second is the size of the stack. The second line sets up an error handler. Bob will call this error handler whenever an error occurs passing it an error code and relevant data.

If you need access to the run-time library functions, that is arranged by Example 1(b). The second line sets up a handler that the interpreter will call to get the address of a function handler, given a function name. This is necessary when the interpreter restores a saved workspace because the saved workspace format on disk contains only the names of library functions, not their addresses. This allows saved workspaces to work correctly even after the DLL has been rebuilt, causing the function-handler addresses to change.

It's now necessary to initialize the compiler, as in Example 1(c). This creates a compiler context with the specified interpreter context. The numeric parameters are the sizes of the compiler-bytecode staging buffer and the literal staging buffer.

The Bob memory manager is a compacting, stop-and-copy garbage collector and can change the address of objects when a garbage collection occurs. Because of this, the memory manager must know about all variables that could contain a pointer to an object in the heap. The variables that the interpreter uses are contained within the interpreter context structure and can therefore be located by the memory manager. However, it is sometimes useful for a memory-manager client to maintain its own pointers into the heap. The Bob memory manager allows for this by providing the function ProtectPointer to register an object pointer with the memory manager. This registers the specified pointer with the memory manager and guarantees that its value is updated whenever the garbage collector moves the object it points to.

This leaves the read/eval/print loop itself; see Example 2. Bob does all of its I/O through "streams." A stream is an object with some data and a pointer to a dispatch table. The dispatch table has pointers to handlers to carry out various stream operations. At the moment, there are handlers for getting and putting characters and a handler for closing the stream. The call to CreateStringStream creates a stream that allows the Bob compiler to read characters from the string. The interpreter context structure contains pointers to the standard I/O streams that must be set up by the client of the Bob DLL. These streams should arrange for characters to be read and written to the standard input and output of the application.

The call to CompileExpr compiles a single Bob expression and returns a compiled function which, when called with no arguments, will cause the expression to be evaluated.

CallFunction calls a function with arguments; see Example 3(a). The arguments after argumentCount are passed to the specified Bob function. They are of type ObjectPtr (a pointer to a Bob heap object), and argumentCount indicates their number. You can also call a Bob function by name, using Example 3(b).

Of course, the Bob DLL has many other functions. It contains a full compliment of object-creation and access functions for creating and manipulating objects of type ObjectPtr, as well as functions to control the interpreter.

This is just my first step in making Bob easier to embed in applications. I plan to extend the Bob language to support full function closures and optional arguments. I'll also add a "fast load" format for storing precompiled Bob code in disk files. This would make it possible to distribute Bob functions without including the source code, a necessary feature for using Bob to build commercial applications.

Example 1: (a) Bob interpreter context; (b) accessing the run-time library functions; (c) initializing the compiler.

(a) InterpreterContext *ic = NewInterpreterContext(16384,1024);
    ic->errorHandler = ErrorHandler;

(b) EnterLibrarySymbols(ic);
    ic->findFunctionHandler = FindLibraryFunctionHandler;

(c) CompilerContext *c = InitCompiler(ic,4096,256);

(d) ObjectPtr val;
    ProtectPointer(ic,&val);

Example 2: The read/eval/print loop.

for (;;) {
    printf("\nExpr> ");
    if (gets(lineBuffer)) {
        Stream *s = CreateStringStream(lineBuffer, strlen(lineBuffer));
        if (s) {
            val = CompileExpr(c,s);
            val = CallFunction(ic,val,0);
            printf("Value: ");
            PrintValue(ic,val,ic->standardOutput);
            CloseStream(s) ;
        }
    }
    else
        break;
}

Example 3: Calling a Bob function (a) by reference; (b) by name.

(a) ObjectPtr CallFunction(InterpreterContext ic, ObjectPtr function, int argumentCount,...);

(b) ObjectPtr CallFunctionByName(InterpreterContext ic, char *functionName, int argumentCount,...);

Figure 1 How the Bob macro library works. Figure 2 Sample dialog box.

Table 1: Bob internal functions.

Function                       Description   
char chr (ascii_value);             Converts the ASCII value into a string.
string date_time ();                Returns the current date and time in
                                    the format "Mon Nov 21 11:31:54 1983"
string downcase (string);           Converts the passed string to lowercase.
string editcase (string);           Converts the passed string to edit case.
                                    Edit case is where the first character
                                    after a space is uppercase and the rest
                                    are lowercase.
val exec_function (fname[,arg1      Executes the passed function name with
                                    the arguments. The number of arguments
                                    must be consistent with the function
                                    that is being called. This function returns
                                    the value that the function would have
                                    returned.
int fclose (file);                  Closes the passed file.
file fopen (fname,mode);            Opens the passed file in the mode: r
                                    for read, w for write.
int gc ();                          Does a garbage collection.
int getc (file);                    Returns the next character from the file.
string getenv (envname);            Returns the string associated with passed
                                    environment variable name.
bool keyboard_quit ();              Stops the current processing. Returns NIL.
list list_functions ();             Returns a list (vector) of function names.
bool load_bob (filename);           Loads the Bob macro file. If the file
                                    is not available, then returns NIL.
                                    The file is not compiled into memory
                                    until the current function is processed.
string newstring (size);            Returns a blank string of the passed size.
vector newvector (size);            Returns a vector of the passed size.
bool nothing ();                    This function does nothing. It could be
                                    used for disabling key translations.
int print (val1 [,val2              Prints the passed values to stdout.
  [,...valn]]);
int putc (file,char);               Puts the passed character to the file.
int sizeof (value);                 Returns the number of elements in a
                                    vector or the length of a string,
                                    or 1 for any other type of value.
int str_to_nam (string);            Returns the passed string as a number.
int strchr (string,char);           Returns the position of char in
                                    string. If returns < 0, then
                                    the string was not found.
int strlen (string);                Returns the length of the passed
                                    string. This is an alias for sizeof.
int strstr (string1,string2);       Returns the position of string2
                                    in string1. If < 0 is returned, then
                                    the string was not found.
string substring                    Returns the substring starting at
  (string,start-pos,[len]);         the position pos for the length len.
                                    If the length arg is not there, then
                                    returns the rest of the string. Pos of
                                    0 is the beginning of the string.
int system (command);               Sends a command to the operating
                                    system. Returns the OS exit code.
string typeof (value);              Returns the type of the passed
                                    string, which is one of the following: 
                                    NIL, CLASS, OBJECT, VECTOR, INTEGER, 
                                    STRING, BYTECODE, CODE, DICTIONARY, VAR, FILE.
string upcase (string);             Converts the passed string to all uppercase.
string val_to_string (value);       Converts any value to the equivalent string
                                    and returns it.
string version ();                  Returns the current version string of Bob.

Listing One

/* example.c: Exemplifies the use of the Bob macro processor library
 * Copyright (c) 1994 Brett Dutton
 *** Revision History:  * 13-Dec-1994 Dutton      Initial coding
 */

/* Description: */
/* includes */
#include <stdio.h>
#include <stdlib.h>
#include "bob/bob.h"
#include <X11/Intrinsic.h>
#include <X11/StringDefs.h>
#include <X11/Xaw/Label.h>
#include <X11/Xaw/Command.h>
#include <X11/Xaw/Box.h>
#include <X11/Xaw/Dialog.h>

/* macros */
#define APPNAME "example"
#define VERSION APPNAME " 1.0 By Brett Dutton"

/* typedefs */


/* prototypes */
void AppCheckBob ( XtPointer cl_data, int *fid, XtInputId *id );
void Quit ( Widget w, XtPointer cl_data, XtPointer call_data );
void Break ( Widget w, XtPointer cl_data, XtPointer call_data );
void LoadOk ( Widget w, XtPointer cl_data, XtPointer call_data );
void ExecuteOk ( Widget w, XtPointer cl_data, XtPointer call_data );
int Message ( int argc, VALUE *arg, VALUE *retval );
int Error ( int argc, VALUE *arg, VALUE *retval );
void ShowError ( char *msg );

/* variables */
/* these resources are usually external */
static String resources[] = {
    "*example.width: 300",
    "*example.height: 400",
    "*quit*label: Quit",
    "*break*label: Break",
    "*Command*background: green",
    "*message*label: Message:",
    "*message*width: 275",
    "*error*label: Error:",
    "*error*width: 275",
    "*value: ",
    "*load*label: Load file:",
    "*load*loadok*label: Ok",
    "*load*width: 275",
    "*execute*label: Execute function:",
    "*execute*executeok*label: Ok",
    "*execute*width: 275",
};

/* global widgets */
Widget message, errordia, load, loadok, executedia, executeok;

/* functions */
 * Function: main -- main function
 * Returns: Nothing   
 */
void main ( int argc, char *argv[] ) 
{
    XtAppContext        app_context;
    Widget              topLevel;
    Widget              box, quit, brkwid;
    int                 bobSock;
    /* create a application */
    topLevel = XtVaAppInitialize ( &app_context, APPNAME, NULL, 0, 
                                  &argc, argv, resources, NULL ); 
    /* create all the buttons and dialogs for the application */
    box = XtVaCreateManagedWidget ( "box", boxWidgetClass, topLevel, NULL ); 
    quit = XtVaCreateManagedWidget ( "quit", commandWidgetClass, box, NULL ); 
    brkwid = XtVaCreateManagedWidget ("break", commandWidgetClass, box, NULL); 
    message = XtVaCreateManagedWidget ( "message",dialogWidgetClass,box,NULL );
    errordia = XtVaCreateManagedWidget ("error", dialogWidgetClass,box, NULL );
    load = XtVaCreateManagedWidget ( "load", dialogWidgetClass, box, NULL ); 
    loadok = XtVaCreateManagedWidget ( "loadok",commandWidgetClass,load,NULL );
    executedia = XtVaCreateManagedWidget ( "execute", dialogWidgetClass, box, 
                                                                        NULL );
    executeok = XtVaCreateManagedWidget ( "executeok", commandWidgetClass, 
                                                           executedia, NULL );
    /* set up all the callbacks for the buttons */
    XtAddCallback ( quit, XtNcallback, Quit, 0 );
    XtAddCallback ( brkwid, XtNcallback, Break, 0 );
    XtAddCallback ( loadok, XtNcallback, LoadOk, 0 );
    XtAddCallback ( executeok, XtNcallback, ExecuteOk, 0 );
    /* initialize the bob interface language */
    if ( ( bobSock = BobInitialize ( ) ) < 0 ) {
        fprintf (stderr,"Unable to initialize Bob\n" );
        exit(1);
    }
    /* add this socket to the event loop for monitoring */
    XtAppAddInput ( app_context, bobSock, (XtPointer)XtInputReadMask, 
                    AppCheckBob, NULL );
    /* register the external functions with BOB */
    BobAddFunction ( "message", Message );
    BobAddFunction ( "error", Error );
    /* load up the application defaults macros */
    BobLoadFile ( "." APPNAME "rc" );  /* user definitions */
    /* this has just been put in to demonstrate calling BOB functions */
    BobExecute ( "print", 4, DT_STRING, "Hi ", DT_INTEGER, 20,
                 DT_NIL, DT_STRING, " World\n" );
    /* create windows for widgets and map them */
    XtRealizeWidget ( topLevel );
    /* loop for events */
    XtAppMainLoop ( app_context );
}
 * Function: AppCheckBob -- The is the work proc called when there is  * input from Bob
 * Returns: Nothing   
 */
void AppCheckBob ( XtPointer cl_data, int *fid, XtInputId *id )
{
    /* Call bob to get the events of the Bob comms socket */
    BobCheckClient ( );
}
 * Function: Quit -- Exits from the windows system
 * Returns: Nothing   
 */
void Quit ( Widget w, XtPointer cl_data, XtPointer call_data )
{
    BobTalkTerm ();     /* shutdown comms with Bob */
    exit ( 0 );
}
 * Function: Break -- Exits from the windows system
 * Returns: Nothing   
 */
void Break ( Widget w, XtPointer cl_data, XtPointer call_data )
{
    BobBreak ( "BOB Inturrupted" );
}
 * Function: LoadOk -- The load dialog is complete and to load up file
 * Returns: Nothing   
 */
void LoadOk ( Widget w, XtPointer cl_data, XtPointer call_data )
{
    String      str;            /* filename to load */
    char        msg[500];       /* Error message */
    Arg         xargs[1];       /* New value */
    /* get the string and try to load it */
    str = XawDialogGetValueString ( load );
    if ( BobLoadFile ( str ) ) {
        /* clear the box if no error */
        XtSetArg ( xargs[0], XtNvalue, (XtArgVal)"" );
        XtSetValues ( load, xargs, 1 );
    } else {
        /* send an error to the error box */
        sprintf ( msg, "Unable to load file: %s", str );
        ShowError ( msg );
    }
}
 * Function: ExecuteOk -- Execute dialog is complete 
 * Returns: Nothing   
 */
void ExecuteOk ( Widget w, XtPointer cl_data, XtPointer call_data )
{
    String      str;            /* function to execute */
    char        msg[500];       /* error message */
    Arg         xargs[1];       /* New value */
    /* get the string and try to load it */
    str = XawDialogGetValueString ( executedia );
    if ( BobExecute ( str, 0 ) ) {
        /* clear the box if no error */
        XtSetArg ( xargs[0], XtNvalue, (XtArgVal)"" );
        XtSetValues ( executedia, xargs, 1 );
    } else {
        /* send an error to the error box */
        sprintf ( msg, "Unable to execute function: %s", str );
        ShowError ( msg );
    }
}
 * Function: Message --  Displays a message in the dialog boc
 * Returns: Tells Bob if it is an error or not
 */
int Message ( int argc, VALUE *arg, VALUE *retval  )
{
    char                msg[500]; /* message to put in dialog */
    Arg                 xargs[1]; /* New value */
    /* make sure that there is 1 args */
    /* make sure that it is a string */
    if ( ! BobExtArgs ( argc, 1, 1, retval ) ) return ( FALSE );
    if ( ! BobExtCheckType ( &arg[0], DT_STRING, retval ) ) return ( FALSE );
    
    BobGetString ( msg, sizeof ( msg ), &arg[0] );
    XtSetArg ( xargs[0], XtNvalue, (XtArgVal)msg );
    XtSetValues ( message, xargs, 1 );
    return ( BobReturnValue ( retval, DT_INTEGER, TRUE ) );
}
 * Function: Error --  Displays a error in the dialog boc
 * Returns: Tells Bob is there is an error or not
 */
int Error ( int argc, VALUE *arg, VALUE *retval  )
{
    char                msg[500]; /* error to put in dialog */
    Arg                 xargs[1]; /* New value */
    /* make sure that there is 1 args */
    /* make sure that it is a string */
    if ( ! BobExtArgs ( argc, 1, 1, retval ) ) return ( FALSE );
    if ( ! BobExtCheckType ( &arg[0], DT_STRING, retval ) ) return ( FALSE );
    BobGetString ( msg, sizeof ( msg ), &arg[0] );
    ShowError ( msg );
    return ( BobReturnValue ( retval, DT_INTEGER, TRUE ) );
}
 * Function: ShowError --  Displays the passed message in the error dialog box
 * Description:
 * Returns: Nothing   
 */
void ShowError ( char *msg )
{
    Arg                 xargs[1]; /* New value */
    XtSetArg ( xargs[0], XtNvalue, (XtArgVal)msg );
    XtSetValues ( errordia, xargs, 1 );
}

Terms of Service | Privacy Statement | Copyright © 2024 UBM Tech, All rights reserved.