Tom is an independent software developer with Lone Peak Technologies. He can be reached at [email protected].
Although dynamic linking, late binding, and shared libraries have been around for years, it wasn't until the advent of Windows and, to a lesser extent, OS/2 that they came into the mainstream. There are a number of reasons for using dynamic linking and late binding as a technique for deploying program features. For instance, say you need to physically partition functionality along logical boundaries. Good modular design logically partitions functionality in ways that maximize cohesion and minimize coupling. Creating separate modules in physically distinct dynamic libraries further underscores their logical separation.
Another reason for placing functionality in DLLs rather than linking into applications that use the functionality is that one workgroup can create the application and another can create the DLL. All that is required is agreement on the functional interface and semantics of the features implemented in the DLL. The module and application can be revised on different schedules and do not have to affect each other. Application writers do not have to be concerned about how the DLL is built. DLL creators can focus on their service, and not be concerned with how the application is built. In fact, it is common for these to be implemented in different programming languages.
Late binding further provides you with the flexibility to decide at run time whether or not a particular feature is available. Or, the application logic can choose from perhaps many different implementations of a particular feature. In fact, careful design of the functional interface between the application and the DLL enables new features to be added to an application after the application has been released without requiring a new revision of the application.
Finally, since dynamic modules are loaded into memory only once and shared among many execution threads, there are savings of memory and load time. On operating systems that don't support virtual memory, such as Netware, this can be extremely valuable.
For the purpose of this discussion, I'll refer to code that uses dynamic link modules as "consumers" of whatever service or services that DLL provides. DLLs, then, are service "providers." (The reason for using "consumer" rather than "application" is simply that it is common for one DLL to consume the services of another.)
Dynamic Linking on Windows
The Windows model of DLLs is both simple and powerful, supporting static and late binding. It is relatively easy for both consumers and service providers to use. Under Windows 3.1, shared DLLs will only be loaded into memory once regardless of the number of consumers. Under Windows 95/NT, this is also possible, though it takes extra care on the part of service providers as well as consumers, and may still not really happen.
The Windows model of dynamic linking requires consumers to use three management functions to control loading and unloading of the dynamic module, and retrieval of function addresses. You make a call to LoadLibrary to retrieve the handle of the module (and possibly to load the module into memory if is not not already there). FreeLibrary is the function used to indicate that a consumer is finished calling functions in the DLL. GetProcAddress lets consumers retrieve the address of an exported function based on its name. This is how late binding is supported.
Dynamic Linking on Netware
When I decided to look for a standard approach to dynamic linking on Netware, what I found was that Netware Loadable Modules (NLMs) can export both functions and variable addresses. These can be imported into other NLMs. But the only model of binding supported is really the static model. It's not that you can't do late binding. The problem is that there is no accepted standard approach to it, and the some half-dozen approaches I came across are all much more complex for service providers, and at least somewhat more complex for consumers, than with the Windows model.
Therefore, my goal in creating a dynamic linking model for Netware was to have an approach that would be simple to use from the standpoints of both the consumer and the service provider. It also had to be powerful enough to support more sophisticated dynamic programming models such as a dynamic component model. Since there is more familiarity with Windows DLLs than any other dynamic link model out there, I wanted my approach to leverage what people would likely know if they knew anything at all about delayed binding and dynamic loading.
When Netware loads an NLM, it extracts the names of all symbols that were exported from the NLM. A symbol can be either a function name or global variable name. You export a symbol from an NLM by giving the EXPORT directive to the Watcom linker. These symbol names are placed in a table of exported symbols that Netware maintains on a server-wide basis. If another NLM is loaded that has an exported name that is the same as one already in the exported symbol table, the loader will write a diagnostic message to the console and not load the second NLM.
What should you name the functions that are to be exported from their modules? One approach that has been used is to mangle the exported name based on the NLM module name. The result is usually something like this: If the module name is Foo and the function name is Bar, the function name is mangled to Foo@Bar. The problem with this approach, of course, is that it requires service providers to jump through hoops to publish their service, and it places a great burden on consumers to dynamically mangle the symbol names it imports based on the source.
This leads to the final goal I wanted to accomplish: creating a system that allows service providers to name their exported service interfaces anything they want and not worry about naming conflicts with other developers. The result of my efforts to meet these goals is a method I call "Dynamic Netware Libraries" (DNL).
Figure 1 illustrates the architecture of the DNL environment. The broken lines indicate control information while the solid lines indicate the actual service interface provided to consumers. The important thing to note about this architecture is that the DNL Manager does not get in the way of the service interface. Once consumers have extracted the function pointers that comprise the service interface that is needed, actual calls across that interface do not involve the DNL Manager. I've represented both control and service interfaces as bus-type communication. Of course, not every consumer connects with every service; this was an easy way to represent that any consumer could connect with any service.
DNL Use: A Consumer Perspective
To collect the addresses of the published functions it needs from a DNL, the consumer XDNL_HDNL hDNL; makes calls to GetProcAddress. Listing One describes the general process.
A consumer makes a call to LoadLibrary to receive a handle to the library. That handle can be used in subsequent calls to GetProcAddress, which is called once for each of the functions that a consumer wishes to call in the loaded library. After obtaining pointers to the functions it needs, consumers can then make calls indirectly through the returned pointers. When it is finished using the services of a DNL, a consumer calls FreeLibrary, one of the exported functions of the DNL Manager. This allows the DNL Manager to release resources associated with the DNL and may result in unloading the DNL. After calling FreeLibrary, the value of the DNL handle passed to FreeLibrary is no longer valid and should not be used. Also, any function pointers obtained from GetProcAddress with the freed hDNL should no longer be used because the DNL itself may be unloaded.
The parameter passed to LoadLibrary can be a fully qualified path to the DNL, or it can be a partial path, or even just the name of the DNL. The DNL Manager will attempt to load the DNL by calling spawnvp, so it simply passes this information to the Netware loader. However, the DNL Manager also strips off the module name and uses it in internal management structures to determine if the DNL has already been loaded.
That is all there is to the consumer's view of dynamic loading. The creator of the DNL can provide header files with type definitions to make it easy for a consumer to declare variables to hold function pointers, and even to get some compiler type checking to look for parameter mismatches, and so on.
DNL Creation: Provider Perspective
One of my goals in designing the DNL technique was to make it easy for programmers to create the libraries. To that end, I hid all the issues of startup code -- registration of the library with the library manager, loading and unloading of the module, and so on. All DNL creators need to do is place the addresses of their published functions into a table provided for that purpose. You don't even need to make any changes to the DNLMain function for startup handling. Typical NLM programs have a main function as the entry point into the NLM. For DNLs, I have written the main function -- an object file called dnlsup.obj ("DNL support"), which is linked with each DNL.
Creators of a DNL simply focus on their service. To build a DNL, create the code you want to provide to your consumers as if it is going to be link-time bound with consumers. However, you should typically avoid creating global data. Global data can be used, but you must recognize that there will be a single instance of your global data provided to all threads that call your library. Therefore, you should either avoid having global data, and package thread-specific data into dynamically created data structures, or you should limit the global data to only those structures that can be created at startup time. Another possibility is to use one of the semaphore primitives provided on Netware in order to serialize access to shared memory. In this respect, DNLs are similar to DLLs on Windows 3.1.
Once you have created the functions that you'd like to "publish" to consumers, you need to place the addresses of these functions into a table that is used for processing consumer calls to GetProcAddress. I've written a template called dnltmpl.c for doing this. All you need to do is make entries into a table, compile the file, and then link the object with your DNL code. Be sure that you do not use the EXPORT directive to the linker to export your functions. In Listing Two, I've declared the functions Foo and Bar to be static. That means that you could not successfully use the EXPORT linker directive even if you wanted to. An example of the modified dnltmpl.c code needed to publish the addresses of functions Foo and Bar is available electronically; see "Resource Center," page 3. The only changes I made to this code are the two entries in the array WorkerFunctionTable.
The initialization of WorkerFunctionTable has a comment as the first line. All that is necessary is to copy that comment, then change the entries to your function names, and, of course, uncomment your new entries. The third element of each entry deserves some mention. This is called the "ordinal." Besides retrieving a function address based on a query of its name, you can also retrieve the function address based on a number. This is optional and may not be supported by any particular DNL. To work, the ordinal must be between one and 0xFFFF (32767), inclusive. Also, ordinals should be listed in ascending order. There can be gaps between values, however. While ordinals are currently supported by the DNL system, it's beyond the scope of this article, so I will not discuss them any further here.
The only function implemented in dnltmpl.c is DNLMain, which is called by the main function located in dnlsup.obj. It is modeled after the DLLMain function used in DLLs to indicate various events on the DLL. Currently, only two events are supported, loading and unloading, though other events can be added in the future by simply adding new iReason codes and adding code in the switch statement to handle the events. I expect to add events to indicate whenever a new connection is being made to the DNL or an existing one is dropped -- in other words, whenever the management functions AddRef or Release get called.
In addition to an event code, DNLMain receives a pointer to a data structure. The actual data type depends on the event type. If the event is DNL_LIBRARY_LOAD, then the parameter pData is a pointer to a DNLStartStruct structure. This structure is defined in dnlsup.h (available electronically).
The reason that DNLMain receives a pointer to the entire start structure, rather than just to the WorkerFunctionTable pointer, is that I wanted DNL creators to be able to control the management functions of the DNL and not necessarily have to see (or be bothered with) their details. For example, if a DNL creator wanted to have a more sophisticated function name lookup than the simple sequential lookup that is provided by default, he could insert the address of his own GetProcAddress function into the management function table. (Typically, though, DNL creators will have no reason to change the code in DNLMain at all.)
Besides its main purpose of loading the address of the worker function table into the start structure, DNLMain gives DNL creators the opportunity to perform any one-time initialization that might be needed at startup. It's generally a good idea to hang all dynamic data off of a context structure allocated in response to some function that your consumer calls. But suppose there is some kind of management information needed by all instantiations of a given service DNL: DNLMain is the place to put that kind of thing.
DNL Example
To show how DNLs work, I'll present a consumer application called catinhat.nlm (that's right -- "The Cat In The Hat"). Now as you may remember (those of you with four-year-olds will certainly know), The Cat has some helpers. I've artificially limited the number of helpers to two -- thing1.dnl and thing2.dnl.
As you can see from Listing Three (catinhat.nlm), The Cat does the following:
1. Loads both DNLs thing1 (Listing Four) and thing2 (Listing Five).
2. Prints a "hello" message.
3. If thing1 is loaded, gets function addresses of DoOneThing and DoAnotherThing.
4. If function addresses were retrieved, calls indirectly through the function pointers.
5. Prints results of function calls.
6. Repeats steps 3, 4, and 5 for thing2.
7. Unloads the two DNLs and says goodbye.
While The Cat doesn't really do anything useful, what it does is significant. Both thing1 and thing2 support the same function names and prototypes. They are dynamically loaded and the functions are bound to variables in The Cat. Calls can then be made to the dynamic modules.
Examining Listings Four and Five, you see that although the functions DoOneThing and DoAnotherThing are supported by both modules, and their syntax is the same (that is, their prototypes are the same), but they differ semantically. Typically, you wouldn't expect such a large semantic difference between two implementations of the same interface, but I wanted to highlight the possibility that two dramatically different implementations of the same interface might exist in two different modules.
Conclusion
The DNL Manager presented here provides Netware programmers with a simple but powerful technique for dynamic linking (loading) and late binding. This can be applied to a number of problems. I am now looking at various ways in which it can be used for even more powerful location and binding methods. For example, DNLs could be used to contain object class factories that could create object instances on demand from consumers. I envision having a locator DNL that can locate class factories based on a DNS name looked up in a registry such as an LDAP or NDS directory.
DDJ
Listing One
void (* pFoo) (void);int (* pBar) (int iParam);
hDNL = LoadLibrary("SomeDNL");
if (hDNL)
{
pFoo = GetProcAddress("Foo");
pBar = GetProcAddress("Bar");
}
if (! pFoo || ! pBar)
{
// handle error condition
}
pFoo(); // Call Foo in SomeDNL
pBar(5); // Call Bar in SomeDNL
FreeLibrary(hDNL);
</p>
Listing Two
static void Foo (void)
</p>
{
...
// Foo code goes here
}
static int Bar (int iParam)
{
...
// Bar code goes here
}
</p>
Listing Three
#if 0 CATINHAT - Cat in the Hat (with apologies..)
This file: catinhat.c
Project Files:
catinhat.c - Source code for catinhat.nlm
catinhat.nlm - Executable implementing the library manager tester
Purpose: To test the dynamic netware library mechanism
Created 21 November 1997 by N. Thomas Creighton.
Copyright (C) 1997 by Novonyx, Inc. All rights reserved.
#endif // File header comments
</p>
#include "dnlmgr.h"
int main (int argc, char **argv)
{
DNL_HDNL hThing1;
DNL_HDNL hThing2;
int (* pDoOneThing1) (int iFirst, int iSecond);
int (* pDoOneThing2) (int iFirst, int iSecond);
int (* pDoAnotherThing1) (int iFirst, int iSecond);
int (* pDoAnotherThing2) (int iFirst, int iSecond);
int iFoo;
</p>
hThing1 = LoadLibrary("thing1.dnl");
hThing2 = LoadLibrary("thing2.dnl");
</p>
printf("\n\nHello from the Cat in the Hat.");
printf("\n\nI need some help from Thing1 & 2...");
</p>
if (hThing1)
{
pDoOneThing1 = GetProcAddress(hThing1, "DoOneThing");
if (pDoOneThing1)
{
iFoo = pDoOneThing1(5, 6);
printf("\n\tpDoThing1(5, 6): %d", iFoo);
}
pDoAnotherThing1 = GetProcAddress(hThing1, "DoAnotherThing");
if (pDoAnotherThing1)
{
iFoo = pDoAnotherThing1(5, 6);
printf("\n\tpDoAnotherThing1(5, 6): %d", iFoo);
}
}
printf("\n\nThing2 does things backwards from Thing1...");
</p>
if (hThing2)
{
pDoOneThing2 = GetProcAddress(hThing2, "DoOneThing");
if (pDoOneThing2)
{
iFoo = pDoOneThing2(5, 6);
printf("\n\tpDoThing2(5, 6): %d", iFoo);
}
pDoAnotherThing2 = GetProcAddress(hThing2, "DoAnotherThing");
if (pDoAnotherThing2)
{
iFoo = pDoAnotherThing2(5, 6);
printf("\n\tpDoAnotherThing2(5, 6): %d", iFoo);
}
}
printf("\n\nUnload Thing2..");
FreeLibrary(hThing2);
printf("\nUnload Thing1..");
FreeLibrary(hThing1);
printf("\nGoodbye!");
</p>
return 0;
}
Listing Four
#if 0 Thing1 - One thing to test dnlmgr
This file: thing1.cpp
Project Files:
thing1.c - Source code for thing1.nlm
dnltmpl.c - Source template for dealing with DNL stuff
thing1.dnl - Actual executable implementing the thing
Purpose: To test the dynamic library manager dnlmgr.
This file is the principle (or only) code source module for thing1.
Created 19 November 1997 by N. Thomas Creighton.
Copyright (C) 1997 by Novonyx, Inc. All rights reserved.
#endif // File header comments
</p>
// The following functions are what Thing1 do.
int DoOneThing (int iFirst, int iSecond)
{
// Return the product of iFirst and iSecond.
return iFirst * iSecond;
}
int DoAnotherThing (int iFirst, int iSecond)
{
// Returns the difference of iFirst and iSecond.
return iFirst - iSecond;
}
// Following is the WorkerFunctionTable declaration from dnltmpl.c
// as modified for Thing1.
static DNLFuncDesc WorkerFunctionTable [] =
{
// Place an entry here for each function you want people to have access
// to. If you don't want an ordinal, use 0. If a function is to have an
// ordinal, it must be in the range 0x1 - 0x7FF.
// Just copy the example entry and adjust the values.
</p>
{"DoOneThing", (void * (*) ())&DoOneThing, 1},
{"DoAnotherThing", (void * (*) ())&DoAnotherThing, 2},
};
Listing Five
#if 0 Thing2 - Another thing to test dnlmgr
This file: thing2.cpp
Project Files:
thing2.c - Source code for thing2.nlm
dnltmpl.c - Source template for dealing with DNL stuff
thing2.dnl - Actual executable implementing the thing
Purpose: To test the dynamic library manager dnlmgr.
This file is the principle (or only) code source module for thing1.
Created 26 November 1997 by N. Thomas Creighton.
Copyright (C) 1997 by Novonyx, Inc. All rights reserved.
#endif // File header comments
</p>
// The following functions are what Thing2 do.
int DoOneThing (int iFirst, int iSecond)
{
// Returns the difference of iFirst and iSecond.
return iFirst - iSecond;
}
int DoAnotherThing (int iFirst, int iSecond)
{
// Returns the product of iFirst and iSecond.
return iFirst * iSecond;
}
// Following is the declaration of WorkerFunctionTable in the
// dnltmpl.c used for Thing2.
static DNLFuncDesc WorkerFunctionTable [] =
{
// Place an entry here for each function you want people to have access
// to. If you don't want an ordinal, use 0. If a function is to have
// an ordinal, it must be in the range 0x1 - 0x7FF.
// Just copy the example entry and adjust the values.
{"DoOneThing", (void * (*) ())&DoOneThing, 1},
{"DoAnotherThing", (void * (*) ())&DoAnotherThing, 2},
};
Copyright © 1998, Dr. Dobb's Journal
