William Nagel is the Chief Software Engineer for Stage Logic. He specializes in developing CORBA-based real-time systems. He can be contacted at [email protected].
While C++ is a powerful language, its lack of built-in language support for dynamic loading is a disappointing shortcoming. There are, of course, ways to get around the limitation, but they are all clunky or nonoptimal from a performance standpoint. Frustrated with the hacked on C-based solution for loading that is typically used, I decided to wrap it up in a C++ wrapper, so that my programs would no longer be kludged together (and error prone) when dealing with dynamically loaded objects.
On many UNIX and UNIX-like systems (such as Linux), there is a dynamic linking library that can be used for loading symbols from a shared object file. This works well for libraries compiled from C, where the symbol name is the same as the function name from the source. C++ compilers, on the other hand, have to deal with additional overhead such as namespaces and class names, which must be encoded into the symbol name. Compounding the issue, there is no standard format for this encoding.
These mangled names, though, only become an issue if the main program is attempting to directly access C++ symbols in a shared object file. Since C++ programs can call C functions, the mangling issue can be worked around by using entry functions in the library file, which are used to access the hidden C++ symbols.
But again, using these functions can be cumbersome and error prone. Therefore, to bring things back into a safer, more C++ paradigm, I created a library that applies object-oriented C++ techniques and a few design patterns to allow dynamic loading inside programs to be addressed with a much more elegant approach.
The library that I describe here is designed to work in UNIX-like environments, where the dynamic loading library is available. Although I don't have first-hand experience doing dynamic loading on other platforms (such as Microsoft Windows), my understanding is that most other platforms have their own, equally painful, methods for dynamically loading an object. The specifics of the dynamic loading library that I describe here may not apply to other platforms, but the techniques used to wrap the dynamic loading library should prove portable.
The Dynamic Loading Library
The dynamic loading library lets you dynamically load symbols from an object file. That's it. Nothing fancy, just loading.
When a source file is compiled, it is placed in an object file, which contains the functions, structures, and any other entities that are a part of the compiled source. To allow other object files to link to a given object file and call its functions (among other things), each of the elements in an object file are associated with a symbol, which acts as a key for looking up the element.
In a compiled C program, the symbols in an object file directly correlate to the ASCII function name for the function they represent. This makes dynamically linking to a shared library that was compiled with C very easy. For example, if you want to call a function named get_foo(), the symbol name would be get_foo. When the source being compiled is C++ instead, the principle is the same, but the symbols are not as clear.
To load a symbol from a shared library file, applications first open the library, using the dlopen() function, then retrieve the desired symbol with dlsym(). The dlopen() function takes a filename, which can be either a fully qualified path name or a relative path (in which case dlopen() searches for it in the normal places where one looks for a library). A successful dlopen() call returns a void pointer handle, which can then be used by dlsym() to load the symbol, with a "symbol" parameter containing the name of the desired symbol. When dlsym() finds a symbol, it returns a function pointer that can be used to call the loaded function. Example 1 shows how you might use the dynamic loading library in a C program.
Loading C++ Symbols With the DL Library
Loading C++ symbols with the dynamic loading library is trickier. Symbols for a C program are simple because C has no namespaces. Since names are all global, it makes sense to just use the function name for the symbol. With C++, there are a number of different constructs that place scope on names, such as namespaces and classes. This means that when a C++ compiler creates the symbol for a C++ function, it uses some sort of mangled form of the function name and namespace identifiers. To compound the issue, there is no standard for what that mangled form will be. The result is that there is no reliable way for C++ programs to load a symbol through the dynamic loading library using only the symbol name.
However, C++ functions can call C functions compiled with C-style symbols. C++ also provides a mechanism for indicating that specific sections of C++ source code should be compiled with C symbols, the extern "C" keyword. Sections of code contained within an extern "C" block are loadable dynamically just as if they were in a library compiled by a C compiler, even if they are mixed in with sections of code compiled with normal mangled C++ symbols. Furthermore, an extern "C" section is otherwise compiled as C++. This means that the code inside extern "C" functions have full access to C++-linked elements. In this way, a C++ function can dynamically load a function with a C-linked symbol name, which can in turn instantiate a C++ object and return a pointer to it.
Listing 1 shows how to load a C++ object through the dynamic loading library. In it, a C++ program loads a C++ shared library, libLowPass, which contains a loading function, loadFilter(), compiled with the extern "C" keyword, and a class, LowPassFilter, that derives from a Filter class known to the program. To get an instance of LowPassFilter, the program loads the symbol loadFilter, then calls it as a function. The loadFilter function then instantiates an instance of LowPassFilter, and returns a pointer to it as a void pointer. The program then has to cast the void pointer to a Filter, using a reinterpret_cast.
Looking at Listing 1, you also see that the shared library has one additional function compiled with extern "C", the deleteFilter() function. This function is necessary because objects that are instantiated inside a dynamically loaded library must be deleted inside the same library. If the object is deleted inside the program itself, the results are unpredictable. It may work fine, or it may cause a memory leak or even a program crash. If your program redefines new or delete, a program crash is almost guaranteed.
A Dynamic C++ Loading Library
Using the DL library to load C++ objects works, but it isn't pretty. What I really wanted was a library that would let me deal with loading and unloading dynamic objects in an object-oriented fashion. Consequently, I solved the problem by applying a couple of simple patterns and some C++ tricks to design a C++ dynamic loading library.
Starting out, I had a couple of design goals for my library:
- I didn't want to directly call any C functions; everything was to be object oriented.
- I wanted to be able to load a class given only the class's name in ASCII.
- I didn't want anything outside of my dynamic loading library to need to deal with a void pointer or reinterpret_cast.
- I wanted to be able to delete an object given only a reference to that object.
To keep everything object oriented, I created a dynamic library wrapper class. The DynamicLibrary class encapsulates the loading of objects from a shared library, hiding all of the low-level details from the rest of the application. A DynamicLibrary is created through the static DynamicLoader class, which loads a library by name, and creates a new DynamicLibrary to hold it. Once a library has been loaded, the DynamicLibrary can be used to create new objects by calling the newObject() function. The DynamicLibrary handles closing the library when it is destroyed by calling dlclose(). Listing 2 shows the C++ declaration and definitions for the DynamicLibrary class.
A library loaded by DynamicLibrary must have a loadObject() function. When the program calls newObject() on an instance of DynamicLibrary, newObject() in turn calls loadObject() on the dynamically loaded library, and passes it the name of the class to instantiate. The loadObject() function acts as an abstract factory for dynamic objects, and is responsible for determining which class to load based on the class name it is given. If the class requested isn't found, it returns a null pointer.
I didn't want to just accommodate the default constructors for objects, so I needed a way to pass an arbitrary number of arguments, of arbitrary types, through to the constructor in the shared library. This turned out to be the one case where I was not able to avoid any use of void pointers in the application using the library. When the program calls newObject() on the dynamic library, it passes an argument counter, with the total number of arguments and an array of void pointers, which contain the actual arguments.
For small libraries with few loadable classes, a sequence of if statements may give sufficient performance when looking up the class to load in loadObject(). The performance of if statements degrades linearly (O(n)), though, and can quickly become a performance bottleneck if you need to load lots of objects or have a library with many loadable classes. If this is the case, I suggest using a tool for generating perfect hash functions (like "gperf," for example), to generate a hash of the loadable class names, which significantly increases lookup performance (O(1)).
I wanted to avoid dealing with void pointers or calls to reinterpret_cast, so I needed to make a common base class from which all dynamic classes can derive. I also need to have my program know something about the interfaces of the dynamic objects it's going to be loading, so it can do something useful with them. To achieve this, the program needs to know about two different classes that a dynamically loaded class needs to derive from. The first is a standard class for every dynamic class, called DynamicObject (Listing 3), which defines a deleteObject() function and operates as a common base class for all dynamically loaded objects. The second class that a dynamically loaded object needs to derive from is a program-specific class, which defines functionality that the program knows it will need to use the object for. The dynamically loaded object should derive from the program-specific base class, which should in turn derive from DynamicObject. If you need to be able to dynamically load subclasses of a base class that already exists, but doesn't derive from DynamicObject, you can easily achieve what you want by creating a subclass that derives from both your desired base class and DynamicObject.
Using delete to destroy a dynamically loaded class doesn't work from the main program, as previously mentioned. A dynamically loaded class must be destroyed from a function inside the shared library it was loaded from. In my design goals, though, I stated that I wanted to be able to destroy an object given only a reference to it. To achieve this, I gave DynamicObject a method for deleting itself. I also gave it a constructor parameter requiring a pointer to a delete method. In this way, when loadObject() creates the object, it passes it a pointer to a deleter method inside the shared library. Then, when the main program invokes DynamicObject::deleteObject() on the object, it calls back to the method in the library, and passes a pointer to itself, which is then deleted.
Using the Library
Using the dynamic C++ loading library is straightforward, and much cleaner than dealing directly with the UNIX dynamic loading library. Since it provides a facade that deals exclusively in C++ objects, the program never sees a void pointer, and is spared from using the unsafe reinterpret_cast to turn a void pointer into an object.
Listing 4 is a program that uses the library to load different filters for an audio effects program. In the example, the program uses the dynamic C++ loading library to load a high-pass filter object, does some processing with the filter, and then deletes it. As long as the shared library returns a valid class that derives from DynamicObject and Filter (which should be easily verifiable when the shared library is written), the program should be entirely type-safe.
Linking Everything Together
Creating your application so that everything links together properly can be tricky. The dynamic C++ loading library is, of course, a shared library. It is especially important that the DynamicObject class be in a shared library, since both the program and the dynamically loaded library need to know about it. What's not necessarily so obvious, though, is the fact that the program-specific base class from whence the dynamic object derives also needs to be in a library, since the dynamic library needs to know about it, too, and symbols that are statically compiled into a program are not exported to shared libraries when they are loaded.
Example 2 shows the commands that would be necessary to compile Listing 4. The first step in compiling the example is to compile the base Filter class into a library of its own. This ensures that the dynamically loaded library will have access to Filter when it is loaded. Once that is done, you need to compile the HighPassFilter shared library, which is dynamically loaded, then compile the actual application. You will notice in the example that the audio_processor program is linked with the Filter, the dynamic C++ loading library, and the C dynamic library, but not with the HighPassFilter library.
