A.J. Musgrove is the Vice President of Strategic Consulting at Cymbal Corp. He can be contacted at ajmusgrovecomcast.net.
Dynamically adding new functionality at runtime greatly enhances the flexibility, robustness, and longevity of applications. While the usual approach to adding functionality is to introduce changes into the core source, it is often easier and more reliable to create plug-ins that fit into a predefined framework. Real-world uses include loadable network transports, protocols, formatting engines, security services, transaction services and management, and administration services.
To see the usefulness of plug-ins, all you have to do is look at Apache and J2EE. Apache has a well-defined plug-in framework and many plug-ins have been created for it, including mod_perl and mod_tcl. From a different platform, Java's J2EE shows the power of a fully developed runtime configuration environment. The entire implementation of J2EE Application Servers depends on the runtime loading and inspection capabilities of Java.
Like everything else in software, using plug-ins starts with design. You must formally define what functionality is plugged in and what functionality is built using traditional methods. The complete source code that implements thisincluding build scriptsis available at http://www.cuj .com/code/.
A plug-in is a compiled module you load into your program, initialize, and use at runtime. The specific plug-in can be created at any time during the application's life cycle, with or without access to the application's source code. Having an application load specific plug-ins is usually just a matter of a configuration change.
Physically, a plug-in is a shared object (.so) or shared library (.sl) in Linux/UNIX, or a dynamic link library (.dll) in Windows. For simplicity, I refer to these as "shared libraries." The shared library has a list of symbols that are normally method and variable names. Each of these symbols is marked as either present in the library or as required from another module (unresolved).
Symbols have a visibility. In UNIX, the visibility of a symbol is Global, Local, or Weak.
- Global symbols are unique across the process and are listed in a data structure called the "Global Symbol Table." Global symbols are visible to the entire program by their name.
- Local symbols are only visible in a given compile unit (file) and are not visible to the rest of the process, at least not by name.
- Weak symbols are functionally the same as Global symbols, except that if they conflict with a Global symbol during program link, they are suppressed in favor of the Global symbol. In UNIX, C-static functions and variables are Local, and everything else (including C++-static methods) is Global.
In Windows, symbols are either exported or local. Exported symbols are like Global symbols in UNIX, and local symbols are like their UNIX counterparts. To make a symbol exported, it has to be tagged as such. In Visual C++, this is done by marking classes and functions with _declspec(dllexport).
When a shared library is loaded, its included Global symbols can be put into the process's global symbol table, allowing other modules to use those symbols by name. Once those symbols are loaded, the shared library's unresolved symbols must be resolved. The loader attempts to match these up with symbols in the process's global symbol table. If any symbols cannot be resolved, the load fails. In UNIX, the symbol resolution can either be immediate or take place on an as-needed basis (called "lazy resolution").
For symbol visibility, symbols can either be found or not by name. Because a function member is Local in a module does not mean that it cannot be executed from another module, it just means that its name is not valid in that file. If you somehow pass the address of the function to a function in another module, it can execute the Local function in another file with the address. For instance, signal handlers can be Local (C-static) but the address of the function is passed using the signal() function.
Writing a Plug-In
The only part that must be written in C is the initialization routine. You could also code that in C++, but then you have to worry about name mangling to find the routine, and name mangling is compiler dependent.
The keyword static is one of those overloaded keywords that means something different in C and C++. In C++, "static" simply means that there is no initial this pointer parameter in method invocations. The affect of static is purely for the compiler and makes little difference in the finished object file (except for the symbol name, of course). C++ static methods have a Global visibility, just like normal C++ methods.
In C, "static" has an entirely different meaning. If a method or variable is declared static, it is marked as a Local symbol within its final object file. This means that any number of files can have C static functions with exactly the same name and they will not conflict. Of course, those functions will only be visible to other functions within the same file.
If you are going to plug in new functionality that you do not know about in advance, how do you know what it will be named and how do you initialize it? You solve this problem using a C static function (and in Windows, that's the _declspec(dllexport) function). All initialization methods for a plug-in have the same name, which you define upfront. The method must be C-static so that the symbol is Local and you won't have conflicts among multiple plug-ins.
To load a plug-in, instruct the dynamic loader on your platform to load the library. Once it's loaded correctly, find the symbol for the initialization method. Remember that you must use a special function to find the function's address, since it will not be exported into the process's global symbol table. Once you find the function, run it.
Listing 1 is the header for an example plug-in manager. For now, ignore everything except the typedef for plugin_init_func(). Notice that the initialization function is responsible for returning a concrete object of type Plugin. Listing 2 shows the source code for the first example module. Notice that it fulfills the contract by returning a new EnglishTestPlugin, a specific type of Plugin.
Loading the Plug-In
Writing the plug-in is straightforward and almost completely platform independent (with the minor irritation of _declspec(dllexport), which I take care of with a #define). Loading the plug-in is not nearly so simple because UNIX and Windows use completely different APIs for these purposes.
Referring to Listing 3, SharedLibrary.h, you see the abstraction of these two different APIs into a common module. The implementations of the class SharedLibrary are platform specific.
Referring to Listing 4, UnixSharedPlugin.cxx, you see how I implement this functionality on UNIX. To open the UNIX shared library, I use the function dlopen. I give dlopen the name of the module I would like to load; if you pass NULL as the name, you get a handle to the currently running global symbol object, giving the ability to process the global symbol table.
Notice that I pass in the flags RTLD_NOW and RTLD_GLOBAL to dlopen. RTLD_NOW tells the loader to resolve all internal references to external code immediately. The opposite is RTLD_LAZY, telling the runtime system to resolve functions on an as-needed basis. RTLD_GLOBAL tells the loader to use Global symbols within this module to resolve unresolved symbols within other modules that are loaded (the opposite, and the default, is RTLD_LOCAL).
For any errors that occur, dlerror retrieves the error text. dlerror can only be called once; any subsequent call without an error will return NULL (two calls in a row to dlerror always returns NULL). Calls to dlerror change its internal state, which itself depends on previous calls to other dynamic loader functions. That, by itself, makes these APIs thread-unsafe. The dynamic loading APIs are not thread-safe and proper locking must be used in a multithreaded program.
The destructor for UnixSharedLibrary calls the dlclose function. This closes a shared library. The shared library will not be unloaded from memory until there are no more handles held open to it and none of its symbols are used by other modules.
dlsym finds symbols in the library. This function returns a pointer to a symbol given its name; it's up to the program to ensure this pointer is used properly (it's just a simple void pointer). If dlsym returns NULL to indicate an error, the error text can be retrieved using dlerror.
To do its work, dlsym requires a handle. The handle is normally one returned by dlopen. The special handle RTLD_NEXT tells the loader to look in each module loaded after the one dlsym is called from for the symbol. This function is not terribly useful for our purposes and is typically used by things such as memory profilers.
Listing 5 is my Windows implementation of the shared library loader. Like most Windows-equivalent APIs, it is a bit more complicated than UNIX. (I'll leave the debate as to whether this adds richness or unneeded complexity to those who enjoy discussing that sort of thing.)
In Windows, LoadLibrary loads a DLL into the process and handles loading-dependent libraries and resolving symbols. It returns a handle to the library. If the handle is NULL, that indicates an error. The error code is returned using the GetLastError API, which can be converted to readable text using the FormatMessage function (see the source for an example of doing that).
The Windows equivalent of our dlclose method is FreeLibrary, while the Windows equivalent of dlsym is GetProcAddress.
The shared-library abstraction provides only the most basic way to deal with plug-ins. A plug-in manager is required to give a useful framework on top of the shared-library abstraction. That manager should return a plug-in given a name (optionally loading it if it is not resident). It should also handle initializing a plug-in when it is loaded. Listing 6 is the source for the plug-in manager I present here.
The important logic is in PluginManager::findPlugin. The basic flow is that when a plug-in is requested, an internal table is consulted to see if the plug-in is already loaded. If it is not loaded, the plug-in manager loads the plug-in and attempts to initialize it. Assuming that all goes well, the plug-in is returned to the caller.
The plug-in manager must be told what plug-in to load. A naive implementation would just give the plug-in manager the physical shared-library name. It is better to have some external configuration that goes from a logical plug-in name (say, _program_text) to a library name (say, _EnglishTextPlugin.dll). When new plug-ins are written in the future, the name in the configuration just needs to be changed.
From Plugin.h, you see I define the class PluginNameMap. Its responsibility is to convert logical names to physical names. The example implementation looks up the logical name in the environment using getenv. It expects to find the physical name there.
Putting It all Together
The plug-in example is enabling a simple program to have multiple languages and load the language driver at runtime. There are, of course, much better ways to enable multiple languages, but this makes a great "Hello World" style example. You have already seen the EnglishTextPlugin. The SpanishTextPlugin is included in the complete source code.
The Text drivers are expected to be of type TextPlugin, which itself inherits from Plugin. During casting, I use C++'s dynamic_cast to ensure the type correctness. Without C++-style casting, other, less-reliable mechanisms must be used to ensure the loaded plug-in is a type that is expected (such as using the Plugin::pluginType method in conjunction with C-style casting). Listing 7 shows the abstract class TextPlugin. The method that must be implemented is getSayHelloString.
Listing 8 is main.cxx, the main driver for this example. It simply loads a plug-in named _language. The plug-in manager resolves this logical name to a physical name using getenv, so the library name (either EnglishTextPlugin.dll/.so/.sl or SpanishTextPlugin.dll/.so/.sl) must be in that variable and be able to be found in the path.
Of course, building this example is different on almost every platform. For convenience, I have included a build script that will build this example on Windows, Linux, Solaris, and HP-UX in the complete source code as build.sh.
If you build and run this example and switch _language between Spanish and English, you'll see that things are changing at runtime depending on the environment.
A few years ago, this article would have been written and published without a mention of security. These days, however, it is important that security be considered. With plug-ins, remember that code will be loaded into a process and then run as that process. If your program can be tricked into loading a mischievous module, the code in that module will get to do whatever that legitimate program could do.
Once you get the hang of it, you can put together applications that are extremely flexible and have a long lifespan. They will also be insulated against requirement changes impacting the core source of a system. When designing your system to load code at runtime, don't forget proper design of generic interfaces. During deployment, configuration and security are the central considerations. In the end, dynamic applications are often more useful and valuable than their static counterparts.