Every programmer is a toolsmith
Kent is director of the Three Rivers Institute and author of Extreme Programming Explained. Erich is with the IBM OTI Labs (Switzerland), leads the Eclipse Java Development tools project, and is a member of the Eclipse and Eclipse Tools project management committees. He also is a coauthor of Design Patterns. They can be contacted at kentthreeriversinstitute.org and erich_gammach.ibm.com, respectively.
Eclipse is an open-source Java platform for integrated development tools. However, Eclipseoriginally developed by IBM, then donated to the eclipse.org open-source projectis more than just a Java IDE. The whole of Eclipse is built out of layers and layers of "plug-ins," with only one layer specific to Java programs. While the plug-in-based architecture seems a purely technical design decision, it turns out to have important social implications:
- Every programmer is potentially a toolsmith, since extending Eclipse is both relatively easy and highly leveraged.
- Distributing and updating tools is easy.
- Creating opportunities for extension makes it possible for toolsmiths to enable others to further extend new tools.
In this article, we examine Eclipse plug-ins and how they address these issues.
Everything Is a Plug-In
When designing IDEs, you are faced with a set of pretty nasty contradictions:
- If you make it easy for others to change the internals, your interfaces and implementations have to be so general as to prohibitively expensive. You'd like to keep the system cheap by providing a fixed set of functionality, but...
- You can't possibly build all the tools most folks need because programming styles differ widely. So you have to give folks the ability to change things, but...
- An IDE is big, so you need scads of up-to-date documentation for potential extenders, which slows development of new features, but...
- Competitors can shave off a little flexibility and introduce features faster than you do. Your attempt to publish details about your internals has slowed your ability to grow the design of the system, which also eventually chokes the flow of new features.
Most IDEs slice this conundrum by providing a large, monolithic core with limited access to points where programmers can add functionality. Eclipse applies a combination of open source, careful segregation of public API and private implementation, and a pure plug-in-based architecture to provide more extensibility to programmers; see Figure 1.
When Eclipse boots, all that initially loads is a small kernel that knows where to find plug-ins and how to load them. As needed, Eclipse loads plug-ins that know how to interact with the operating environment, plug-ins for the basic user interface, the Java tooling plug-ins, and whatever plug-ins you've written.
Obviously, since everything is a plug-in, there are lots of plug-ins in Eclipse. A key challenge of this architecture is its scalability to hundreds of plug-ins.
Extensions and Extension Points
As Figure 2 illustrates, the menu items that appear when you pop up a menu can be extended by plug-ins.
Extensions need to know where and how they appear in the system. Extension points are the dual of extensions. An extension point is like a powerstrip, giving anyone the ability to plug-in new functionality. You tell Eclipse about extensions in the plug-in manifest file. In the manifest, you describe extensions in XML. If you want to add a new item to the menu in Figure 1, you can do so by declaring Example 1 in the plug-in manifest. When reading this, Eclipse notes that when the time comes to pop up a menu, if an object with the type IFile is selected (defined by the attribute objectClass), it should add a menu item labeled "Hello." When "Hello" is selected, it should create and invoke a HelloAction (defined by the attribute class), using the usual menu-action protocol.
Remember that all of Eclipse is built out of extensions to published extension points and all of those points are available for you to extend. You can add a new compiler, an interface to a new code repository, or even whole new applications as plug-ins. Looked at from this perspective, Eclipse isn't a Java programming environment (or a programming environment at all)it is a place to declare extension points and add extensions.
The goal of the Eclipse architecture is to scale to hundreds of plug-ins and for system startup to be proportional to the number of used plug-ins and not to the number of installed plug-ins (O(1), not O(# plug-ins)). The most important architectural decision to meet this goal is splitting plug-ins into a declarative part (the plug-in manifest) and an imperative part (Java code in a JAR file).
Each plug-in is represented as a directory. In the directory is a manifest file named "plugin.xml" that contains the declarative description of the plug-in. Most plug-ins also contain code (referred to in the manifest) in one or more JAR files.
All the plug-ins usually sit in the plug-ins directory. When Eclipse boots, it reads all the manifests and creates a map of all the plug-ins.
The code is not loaded at boot time. Reading a JAR file and loading its contained classes takes significant time. When your whole environment is built out of tens or hundreds of plug-ins, reading them all at boot time would result in glacial start-up times and a monstrous memory footprint. Consequently, Eclipse waits until a contribution is invoked before loading the code for its plug-in. In the example, we could monitor and see that our HelloAction class wasn't loaded until the Hello menu item was selected. Even just popping up the menu with the item on it isn't enough to cause the class to load. By design, the manifest contains enough information to display a contribution to users without invoking any contributed code.
The goal of this architecture is to preserve fast booting independent of the number of plug-ins. The cost of loading the code is spread out over the initial uses of features in the system.
Eclipse enables programming-as-tool-building by providing a rich set of raw materialsin this case, Java APIsfrom which to construct tools. Wise tool-building programmers are powerful, though, because their time is leveraged.
For instance, we were working in a Seattle hotel room the other day when we discovered we needed to know what the available built-in icons were and what they looked like. Two minutes later we had the answer. Figure 3 is the view we built that displays the standard icons and their names.
In a way, Eclipse reminds us of Emacs, in that folks are writing plug-ins for e-mail, news, Google, and chat. The resulting programs may not be any great shakes as standalone programs, but don't have to be. By taking advantage of the fact that they are inside a programming environment, you can create a communication tool that is more useful to programmers than any general-purpose tool, no matter how rich. There are already many plug-ins available (see http://www.eclipse.org/community/index.html).
The Eclipse story isn't done with contributing to an extension point. You need to also think about how to structure your plug-in so other people can add their extensions. You do this by declaring your own "extension points."
For example, suppose Kent wrote a tool, "Spider and Eclipse," to draw diagrams of objects. When using this tool to analyze our quick hack icon view, we can see: SampleView has a TableViewer, which is configured with our ImageContentProvider that fetches the list of icons and the ImageLabelProvider that presents an icon as an image in one column and a name in the second; see Figure 4.
"Spider for Eclipse" is just one example of an introspection plug-in that helps you gain insight into Java programs. When you click on an object in Spider, a set of handles pops up around the object. In Figure 4, you see three handles attached to the TableViewer:
- Expand a field.
- Expand an attribute (no-argument method.)
- Delete an object.
Now let's see how to make the Spider extensible. Erich sees the Spider View and says, "Sauglatt! I'd like a handle that opens an editor on the source code of the object." Kent has thought about this in advance. He has opened the plug-in for such extensions by defining an extension point for handles:
<extension-point id="handles" name="Handles"/>
Spider's own handles are written using this extension point as well. Erich can now implement a plug-in and declare the presence of his handle, as in Example 2. After installing and loading the new plug-in, his handle appears as in Figure 5.
To implement the behavior of his handle, he writes a ShowSourceHandle class including the code like Example 3 (using the Eclipse APIs provided by the Java tooling plug-ins). Now when users click on the Show Source handle, the source code appears. Here are a few things we notice about this scenario:
- Kent didn't have to do anything specific to prepare for Erich's extension of Spider. The work required to write the original tool as extensions of extension points paid off immediately for Kent.
- Erich didn't have to communicate or coordinate with Kent in any way to add the functionality he wanted. The declaration of the extension point and the interface that defines how a handle is treated were enough.
- If someone else wants to add their own special handle, they can do so without coordinating with either Erich or Kent.
- Kent, however, is deeply constrained about the changes he can make to his interface. The XML definition can be extended but not changed, and the common Handle interface can't be easily changed at all.
All this extension/extension point configuration was done with the help of the Plug-in Development Environment (PDE). Kent could have helped Erich even further by defining a schema for the handles extension point. Then PDE would have even guided Erich when defining his handle extension; see Figure 6.
All this talk of tool writing brings up another pointhow do tools get distributed in Eclipse?
First, the simple way of zipping up a plug-in and making it available on the Web for others to download and unzip still works. However, there is a better way. Installing plug-ins should be straightforward. In particular, given the modularity of Eclipse where everything is a plug-in and there are many plug-ins, it is critical that the set of plug-ins is still manageable. Otherwise, an Eclipse installation ends up as a morass of plug-ins. The implementation of a tool can range from a single plug-in up to dozens of plug-ins. To simplify the plug-in download, installation, and management, Eclipse has "features." A feature consists of a set of plug-ins. Features are the unit of download and installation. Example 4 shows a simple feature description for distributing the Spider plug-in. You won't have to remember all the details of the XML. PDE provides tools for all these plug-in development tasks.
The Eclipse Update Manager (Help|Software Updates|Find and Install...) uses the feature information to install or disable a feature. By doing so, you are protected from ending up with inconsistent Eclipse installation configurations.
Once you have the plug-in written and the feature described, you can publish your tool for easy installation on an "update site." An update site is a URL addressable locationfor example, a directory on your web site (in our case, it will be http://www.javaspider.org/). The storage mechanism for an update site is simple. It consists of features and plug-ins packaged as JARs whose names include a version identifier (1.0.1, for instance). The content of the site is described in a site.xml file. Example 5 is a simple site.xml for an update site offering the Spider feature.
Now, all you have to do to make your tool available is to create JARs for your plug-ins and the feature and update or create a site.xml file. Eclipse supports signing feature JARs. Figure 7 shows the Eclipse Feature Updates dialog you use to install or update features.
With the Update Manager, you can create bookmarks for sites you are interested in. Once you've selected a feature, you can install it at the touch of a button. Tools are usually continuously refined and improved. Therefore, the Eclipse update manager supports you in finding and installing the latest and greatest feature updates, including automatically checking for updates when Eclipse starts up.
We began this article with a list of conundrums for writers of development environments. How does Eclipse resolve these seemingly contradictory constraints?
Clearly, Eclipse is easy to extend. Because plug-ins were used to build all of Eclipse, the extension point/extension mechanism is polished by long use. Because Eclipse is built in terms of extension points, all those points are available for tool builders to use.
Eclipse is also well documented. Here is where Eclipse breaks with traditional tools. The online help provides a thorough overview of the available extension points, and Javadoc provides an outline of the APIs. Much of the burden usually assumed by printed documentation, though, is carried by the availability of source. As an extender, you will spend a fair amount of time just reading how other folks have already used an extension point. While this may not be the most perfect form of documentation theoretically, practically speaking, it's fantastic. You know the examples are up to date and, since you use the same extension points to write tools as Eclipse uses internally, there are always examples. Finally, Eclipse comes with a full-blown Java IDE that helps you with searching and navigating the source.
We appreciate the gentle, nurturing editing of Andrei Weinand, Greg Wilson, John Wiegand, and John Kellerman.