Is Open Source for You?

If your team is anything like mine, everyone has a nickname. In some companies, a newcomer practically needs a shadow phone book just to follow a conversation. Me? Honest, I have no idea why they call me Gadget Boy. Sure, I like cool stuff, but who doesn't?

I didn't bring open source software to our group; Science Girl was using Perl long before I came on board. I was responsible for our first Apache installation, though, back before Time and Newsweek had ever heard of open source. Then The Finn needed an SQL database; MySQL was stable, open and free. I tried out Linux on a server. Pretty soon I couldn't stop. Every day, you could hear the Gadget Boy Battle Cry: "Oooh, I NEED that!"

We have JServ, Samba, Logcheck, Tomcat and Tripwire. Cocoon, Portsentry, JDOM, OpenSSH and Ethereal. Xerces, Xalan and all the rest of the X files. Most of it works quite well, and lets the small, university-based group I'm part of do things we could never accomplish if we had to write everything ourselves or buy commercial (the Free Software Foundation prefers the term "non-free") alternatives.

But what about you? Do you have the luxury of selecting your own architectures and technologies? Can you afford to give your source away to your competitors? The answer, of course, is "it depends." In the first half of this article, I hope to give you some flavor for what "it" depends on. I'll talk about the viability of open source as a development model. Next month, I'll examine a sample project, implemented with open source tools.

What's the Buzz?
So what does "open source" mean, exactly? The Open Source Definition (www.opensource.org) and the GNU Public License (www.gnu.org) go into a great deal of sometimes conflicting detail on this, but in the end, open source boils down to software whose license terms allow you to read, compile, redistribute and modify the source code. (Contrast this with "public domain" source, which has no restrictions whatsoever.)

In 1985, GNU Project founder Richard Stallman created the Free Software Foundation, a tax-exempt charity for what he called "free software." The term confused many of us. Did that translate to "you don't have to pay money for this," or "you're free to copy and modify the source code"? It turned out they mostly meant the latter, as explained in the famous "it's not free as in 'free beer' but free as in 'free speech'" analogy.

Determined to circumvent the confusion and the combativeness that was perceived to emanate from the free software camp, in 1998 a group of hackers met in Palo Alto, California and brainstormed a new name for nonproprietary software, to Stallman's dismay. To read Stallman's refutation of the "open source" label, go to www.gnu.org/philosophy/free-software-for-freedom.html.)

Open source projects have undeniable counterculture cachet. It's fun to be fashionable, but just what does it get you? First, let's look at the perks and pitfalls of using open source tools. Taking your own projects into the open source realm is another can of worms, entirely.

Shallow Bugs
According to its advocates—and setting aside the social agitprop for now—the strongest advantages of open source software are quality and reliability. If a project can attract enough users and developers, it can leverage The Cathedral and the Bazaar author Eric S. Raymond's famous epigram: "Given enough eyes, all bugs are shallow." My experience has been mixed. Infrastructure software tends to be world-class and rock-solid. End-user applications, however, often haven't yet had enough trips around the spiral, and many of them are flakier than their proprietary counterparts. But they're disarmingly frank about it—unlike some of their proprietary brethren with a penchant for "gamma testing" (shipping known buggy software). If you're using version 0.2 of a tool, you know there are going to be bugs.

Another cited advantage to having the source is the ability to debug it, add your own features or learn how things work. I have to admit that diving into, say, the XFree86 X Window server is a pretty daunting prospect to me. But if my business depended on it, I'd don my snorkel and flippers.

Finally, when you use open source software, you're not locked into a single company. You do become somewhat beholden to a community, but on the whole, they're usually willing to take suggestions. To me as an in-house developer, this is a compelling argument, and at least one major tools developer sees it the same way. While developing the Kylix rapid application development (RAD) environment for Linux, Borland had to work closely with open source projects (particularly the system-libraries group).

"From our standpoint, we're not making a religious statement in open source," says Michael Swindell, director of product management for RAD products at Borland in Scotts Valley, California. "We're a builder of tools, and we need to make sure that our tools will fit whatever model the user wants to use. One of the great things we've found about the open source/free software community is the willingness to listen, to take fixes and to talk about them."

Of course, every silver lining has its cloud. The level of documentation and support for open source tools varies wildly. For example, Apache, Linux, MySQL and PostgreSQL all have manuals, books, consulting and phone support to rival their proprietary counterparts. At the other end of the spectrum, suppose you want to work with the Cocoon ESQL tag library (I certainly do, as it's a terrific little thing). The single Web page simply refers to two example files, then chills the blood with "The ultimate reference is, of course, the source code…" Gulp.

This points out another common problem with open source software: great programming, but not much process until later in the life cycle. As a product's popularity climbs, it attracts more resources and can move from "inspired hacking" to "software engineering."

Finally, what about Windows? Open source is closely identified with the GNU project, Linux and Unix in general—do any of these things work in Windows? Indeed they do—some of them, anyway. Some (Mozilla, for example) even use Microsoft's compilers, while others use the Cygnus Windows Toolkit (http://sources.redhat.com/cygwin/) to provide Unix-derived programs the environment they expect. In general, the big, popular products such as Apache and MySQL tend to have direct Windows ports available, even if the Windows support lags; the Apache Server Web page, for example, currently says that the Win32 port of Apache is "not as stable as the UNIX version." The Macintosh isn't completely ignored, but it's by no means as easy to find as Windows support or Unix ubiquity.

Property, Shmoperty
The list of open source projects that have achieved at least some modicum of success is impressive, and there's a boatload of wannabes. SourceForge (http://sourceforge.net) is a free hosting service for open source projects. Its Web site lists 16,620 hosted projects as I write this (having grown by some 1,500 projects since I started researching this article in January).

Proponents argue that such products as Apache, Cocoon and Linux are not just useful, they're best-of-breed—some define whole new categories. Some of the more intemperate boosters, intoxicated by this success, claim that proprietary software is evolutionarily obsolete, that the very idea of intellectual property rights in software is a dead end, that it's immoral to keep your source code secret and that cooperation will win over competition everywhere, in every domain, every time.

Well, maybe. I'm dating myself terribly here, but my generation heard a similar refrain 30 years ago: "It'll all be beautiful if we just join hands…"

Harsh I'm not. I really am a believer in what free (as in speech) software can accomplish. Frankly, whenever I have a choice, I go for the open source process and its products every time. But don't try to tell me that there's only one model for developing software—for one thing, everybody in the industry has heard that tune before. "It's structured programming! It's artificial intelligence! It's functional programming! It's objects! It's the Capability Maturity Model! It's components! It's UML!"

For another, counterexamples are just too easy to find. The One True Way? Tell that to management, when you explain why it took your team eight months to create a robust, scalable e-business infrastructure, and your biggest competitor did it in three weeks. "Well, see, they downloaded our code and…" Don't let the door (more likely, your CTO's foot) hit you in the fanny on the way out. Or perhaps you really have devised some high-flying, better software mousetrap for end users, and want to reap the rewards for awhile before you fall back down in the service-model weeds with the rest of us. More power to you.

But clearly, once we've dodged the wilder-eyed warriors of the Free Software Jihad (you can even run Windows if you want—honest!), there's plenty to be gained from the movement. Many of us can even contribute without crippling our companies' competitive positions. How?

It's a Service, Not a Product
For one thing, you can just use the tools. In the second part of this article, I'll show you an example of doing just that, in a real-world, three-tier project. If you can get the job done this way, you really can reap lower overall cost, crash-resistant systems and freedom from vendor lock-in. As for your contribution, simply by expanding the ubiquity of, say, Apache, you'll be helping the project, especially if you remember those bug reports.

If you want to get more involved, you can participate in development. Many open source authors have "real" jobs just like you and me, but they either work on open source products in their free time or get their employer's permission to do a bit on company time. These projects require more than programmers, too—most have a crying need for writers, testers, analysts and designers.

Finally, you might take the plunge into making your company's projects open source. This is the scary one, and it works only for certain products—those you want to promote as a platform to leverage other products, perhaps, or ones for which you can invent a business model based on services, or ones without much of a direct-sales revenue stream. In Eric Raymond's essay, "The Magic Cauldron", he argues that today, "software is largely a service industry operating under the persistent but unfounded delusion that it is a manufacturing industry."

Scary, indeed. Still, it worked for TCX, the company that wrote MySQL, and it seems to be working for Netscape. Borland should be shipping Kylix ("RAD for Linux") by the time you read this, with its open source CLX component library, and has brought Interbase into the open source domain.

"It's certainly a challenging road," says Ted Shelton, a Borland senior VP and chief strategy officer. He's confident, however, that the commitment to open source was the right decision: "We've already seen an enormous return on open source—they can do it much more rapidly and cost-effectively than we'd ever be able to do internally." For example, he says, when Interbase went open source, the community pounced on a security problem in Interbase "that had never been uncovered by our developers. It is different, though, the main difference being that the community talks back to you and tells you 'No, I want to work on this instead.' [Laughs.] At the end of the day, when you incorporate changes back into your final version, you have the chance to decide whether or not to include it."

Raymond points out that most of us are working on in-house projects anyway, and on maintenance at that—what's the harm in open sourcing the code, so long as it doesn't contain business secrets? (See "The Magic Cauldron" for some thought-provoking details.) Borland's Shelton also points out that if you're doing proprietary in-house development, "turnover in engineering can lead to mysteries in your code;" having an external community, too, can also insulate you to some extent.

Ready to Open Up?
So, which projects are suitable candidates for migrating to open source? Four principles define whether throwing competitive caution to the wind—at least with regard to writing or getting your code or tools—is advisable.

Rule 1: Horizontal Is Good. Generally speaking, the broader the market is for a piece of software, the better chance an open source flavor of it can succeed and perhaps even make money. Linux and Apache are the most obvious examples. Everybody needs an operating system, and darned near everybody needs a World Wide Web server. Not only does the huge potential user base mean a large pool of developers, but the larger the added-value services market, the smaller piece you need of it to survive. Look at Red Hat and Mandrake; not everybody who runs Linux needs the services they provide. But there are a lot of Linux users out there; even a few percent of those millions is still a healthy living. (Figures on the installed base for Linux are notoriously messy—one study by International Data Corporation put purchased, shipped Linux distributions last year for servers alone at around 1.5 million; Red Hat and Mandrake claim anywhere from a quarter to over half of that. Add in the downloaded copies, desktop purchases and multiple installations from one purchased CD-ROM, and the number of Linux and GNU users might reasonably be 20 million, according to Stallman.)

In fact, the larger the market, the more chance an open source product has to outplay, outwit and outlast its proprietary competitors. As the market expands and matures, the margins get thinner and thinner, and the value of the shrink-wrapped box pales in comparison to the support services required by either product. In the end, it gets easier and easier to vote the priciest competitors out of the tribe.

Rule 2: Applications Are Hard. Hundreds of open source programmers have already pounded on infrastructure: operating systems, networking and such. Their work is debugged, in place and free (as in free beer). If your products are in this space, you might as well open source 'em. But what about a video-editing suite or a new game? Your company has probably put a great deal of effort into novel algorithms, not to mention the GUI, and wants to see some return.

Caleb Pourchot is the director of engineering for Sonic Foundry in Madison, Wisconsin, which makes shrink-wrapped audio- and video-editing applications. "There's no incentive for us to go open source," he says, since "there's no really valid business model there." Aside from the revenue considerations, Pourchot doubts that his company could build much of an external developer community: "The code base is such that there is probably only a handful of people in the world who would, one, care enough to try, and two, actually have the chops to accomplish something in the code."

Rule 3: Hackers Hack Hackware. Every programmer needs an operating system; most want a better one, which explains why Linux can draw on a huge, motivated talent pool, year after year. Programmers can't work without programming languages, either, so compilers, editors and IDEs tend to improve in a real hurry, under the "eat your own dog food" model of using what you write. If you start an open source project whose end product is useful to programmers, you'll rapidly achieve a critical mass of dog-food gourmets. Conversely, tools for "real users" languish by comparison. For example, Linux and Apache are mission-critical systems for us; they run reliably for months without my having to touch them. I also have AbiWord and KWord on my personal Linux workstation, but to write this article, I had to reboot into Windows and use Word. Heresy? No, utility. For one thing, AbiWord crashed, and upon restarting, haughtily informed me that this article was "a bogus document." (From my editor, that's painful; from some bag of bits, it's intolerable.) For two more: outlining and word counts, folks. I need those and the open source alternatives don't have 'em. So why haven't I put the features I need into AbiWord or KWord? My only excuse is that to me, the features aren't worth the time commitment when alternatives are available.

Rule 4: Services Become Tools, Tools Become Commodities, Commodities Become Services Again. If you're an in-house developer like me, much of your work consists of one-off projects. You're not really working on products, you're providing a service: solving your users' problems. But over time, the same problems crop up again and again, and it becomes cost-effective to create (and perhaps sell) a reusable tool to solve them. If enough people need the tool, competition drives down prices, thinning the margins until little profit can be made selling the tools itself; users look for added value from services (Sun, Microsoft and especially Oracle have been talking this idea up lately), or tend to rely on a brand name. At this point, consider taking the project open source—there's little profit to lose, so you're better off with a service model anyway.

Oddly Enough, It Works
Not every project is suitable for open source, and certainly not every open source project succeeds. But the ones that work tend to work very well indeed, delivering feature-rich, reliable software like Linux, Apache and Samba. Largely, that's due to the skill and dedication of the developers, who are drawn not from the ranks of the unemployed, but from corporate engineering departments.

As Borland's Shelton says: "We talk about them as wild-eyed, hairy independents, but a lot of developers have corporate jobs during the day, and do open source on their own time. Corporate developer by day, Spider-Man by night."

Next Time
We'll look at putting a project together. Our goal will be to serve documents from a single XML source to any of a variety of formats and platforms (the Web, static pages on a disconnected machine, hardcopy, PDAs). We'll collect and supply data with a three-tier system, making use of the Apache Web server, Apache Tomcat, an SQL database, Xerces, Xalan, FOP and Cocoon. See you then!