It's Good Work When You Can Find It
Al Stevens has long been Dr. Dobb's eminent C/C++ programming guru. In this 2001 installment from his long-running column, Al discussed, among other things, open versus closed-source development--a topic that might as well be called: "The differences between Linux and Windows development," and his long ride on the "dependency carousel."
It's Good Work When You Can Find It
by Al Stevens
My exploration of Linux as a development platform continues this month. Most of my activities have involved reading books and talking with other Linux users and programmers in Usenet newsgroups. This month, I'll discuss some of the books, the perils of newsgroup participation, and some of the issues I've uncovered. You'll also realize why I probably have to park the Dobbsmobile (the roving DDJ reporter's motorhome) outside my friend Lou Grinzo's house and live there for a while. Lou is a frequent contributor to DDJ and editor of the Linux Programming e-zine (http://www.linuxprogramming.com/). He's also the source of much support and help as I get in further and further over my head with Linux programming.
Usenet is an essential resource for Linux users and developers. Although books are plentiful, there are so many small, specific issues related to installing and using Linux, that your only hope for help is to find someone else in this huge community who can shed light on your problem. Linux runs on hardware platforms with potentially millions of possible unique configurations. Not all devices have Linux support, and some of those that do are not that easy to get running properly. Installing the latest software can pull you into what Lou calls the "dependency carousel," which I'll address later in this column.
But discussing Linux on Usenet is an experience unlike what you might be used to. Linux newsgroups are populated by many learned, helpful, polite people. But an element lurks there that prefers insults and flames to the civil exchange of information. Be prepared to have your questions ridiculed, your competency challenged, and your integrity impugned. You need a thick skin. On one occasion, Lou stepped in and defended me from a particularly viscous affront. The moral? Don't go in there alone. Thanks, Lou.
I said two months ago that my preliminary selection of KDE as a development platform was based partially on the availability of a C++ application framework for KDE. Readers wrote to tell me about gtk-a C++ wrapper around the GTK API upon which GNOME is based. Readers also pointed me to wxWindows, a multiplatform GUI C++ framework (which coincidentally is examined elsewhere in this issue). I'll report more about these libraries in a later column.
KDE endures a problem well known to Windows users and programmers-bloat. The KDE 1.1.2 that ships with Red Hat Linux 7.0 performs reasonably well with low-end hardware. I installed Mandrake 7.2, which uses KDE 2.0, and observed a significant performance degradation. Other users on the Mandrake news group (alt.os.linux.mandrake) report similar observations. To use the kdevelop IDE, which was one of the criteria for which I selected KDE as a development and target platform, I must install KDE 2.0 or higher. Obviously, Linux is not immune to the Windows syndrome that consumes processor resources as quickly as PC builders can add them.
Winmodems and Linux
Winmodems and Linux are not a happy couple. A winmodem (in the generic sense-if someone has trademarked "winmodem," I am not referring to that specific product) is an internal modem device that implements some of the traditional modem functions in software. Usually, the driver interprets the modem command set and translates the commands into low-level protocols that the winmodem understands. I am told that some winmodems also perform the digital signal processing that translates between digitized audio waveforms and digital modem data.
A winmodem has obvious advantages over traditional modems that implement these things in firmware and hardware in the modem itself. Those devices include serial port line drivers and UARTs, a microprocessor to interpret the commands, and memory for the firmware and data. The advantages are cost, weight, and size-all valid concerns for manufacturers, particularly laptop manufacturers.
In the old days, devices were dumb and the CPU had to do a lot of stuff. But slow CPU cycles and limited memory were precious, so devices got smarter and device-dependent functions were offloaded as much as possible onto the device.
The winmodem concept comes full circle. By exploiting today's fast CPUs and huge memory resources, the winmodem "onloads" device dependent functions back onto the host computer's CPU by implementing them in device drivers. The advantages? As I said earlier, cost: You pay for software once. Size and weight: More functions are crammed into smaller packages. The disadvantage? Nonstandard implementations. Since there are no standards for implementing protocols, each unique winmodem has a custom device driver, protocols are proprietary, and manufacturers usually deliver drivers only for Win32 platforms, which represents virtually their entire market. Consequently, most winmodem devices have no Linux support. As vendors publish the protocols, enterprising Linux driver programmers try to support the devices. But not all vendors do that, and the only way to get support is to reverse engineer the Windows drivers. That takes time and a desire to do it.
Other devices are following the winmodem model. I have an HP color printer designed to work only with Windows. I have two sound cards in the same boat. All three devices depend on software implementation of functions that are traditionally managed in the device. Linux users tend to react negatively to such devices because the unavailability of Linux drivers makes it difficult to install Linux on laptops and desktops that use them. But PC builders need to reduce per-copy cost to stay competitive and profitable, particularly as appliance computer prices plummet. The HP unit next to the Emachines unit on the shelf at Best Buy needs to compare favorably in features and price.
If Linux can ever support all those devices (no reason it can't, it's just software), OEMs might begin to look favorably on Linux as an alternative OS for their appliances. It's free, it's getting really friendly, and MicroSmurf can't push OEMs around the way they used to. Linux can have a really desirable effect on the per-unit cost. We're not that far from having an OEM configurable, shippable Linux. Eventually, there will be more VA Linux kinds of companies and their names will be Dell, Sony, Gateway, Toshiba, IBM, and so on.
(Then Linux will be mainstream, won't be considered cool anymore, and the next generation of geek-rebels will find some other platform to champion, publicly disparaging Linux and its proponents.)
The Cathedral and the Bazaar
A substantial part of my exploration into Linux has been into the culture of Open Source software development. More than a culture, Open Source has become a religion. I regret repeating that cliché, but there seems no better way to define it. Open Source has doctrines, scripture, and a self-proclaimed spiritual leader. Open Source advocates are sensitive to challenge and circle the wagons when outsiders ask obvious and penetrating questions about the validity of their doctrines and beliefs. To the outsider, Open Source looks like a religion.
Not that there's anything wrong with that.
Open Source's leader is Eric Raymond, the author of an Open Source program named fetchmail and of a series of essays that define the movement. These essays have become the official doctrine of the movement and are available in print as The Cathedral and the Bazaar in its second edition published by O'Reilly & Associates (http://www.oreilly.com/). They are also available online at the author's web site (http://www.tuxedo.org/~esr/writings/cathedral-bazaar/).
This book is a wonderful collection of essays. The first two essays, "A Brief History of Hackerdom" and "The Cathedral and the Bazaar," are well written and a delight to read, whether you agree with them or not. Later essays, however, are written in a more ponderous, verbose style leading me to wonder if the author decided not to do any rewrites or, enamored of the acclaim he properly received about the earlier ones, decided not to expose his new deathless prose to the close scrutiny of a good editor who might dare to paint a mustache on his Mona Lisa. I've done that.
In Raymond's thesis, traditional software is built much as a cathedral would be, with management, planning, goals, strategy, teams, and so on, whereas Open Source software is built in the environment of a bazaar, with many independent programmers reviewing, debugging, and contributing corrections to an ongoing software-development effort. In the bazaar, many more volunteer eyes view the bugs than cathedral developers can afford to employ. Consequently, and because the beta testers are themselves developers, most bugs are revealed and corrected in short order. This concept is the foundation of the Open Source movement, and Raymond goes further to examine its social and economic implications, not necessarily to my complete satisfaction, but nonetheless thoroughly and interestingly.
I won't try here to repeat all the tenets of the Open Source movement-you can read the essays yourself. But one of them is that to be successful, a bazaar project must avoid the traditional mistake of irrevocably tying release schedules and feature lists. Release often, and add features only when they are ready to be released. Sacrifice features for timely releases. This is music to the ears of any programmer who has worked on a big project against a deadline within which some feature list has been promised. I don't know if anyone can make any money this way, but it's the way I'd rather write programs.
In the first edition, Raymond made some predictions, a dangerous thing to do in print because you often have to eat your words. One of his predictions was that Windows 2000 would never "ship in a usable form." Well, he had to eat those words in the second edition, but rather than poking fun at himself for getting one wrong, he tried to salvage his credibility by saying, "Microsoft managed to ship Windows 2000 by drastically curtailing its feature list." This, I think, contradicts the release-often doctrine.
Another Open Source cultural rule that Raymond breaks states that the proprietor of a project must be humble and self-deprecating, downplaying his own importance and the importance of his contributions (or hers), deferring instead to the members of the team. You won't find a lot of humility in this book. You will, however, read about Raymond's many achievements, his close association with the famous and their admiration for him, and his importance to the movement. To his credit, I must acknowledge that it's always difficult for a major contributor to write about his own work, remain humble, and still get the job done.
Finally, addressing the issue of whether software for sale is all that much of a concern, Raymond tells us that as he talks to programmers at software conferences, he asks how many people are paid to write software and how many of them have salaries that depend on the "sale value" of software. He always gets lots of hands for the first question and not very many for the second, which, he says, surprises everyone except him (having come to expect it, I suppose). Obviously, he has never spoken at a shareware convention.
Those small criticisms notwithstanding, The Cathedral and the Bazaar is required reading for anyone who wants to join the movement. Besides explaining the technical and social implications of membership, Raymond associates the movement's attitudes and mores with philosophical, sociological, and anthropological theories from the past and many nods to the great ones who went before him-Brooks, Weinberg, and others.
Earlier I told you about The Complete FreeBSD, by Greg Lehey, from which I learned what I need to know to build my Linux network. Now I have The FreeBSD Corporate Networker's Guide, by Ted Mittlestaedt (Addison-Wesley, 2000). I haven't finished reading the technical chapters. I did leaf through the book looking at the diagrams. Based on the pictures and the "Preface," the book looks promising. Naturally, it includes a CD-ROM with the operating system. But the real gem in this book is Chapter 10, "FreeBSD Advocacy," which the author says is "political." He first gives the history of UNIX, then FreeBSD. FreeBSD began as a project called "386BSD," which its authors, William and Lynne Jolitz, described in DDJ in the early 1990s. Mittlestaedt explains how 386BSD forked into three independent projects, FreeBSD, OpenBSD, and NetBSD. He compares FreeBSD to Linux and tells why he thinks FreeBSD is a better choice. Chapter 10 addresses Open Source, Microsoft's antitrust problems, and many other things. This chapter alone is worth the price of the book.
To develop applications for UNIX-derived platforms, you might consider building a network with machines running Linux and the three BSD knockoffs. That way, you can determine whether your code compiles and runs on each of the platforms. The hardware investment would be minimal because you can dual-boot in some cases. Besides, Linux and its relatives don't consume the kind of horsepower required by that other operating system.
Open versus Closed Source Development
This discussion might well be titled: "The differences between free and costly software." It might also be titled: "The differences between Linux and Windows development." Let's go with that last one.
When I talk with for-profit Windows developers, I often ask their attitude toward the Open Source development model. If they have never heard of it, they are aghast. If they have heard of it, they are skeptical. I then ask if they have any thoughts about porting their applications to Linux platforms. The typical response is immediate: They have no such notions because Linux users are unwilling to pay for applications software.
In my meanderings through the Open Source community, I have observed a dearth of formally submitted Open Source Windows applications. The Open Source community seems dedicated to the Linux platform.
With no more to go on than these observations, I might draw these conclusions: If a Windows application has merit, people will pay money for it. Since developers are people too, and people like money, Windows application developers are unwilling to simply give away their work and forsake the money. Whether a Linux application has merit or not, trying to profit from it is useless because Linux users do not pay for software, good or bad. You might as well give it away and, if profit is a motivator, charge for support.
Next I wondered about the Open Source mentality among Windows (and before this, MS-DOS) users and developers. Does it exist? I draw upon my own experience with this column for some conclusions. Over the years I have, through this column and books, released many programs that fit the model of an Open Source project as defined by Raymond.
For this discussion, I'll address a program named "Quincy," an open source Windows IDE front end to two compiler suites, the mingw32 port of gcc and the free Borland 5.5 C/C++ compiler. Quincy is well known to readers of this column and my recent books. This discussion would apply to other programs I released, but Quincy is the most widely known, so I'll discuss it.
When I say that Quincy is open source, I do not mean that it is formally listed among the projects on freshmeat.net or announced in slashdot.org. Furthermore, Quincy is not covered by an officially sanctioned Open Source license that defines terms for its use and redistribution. Quincy and its source code are in the public domain, the open-source distribution model that I prefer.
I'll quote from Eric Raymond's list of things he did with fetchmail to test his theories about why Linus Torvalds's open source development model works:
- I released early and often...
- I grew my beta list by adding to it everyone who contacted me...
- I sent chatty announcements to the beta list whenever I released, encouraging people to participate.
- I listened to my beta-testers...
Quincy did not, however, benefit much from the bazaar model of software development that Raymond describes, even though Quincy seems to meet all the criteria for an open-source bazaar model. There are many users on my list of Quincy testers. I can presume that they are programmers because Quincy is an application for programmers. But whereas I received many bug reports, virtually none of Quincy's users submitted corrections to the source code. Most such help came from a small number of people who are developing similar programs. Usually, they explained how they addressed some particular problem I was having, often with fragments of source code from their applications, but never with actual corrections to Quincy's code. With their help, I was able to get past a number of obstacles, but even so, all the source code in Quincy is mine.
Most of the communications I receive about Quincy either report suspicious behavior, leaving the debugging work for me, or simply ask how to use the program. Yet Quincy was originally released six years ago, has gone through four major version releases and many more minor ones.
Conclusions? I don't know what to conclude. Perhaps Windows programmers are not interested in participating in the bazaar development community. Perhaps they don't understand how it works or that fixing bugs and submitting code is an option. Perhaps I didn't properly explain that I am willing to receive and review such contributions. I find no such explanation in any of the release history, so I probably did not. I don't even know that the possibility occurred to me as I released new builds.
I do know that I often reacted to some pesky bug report with a knee-jerk mutter like this: "Quit whining. You're a programmer. Fix it yourself and tell me how you did it." To myself, of course. I never said that to anyone, and shame on me for thinking it.
Here's one possibility. Quincy is a Visual C++ MFC program but, since it also does a lot of what VC++'s Developer Studio does, users might be getting Quincy and the compilers for free instead of buying VC++. Wouldn't that be nice? Perhaps most Quincy users, who receive binaries and source code, do not have the VC++ development environment to compile Quincy because having Quincy makes it unnecessary. Quincy cannot compile itself. Neither of the two free compilers includes MFC libraries. Perhaps most Quincy users cannot debug and fix Quincy.
The Dependency Carousel
The dependency carousel, according to Lou Grinzo, is a serious issue that doesn't get addressed much except when someone has a specific Linux application installation problem, which happens more often than it ought to. Anyone who tries to upgrade an installed Linux distribution with some new package is likely to run into the difficulty.
Here's how the carousel spins. Linux applications are released to depend on certain versions of supporting compile-time and run-time libraries. Those libraries themselves depend on other libraries. For example, kdevelop might require a certain version of KDE, which might require a certain version of QT, which might require a certain version of X, all of which require certain versions of the compiler headers and libraries.
Fortunately, installation of a package automatically senses when a dependent package is either not installed or not the correct version. Unfortunately, you are not informed of nested dependencies from the git-go. Each successive installation advises you that you need something else upgraded, and these dependencies can nest deeply. At best, you are in for a long session of downloads and installs. At worst, you get way into the process, with much time invested, only to be told that replacing an earlier version of something violates an earlier dependency because something else is installed that depends on the installed version of the library you are trying to replace. Whew. That's as difficult to say as it is to understand.
If you are lucky, you can vector over to the other thing that needs to be upgraded and start over again from there. If you are not so lucky, you don't recognize the name of the thing that refuses to be preempted, don't know where to get an upgrade, and aren't sure whether you ought to.
The installation procedures include options that permit you to ignore or override dependencies, but nothing tells you what effect that has on your system.
This is an area of Linux installation and maintenance that needs some attention. It reminds me of DLL version problems in Windows, only much worse and much more out of control. At the very least the problem needs to be well documented with careful procedures for installing particular upgraded packages. At best, the cause of the problem needs to be examined and alternative approaches worked out.
What to do? I work in an impoverished region where cable modem does not reach. Actually, it could reach my house, but after the law permitted us to install our own cable extensions without paying Ted Turner for them, I ran about 100 feet of underground TV cable from the house to the office. Reception was poor, so I added a Radio Shack video amplifier. It suffices for watching Gilligan reruns, but the signal isn't strong enough for Roadrunner (the cable-modem service, not the cartoon). Okay, now that you're feeling sorry for me, you will understand why I do not download the latest of everything at 31,200 bps, particularly when downloads run many megabytes and disconnects are frequent. (Another travail of the bucolic lifestyle is unstable phone lines.)
Therefore, I wait for things to become available on CD-ROM distributions. I have installed Red Hat 7.0 and Mandrake 7.2. Neither one fully supports kdevelop as of this writing, and my first several attempts to upgrade introduced me to the dependency carousel. I don't like getting dizzy, so I'll wait for a distribution that provides what I need. After all, the Open Source development model has no deadlines for delivering working code. I think I'm going to enjoy this work.