Takin' Care of Business

Despite the firm opinions of some now-departed companies, you can't earn a living by giving away free software. You must actually sell something of value to customers while giving away the software, otherwise your business won't clear the gantry. While this may be obvious now, it was derided as old-think just a year or two ago.

Embedded systems businesses may be in a better position than most because they deal with hardware devices — often highly customized gizmos — on an intimate basis. They can now supply, say, a Linux device driver along with their gizmo and leverage (makes me think of Archimedes every time) the rest of the software. Selling a service to customers who prefer a complete system solution makes sense, as does selling a package of custom hardware, software, and services.

It's not that simple, of course, but let's examine two data points to get a feel for the issues where hardware and software meet money and users. With those in hand, perhaps we can reason our way to some conclusions.

Free Terminals?

The Linux Terminal Server Project (http://www.ltsp.org/) solves a problem most people think went away years ago — connecting users to a central computer. After all, haven't PCs replaced mainframes?

Even today, data processing operations hidden away in back offices form the foundation of many businesses. Key-pounding remains difficult to automate, labor intensive, and because each set of hands and eyeballs requires PC-oid hardware, somewhat capital intensive. Reducing the cost of those workstations can go a long way toward improving a firm's financial situation.

Each terminal requires little computing power, which is why such operations were easily computerized in the early days of computing: Many dumb terminals linked to a large central computer made perfect sense at the time. Replacing the terminals with PCs didn't improve performance much, while increasing maintenance, power consumptions, user training, and system intricacy. Worse, some installations wound up supporting two or more incompatible terminals on each desktop.

In the mid '90s, a hospital supply firm tapped Dynamic Results (http://www.dresults.com/) to modernize its MIS setup. After evaluating commercial products and finding them lacking, then experimenting and flailing around with Windows PCs, Dynamic Results discovered that a stock Linux distribution had all the tools required to convert an otherwise obsolete PC into a diskless workstation networked to a Linux server that was, in turn, connected to the disparate mainframes.

By the time they finished tinkering, they'd flensed that PC to a vestigial stub — keyboard, mouse, monitor, system board, network card, power supply, and case. No hard/CD-ROM/floppy drives, no sound card, no ports, literally nothing else. It ran a terminal emulation program that presented a familiar face to operators.

Briefly stated, a boot ROM in the PC's NIC gets control after a hardware reset and broadcasts a DHCP request. A DHCP server fires back a boot image, which pulls in a larger image, which eventually flowers into a Linux kernel connected to a Network File System drive. The kernel then becomes a standard, albeit specialized, Linux system with an X server window on the display and starts the relevant applications. The user is up and running in a few seconds, even on a low-end 486.

The server machine supports DHCP, TFTP, NFS, an X client, and all the usual GNU/Linux applications. Any current Linux distribution contains all this stuff, so in principle, there's nothing new here. Dynamic Results was delighted to discover that a relatively low-end machine would suffice as a server, as well. Its clients were even happier to discover that each workstation no longer needed separate OS and applications licenses, which eliminated a large fraction of the per-unit cost. The server ran predominantly GNU apps, greatly reducing that cost as well.

Users log on to the server just as they would to a stock Linux system on their desktop, minus the administrative hassle of local configuration and maintenance. In fact, it seems many users aren't aware of the "computer" on their own desktop, a condition that's just about the definition of an embedded system.

During the course of this adventure, Dynamic Results discovered two things:

• Actually knitting together all the bits and pieces into a workable configuration required a considerable degree of broad-based Linux and network expertise, some of which they invented on the fly.
• Many other people were interested in doing the same thing.

Dynamic Results came to the obvious conclusion — there's gold in them thar terminals! But how to extract it?

The original GNU/Linux software came from the Open Source community, so Dynamic Results elected to release all of its code and documentation efforts under the GPL. They formed the Linux Terminal Server Project to return that work to the rest of the world, so if you want to build a thin-client system, you can start from the LTSP without paying a cent to anybody. In fact, you'll find an active and supportive mailing list as well as an IRC chat group to help you out.

Dynamic Results estimates that the LTSP code has tens of thousands of installations worldwide, so the do-it-yourself method must work reasonably well.

However, because many companies prefer pretested and known-good components over scrounged PCs, they also set up a new, commercial division called DisklessWorkstations.Com (at the obvious, albeit lengthy, URL) for exactly that purpose. This group sells all the components, preconfigured minimalist PCs, and embedded-style PC-ette bricks that you affix beneath a (wooden) desktop.

And, of course, companies in need of a turnkey solution often hire DisklessWorkstations.Com as consultants, too. They can obviously claim to have written the book (well, the online doc) on the subject, use proven hardware configurations, and deploy experienced people to get the job done. All in all, an enviable set of credentials, based on free software with a generous dollop of business acumen.

As nearly as I could tell from the crowd around the LTSP booth at LWE, things are going just fine.

Free Firewalls?

It seems the number of computers commandeered by crackers is asymptotically approaching the number of computers not protected by firewalls. Looking at my firewall logs tells me that even dial-up Internet connections have become desirable property: Stay online long enough and someone will drop by to sip through your straw.

Even though DSL and cable modems haven't reached my corner of the universe, I decided that a separate firewall box made sense. Although there are other options for a small network of Linux and Windows machines, a separate firewall centralizes all the hocus-pocus and simplifies sharing even a painfully slow Internet connection among all the nodes.

After some poking around, I decided on SmoothWall (http://www.smoothwall.org/), a GPL product that converts an obsolete PC into a firewall. You download the latest ISO image, burn a CD-R, pop it in the slot of your oldest PC, answer a few questions, plug in phone and network cables, and you've got a firewall. That's literally all there is to it; after it's set up you can remove the keyboard and display, tuck it under a table, and ignore it.

SmoothWall is essentially a GNU/Linux distribution, highly specialized for a specific (albeit common) hardware environment and designed for hands-off operation. You can achieve much the same result by sawing down a stock distribution, if you're willing to do a lot of groundwork and research.

SmoothWall got its start in mid-2000 as a free, GPL alternative to commercial hardware firewall boxes. Newer Linux kernels include packet filtering functions and the SmoothWall team added custom code atop that for web-based monitoring and configuration. They released that code under the GPL and soon discovered they had a winner.

At least a winner in terms of downloads and installations: a sustained rate of several hundred downloads per day with an installed base of nearly half a million. That's a lot of installations and a testament to both the quality of their work and the opportunity out there for such a product. They were delighted to find that their code was displacing commercial firewalls around the world.

The team provided an opportunity for users to contribute financially to the firewall's future development, whereupon they discovered that, to a good first approximation, nobody did. They accumulated a few thousand dollars of donations, a sum that fell an order of magnitude short of what they'd already poured into the project.

After I downloaded the code this past January, I contributed what seemed like a fair amount — roughly what I'd been paying annually for a commercial PC software product of similar capability. It seems I accounted for a third of the December and January donations and, no, I'm not a big spender.

The SmoothWall team formed a company (http://www.smoothwall.co.uk) to develop a more fully featured commercial firewall, holding its new code as proprietary IP. Because the initial SmoothWall project was launched in opposition to commercial firewalls, it may well be that this won't work out. It's simply too soon to tell, but the track record says that the GPL version does exactly what they designed it to do: run reasonably well in commercial situations at zero cost.

It is also fair to say that users who attempt to deploy the GPL version in a business setting are not received warmly on the SmoothWall GPL mailing list. Despite the fact that GPL code should be freely available for any purpose, at least one of the SmoothWall developers reacts harshly to anyone who asks questions that seem unrelated to a strictly personal installation.

There is an obvious conflict between the GPL code and the commercial code, as each new GPL user may represent either a potential customer or lost sale. Because GPL code is free, their natural bias assumes the latter situation and, unlike LTSP, there's no separate hardware or service that can serve as a revenue source.

Free Advice?

I contend that the LTSP and SmoothWall projects can tell us about tomorrow's embedded marketplace by their differences from yesterday's embedded products. Even if the times are a-changing, business principles remain constant. Let's work outward from the basics.

First, if you don't have profits, you don't have a business. You may have a nonprofit organization, you may have a charity, you may be making a small fortune from a large one, but you don't have a business. Further, you must sell something to customers to produce those profits; you can't give stuff away and make it up with volume.

The LTSP group spun off DisklessWorkstations.Com to establish a brand name for a product line, as well as to separate the GPL code from their (presumably) money-making effort. SmoothWall is in the difficult position of trying to distinguish a profitable name from a very similar GPL offering.

Second, there's no profit in a completely generic product, be it hardware or software. If anyone can build it, someone will do so with a lower cost and undercut your price. You can only lose money if your product isn't readily distinguishable from the competition.

As supporting evidence, note that Red Hat recently prevented CheapBytes from distributing low-cost CDs bearing Red Hat's trademarked name. At issue was not the content of the CD, but its label. Although Red Hat's GNU/Linux software may be freely distributed under the GPL, their name is a valuable, very proprietary asset.

Third, satisfied customers really are the best advertisement. Pure software, particularly in this Internet Age, lets a company have many enthusiastic users with no customers whatsoever. Rule One eats you alive if you sink all your efforts into keeping your users happy without converting them into customers.

Conversely, unhappy users can destroy an otherwise excellent product. Open Source software requires a fine balance between Rule One and Rule Three, with Rule Two waiting for the unwary.

Free Embedding?

Classical embedded systems are isolated devices dedicated to a specific purpose, generally short on CPU power and lacking in memory. Contemporary embedded systems tend to be networked, if not Interneted, and sport peppier CPUs with far more RAM. While systems remain resource constrained, their limits have become a bit easier to live with.

However, anyone who has ever written even a little, teeny piece of a network stack can attest that not writing that code makes a whole lot of sense. Unless you have an unusually well-rounded batch of designers and developers, you're best off using someone else's well-tested code wherever possible.

The future of embedded systems thus involves a small percentage of unique software amid a sea of Other People's Code. Call it component software or open source, the model represents something new and different in embedded design. A successful product must distinguish itself using its own IP while downplaying its OPC content.

Hardware now follows the same trend. Classical embedded systems demanded exceedingly customized hardware, but current systems use generic hardware with a small amount of customization. In fact, Other People's Code tends to be optimized for a certain class of machinery and, if you use the code, you must use the hardware. For example, Linux-capable PDAs look remarkably alike under the hood not because they can be similar but because Linux imposes a certain structure.

Embedded systems in the classical mode remain perfectly viable, at least for large-scale projects. What's new and different is that smaller, more generic widgets form an increasing share of the embedded market. The business model that used to apply to embedded systems must now undergo some serious adaptation.

Both LTSP and SmoothWall use a small amount of custom code atop a tweaked GNU/Linux base. Their hardware is, by definition, generic and essentially free, but they perform much the same function as proprietary boxes that we'd surely recognize as embedded systems.

DisklessWorkstations.Com can support a steady stream of GPL users on the revenue from the few who go on to buy their hardware and services. It's not clear to me that a pure software product such as SmoothWall can do the same because the free GPL code is, by design, a serious competitor to their commercial offerings.

The new embedded systems design paradigm (if I'm permitted to use that word) poses a severe challenge both to companies attempting to leverage free software on generic hardware and to companies attempting to leverage their valued IP. Resolving the conflicts should produce some truly interesting products, indeed.

Reentry Checklist

It's surely an accident of history that X servers run on the client workstation and X clients run on the server. Keep that in mind when you're considering graphics on thin clients.

Detailed information on Red Hat's trademark policies resides at http://www.redhat.com/about/corporate/trademark/. You can see the effect at http://www.cheapbytes.com/ if you search for a product called "Xxx Xxx."

A background interview with Richard Morrell, SmoothWall's founder, is at http://www.fosdem.org/interviews/1649.html with a discussion link at the bottom of the page.

The IPCop project (http://www.ipcop.org/) forked from the SmoothWall GPL code base with a different set of developers and a somewhat different world view.

DDJ