ESC Boston

Ed reports on the goings-on at the Embedded Systems Conference.


February 05, 2007
URL:http://www.drdobbs.com/embedded-systems/esc-boston/197003357

Ed is an EE and author in Poughkeepsie, NY. Contact him at [email protected] with "Dr Dobbs" in the subject to avoid spam filters.


Trade shows survive despite the Internet, although the Internet has eaten their original rationale of distributing information. There's just something about listening directly to a speaker that's hard to duplicate online, even for techies.

So I spent a week at the Embedded Systems Conference (a CMP event) in Boston last fall, listening to folks explain what they've learned. As always, I listen for what's not being said and how the audience reacts: Sometimes, silence and mutters can be rather informative.

What VCs Want

I sat in on the Disruption Zone lunch meeting between venture capitalists seeking The Next New Thing and entrepreneurs pitching their products and companies. This event resembled nothing so much as business-class speed-dating: Make your case in five minutes flat before a very critical audience. Much to their credit, the organizers imposed a strict time limit regardless of PowerPoint progress.

Essentially by definition, venture capitalists are only in it for the money, so they're naturally attracted to business opportunities that maximize their return. The introductory speeches clarified two points, if only by what was left unsaid: The entire embedded-systems market represents such a small opportunity that it's essentially irrelevant and these VCs see no way to make money from open-source/free software projects.

In round numbers, the entire embedded-systems market, whatever that means, offers at most a few tens of billions of dollars in revenue. While that's big money for me, it's just slightly more than the American pet food market and perhaps a factor of five larger than global ringtone sales. Worse, because embedded systems companies have the cottage-industry mentality common to tech-based biz in general, there's no single dominating company.

The ideal VC opportunity, the Next New Thing, must promise two decimal orders of magnitude return on their investment: Put $1 in, get $100 out. That requires creating a market where none exists, convincing customers that they must buy the product, building a company around that product, and then fending off competitors drawn by the frenzy. Fairly obviously, few embedded-systems business plans come close to that goal.

The next level, Disruptive Technology, can provide one order of magnitude return by eliminating inefficiencies, side-stepping roadblocks, being on the right side of regulatory controls, or otherwise innovating an existing business. Essentially everyone claims to offer disruptive technology these days, but, from what I can see, the bar is set rather low.

The lowest level, Buyout, must yield a binary order of magnitude return for merely buying a company, tarting it up or flensing some flab, and flipping the result to another buyer. A buyout can be easy, cheap, and low-risk, but it tends to chew up tech-class employees who see their company changed beyond recognition as it makes someone else rich beyond their wildest imaginings.

One chart showed that software now accounts for half of total product cost, which implies a strong motivation to somehow get that unruly mess under control. Worse, forecasts show code size increasing at a 46 percent compound annual growth rate against staffing at 14 percent. That's obviously not a sustainable situation: Five years from now twice as many developers must produce seven times more code.

As I see it, the bottom-line reason that VCs avoid embedded-system and, most likely, small software companies boils down to the fact that there's no economically viable solution to the software problem. They can't hire enough code monkeys (even Over There), nobody has a silver-bullet tool or technique, and analysis products simply add more cost to an already expensive part of the problem.

Depressing news, to be sure, but that's how it looks from their side of the desk. At least they're realistic, which is more than I can say for some folks.

Pop Quiz: What's missing from this picture?

The Missing Elephant

It seemed that each Disruption Zone presentation included a slide or two describing how the small company owned a key patent or technology that gave them control over their market niche. VCs love vendor lock-in and favor companies with strong patents, perhaps because they can still make money though licensing even if the company goes under.

As a result, not one mention of open source or free software occurred during the entire lunch. In particular, the VC slides on embedded operating systems completely omitted Linux. As nearly as I can tell, these VCs have decided there's no way to make money from cottage industries using software anybody else can distribute.

A show-floor panel discussion on "The End of Embedded Linux?" provided some interesting data points based on a survey of North American developers by Embedded Systems Design (another CMP magazine). As nearly as I can make out, 45 percent of deployed operating systems are various flavors of Linux, Windows CE, and Windriver's VxWorks. The trend lines indicate that in a few years the choices will be Linux, WinCE, and debris.

The discussion's deliberately inflammatory title came from the year-to-year trend line for Linux usage, which was downward, in contrast to its previous upward slope. Bill Gatliff noted that most embedded Linux developers don't read trade magazines and probably don't bother with surveys. The audience seemed to agree with his assessment.

I'd wondered about that, because none of the measurements have error bars or confidence limits. That's understandable, given the nature of an online survey, but it certainly means the results don't have many significant figures.

I checked a different survey performed by linuxdevices.com that wasn't mentioned at the panel. According to that survey, Linux accounts for nearly half the market and shows strong year-to-year growth. Got that?

Regardless of the details, somewhere around a third of the embedded OS market uses Linux. Any VC in the field with any interest in software should have that up on a slide somewhere, I'd say. Perhaps it was there and I simply missed it amid all the talk of patent holdings and market control?

Experience Matters

Many of the presentations and tutorials I attended had a common thread: Knowing what you're doing really matters. Perhaps it's better stated as "You can't buy success" or, as Louis Pasteur probably said, "Fortune favors the prepared mind."

David Kalinsky, talking on "The Architectural Design of Device Drivers," noted that it's easy to write a single device driver, but designing an OS driver architecture that works for many different gizmos requires broad and deep experience. In short, your first two or three versions just won't work, simply because each new device requires functions you didn't anticipate.

Jack Ganssle returned from a Singapore trip and reports that their typical developer is 28 years old with 4-5 years of experience. That's in contrast to the U.S., where embedded developers tend to be graybeards with enough experience to know what doesn't work. He notes one reason for the age disparity: The U.S. no longer graduates enough engineers.

Bill Gatliff, who knows more about stuffing Linux into small systems than any one person really should, observes that the kind of company which steals your code also tends to be unable to write it, so you can stay a few features,perhaps whole releases, ahead of them. I infer a business opportunity for anyone willing and able to run with the Red Queen, as a small business can certainly out-innovate larger ones in this arena.

Overall, it seems as though the industry has collided with the fact that skilled developers don't grow on trees. The reports I read indicate that, while the overall quality of Indian and Chinese engineers leaves a lot to be desired, there simply aren't enough U.S. engineering graduates these days. The pipeline that produces college graduates mandates a half-decade lag, no matter what the demand, and all indications show that our undergrads now avoid engineering majors in droves.

You might expect companies to pay a premium for developers and you'd be almost right. While new electrical and computer engineering grads tend to earn 50 percent over the average for all tech fields, Ganssle observed that wage compression limits the premium for additional experience. The average seems to be twice the starting salary during an engineer's entire career, which may grievously understate the value of knowing what works.

I haven't seen a viable solution, let alone a good solution, to this problem. If you have one, you can get rich pretty quickly.

Feature Creep

Only recently have embedded systems sported operating systems, if only because low-end hardware has grown large enough to support such overhead. That ESD survey showed 29 percent of embedded systems still have no OS, with consumer electronics hitting 40 percent. Although it's not explicitly stated, I assume the survey was aimed at developers of systems capable of running an OS, rather than PIC-class microcontrollers.

Kavitha Shanmuga Sundaram presented a fascinating (if you're that kind of techie) talk on "Migrating from 8- and 16-bit to 32-bit: Lessons Learned the Hard Way." Her experience in the textile industry, where intricate machines produce uniform products in huge volumes at incredible speeds, includes process control and monitoring devices. Many of those gadgets started out as mechanical linkages, used electronics when the cost advantages became compelling, and have since migrated into the RTOS and Linux arena.

One of her projects jumped from an 8-bit microcontroller to a 32-bit ARM, not so much due to feature creep as the fact that a single-chip ARM7 costs less than the high-end PIC microcontroller they'd been considering. The application involved a set of spindle status indicator lights on a cloth loom, a task implemented with mechanical switches and parallel high-voltage wiring back in the Good Old Days of cast iron machinery.

By the time they switched to the ARM, the features included maintaining a lamp's state across power failures and serial communication to microcontrollers at each lamp. But the overall complexity remained at an 8-bit level and didn't require any 32-bit OS features.

She pointed out that many bare-metal programming techniques simply don't transfer well to the strictures of an operating system and, in particular, to the Linux kernel's isolated and well-protected memory and I/O. Rewriting an existing application may not be practical for what's seen as a small upgrade, even if you're switching CPUs in the process.

She suggests simply continuing bare-metal programming in such a situation, at least until feature creep demands support for complex devices that simply can't be handled by your staff. For example, hammering out your own networking, graphic displays, and USB device support just doesn't make sense, so that's when you accept the overhead of an OS and rewrite the app.

Her talk was, by my informal tally, more lightly attended than I expected, particularly given the number of low-end microcontrollers sold each year. Perhaps there's a mismatch between the relatively high-end ESC audience and the preponderance of low-end apps? It may be that the majority of folks in the low end of the market don't need any of the fancy offerings found at ESC, perhaps because there's no money in the budget for such frills.

Root the Vote: Early Returns

On my timeline, it's just after the November elections and too soon to tell exactly what happened. The expected failures did occur, most particularly in Florida where 18,000 of 153,000 votes seem to have quietly vanished with no independent audit trail, no fallbacks, and no possibility of verifiable recounting. Other polling places across the nation reported less-severe problems, but enough failures occurred to be worrisome.

Here in Dutchess County, an ancient lever-action voting machine failed at the polling place monitored by my voting-inspector wife. She had called for a "custodian" when the machine refused to print the zero total sheet at the start of the day; he promised to return to help close the machine. It also failed to print the final totals and they manually copied tallies from the dozens of odometer-style dials inside the machine. Yes, it failed, but no, the result wasn't catastrophic.

More on this after the dust settles.

Last Tab

Those embedded OS surveys are at www .linuxdevices.com/articles/AT7070519787.html and www.embedded.com/showArticle.jhtml?articleID=187203732. Jonathan Coulton's classic Code Monkey tune is at www.jonathancoulton.com/music/thingaweek/ CodeMonkey.mp3. Lewis Carrol's Red Queen defined the software release treadmill in Through the Looking Glass, which you can read at www.gutenberg.org/etext/12.

Jack Ganssle's recommendations on becoming an embedded developer could be your ticket to a new career at www.ganssle.com/geek.htm. Read the NSF report on engineering salaries at www.nsf.gov/statistics/infbrief/nsf06303.

Reports on the election appear in Bruce Schneier's Crypto-Gram are at www.schneier.com/crypto-gram-0611.html.

I played hooky from ESC's last Thursday session to visit the MIT Museum, which is something of a geek Mecca. You can see some hints at web.mit.edu/museum. Yes, I counted Smoots as I walked across the Harvard Bridge (web.mit.edu/spotlight/smoot-salute).

Terms of Service | Privacy Statement | Copyright © 2024 UBM Tech, All rights reserved.