It's foolish to be invisible.

At the other extreme, the Bad Taste award goes to a company, which shall remain nameless, that used cheerleaders and an oaf bellowing through a megaphone to impress show attendees. Everyone I talked to said at least one discouraging word. The surrounding companies prevailed on them to turn the volume down by the second day, but I still didn't see much booth traffic. For sure, I didn't stop.

If you want to look cool, don't be a fool.

On the way back, another cell phone user with baggage completely blocking a Philly airport slidewalk turned to snarl "Where do you think you're going?" at an overtaking pedestrian. The pedestrian, a pilot in full regalia bearing a package and a very harried look, did his best to not say anything rude. The cellman looked abashed and compacted his heap.

Embedded systems can be fool enablers.

This month, I have short takes on interesting events, companies, and products at ESC/Boston. In upcoming months, I'll present more extensive looks at some common themes extending well beyond the show.

Hardware Is Hard

Given the evanescent nature of the PC market, the longevity of embedded systems poses a real problem for both users and suppliers. How do you design with PC components that have a production lifetime of a year, tops, in systems that must function for decades? How can you build software that must cope with constantly changing hardware, stuff that probably won't be exactly backward or forward compatible?

One ESC exhibitor, who shall also remain nameless, showed a nice line of plug-in, industrial-control CPU boards built from, basically, last year's desktop CPUs and chipsets. Nothing wrong with that, but when I asked how they handle delivery orders for the same part number spaced over half a decade, he allowed as how he was from the semiconductor branch of the company and really didn't know. I left my card and haven't heard back yet.

However, if you're ready, willing, and able to ride the crest of the PC-hardware wave, Sweetcircuits (http://sweetcircuits.com/) provides a web-based CPU board design system using its stock of tested components and layouts. Basically, you pick the bits and pieces required for your widget from a menu, an automated layout program stitches the blocks together with copper traces, and your boards pop off the production line with no manual tweaking. You get an instant quote, prototypes in short order, and as much of a production run as you need on a schedule you define.

It's a great example of how to make money using the Internet: Sell each customer a unique, highly customized product through a direct manufacturing interface. The Sweetcircuits web interface lets you adjust things and compare prices until you're satisfied, at which point an actual human being gets involved in the process.

However, the Sweetcircuits selection of parts and layouts constrains the design to a corner of the x86 universe and does not free you from a part's lifetime issues. Should you need a dollop of custom circuitry, the company can accommodate you with more intervention than the automated web tool allows. It's definitely worth a look for moderate-volume production.

You've probably noticed the deprecation of serial and parallel interfaces in new PC peripherals. What if you're responsible for an installed base of sensors and widgets, each sporting an RS-232 connector and none with the slightest prospect of an upgrade? Lantronix (http://www.lantronix.com/), which started doing PC networking in the earliest days when networking was both novel and nonstandard, has moved into the realm of interfacing nearly anything to the TCP/IP tentacles found everywhere. To that end, Lantronix has a line of little boxes with a serial or parallel connector on one end, an RJ-45 jack on the other, and some smarts in between. Plug it in and presto your gizmo appears on the Internet. Somewhat larger boxes sport multiple I/O ports, switches, routers, and terminal servers.

As a natural consequence, Lantronix is marketing its DSTni (pronounced "Destiny," of course) chip specifically for legacy serial and parallel widgets that desperately need a network interface. In principle, you simply design it in next to your existing hardware and plug in a network cable. Of course, there's a dash of software required, much of which appears in Lantronix's OEM developers' kit.

It's a great example of making money from an installed hardware base that can't or, perhaps, shouldn't keep up with desktop technology.

Software Is Harder

The number of mergers, absorptions, takeovers, buyouts, restructurings, and related corporate events in the embedded field presents a bit of a challenge for ordinary mortals. Familiar nameplates from years gone by have, well, gone bye-bye, while others appear under a different logo.

That seems to be in contrast to the expanding number of widgets claiming the "embedded" mantle, though. It takes much more than just a company with a good product or technology to survive.

The only conclusion I can draw is that there isn't room in the embedded arena for all contestants. Those with deep pockets, a wide customer base, and a continuing string of innovative new products will survive, with the remainder seeking shelter under another's wing.

What's interesting to note is that old code not only never dies, it rarely even fades away. In the desktop world, it's not uncommon to own an orphaned program. In the embedded world, someone absorbs the orphans with the hope of making at least a bit of profit on lifetime support. Whether that actually works out in practice remains a good question; I've seen some products glide through several owners on their way to a final resting place.

It's yet one more thing to worry about when you're building long-lived systems: What to do about vanishing software vendors?

While I can hear chants of "Open Source, Open Source!" in the background, that may not be the answer. The consensus of several operating-system vendors I talked to was that their large customers prefer to concentrate on whatever their core competencies may be and leave infrastructure software design and maintenance to others.

That's obviously a self-selected sample, because we don't hear from customers who decide to roll their own or adopt an open-source solution. Nevertheless, if you take control by owning all the code, you've pretty much traded one set of problems for another.

Nothing is free, not even software.

Debugging Is Harder Yet

People tend to forget — especially early in the schedule-planning stages — that when first-iteration hardware finally arrives for integration, it's likely not to work. Worse, it may seem to work, but fail in mysterious, confusing, data- and time-dependent ways. Figuring out what's wrong requires the combined talents of hardware, software, and technician mindsets.

This is entirely different from the established PC world, where application programmers can safely assume all PCs work perfectly (if not identically), so whatever problems occur must be due to software errors. In the embedded world, you literally can't trust anything at first.

Green Hills Software (http://www.ghs.com/), of compiler and RTOS renown, introduced a BDM (Background Debug Mode) pod-and-probe that jacks into the BDM/JTAG/etc. connector you've presumably had the good sense to include on your Power PC/ARM/etc. board. It interfaces your system's CPU with Green Hills's MULTI development environment, providing a porthole view of your code's inner workings and some control over its wanderings.

The probe incorporates a 32-bit CPU with a generous slug of Flash and an FPGA interface to the target CPU. In theory, it has incorporated everything you'll need for quite a few years into one box, with downloadable firmware to configure it to the target architecture of your choice.

From Green Hills's standpoint, the company now has complete control over the functions in the probe and can more easily configure it to work with its software. That's much easier than trying to adapt their code to the peculiarities of Other People's Probes, it seems, as you already have a considerable range of BDM probes to choose from.

However, all those probes have one major failing — they must halt the CPU to read out its current state. Yes, there are tweaks and twiddles to minimize that, but the net effect resembles a stick in the spokes of embedded code. As a case in point, you can't simply halt a program that's controlling even a slow real-time process without causing horrible sounds out on the factory floor.

If you can't halt the system, you can only observe what goes on as it's running and hope to figure out what broke at some later time. That's the domain of logic analyzers and oscilloscopes, once found only near hardware engineers but now entering the mainstream of debugging.

Tektronix (http://www.tektronix.com/), of high-end oscilloscope fame, continues slugging it out with Agilent (formerly Hewlett-Packard, but HP's test-and-measurement group got the short end of the name-recognition wishbone) for control of the test equipment market. Tek's latest logic analyzer offerings combine its analog data capture expertise with the hardware required to collect and display vast amounts of digital information, all running under control of a Windows NT PC-oid controller built into the analyzer. Tek's centerpiece TLA721 logic analyzer displays timing information, waveforms, source code, and debugger traces through a quad video board. The Tek booth featured four 21-inch, full-color, 1600×1200, flat-panel displays above the data-capture mainframe.

Hardware folks were easy to pick out of the crowd: They all had the glazed expression that comes from severe pixel envy and were swaying slightly in the breeze. Not a pretty sight.

Unfortunately, the days of logic analyzers having direct access to all of the CPU's state information are long gone. In fact, the latest sea-of-gates ASICs feature multiple CPU cores with on-chip RAM and peripheral controllers, none of which are visible from the chip pins. As a result, BDM with all its shortcomings may be the only way to figure out what's going on in there.

The folks at Tek point out that, regardless of what happens inside the chip, eventually the signals must show up on the pins in order to do anything useful. While true, those external pins may have little or nothing to do with the high-speed logic going on inside.

Worse, Rent's Rule gets in the way. Back in 1960, E.H. Rent observed that to a good first approximation, the number of pins on a package varies as a function of its gate count like this:

Pins=K×gatesp

For relatively large designs typical of current SOC (System-On-a-Chip) offerings, p runs somewhat under 0.5. That means the number of pins varies as the square root of the gates. If a 10 K-gate design has 100 pins, you should expect at most 1000 on a 1 M-gate part.

Obviously, pin count can't possibly keep up with internal gate count, which means your access to and visibility of the system's state decreases dramatically with each chip generation. There simply aren't enough pins to go around, even on packages sporting a field of miniature ball-grid array bumps.

Rent's Rule is most applicable to sections of an entire system, which makes sense only if you can see the interfaces. That's decreasingly possible with SOC-level integration and you can imagine the limiting case quite readily: one analog input, a fast DSP, and an analog output. No matter how clever your probing technique, you simply can't see any digital logic from the outside.

The solution requires debugging and tracing hooks integrated directly into the system's hardware, firmware, and operating system. By saving state information in RAM, then piping it elsewhere (through a dedicated channel) for later analysis, you can at least figure out when and where the system fell off the rails. Green Hills, among others, offers that sort of ability in its MULTI IDE, with glue for a variety of embedded operating systems.

What's obvious, though, is that you can no longer design a system without accounting for debugging overhead right from the start. Assuming that all the I/O and all the RAM and all the ROM and all the CPU cycles will be devoted strictly to the job at hand simply won't get the job done.

Substrata

A while back I mentioned OnCore's embedded operating system (http://www.oncoresystems.com/) as a means of achieving real-time performance beneath a bone-stock OS. This is neither a new idea nor one limited to Linux, as shown by the number of other ESC vendors with mature products that do essentially the same thing for various environments.

Tenasys (http://www.tenasys.com/) may be the most mature, now having custody of Intel iRMX. In one of its current incarnations, iRMX III provides a solid real-time substrate beneath Windows NT, allowing a vital split between fancy GUI features and closed-loop I/O processing. Another incarnation, for systems where a GUI would just get in the way, dispenses with Windows.

Those of long memory may recall iRMX from the mid '80s, where it ran on embedded systems with neither MIPS nor megabytes. The full virtualization and memory protection offered by the 386 CPU gave iRMX a real shot in the arm, creating a code base that still controls processes around the world. Once again, code never dies.

QNX (http://www.qnx.com/), another survivor from the old days, has morphed from a teeny package that ran quite happily on Motorola 6809 microcontrollers into a full-bore, real-time, embedded OS suited for the largest imaginable systems. As nearly as I can tell, the hardware has finally caught up with the QNX message-passing architecture, making its performance effects largely irrelevant. In fact, these days nearly everybody claims to use message passing, so its time may have come.

Not surprisingly, all RTOS vendors claim design wins with more or less the same set of high-end telecom and networking manufacturers. I suspect that those manufacturers have enough projects ticking along that nearly everybody gets a chance to play, even if not all projects eventually see the light of day.

Reentry Checklist

I have a stack of evaluation kits and CDs to plow through, so you'll see more details in upcoming months. Product reviews aren't in the lineup, but I saw some fascinating developments that merit further attention.

Although my last ESC/Boston session ended on Friday afternoon, no airline flies directly to Poughkeepsie in the evening. I left Logan at 7:00 am on September 8, 2001, after the ground crew topped off the tanks of each plane at the US Airways gates. We rumbled aloft in a lightly loaded US Airways 767 bound for Philadelphia with continuing service to the Dominican Republic. The scenery was spectacular along the Massachusetts coast, southern Long Island and, of course, The City.

Timing is everything.

Shawn Carlson, who wrote Scientific American magazine's "Amateur Scientist" column, proposes a unique way for technical folks to combat terrorism. Find out more at http://www.sas.org/, which has absolutely nothing to do with the SAS company of software fame.

E.F. Rent first presented his rule describing pin count as a function of gate count in IBM hardware circa 1960. You'll find more details and caveats at http://www.cedcc.psu.edu/ee497i/rents_rule.PDF. Due to the size of current silicon parts these days, Rent's Rule now predicts wiring locality between sections of on-chip designs.

Several alert readers reminded me that the correct pronunciation of "sidereal" has four syllables: "sigh-DEER-ee-al". Evidently I found the only incorrect source, as all the dictionaries I consulted after the fact agree with them. Never trust anything you see in print!

DDJ