Last month's Dr. Dobb's Developer Update featured a look at recent attempts to rationalize the software-development process, focusing on large projects.
One well-known case is the in-flight software for the space-shuttle program, a project which has garnered kudos from all quarters for its enviable record of software quality, productivity, and predictability. Some theoreticians see the project as highly "mature" - reaching level 5 on a maturity scale devised by the Software Engineering Institute at Carnegie Mellon. A hallmark of such maturity is the ability to successfully apply the same development methods to another project. Sadly, it turns out that whatever maturity was achieved in the shuttle project is not readily transferable to other projects. Witness the spectacular failure of the proposed FAA air-traffic control system, a $6 billion project recently scuttled by the government due to budget overruns and delays in delivery. (Both projects come from the same Loral division.)
The Mirage of Maturity
A maturity of level 5 seems to be like the fabled unicorn: a theoretical construct devoid of real-world instances. This is not to take credit away from the shuttle project's achievements, which are undeniable. Its near-zero defect rate and consistently on-time releases are unmatched in the software industry.
However, that particular project has enjoyed certain advantages rarely encountered by other development efforts: a hardware platform that is stable for two decades, a no-surprises set of requirements, an increasingly well-understood problem, a deep-pockets budget that allows for creating throwaway prototypes, a management commitment to gargantuan levels of testing, a civil-service-style lack of turnover in personnel, and a delivery schedule unmoved by competitors' actions or market changes. Given this rather unique set of advantages, it is not necessary to posit stratospheric levels of "maturity" to explain the shuttle project's order-of-magnitude improvements in productivity and quality over 20 years.
There is no doubt that a large part of the project's success is due to a highly skilled technical staff with a strong esprit de corps and a robust culture of excellence. But methodologists don't like to highlight these kinds of project-specific intangibles, because the existing methods don't delineate a specific path toward building such a team.
CASE Came and Went
In the mid-1980s, about the same time Frederick Brooks predicted the coming decade would not bring order-of-magnitude improvements in software quality and productivity, CASE tool vendors were cranking up a wave of products and white papers promising something close to that.
In fighting the werewolf of software development, any chance at a silver bullet, no matter how slim, has great appeal. The CASE wave that washed over the industry left some gullible development managers, who spent $10 - 50K per seat, high and dry. That wave has now receded, and many of those vendors that remain have, like witnesses in a federal protection program, changed their names and assumed new company identities (the marketing term for this is "repositioning"). Some of those players are now riding high on the current wave: object-oriented methodologies and patterns.
Although the tools and techniques of CASE and its object-oriented descendants are not without value, many constructs still seem cooked up in a vacuum. Much of the classic literature on software engineering and the development process does not seem applicable to real-world projects where requirements change every time the marketing department comes back from lunch, much of the operating environment is undocumented and still in beta, algorithms have yet to be discovered, and management pulls the plug on testing because the calendar ship date has arrived.
Even projects that are a safe distance from the bleeding edge encounter difficulties. Everyone has heard about problems in the automated baggage-handling system at Denver International Airport. Less well-known examples abound. The automated system for the Bay Area Rapid Transit (BART), a San Francisco area subway system, was declared inadequate and obsolete four years before it was due to be completed, and was canceled, resulting in a loss of $40 million. The Bank of America suffered a $100 million blow from delayed and bug-ridden trust-account management software that had to be removed from service. Health-care giant Kaiser Permanente has spent millions over the past decade trying to automate patient records, with no visible results. Kaiser's Southern California division not long ago installed a pneumatic-tube system to ship hardcopy documents around its buildings the old-fashioned way.
Consequently, many programmers are soured on the empty bureaucratic procedures that many CASE tools and techniques seem to imply. In a Computerworld article (September 4, 1995), Jean Stanford describes how programmers on a project dealt with the source-code-management system:
They proudly demonstrated a clever tool they had written that faked out the central repository and let them avoid checking for bugs with each change. They were supposed to be checking in files as they completed each change. Instead, they kept all their working files in their home directories. Once every three months, their ingenious little tool swept their directories and checked everything en masse for the whole subsystem. No wonder the test schedule is trashed.
Larry Wall, creator of the Perl language and author of the book Programming Perl (O'Reilly & Associates, 1990), writes about the "three great virtues of a programmer: laziness, impatience and hubris." Although Wall's statement does not mean what a surface reading may imply (he is actually talking about how these qualities motivate programmers to write great programs), the spin is definitely against buttoned-down dictates from a "reductionist" management, who have a "fervent but misguided desire to squash your mind to fit their mindset, to smush your patterns of thought into a joyless existence."
Apple's Guy Kawasaki has described many programmers' notion of the software-development process. The typical sequence of events (slightly enhanced by the present author) goes something like this:
Design the tee-shirt, write the code, write the documentation, ship the software, look for a new job, test the software, and, if time permits, write the specification and comment the code. Repeat the cycle endlessly with turnover in personnel.
With attitudes like these among the best and most creative programmers, who presumably serve as role models for the rest, it's no wonder that 70 percent of CASE tools sit on the shelf and become decorations for clueless managers' offices.
A New Realism
Recent writings on software engineering and the development process acknowledge a few of these concerns. Patrick Loy, in a 1993 article in ACM Software Engineering entitled "The Method Won't Save You (But It Can Help)" writes:
"There is no silver bullet to be found. The search for the perfect method or CASE tool is a futile one. New methods and tools might help, but they are unlikely to solve the root problem."
One recently published book, Program Smarter, Not Harder: Get Mission-Critical Projects Right the First Time, by Jay Johnson, Rod Skoglund, and Joe Wisniewski (McGraw-Hill, 1995), is a compact summary of current conventional wisdom on all aspects of the software process: requirements management, methodology, team building, prototyping, concurrent implementation, quality assurance, configuration management, and automated testing. Most of the material will be familiar to anyone who has read Yourdon, Booch, Boehm, Jackson, DeMarco, Parnas, and Weinberg. Though the book does not break new ground, there are spots where the political realities of instituting a formal process are acknowledged. This is in contrast to books such as Watts Humphrey's recently published textbook, A Discipline for Software Engineering (Addison-Wesley, 1995), where the ivory-tower approach goes unchallenged.
Another newly published book that is grounded in reality is 201 Principles of Software Development, by Alan Davis (McGraw-Hill, 1995). This book is a short collection of principles about all aspects of the software-development process: requirements, design, coding, testing, management, and quality assurance. Each principle is presented on a single page, and consists of an aphorism-style heading followed by a paragraph or two of elucidation, and the reference at the bottom of the page. Although some principles are the author's own invention, most come from other sources, such as the established software-engineering authors previously mentioned.
Many of the principles are common-sense axioms you are probably already familiar with, such as:
- "Get it right before you make it faster,"
- "Don't integrate before unit testing," or
- "Build software so it needs a short user's manual."
Others reflect the new realism in the field:
- "There is no perfect way to measure productivity,"
- "Use tools, but be realistic,"
- "Though copious errors guarantee worthlessness, zero errors says nothing about the value of the software," and
- "A few good people are better than many less-skilled people."
The explanations are lucid, concise, and, unlike some of the primary sources to which they refer, guaranteed not to glaze your eyes over before you finish the page.
These books, though worthwhile additions to your library, seem strongly rooted in the software-engineering community and its particular terminology (acronyms such as SRS, SCM, V&V, COCOMO, and SLOC/PM). The books attempt to reach out to those who are not part of this community, but the overall effect is of a "Dilbert"-style techie loosening his tie a few millimeters.
So there remains a strong need for street-smart guides that provide an alternative way of looking at the software-development process, go beyond the traditional concepts and lingo, and start from the trenches on up. It turns out there is a slim book just published that provides exactly this kind of perspective. It's from Jim McCarthy and is so enlightening that it's worth spending the remainder of this article discussing its contents.
Dispatches from Redmond
McCarthy's book is entitled The Dynamics of Software Development (Microsoft Press, 1995). You may know the author through his prior affiliation with the Whitewater group (developer of the Actor language), a company acquired by Microsoft several years ago. He has a technical background and worked in the Visual C++ marketing team before becoming development manager for recent editions of Microsoft Visual C++ (MSVC).
MSVC is a product with a checkered record, one that in recent years has made a surprising comeback from an embarrassingly low-quality release (C7). Today, MSVC enjoys the dominant position in the market and continues to garner favorable reviews from the critics - for which McCarthy claims his rightful share of credit.
Visual C++ is a large, complex, and powerful program. The recently announced Version 4.0 consists of around 3.25 million lines of code, according to McCarthy. He points out that MSVC is one of the few major software products sold by subscription, which means that the company makes a commitment not just to ship a new version by a certain date, but also to repeat that feat several times a year. These releases are not incremental tweaks; if you've followed the product history, you know that each one adds significant new functionality (OLE, MVC architecture, OCX, Macintosh support, and so on), and has a pretty good reputation with regard to bugs. With such a track record, any insights into software development from the MSVC development manager deserve your full attention - regardless of how you may feel about Microsoft the company, its aggressive marketing, or even how well Visual C++ stacks up against its competitors.
The book consists of "54 rules of thumb for delivering great software on time." Each rule is explained in one or several pages. Unlike the principles in the Davis book, you likely haven't heard these rules before (unless you've listened to one of McCarthy's talks over the last couple of years). The concepts are not always original, but are usually expressed in an original way. The book offers an insightful epigram on just about every page. For example, on consensus:
[This does not mean] some namby-pamby consensus and bogus good cheer. The only consensus worth having is a creative one achieved in the combat of fully-engaged [sic] intellects.
On software-development managers:
The novice program manager will often think that everybody else will implement, test and document the program manager's grand vision. This is at best a naive view and at worst a foolish and destructive one. Before the program manager can be worth anything to the team, he or she must be thoroughly disabused of the notion that he or she has any direct control.
On the market:
If we view the competition as our enemy then we must view the market as the prey that both we and our enemy pursue. If we capture the market, we deprive the enemy and drive them to some other land, or destroy them altogether.
There are specific references to the history of Visual C++, an honest acknowledgment of the shortcomings of the C7 product, and an inside description of the reasoning behind later MSVC strategies; for example, why they held the line on not implementing templates and exceptions, even in the face of mounting criticism from the press and analysts.
The revealing look at the company culture of Microsoft is worth the price of the book - although generalizing from one individual to a division, much less a whole company, is a treacherous game. This book is a good complement to two other good books on software development by Microsoft authors released in 1993, Steve McConnell's Code Complete and Steve Maguire's Writing Solid Code, both published by Microsoft Press. In conversation, McCarthy says that these three books provide a reasonably complete picture of the best practices at Microsoft.
Real-World Psychobabble
McCarthy's book is as much about developing great software teams as it is about software development. It is chock-full of hard-earned insights into the psychology of programmer teams, which is what ultimately drives the software process. A lot of the language in the book is larded with words like gestalt, empowerment, passion, psychopathology, self-esteem, and personal growth. These are words whose currency has been devalued in recent years by glib usage. You may be allergic to this New-Age terminology, but you'll do yourself a disservice if you dismiss the content of this book. Such a dismissal is what McCarthy calls "flipping the bozo bit."
McCarthy talks about how to not switch the bozo bit on with respect to others - and thereby limit their performance if you are a manager - and also how not to set the bit on with respect to yourself, and thus block your own potential. In conversation, McCarthy says "the key issue is getting people to communicate." McCarthy is not a woolly-headed, starry-eyed, Stuart Smalley (Al Franken's New-Age caricature on "Saturday Night Live"). His cynicism and insight into department politics can match the best of Scott Adams' Dilbert. In the book, he speaks affectionately of the "Death March" stage of the development process (the final push to completion) and makes provocative but informed statements such as "most software sucks." McCarthy's book goes beyond theory to provide useful tips about the process, such as how many milestones to schedule for a project, and how to reschedule a milestone when it slips.
The Man in the Room
One indicator that McCarthy has been around the block several times is his rule #31: "Beware of a guy in a room." Say what? What he means is that you should avoid assigning your group's best programmer the most difficult task of the project, encouraging that person to sequester him or herself in a room for weeks or months, while outside that room, work eventually comes to a standstill. McCarthy writes:
"[The programmer] will finish if he doesn't die or quit first [The only thing you can do as a manager] is simply buy large quantities of Mountain Dew."
Reading this rule made me recall a sad experience. Some years ago, I worked for a small, technology-intensive company that had evolved a macho culture of software heroics, a trait that definitely placed them at the bottom rung of the maturity scale. No programmer would touch a project until it was already in crisis.
One multimillion-dollar project was assigned to the development staff after it was sold to a customer (who was laboring under the illusion the product was complete and ready for delivery). The task was given to our best and brightest programmer, who had to begin by writing an assembler, then a real-time operating system, and only then could start on the actual application (which was a multiuser, high-volume, mission-critical program running on custom hardware). As months went by, the delivery date got stretched past the breaking point, and likewise, tragically, did the programmer. One day he stopped coming to work and ended up staring at the walls of his apartment for the next year and a half, refusing to answer the door. It took years for everyone to recover, especially the hapless guy in the room.
These kinds of real-life situations are not found in most software-engineering texts. Jim McCarthy's book provides you with useful street wisdom to get you through to the last milestone.
Ray Valdés is senior technical editor at Dr. Dobb' Journal and can be contacted at [email protected].
