Anatomy of a Failed Agile Adoption

Most articles focus on the success stories and few on the failures, a wrong that Scott hopes to right this month.


March 10, 2008
URL:http://www.drdobbs.com/open-source/anatomy-of-a-failed-agile-adoption/206902744

There is mounting evidence that, in many cases, agile software development approaches work better than traditional ones. However, some organizations are still struggling to successfully adopt agile techniques and philosophies. Sometimes it's simply because an agile approach isn't a good strategy for a project team or even the entire organization, but usually it's because cultural barriers within the organization are too difficult to overcome or because the organization doesn't invest sufficiently in agile training and mentoring. Sadly, most articles focus on the success stories and few on the failures, a wrong that I hope to right this month.

Last year, I was brought in to UK-based Gorwell Financial Group to assess a failed software process improvement effort. A gentleman whom I'll refer to as Winston Smith (not his real name) had attempted a grass roots, stealth agile adoption effort within his project team. Winston was an up-and-coming senior technical lead with over 10 years experience in IT. He was well connected with Gorwell because his big brother, whom I'll refer to by his nickname of "BB," ran the IT department. This proved to be both an advantage and disadvantage because although it gave Winston a bit more leeway than most, it also meant middle management was watching him closely. Be that as it may, in the end Winston proved to seriously misjudge Gorwell's organizational inertia.

The Victory Process

It's critical to understand the context. Gorwell had developed the Victory Software Process in the early-1990s, based on the serial V-model process that was becoming popular at the time within the U.S. government. The basic idea was that up-front activities such as requirements analysis, architecture, and design would be validated through back-end activities such as user-acceptance testing (UAT), system-integration testing (SIT), and functional testing (FT), respectively. It definitely wasn't the fastest way to work, but at the time it was believed to be low risk and result in high quality. BB, who leaned towards the melodramatic, came up with the slogan "Victory produces glorious results every time," which appeared on posters throughout Gorwell's IT department.

In parallel, Gorwell developed an IT governance program that provided rewards for conforming to the Victory process. Being the early 1990s, they were just beginning to adopt C++ for new development, and as a result of their OO fervor they created the ++GOOD (GOOD stood for Gorwell Object-Oriented Design) point system. You were rewarded ++GOOD points each time you attended a meeting, wrote a document, reviewed a document, or met a major Victory milestone. Every Friday morning BB would publicly congratulate the top three ++GOOD point earners that week over the PA system and take them out for lunch that day.

The key to the Victory process and governance program was Gorwell's project-management office (PMO). The PMO adopted both the UK's PRINCE (now PRINCE2) project-management guidelines and the U.S.-based Project Management Institute's book of knowledge (PMBoK) to ensure a comprehensive approach to project management. Its slogan was "Create the plan, work to the plan, and measure against the plan," a philosophy that drove all Victory project teams. To educate junior staff, they had a mentoring program called the "Youth League" where senior project managers indoctrinated new hires in the Victory process for their first two weeks at Gorwell, followed up by one-day refresher training courses once a quarter for their first five years.

After a decade of Victory, and after reading about agile software development, Winston realized that the results really weren't as "glorious" as BB was claiming. Projects were routinely late and over budget, although because they took so long and because the PMO kept updating the original estimates few others seemed to notice. Quality was slowly eroding away, but because Gorwell management kept reorganizing the reporting structure, everyone quickly forgot that things had worked better in previous years.

Compounding the problem was the political infighting amongst their business stakeholders, making it virtually impossible to set system requirements in stone at the beginning of projects as the Victory process insisted on. Gorwell had three business divisions based on geography—Oceania, Eurasia, and East Asia—which were always changing their priorities, waging internal political power struggles, and worse yet changing allegiances on a regular basis. The environment was chaotic, and when applying more paperwork into the process didn't help, BB simply decided to start claiming success via internal propaganda. That was the turning point for Winston.

Stealth Agile Adoption

Winston realized that if there was any hope at all that it lay in the programmers—without responsible, disciplined programmers, all other aspects of Gorwell's software development strategy were for naught. So, after a bit of research Winston discovered that a wealth of agile techniques existed that the Victory process didn't include, and more importantly it appeared that these agile techniques worked significantly better in practice. Winston started sharing some of these agile concepts with his colleagues, whom he knew to be open minded and could be trusted not to report his subversive thoughts to senior management.

This group of people formed a development team and began to gain experience with test-driven development (TDD), continuous integration, architectural envisioning, agile requirements change management, iterations, and nonsolo development. They quickly discovered that these techniques did, in fact, work better than the traditional Victory techniques. Initial requirements and architectural envisioning gained the benefits of up-front modeling without the development and maintenance cost of comprehensive static documentation that resulted from Victory's up-front modeling activities. Nonsolo development, particularly with stakeholders who actively participated in requirements elicitation activities, also proved superior to Victory's documentation-heavy approach. TDD and continuous integration enabled developers to produce greater quality software in a shorter time period than did Victory's V-model approach to testing. They found that by becoming generalizing specialists, they were able to work together more effectively, and thereby produced greater quality work in a shorter period of time while requiring fewer work products compared to similarly sized teams comprised of specialists.

A remarkable accomplishment for Winston's team was that they avoided Gorwell's 2007 data management (DM) improvement effort dubbed "Operation Blackhole." The DM group wanted a more repeatable approach to what they did, and although they had been collecting comprehensive and accurate metadata using the Newspeak Data Repository toolset, they found that development teams weren't satisfied with the two-to-four month time required for simple database changes. Over a nine-month period, the Blackhole team managed to migrate the Newspeak metadata into the newly acquired TruthMinister 2007 repository and rewrite the 1500-page metadata management procedures and standards manuals. The end result was a consistent data-management process where all change requests were either denied outright or took exactly three months to respond to. Because Winston's team had gained experience with agile data-modeling approaches, database regression testing, and database refactoring they exceeded the quality requirements set by Operation Blackhole in a fraction of the time. They didn't need to request help from the DM group and thereby managed to stay off their radar scope.

Agile Victory?

As time progressed, more and more developers heard through the grapevine about the greater levels of success following agile techniques. Unfortunately, so did some of the more political people within Gorwell's IT department—remember, people kept an eye on Winston because he was related to BB. In particular, the process engineering group took notice and decided to embrace this growing threat to their existence. One afternoon, they sent out a forced instant message to everyone in Gorwell's IT department that locked up everyone's machines until they read the announcement about the new version of their software process, called "Agile Victory." This occurred on the 198th day of Winston's stealth agile adoption effort; he remembers it well because it broke the fourth build of the day that was referred to as build 198.4.

A new group, called the "process police," had been created with the specific goal of ensuring that project teams followed the new Agile Victory process. Nonsolo development, such as pair programming and modeling with others, was declared to be a process crime because management felt that it was obvious that collaborative techniques such as this would reduce overall productivity. It was also a process crime to become a generalizing specialist because not only would it be difficult to retain anyone with marketable skillsets, thereby increasing the burden on the human resources (HR) team, it would also threaten the existing political empires currently based on specialized job functions. People who committed process crimes were fired on the spot and simply disappeared, and BB put a reward program in place so that you could even earn ++GOOD points by reporting process crimes.

Although this was a bit harsh (but how else could you possibly govern IT professionals), the process engineers did manage to improve things. After analyzing the values of the agile alliance, the process engineers determined that they were insufficient to meet Gorwell's needs and thereby introduced the Victory Doublethink slogans. The agile value of "working software over comprehensive documentation" was replaced with the simpler "Documentation is Software" slogan, an obvious fact that any bureaucrat can explain to you. Similarly, the slogan "Metadata is Quality" communicated the importance of Operation Blackhole and also made it obvious that regression testing and continuous integration were clearly a waste of time. The slogan "Plans are Progress" enabled project teams to report earned value when compared to the agile strategy of delivering high-quality working software on a regular basis—business stakeholders could trust the word of IT project managers and didn't need the concrete visibility provided by real agile teams.

The human resources department supported the Agile Victory process by establishing the "Victory Alliance," a group of trainers that ran what they called the Certified Victory Master (CVM) program. Anyone could become a CVM, regardless of experience or ability, simply by taking a two-day CVM training course. Granted, you couldn't become a certified master of anything, even something as simple as tying your own shoes let alone leading a software development project, by taking a two-day course. The Victory Alliance claimed that what they were really doing was certifying that you had taken the class, but this didn't stop anyone from claiming that they were Certified Victory Masters on their business cards and in their e-mail signatures. Of course it didn't help that the trainers provided the CVM logos to anyone who wanted a graphical instead of textual representation of their awesome certification accomplishment.

In the end, Winston's agile adoption effort was squashed. By putting an agile façade on top of traditional strategies, Gorwell managed to derail the productivity improvement potential of actual agile techniques. Inadequate training, mentoring, and coaching prevented staff experienced in traditional approaches to learn the techniques and nuances of agile development. In short, Gorwell's existing culture and bureaucracy had successfully repelled the agile onslaught.

Parting Thoughts

If you haven't figured it out by now, April Fools! Although some agile adoptions do, in fact, fail, this column is completely fictitious and is an homage to George Orwell's 1984. Although fiction, I suspect that many of the challenges that I described within Gorwell Financial Group will strike close to home for a lot of readers. It would be doubleplusgood if you were to consider many of the concepts promoted by the Victory process to be thoughtcrimes—oops, I mean process crimes.

Terms of Service | Privacy Statement | Copyright © 2024 UBM Tech, All rights reserved.