Dr. Dobb's | Survey Says...Agile Has Crossed the Chasm

When the Agile Manifesto was published in 2001, it gave name to a collection of methodologies that had been growing in popularity for several years. These methods, many of which are now in common usage worldwide, had strange names such as Scrum, Pinball, Extreme Programming (XP), and Dynamic System Development Method (DSDM). My 2007 Agile adoption survey (see sidebar) shows that agile techniques have been successfully adopted within a majority of organizations and often at scale. So now, six years later, I think that it's clear that Agile has successfully crossed Moore's technology-adoption chasm.

Figure 1 overviews the adoption rate for agile techniques. When asked whether their organization had adopted any agile techniques, or if not when they thought it would happen, 69 percent responded that their organization was currently doing agile and an additional 7.3 percent indicated that they believed it would happen within the year. More importantly, as you can see in Figure 2, many organizations that have adopted agile approaches have clearly gone beyond the pilot project stage. Of the 427 people who indicated the number of agile projects that their organization has performed, 45 percent indicated that their companies had done two to five agile projects and 39.6 percent indicated five or more projects (unfortunately, there was an overlap in some categories in the wording of the question, although respondents were only allowed to select one option).

The success rate of agile projects appears to be very high—77 percent of respondents indicated that 75 percent or more of their agile projects were successful, as you can see in Figure 3. For this question, the definition of success was left up to the respondent as the definition of IT project success varies by organization and often even by project (more on this in a future column).

Agility at Scale

Agile is being used for various-sized projects. Table 1 summarizes the results when respondents were asked two questions about team size: What is the largest agile project your organization has attempted, and what is the largest project that agile techniques have been successful on? There are two interesting observations about these results. First, although the majority of organizations are applying agile techniques on projects of 10 or fewer people, many are, in fact, trying agile on larger projects. A few years ago, Jim Highsmith estimated that 70-80 percent of all software-development project teams are 10 people or less, so these numbers aren't out of line. Second, organizations seem to be more successful with smaller project teams than larger ones. This is to be expected: The bigger the team, the harder it is to succeed regardless of the development paradigm, and much of the agile literature has focused on colocated teams.

Team Size (People)	Largest Successful Agile Project	Largest Attempted Agile Project
1 to 5	165	135
6 to 10	144	135
11 to 20	73	90
21 to 50	30	41
51 to 100	6	11
101 to 200	3	6
201+	2	5

Table 1: Applying agile techniques on various sized projects.

The survey also asked about the success rate of agile when applied in various situations: 296 respondents indicated that they had tried agile techniques when the team was colocated, 251 tried agile without colocation, and 130 had applied agile in offshoring situations. Figure 4 shows that the average success rate is highest for colocated teams and lowest for offshoring situations. There are several implications of Figure 4. First, the agile rhetoric about communication and collaboration being critical success factors seems to hold. Second, although the success rates are clearly lower, many organizations are succeeding at agile projects that aren't colocated. Organizations are, in fact, succeeding at scaling agile software development techniques.

Agile Practices

The survey explored the effectiveness of various practices on agile projects, both agile and not-so-agile, to determine what is actually working in practice. The results are listed in Table 2, with the most effective practices listed first. The score is the average taken from respondents who responded with a numerical score; the applicable values were 1 to 5, with 5 being very effective, and values of "Don't Know" and "Not Applicable" were not taken into account to calculate the average.

Practice	Average Score out of 5
Iterative Development	4.38
Regular Delivery of Working Software	4.19
Configuration Mgmt	4.06
Whiteboard Modeling	4.01
Customer Tests	3.94
TDD	3.91
Daily Stand Ups	3.88
Active Stakeholder Participation	3.88
Coding Standards	3.83
Code Refactoring	3.81
Prove Architecture Early	3.80
Simple Design	3.78
Flexible Architectures	3.74
Evolutionary Design	3.70
Self Organizing Teams	3.61
Initial Agile Arch Modeling	3.60
Initial Agile Req Modeling	3.57
Regular Status Reports	3.54
Independent Testing	3.53
Architectural Spikes	3.52
Paper-Based Modeling	3.43
Code Inspections	3.43
Pair Programming	3.40
Database Testing	3.37
Database Refactoring	3.31
Model Reviews	3.14
CASE Modeling	3.03

Table 2: Effectiveness of various practices on agile teams.

I want to share a few observations. First, the high-scoring practices really don't come as a surprise, particularly iterative development and regular delivery of working software. Once again, the agile rhetoric seems to hold. Second, every practice that I asked about was rated above average, although I didn't explore many traditional practices such as detailed up-front modeling and detailed up-front planning because they were covered by questions about the effectiveness of work products. Third, pair programming didn't rate as well as I had expected, probably because many organizations often don't give it sufficient time to take root on a project team. In a conversation with Laurie Williams and Bill Krebs, two people who have done research into pair programming, we decided that the survey should have distinguished between promiscuous pairing where you swap pairs regularly and nonpromiscuous pairing. We suspect that promiscuous pairing would do much better. Fourth, I was a bit disappointed in how database refactoring and database testing did, although, to be fair, this is likely a reflection of the current lack of tool support for these concepts. Tooling is quickly getting better, so I expect the rankings of these practices to rise in future surveys.

The survey also asked about average iteration length on agile projects. Five percent of respondents indicated the last agile project they were on had iterations of less than one week, 17 percent one week, 32.6 percent two weeks, 12.5 percent three weeks, 21 percent four weeks, 10.4 percent five or more weeks, and only 1.4 percent indicated that they didn't do iterations at all. So, it seems that the vast majority of agile teams do reasonably short iterations of between one and four weeks.

Agile Work Products

The survey also asked about the effectiveness of various work products, summarized in Table 3 (the rating strategy from Table 2 is used). It was interesting to see that the agile rhetoric around the importance of working software and source code was validated by the survey. More interesting, at least to me, was that developer tests and whiteboard sketches pretty much received the same score-developer testing seems to receive at least an order of magnitude more discussion on agile forums than does whiteboard sketching, confirming my observation that agilists sometimes fail to describe what they actually do in practice.

Work Product	Average out of 5
Working Software	4.22
Source Code	4.22
Developer Tests	3.96
Whiteboard Sketches	3.90
Iteration Task List	3.87
Customer Tests	3.87
Arch Spec—High Level	3.66
Defect Reports	3.61
Velocity (Metric)	3.56
Requirements Spec— High Level	3.52
Use Cases—Light	3.52
Test Plan	3.39
Burn Down Chart	3.35
Paper Models	3.22
Use Cases—Detailed	2.96
Requirements Spec— Detailed	2.90
Arch Spec—Detailed	2.88
Gantt Chart—High Level	2.70
Gantt Chart—Detailed	2.21

Table 3: Effectiveness of various work products on agile teams.

There's clearly a loud message that detailed documentation (in particular, requirements and architecture specifications) have little value to add on agile teams. High-level architecture and requirements models, as promoted by Agile Model Driven Development (AMDD), do seem to be effective in practice, as do whiteboard sketches. I was surprised that paper models didn't seem to be as effective, although in hindsight, I suspect that had I asked directly about user-story cards, CRC cards, and paper UI prototypes, user-story cards would have done well. There's always next year.

On the project-management side of things, both detailed and high-level Gantt charts came in dead last, something our good friends at the Project Management Institute (PMI) should pay attention to. Task lists were very close to the top, an indication that agilists prefer simpler approaches to project planning. Burn down charts, for all the attention that the Scrum community gives them, don't prove as useful in practice as we would be led to believe, something I wouldn't have expected.

The Future's So Bright We Gotta Wear Shades

Although there is an increase from last year's agile adoption rate, I'm reticent to compare the figures because I asked the question significantly differently. Having said that, I think it's safe to claim that Agile has crossed the chasm and should now be considered mainstream. It's time for traditional holdouts to take notice, because as many agilists have likely told you, you're clearly missing out on a good thing. DDJ