Dr. Dobb's | Newsflash: Agilists Write Documentation!

Who said Agilists don't do documentation? Scott looks at what agile teams actually do in practice.

You've likely heard, and worse yet spread, some of the common myths around modeling and documentation on agile projects. For example, there's the myth that Agilists don't do any architecture, something that flies in the face of the fact that all systems have architectures regardless of the development paradigm used to create them. Another myth is that Agilists don't do up-front requirements modeling, yet when was the last time you saw a software development get funding without an indication of what was going to be built? Agilists apparently don't write documentation either, having magically convinced end users that they don't need user manuals, operations and support people that they don't need any sort of documentation, and the maintenance team that they don't need any sort of overview documentation. The list of things that agilists are supposedly getting away with is truly impressive, so needless to say, I believe that it is abundantly clear that these myths and misunderstandings need to be cleared up. This month I summarize the results from two surveys, one that focused specifically on modeling and documentation activities on software development projects and one that explored what agile teams actually do in practice. Reality is definitely far from the rhetoric.

Up-Front Modeling

Dr. Dobb's 2008 Modeling and Documentation survey discovered a few interesting things about how teams approach modeling. It found that 18 percent of traditional teams did no modeling at all, compared with only 5 percent of agile teams, implying that agile teams are arguably more likely to model than traditional ones. It also found that the primary approach to modeling on agile teams was that 55 percent of teams preferred to sketch to think things through or to communicate ideas, and an additional 30 percent both sketched, then captured critical sketches when it made sense to. The majority of traditional teams also focused on sketching, with rates of 33 percent only sketching and 33 percent sketching and then capturing critical ones. Seven percent of agile teams use software-based modeling tools (SBMTs) such as ER/Win or Rational System Architect (RSA) for documentation purposes and 5 percent for full round-trip engineering, compared with 16 percent and 0 percent for traditional teams, respectively. Figure 1 summarizes the results for all four types of teams that the survey looked at: ad-hoc, traditional, iterative, and agile.

The Ambysoft 2008 Agile Practices and Principles survey explored how well agile developers agreed with various development strategies. To explore whether agile teams were doing up-front requirements modeling, it asked "We do some initial requirements modeling at the beginning of agile projects for scoping and planning purposes." The survey found that 74 percent of respondents either strongly agreed or agreed with this statement, 18 percent were neutral, and only 8 percent indicated disagreement or strong disagreement. Similarly, to explore up-front architecture modeling, it asked the question "We do some initial architecture modeling at the beginning of agile projects to get going in the right technical direction." The results were almost identical, with scores of 73 percent, 19 percent, and 8 percent, respectively. In short, two different surveys to two different communities both showed that agile development teams do, in fact, model.

[Click image to view at full size]

Figure 1: Primary strategies for modeling for various software development paradigms.

Agile teams are doing up-front modeling because they still need to answer questions around the scope that they're addressing, the relative cost and schedule, and what their technical strategy is. Without coherent answers to these questions, the project simply won't get funding. Although you need to do some modeling early in the project lifecycle, that doesn't imply that you need to write mounds of documentation to do so. You can gain the benefit of modeling, which is to communicate and think things through, without accepting the costs of unnecessary documentation.

Another reason why agile teams are doing up-front modeling, contrary to the popular belief of extremists in the agile community, is because architectural metaphors are not sufficient. System architectures are complex, addressing a myriad of nonfunctional requirements and technical constraints such as performance, security, scalability, usability, legacy assets, integration, and many more (see my October 2008 column on this topic). An architectural metaphor simply isn't going to get the job done. For example, consider the website for the Agile 2008 conference (www.agile2008.org), which was built around the metaphor that organizing a conference is just like running a stage production. This metaphor clearly didn't work well because people didn't find the website very usable. For example, to put together a schedule for yourself you needed to navigate the 19 stage pages to get detailed listings of what was being offered on each "stage." Yikes. A little bit of up-front requirements and architecture envisioning could very likely have avoided this sort of problem.

Agile Documentation

Dr. Dobb's Modeling and Documentation survey revealed that there's not a lot of difference between the various development approaches when it comes to deliverable documentation. After all, a deliverable is a deliverable. So, the fears concerning the mythical lack of documentation on agile projects appear to be unfounded, forcing the traditionalists who are still holding out to find another excuse for not adopting agile strategies. As you see in Figure 2, the survey did show that agilists are slightly less likely than traditionalists to produce system overview documentation, although slightly more likely to produce operations documentation. I suspect that there is less of a need for system overview documentation because agilists have a greater tendency to produce higher quality code as the result of refactoring and test-first practices. Agile teams may also require less documentation due to the practice of writing executable specifications.

The Practices and Principles Survey explored how people communicate with one another on agile project teams, asking how effective various ways for communicating with stakeholders are. In order of most effective to least effective, the score for each technique is a weighted average between 5 (very effective) and -5 (very ineffective), it found the following:

There are several interesting observations to be made. First, the belief that traditionalists have in the value of detailed documentation can be understood as it has neutral value, regardless of all the rhetoric that we hear in the agile communities surrounding the evils of detailed documentation. Second, sharing overview documentation with stakeholders does seem to have some merit, although it is nowhere near as effective as direct communication, clearly belying some agile prejudices against documentation. Third, media richness theory, which Alistair Cockburn introduced the Agile community to (see www.agilemodeling.com/essays/communication.htm), pretty much holds. The "hotter" the communication channel, generally the more effective it appears to be. Fourth, these results may explain why distributed agile projects appear to be riskier than colocated projects because the greater the distribution, the more restricted your communication options become. Dr. Dobb's 2008 Agile Adoption Survey (www.ddj.com/architect/207600615) found measurable differences in the success rates of distributed agile projects—83 percent for colocated agile teams, 72 percent for near-located agile teams, and 60 percent for far-located agile teams.

[Click image to view at full size]

Figure 2: The chance that a key documentation deliverable will be produced by teams following various software development paradigms.

Agile Modeling and Documentation Best Practices

So now that we know that agile developers are in fact modeling and writing documentation on a regular basis, it begs the question why so many myths still exist. I believe that there are three primary reasons. First, the agile community struggles to coherently describe what they do in practice. For example, if you mention up-front modeling on many popular agile mailing lists, the conversation almost instantly devolves into ranting about the evils of big design up-front (BDUF) and extraneous bureaucracy with virtually no mention of what people are actually doing. Second, many traditionalists fear agile software development approaches because they require a different skillset and often greater discipline to succeed at agile. The "new rules" around agile development are naturally scary to anyone not familiar with them or anyone who hasn't taken the opportunity to make the transition yet. Third, the lack of detailed written specifications early in the project can make it seem that agile teams aren't modeling to traditionalists who aren't familiar with Agile Model Driven Development (AMDD)-based approaches. For example, we saw in Figure 1 that agile teams are more likely to model than traditional teams, that they prefer sketching rather than creating detailed documentation using SBMTs. Differences such as this can be confusing for anyone not familiar with disciplined agile strategies.

There is a wealth of material available online about how agile development teams approach modeling and documentation. The AMDD best practices are to do some initial requirements and architecture envisioning early in the project to write executable specifications via a Test-Driven Development (TDD) approach, to single source information whenever possible, to write documentation later in the lifecycle, to promote active stakeholder participation, to implement requirements in priority order, to include modeling in iteration/sprint planning activities, to create models and documents that are just barely good enough for the situation at hand, to model storm the details on a just-in-time (JIT) basis, to sometimes model a bit ahead to explore complex requirements, and to take a multiview approach via multiple models. A description of these best practices can be found at www.agilemodeling.com/essays/bestPractices.htm.

We Need to Do Better

It is possible to achieve value from effective modeling and documentation practices, but unfortunately many organizations aren't taking advantage of these practices. Dr. Dobb's Modeling and Documentation Survey found that 80 percent of people are learning to model through on the job experience, 28 percent from mentoring by an experienced modeler, 18 percent through university or college courses, 9 percent from modeling training, and 5 percent by modeling tool training. It seems to me that if our approach to learning how to model was a bit less haphazard, then there wouldn't be as many myths and misunderstandings around modeling, and more importantly that we could leverage modeling practices and tools more effectively.