Channels ▼


Scaling Test-Driven Development

Not So Shocking...

TDD is incredibly popular, probably because it works very well in practice—so well that it has a near-religious following within the Agile community. DDJ's 2007 Agile Adoption Survey showed that customer testing is the fifth most valuable practice for agile teams and that developer-level TDD is sixth. When it comes to work products, developer tests are the third most valuable, after working software and source code, and customer tests are the sixth. The point is that there is broad-based support for TDD among Agile developers—that we're actually doing what we claim to be doing.

However, TDD isn't perfect. At the CEE-SET conference in Poznan, Poland in October 2007, a paper written by Maria Sinaalto and Pekka Abrahamsson entitled "Does Test Driven Development Improve the Program Code? Alarming Results from a Comparative Case Study" was presented. This paper summarized a research study into the effectiveness of TDD, and it found that TDD may produce less complex code but that the overall package structure may be difficult to change and maintain. Although other studies into the effectiveness of TDD had looked at code-level issues, and found that TDD helps to increase quality, this was the first one to look at higher level architectural issues. This new study appears to show that TDD is effective for "design in the small," but that we need another approach for "design in the large." These results aren't shocking at all because the focus on TDD is on detailed specification and validation, not on higher level issues. Clearly, we need to look beyond TDD for a solution to this problem.

Specification at Scale

Although TDD is great at specifying code at a fine-grain level, tests simply don't scale to address higher level business process and architectural issues. AMDD enables you to scale TDD through initial envisioning of the requirements and architecture as well as just-in-time (JIT) modeling at the beginning and during construction iterations. The principles and practices of the Agile Modeling methodology describe lean strategies for gaining the benefits of modeling, which are to think things through and to communicate ideas, without the pain of excessive documentation.

Yes, I realize that modeling is one of the dirty words in the Agile community, and any discussion of up-front modeling early in the agile lifecycle is tantamount to heresy. Yet, when you ask people in private what they actually do on real-world Agile projects, you discover that the vast majority of us are apparently heretics. DDJ's 2007 Agile Adoption Survey found that 77 percent of all Agile teams do a bit of up-front requirements and architectural envisioning to identify the initial scope and technical strategy respectively. Furthermore, it found that whiteboard modeling is the fourth most valuable practice and that whiteboard sketches are the fourth most valuable work product.

To scale requirements-level TDD, you must recognize that customer tests are very good at specifying the details, but not so good at providing overall context. High-level business process models, conceptual domain models, and use cases are good at doing so, and these work products are often created as part of your initial requirements envisioning and iteration modeling activities. Process descriptions and use cases can easily reference customer tests, or even be written as customer tests that in turn invoke other customer tests, to provide the business context required to understand the overall requirements for your system. The details are still captured in the form of tests, but the models provide the high-level context required to think through critical scoping and planning issues.

Similarly, to scale design-level TDD, you must recognize that developer tests are very finely grained but once again do not provide overall context. High-level architecture sketches created during your Iteration 0 envisioning activities help set your initial technical direction—a little bit of up front design work helps to avoid structural problems cropping up later in the lifecycle. During each construction iteration, you'll do more detailed design modeling, often sketching or using index cards to explore issues that tests aren't well suited for. Highly skilled developers will even use software-based CASE tools to do this sort of work, although the majority of developers will use IDEs such as Eclipse, Visual Studio, or Jazz.

Related Reading

More Insights

Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.