Scott Ambler is chief methodologist/Agile and SOA for IBM Rational.
In This Issue
- Tragic Mistakes When Adopting Test-Driven Development (TDD)
- Take the "State of the IT Union" Survey and potentially win a copy of "The Art of Scalability"
- Hot Links
Tragic Mistakes When Adopting Test-Driven Development (TDD)
Test-driven development (TDD) is likely the most commonly discussed agile practice, probably because it offers the single biggest opportunity for productivity improvement on software delivery teams. With TDD you write a single test, then write just enough production code to fulfill that test, refactor where needed, and then iterate. With this approach your tests do double duty in that they represent a just-in-time (JIT) specification for your code and perform confirmatory validation on it. Sadly, over the past few months I've run into several organizations that were struggling to successfully adopt TDD. Actually, I've seen this problem for a long time now, but as luck would have it I've just happened to see it a lot lately and that's motivated me to write this article.
There are two forms of TDD:
- With "developer TDD"you typically use a xUnit tool, such as JUnit for Java or VBUnit for Visual Basic, to write developer tests. Developer tests, which include unit tests and simple integration tests, specify and validate the design of your code.
- With "acceptance TDD" you use more sophisticated tools, such as the Fitnesse framework, to write what's often referred to as agile acceptance tests or story tests. These tests specify and validate your code.
The 2008 Dr. Dobb's and Ambysoft Test-Driven Development (TDD) survey found that developer TDD is more common than acceptance TDD within the agile community, perhaps because of the greater technical focus of agile developers.
You will face several challenges when adopting TDD.
- First, developer TDD requires programmers to have testing skills and acceptance TDD requires analysts to have testing skills. In other words, TDD pretty much requires people to break out of their traditional specialties and move towards becoming a generalizing specialist.
- Second, TDD requires significant discipline on the part of practitioners. It's easy to say you're going to consistently write a single test before writing just enough production code to fulfill that test, but not as easy to actually do so.
- Third, for most organizations TDD requires new tooling, albeit tooling that is often available via open source.
So why go to all the hassle? As I said, there are significant benefits to TDD. With TDD you're writing executable specifications on a just-in-time (JIT) basis, which reduces the documentation burden on your team and thereby potentially lowers project cost. Traditional project teams invest an significant percentage of their budget creating, reviewing, and maintaining interim specifications which often do little more than gather dust once the project is completed. Another obvious benefit of TDD is the potential for increased quality -- agilists do more testing and do so earlier in the lifecycle, providing greater opportunity to address defects when they're easy and inexpensive to fix. A third benefit is improved time to market because you can avoid most of the up-front specification effort which plagues traditional teams and because it's easier to work with high-quality code than it is to work with low quality code (think about how much time you spend debugging code on traditional projects) which speeds up development. Before you scoff at these claims, the Dr. Dobb's 2008 Project Success survey found that agile teams were in fact perceived as being more cost effective, able to produce higher quality systems, and able to deliver faster than traditional teams. These benefits clearly can't be attributed solely to TDD, but I suspect some percentage would be.
There are several reasons why organizations are struggling to adopt TDD effectively. First is a lack of investment in opportunities for individuals to learn the technique, perhaps because organizations underestimate the complexity of the required skills or more likely due to a lack of funding. The TDD survey found that for both acceptance and developer TDD that pairing with an experienced person and mentoring were rated as the most effective means of learning the technique. Training came in a distant third followed by other techniques such as pairing with another learner, reading, and online forum discussions. My experience is that even with training and mentoring it can take a few months before someone gets good at TDD, so a bit of patience is required. So few organizations promote the practice of pair programming, typically because of fears that their productivity will drop if they do so, that it's pretty much futile to even suggest it any more. I've found that organizations which give pair programming a fair try, my suggestion is to have a team decide that for the next month they will not tolerate solo programming at all and then afterwards decide whether pairing works for them, discover that pairing leads to greater overall productivity over time.
Another serious challenge is that many organizations are still writing detailed specifications, what I like to refer to as detailed speculations, on their projects even after adopting TDD. Remember that TDD isn't just a validation technique, it's also a specification technique. The implication is that if you're doing just-in-time (JIT) requirements and design specification via acceptance TDD and developer TDD respectively that you can reduce, often dramatically, the amount of detailed speculations that you write at the beginning of project. Yes, you'll still need to follow Agile Modeling techniques of high-level requirements envisioning and architecture envisioning to provide overall direction for your project, but the details can and should be captured via TDD.
Similarly I see organizations that are still supporting comprehensive testing by independent testers, either in parallel to development or at the end of the life cycle, even though the development team is doing TDD. With TDD the developers do the majority of the testing, in particular they should be doing most if not all of the confirmatory testing, alleviating the need for a comprehensive testing by your traditional testing/QA group. There will, however, may be a need for some parallel independent testing, particularly at scale which focuses on more complex forms of testing such as investigative testing, systems integration testing, usability testing, security testing, and more. The implication is that when you adopt TDD you can and should reduce some of your existing testing efforts.
The fundamental mistake that organizations make is that they assume that when they're adopting a new practice that all they need to do is add new techniques, not realizing that the new practice often alleviates the need for other work currently being performed. A contributing factor, often the primary factor, to this problem is an unwillingness to deal with the political ramifications. The perception is that if you reduce the amount of detailed speculations that you write it reduces the need for people, such as business analysts and data analysts, who specialize in that sort of activity. Similarly, the perception is that if you reduce the amount of independent testing being done it reduces the need for testing and quality assurance people who focused on that. The reality is that TDD is removing the need for some of the more tedious aspects of analysts and testers jobs, freeing them up to focus on value added activities. This often implies that these people need to choose to become active members of development teams, sharing their exiting skills with other team members while picking up new skills from them. In other words, they need to move away from being narrowly focused specialists to more productive generalizing specialists.
Another common mistake that some organizations make when adopting TDD is doing so in too extreme of a manner. They believe that with TDD you don't need to do any modeling or independent testing at all, but nothing could be further from the truth. Numerous DDJ surveys, particularly the 2008 Modeling and Documentation survey, have found that the vast majority of agile teams do some sort of up front modeling and that they model throughout the project. TDD addresses detailed specification but isn't very good at high-level specification, that's what agile modeling practices are for. Similarly, the TDD survey itself found that agile teams were still doing some sort of independent testing even after adopting TDD. TDD only addresses confirmatory testing issues, you'll still need other techniques to address the full range of agile testing issues.
If your organization is thinking about adopting TDD, or is currently struggling to do so, I hope that this article has given you a few insights into the challenges that you face. TDD is about working smarter, not harder, and that requires that you change the way you're you organize teams, the skills that your IT staff have, and the tools that you work with. This can be painful at first, but the benefits of TDD more than seem to make up for this. Many organizations are in fact succeeding with TDD and so can your's.