There's No Room for Deadlines
One of the things that people new to agile thinking have a hard time getting their heads around is that time (and estimates, and deadlines) is simply not a factor in an agile shop. They say that deadlines are a necessary part of the "real world." Yes, there are real-world issues that surround time — you may have to present at an upcoming trade show, for example — but time is not a central part of the agile-planning process.
Let me explain.
Agile planning falls out of several basic principles, the most important being:
Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
That notion of a constant pace means that a standard, deadline-based management process simply can't work. There's no room for overtime to meet a deadline, for example. Kent Beck characterized overtime as "borrowing from the future:" You get so burned out that you can't work effectively once the deadline is met, so the pace of development slows considerably after the push. This is not a zero-sum game, with extra productivity before the deadline compensating for lower productivity after. In the best case, you never gain back the lost productivity. In the worst, burned-out people leave the company. If the team is punished for not meeting their deadlines (and overtime is a kind of punishment), then they'll pad the estimates to avoid the pain. Padded estimates both kill productivity and undermine effective planning because they budget for wasting time.
Moreover, deadlines are based on long-term estimates, and estimates of that sort never work. Take that trade show as an example. A deadline-focused manager typically starts with the assumption that certain features are essential, and if you can't build these features by the deadline, then you've failed. An agile team would say that your notion of "essential features" will change as the product develops, so any estimation that you make at the beginning of the product development cycle will be incorrect, because the set of things on which you base the estimate will change. You always find flaws in the specification or encounter unforeseen implementation problems. More importantly, you always uncover major flaws in your thinking about the product.
In other words, you're estimating something concrete — a fixed set of features. To insist on meeting an estimate at a fixed deadline is to commit to building that set of features. That set is bound to be wrong, however, so the estimate is bound to be meaningless. Committing to the deadline is effectively a commitment to build the wrong product.
I'm not sure about you, but in my "real world," building a product that nobody's going to buy isn't a particularly important goal.
So, why all the insistence on deadlines? In a waterfall world where it takes a year to get something that works, a deadline provides the illusion of security. A traditional deadline is a way to measure progress in order to manage budget. The reasoning is: The product won't be finished for a year, but at least we know that we're X% done. You have to be able to plan. Budgets are not infinite. If you budget based on estimates and you can meet those estimates accurately, then you can plan effectively. Nice theory.
Which brings us back to constant pace.
When you're working at a constant pace, the productivity of the team is constant. Moreover, the team can accurately predict how much work it can finish in a short iteration. That's a kind of estimation, admittedly, but it's a reliable one. I've found that experienced teams make pretty good guesses based on observation of past iterations, and when they happen, errors don't have a large impact. We're talking a day or two, not months.
When you couple a constant pace to short iterations, you can guarantee that you can deliver a working product every couple weeks.
The software is guaranteed to work when that trade-show arrives, because it always works.
The goal changes from "building a specific set of features by December" to "building the best possible product in the time we have."
The question changes from "Will it work?" to "What will it do?"
Since we have a list of the unfinished stories (the Scrum guys call it a backlog), we have a good notion of how much work needs to be done:
We know how big the stack of stories is.
We know how many stories we can finish in an iteration.
We know exactly how long an iteration takes.
Consequently, we can project how long it will take to finish the current backlog.
If we lay out the stories in a story map, we can draw a line that represents "
N months out" and see what will be done at that point in a glance. More to the point, if we add stories, we can see what gets pushed past that line and make intelligent decisions about what to build. If we don't like what's above the line, we can change it.
Of course, there are situations (regulatory requirements, for example), where you can't simply declare the current version as "done" and march off into the sunset. A constant pace gives us a more effective way to manage that problem, too. If something critical gets pushed over that line on the map, we have two options: eliminate something unessential that's above the line or add another team. More to the point, we can see that problem very early, so we can add another team. (Brooks's Law says that adding people late in the project won't help.)
The process I just described satisfies all the goals of a waterfall style manager. We can monitor progress, control costs, allocate people and physical resources, and manage our budget effectively. We've lost nothing except the pressure of an unreasonable deadline.
I'll finish up with a theme that should be familiar to you by now. A company with a culture of deadlines can't work the way I just described. They'll undermine the teams with a constant push towards an illusory goal, and any attempt to work in an agile way will fail as a consequence. They'll also build products that nobody wants. Agile thinking must extend way beyond the boundaries of the Engineering Department for it to work at all.