In This Issue
- Lies, Great Lies, and Software Development Project Plans
- Hot Links
Lies, Great Lies, and Software Development Project Plans
In last month's agile update, and in Jon Erickson's blog, we invited you to fill the June/July 2009 edition of my "State of the IT Union" survey, one of an ongoing series of surveys which explore what IT professionals are actually doing in practice. One of the themes which this edition explored was software development project planning, particularly initial project estimating, tracking of actual results, and what strategies did teams employ when they ran into trouble.
One of the questions explored how precise initial estimates had to be, giving options such as the initial estimate needing to be within an X% range, where X was 5, 10, and so on. 21% of respondents indicated that they weren't required to provide initial estimates, and for those that did the weighted average was that their estimates have to be +/- 11%. Interestingly, 39% indicated that they didn't track the actual costs against their original budget and of those that did on average they came within +/- 19%. On the surface this difference doesn't appear so bad, until you realize that many project teams apply very questionable practices to make it appear that they come in reasonably close to plan.
The survey explored the management practices applied to either stay out of trouble or to address the problems once they were perceived. The survey found that:
- 72% indicated that their project teams will extend the schedule so as to deliver on the promised scope
- 63% descope towards the end of the project to meet the deadline
- 39% avoid "scope creep" whenever possible via a "change management" process
- 34% will ask for extra funds
- 32% have a flexible scope from the beginning of the project
- 26% have a flexible schedule from the beginning of the project
- 18% pad the budget
- 18% change the original schedule to reflect the actual results
- 13% have a flexible budget from the beginning of the project
- 12% take a stage-gate approach to funding
- 10% change the original estimate to reflect the actual results
All of these strategies are deemed to be project management best practices within the IT community -- you can easily find project management books telling you so, attend project management courses which teach you how to perform these practices, and even gain industry-recognized certification in project management. Rather than congratulating ourselves for our clever strategies for getting out of trouble, let's instead look at these practices from the point of view of our business stakeholders. When we do that, we soon begin to realize that many of these project management "best practices" are little more than euphemisms for lying, and in one case may even be unethical. Here's why.
Let's start by examining the practices which are highly questionable from the point of view of our stakeholders. Our worst behavior is encompassed by traditional change management practices which are designed to prevent scope creep. When we commit to an initial schedule and cost based on a given scope we are highly motivated to ensure that the scope doesn't change because that will very likely cause our project to be late or over budget. To avoid this problem we institute a "change management" process which makes it difficult for stakeholders to change the scope. This in effect forces them to accept a system which has been built to specification instead of a system which reflects their actual needs, or in other words we're charging them for functionality that they don't really want. I personally struggle to see how this is ethical.
Furthermore, consider the practice of extending the schedule towards the end of the project in order to deliver the promised functionality. If this is a rare occurrence, perhaps for less than 5% of projects, then I wouldn't take issue with it. However, if this is common practice within your organization then the next time you "commit" to a schedule you are likely in effect lying to your stakeholders (and doing so knowingly). Similarly, if you regularly descope towards the end of the project then any commitments to scope on future projects also run the risk of being lies in effect, and asking for extra funds on a regular basis makes your cost estimates lies. When you purposely pad the budget you're also lying about what you think the actual costs will be, although this is nowhere near as bad as going back and updating the original estimate/schedule based on the actuals after the fact.
Sadly, we likely wouldn't need to apply many of these questionable strategies if we could only choose to be more realistic in our approach to project scheduling and estimating to begin with. In The Economics of Iterative Software Development, by Royce, Bittner, and Perrow (Addison Wesley 2009), the authors report that COCOMO II, an estimation technique, is accurate 74% of the time to within 30% (the other 26% of the time the estimate can be wildly off). Other traditional estimation techniques have similar accuracy rates, typically because they're all based on the same basic strategy -- identify the requirements, identify a technical solution, put this information through an estimation formula, and thereby produce an estimate. The problem is that our stakeholders can't tell us up front what they want in detail, regardless of all of our efforts for getting better at requirements analysis, and because of that we can't even begin to hope to put together a reasonably precise estimate.
This begs the question that if our initial estimation techniques are only good to within 30%, then why are we striving to predict within an average 11%? I believe that it's a reflection of the poor relationship that many IT departments have with the business stakeholders whom we're supposed to be serving. Our business stakeholders are trying to reduce the risks associated with IT projects, and they believe that by requiring us to develop, and then commit to, precise estimates and detailed schedules that they will achieve this. This would be a reasonable strategy if stakeholders were able to accurately define what they wanted, if the underlying technology platform remained stable, and if software development was actually an engineering endeavor. Unfortunately people aren't good at defining what they want, the technology evolves rapidly, and software development appears to be far more art than it is science. When we don't deliver on our commitments our stakeholders often insist on even greater precision and detail in our initial project plans the next time, not realizing that they're very likely increasing project risk by motivating us to behave in less than professional ways. The situation is further exacerbated when we don't have honest conversations with stakeholders explaining the realities of software development, or worse yet continue to hold out the hope that we'll someday find the right magical precise estimation and detailed planning techniques.
All is not lost, however. A minority of respondents clearly indicated that they were following project management practices which were based on open and honest communication with their stakeholders. These practices include taking flexible approaches to the scope, to the budget, and to the schedule from the very beginning of the project. In other words to actively work with them to understand their changing needs throughout the project, to help them actively manage their investment in IT projects, and to provide concrete feedback as to the actual state of the project so that they can determine when the system is ready to ship. For these strategies to work you must have a healthy, respectful relationship with your business stakeholders -- something that takes years to build.
There were only 125 respondents to this survey, so to be conservative we should consider these results indicative of trends but not the final word. Having said that, none of the results seem unreasonable. For everyone who filled out the survey, thank you very much. For those who didn't fill out the survey, please consider doing so in the future. As a community we have an opportunity to find out what is actually happening in IT departments around the world, and based on this knowledge perhaps we can change things for the better. I keep these surveys as short as possible and as you can see the potential to get meaningful results is clearly there. Next month I will summarize the results of questions which were related to development governance issues.
The July 2008 Dr. Dobb's Agile Update describes five strategies for reducing the inherent risks associated with fixed price projects. If you can't avoid fixed price then at least be smart about how you go about them.
The August 2008 Dr. Dobb's Agile Update asks the questions "Is Fixed-Price Software Development Unethical"?
The March 2003 Something's Gotta Give argues against a fixed price, fixed schedule, and fixed scope approach and describes agile strategies for avoiding this serious mistake.
The September 2007 Agile on a Fixed Budget, describes strategies for running agile projects for all combinations of fixing, or not fixing, the schedule, scope, and budget.
The NoNovember 2007 Governing Agile Software Development Projects shows how it is easier to govern agile projects compared to traditional projects.
The Surveys Exploring the Current State of Information Technology Practices page links to the results of all the Dr. Dobb's surveys which I've run over the years.
The details of this survey, including the original questions, source data, and a summary slide deck, will be posted at State of the IT Union: July 2009 in late August 2009.
My Agility@Scale blog discusses strategies for adopting and applying agile strategies in the complex environments.