One of the challenges faced by IT professionals around the world is that we're often not trusted by the people we're meant to serve. There are several reasons why we're not trusted, but the primary one is that we're often perceived as liars — and rightfully so considering our software development track record. I explored this problem two years ago in my July 2009 Agile Update: Lies, Great Lies, and Software Development Plans. The article summarized results from a survey we ran earlier that month that explored how project teams got themselves out of trouble. We discovered that many teams exhibited ethically questionable behavior. I received interesting feedback about this newsletter, particularly because several of the ethically questionable strategies are promoted as best practices within the project management community. So, I decided to repeat this effort with a new survey. I explain the results herewith.
This survey ran in late December 2010 and early January 2011 and had 235 respondents. I wanted to get the most timely information possible, so the survey asked people to describe their most recently completed project. The previous survey had asked about all projects within their organization, which in turn inflated the response rate. The new survey explored scope, schedule, cost, and process-related issues. Respondents were presented with a collection of practices and strategies that their teams might have used at some point in their project to get back on track. Some of the options were good ideas, some questionable, and a few downright dishonest.
For example, when it comes to schedule, a good idea would be to negotiate a variable schedule at the beginning of the project, a bad idea would be to commit to a "firm" end date and then negotiate a change to it late in the lifecycle (in some cases this could border on extortion), and a downright dishonest strategy would be to cook the books at the end of the project to make it look like your original schedule predicted the actual end date even when it hadn't.
After exploring the behaviors and survey results, I discuss preferred alternatives when digging out of a problem project.
Ethically Challenged Behavior
The ideas in this section are straightforward, but for many readers they may be the exact opposite of what you've been taught over the years. The problem is that the survey explored many strategies that are touted as project management best practices. And to be fair, when they're considered in exclusion, they arguably are. The problem is that these practices aren't exclusive, they're part of an overall toolkit that project teams pick and choose from appropriately.
For example, there is a low probability that any given team will try to extort more money out of their stakeholders near what should be the end of their project, so low that we're tempted to accept this indiscretion as "business as usual" when it does occur. The issue is that these probabilities add up, as you'll soon see, until there is a reasonably high probability that the IT project team will purposely deceive stakeholders about one or more of the cost, schedule, scope, or quality of their work. When we continue to do this from project to project to project, it constitutes unethical behavior. We set expectations at the beginning of the project, often because we're asked to by our stakeholders (a flimsy excuse at best), but know full well that we're likely to do something different when the going gets tough. If you were on the receiving end of this behavior, how would you feel?
When it comes to budgetary practices, 15% of respondents indicated that their team had motivated changes to the budget in an ethically questionable manner, even though a flexible budget had not been negotiated for the project. They had done one or more of the following: padding the initial budget (effectively deceiving stakeholders as to the actual expected costs); requesting extra funds at the end of the project even though they'd committed to a fixed budget; or reworked the initial estimate to align with the actual results at the end (effectively "cooking the books"). Analyzing the results by development paradigm, we find that 9% of agile teams, 20% of traditional teams, 22% of iterative teams, and 12% of ad hoc teams followed these questionable strategies. I'm not completely sure why there are such differences. Granted, agile teams are more likely to initially negotiate a variable budget than traditional teams because they recognize that the requirements will evolve over time, but the same should be said of iterative teams as well. Expectations for ad hoc teams are lower, so it's less likely that they'd be asked to provide an initial estimate anyways — or, if they were asked to commit to a budget it might motivate them to move away from ad hoc to begin with. The traditional paradigm assumes that it should be possible to define the budget up front, but in practice this does not work out for a significant percentage of the teams.
A similar rate of questionable behavior occurs when it comes to requirements practices, with 15% of respondents indicating that they descoped at the end of the project even though flexible scope wasn't initially negotiated. By paradigm, the survey found that 17% of agile teams, 11% of traditional teams, 17% of iterative teams, and 15% of ad hoc teams took this strategy. The fact that the results were so close and so low is an indication to me that teams are consistent in their desire to deliver what their stakeholders want regardless of paradigm. Scope management was the only instance in which traditional teams acted in a more ethical manner than agile or iterative teams. This is likely a reflection of the underlying rigidity of the traditional approach to requirements.
The rates of ethically dubious schedule practices were significantly worse, with 31% of respondents indicating that they had done at least one of padding the schedule, extending the schedule at the end, or reworking the original schedule at the end to reflect the actual schedule even though a flexible schedule was not initially negotiated. By paradigm, 21% of agile teams, 42% of traditional teams, 35% of iterative teams, and 32% of ad hoc teams exhibited questionable behavior. Once again, agile teams seemed to work in a more ethical manner than teams following other paradigms, although to be fair agile teams are more likely to negotiate a flexible schedule up front and therefore are less likely to run into trouble.
When it comes to the willingness of teams to compromise quality the good news is that the numbers are fairly low, with 15% of respondents indicating that their teams had done so in order to get themselves "out of trouble." By paradigm, 4% of agile teams, 16% of traditional teams, 17% of iterative, and 24% of ad hoc compromised quality. When it comes to agile I suspect that this is the result of the greater focus on quality that we see in that community, although I'm surprised that iterative and traditional had similar rates because iterative teams tend to test earlier in the lifecycle.
To summarize, based on our reported behaviors, there's a good chance that our stakeholders will perceive that we're lying to them at the start of a project when they're making critical funding decisions. There is a 15% chance that we'll deceive them about the cost of a project, 15% chance of deceiving them about what we're going to deliver, a 31% chance that we're lying about the schedule, and a 15% chance that we'll promise greater quality than we'll actually deliver. The bad news is that there's a 44% chance that we'll do at least one of these things (by paradigm, 32% for agile teams, 53% for traditional teams, 48% for iterative teams, and 50% for ad hoc teams). Looking at it from the point of view of our stakeholders, there's a 44% chance that a given project team will deceive them about one or more important success factors. Doing some simple combinatorics analysis, if your organization is currently running five projects there's a 95% that at least one of the teams has committed an ethically questionable act. Or, if you've been on five projects in your career, there's a 95% chance that you've been involved with such a team. Is it any wonder that business stakeholders consider IT untrustworthy?
Why are these things happening? My experience is that many of these improper IT practices are adopted to protect us from bad business practices, such as the desire of business leaders to have "accurate" cost and schedule estimates early in an IT project. For example, the survey found that we're being asked to give initial estimates on an average range of +/- 12%, which seems reasonable from a business point of view. Yet research by Barry Boehm, based on data from thousands of projects, shows that initial estimates for IT projects should be on a range of -25% to +75%. I believe that this discrepancy between what we're being asked to do and what is actually realistic to do motivates poor practices on the part of IT professionals.
The survey asked about two requirements practices that I believe to be dysfunctional although not ethically improper. Actually, it really depends on the motivation behind following the practices. The first practice is a strict approach to change management that makes it difficult for stakeholders to change their requirements. This approach is employed by 29% of project teams. Strict change management processes are, in effect, change prevention processes: While they enable a team to deliver to the specification, they ensure that what you're delivering isn't what your stakeholders actually want. This is clearly not good for your stakeholders and isn't a very good career move on your part either.
The second dysfunctional practice is writing a detailed requirement specification at the beginning of the project, followed by 27% of respondents. This is something called "big requirements up front (BRUF)" within the agile community and it is problematic because it results in a significant level of waste on IT projects.
Interestingly, the number of respondents doing at least one of these practices was 44% on average — by paradigm Agile (36%), iterative (54%), traditional (53%), and ad hoc (29%). Not surprising that the ad hoc teams hadn't adopted more formalized strategies for requirements management.