Governing Agile Software Development

Many traditionalists will claim that Agile projects are difficult to govern, but Scott says nothing could be further from the truth.


October 10, 2007
URL:http://www.drdobbs.com/architecture-and-design/governing-agile-software-development/202401107

Scott is a DDJ Senior Contributing Editor, Practice Leader Agile Development with IBM Rational, and author of several best-selling books. He can be contacted at www.ambysoft.com/scottAmbler.html.


Now that Agile has crossed the Moore's technology adoption chasm, I'm seeing more and more organizations asking serious questions about Agile software development. According to the Dr. Dobb's 2007 Agile Adoption Survey (www.ddj.com/architect/200001986), 69 percent of organizations in North America have run Agile pilot projects and 85 percent of them have run several. Now that organizations have gotten their feet wet, they're starting to think about how to scale Agile approaches to meet their real world needs. A critical question that many people are asking is, "How do you govern Agile projects successfully?" The answer, which often surprises people, is that it's a heck of a lot easier to govern Agile projects than traditional ones.

Many traditionalists will claim that Agile projects are difficult to govern, but nothing could be further from the truth. To be fair, most of these people are likely confusing code-and-fix projects with Agile projects, and without a doubt, code-and-fix projects are virtually impossible to govern. There are two aspects of Agile software development that promote superior levels of governance when compared to traditional software development. First, the Agile approach of producing working software on a regular basis provides stakeholders with greater visibility into what a project team is actually doing. Second, the greater level of involvement provided to stakeholders—they control the budget, scope, and schedule on Agile projects—enables them to direct the project teams effectively.

Informal Visibility Into Projects

Figure 1, modified from OpenUP (www.eclipse.org/epf), overviews the strategy for scheduling work on an Agile project team. This strategy is an extension of Scrum's requirements management strategy to take into account all types of work items that software development teams perform. Not only do we implement requirements, represented as ovals in Figure 1, we also set up workstations, experiment with new technologies, take vacations, attend training courses, and other team health activities. Obviously, there are trade-offs involved with prioritizing nonrequirements-related work into the work stack.

During each construction iteration, Agile teams develop working software, fulfilling the highest-priority work items from the stack and thereby providing the greatest return on investment (ROI) possible. By delivering working software each iteration, stakeholders get clear feedback as to what the team is achieving. This "informal visibility" into development projects is far more valuable than the traditional documentation-based "earned-value management" (www.earnedvaluemanagement.com) because it's concrete. Stakeholders don't have to rely on the promises made in traditional status reports, or in comprehensive requirements documentation, but instead work with actual software and determine if the team is actually delivering value.

[Click image to view at full size]

Figure 1: Scheduling work on an agile project.

Stakeholders can provide immediate and valuable feedback to the team, steering them in a better direction if the team gets off track. As I discussed last month, we find that Agile teams must be far more disciplined than their traditional counterparts to actually make this work.

Greater Stakeholder Control

With the approach in Figure 1, stakeholders can change their requirements at any time, they can add new ones, and they can reprioritize existing ones, putting them in complete control of the scope. This works when the development team writes high-quality code that is loosely coupled and highly cohesive, when they refactor when required to keep their design of high quality, and when they have a regression test suite, which they run regularly. It's very easy to add new functionality to an existing code base in this sort of situation, particularly when the team invested a bit of time at the beginning of the project envisioning the architecture with a bit of initial Agile modeling.

Because Agile teams ask stakeholders each iteration how much they want to invest, our stakeholders are effectively in control of the budget—and rightfully so seeing as it's their money that is being spent. When stakeholders have insight into the actual status of a project team, and when they can change the funds that each team receives each iteration, they're in a position to actually govern IT projects. Figure 2 shows the current status of a collection of software development projects. The height of each stack indicates the relative amount of work the individual teams have left, the color of the stack indicates the current status, and the number of the top of the stack indicates the expected ROI of the system under development. With this sort of information, stakeholders can make intelligent governance decisions. For example, the projects with red status clearly need to be redirected, and I'd even consider canceling the low ROI one at this point. The yellow status projects also could use some help. From a portfolio-management point of view, I'd consider increasing the funding to the high-ROI green teams because I want to put my money into successful teams that provide the greatest value. In other words, with an Agile approach, stakeholders can start treating their IT investment as an investment. Following a traditional approach, it often seems that money spent on IT is more of a gamble than an investment, and I think that it's because of the lack of visibility and control that traditional teams provide to stakeholders.

[Click image to view at full size]

Figure 2: Continuous monitoring of teams.

Stakeholders are put in control of the schedule as a side effect of producing working software each iteration: We can be six months into a project and the stakeholders can tell us that we've delivered sufficient functionality to justify delivery into production. In combination with working in priority order, they can drive Agile development teams to deliver a system to the marketplace at the earliest opportunity.

Formal Visibility

Sometimes the informal level of visibility provided by working software each iteration isn't enough for some organizations, usually because they find it difficult to measure what they got. Although you could count the number of function points delivered, it's doubtful that the benefit of knowing the count is greater than the cost of doing the count in the first place. In my opinion, just the fact that the business stakeholders are willing to continue funding the project should be a sufficient metric.

Traditionalists will often claim that Agilists are flying blind, but nothing could be further from the truth. At the beginning of each iteration, the team plans their work for that iteration in detail, producing an iteration task list (our form of a detailed project plan). According to the Agile Adoption survey, iteration task lists are the fifth most valuable work product of Agile development teams (the first four were working software, source code, developer tests, and whiteboard sketches). Agile teams, particularly those working at scale, will also create high-level schedules that communicate major milestones such as proving the architecture, dependencies on other teams who put together training materials or install production hardware, and on release windows available to the team. This is particularly important for release/transition phase activities (www.ddj.com/development-tools/198800565).

Some Agile processes, particularly those based on the Unified Process such as OpenUp or Rational Unified Process (RUP), include explicit control points in the form of milestones. Figure 3 depicts the lifecycle of the OpenUP, which includes four specific project phases. Many extremists within the Agile community will denounce the concept of phases, often in fear of a waterfall approach to development, but the reality is that when they're done right, phases enable good governance and scaling of Agile techniques. For example, during the Inception phase, you do some initial modeling to envision the requirements and architecture strategy and drive stakeholders to a general agreement as to the scope and vision of the effort. This reduces major business risks and puts your project on a sound footing. During the Elaboration phase, you build a working, end-to-end skeleton of your system, which addresses architecturally difficult requirements, thereby reducing technical risk. And architecturally difficult requirements often prove to be high-priority business requirements, so there is very little reordering of your initial requirements. The Construction phase proves to be the vast majority of the effort where you build the system and the focus of the Transition phase is where you release the system into production.

[Click image to view at full size]

Figure 3: Control points throughout the lifecycle.

Explicit demarcation of the software development lifecycle phases provides business stakeholders with the opportunity for formal checkpoints into the team's progress. For everyone still afraid of the waterfall boogeyman: Just because these checkpoints are "formal," doesn't mean that they have to be dysfunctional. It could be as simple as inviting a few extra senior stakeholders to your end-of-iteration demo and wearing a pressed shirt that day. More importantly, explicit phases enable stakeholders to take a phased-based approach to overall funding. Many organizations can't tolerate the iteration-based "allowance" approach to funding the project described earlier, yet setting the budget at the beginning of the project is clearly an unnecessary business risk that should be avoided. An effective compromise is to fund each phase at a time, and the secret to making this work is to streamline the funding request effort as much as possible.

Agile Governance Works

It shouldn't be a surprise that Agile projects are easier to govern than traditional projects, although it clearly isn't what some traditionalists want to admit. Agilists work closely with stakeholders on a daily basis and the stakeholders have active roles on the team. Agile project teams provide stakeholders with concrete and useful information as to what they're doing, and provide stakeholders with the ability to easily direct teams appropriately. If your organization wants to improve its ability to govern IT project effectively, perhaps you should really focus on adopting Agile approaches to development on all of your software development projects. Once you've done so, I suspect that many of your governance problems will have gone away all on their own.

Terms of Service | Privacy Statement | Copyright © 2024 UBM Tech, All rights reserved.