In This Issue
- Product Backlogs at Scale
- Hot Links
Product Backlogs at Scale
A common practice of agile development teams is to manage their requirements as a prioritized stack, something called a product backlog in Scrum. The requirements on the product backlog can be captured using any technique, although in the last few years user stories have become the requirement type of choice due to their flexibility and simplicity. These user stories are often captured on index cards, although some teams will use electronic tools such as spreadsheets, wikis, word processors, or (egads!) requirements management tools. Managing requirements as a prioritized stack is a great strategy, but common questions that come up time and again is how do you scale product backlogs when the agile team isn't small, or isn't co-located, or isn't developing a reasonably straightforward system?
Let's begin by exploring the range of complexities which agile teams may find themselves in. Recently at IBM I've been working on what we call the Agile Process Maturity Model (APMM), the goal of which is to provide a framework which provides context for the plethora of agile methodologies and practices out there today. The objective is not to provide an evaluation framework, something that the staged version of the Capability Maturity Model Integrated (CMMI) is well suited for: APMM is meant to be complimentary to CMMI, not a competitor. The APMM is important because we've found that many of our customers find the mainstream agile message confusing, in part because of the multitude of voices within the agile community but more so because the agile rhetoric often seems to ignore or gloss over many important issues commonly faced by development teams. The APMM defines three levels, each of which builds upon the previous, to help people map the challenges that they face to the processes and practices which are meant to address them. These levels are:
- APMM Level 1: Core Agile Development. Level 1 agile processes, including Scrum and my own Agile Modeling, address a portion of the overall system life cycle. These processes, in turn, will define a collection of practices which can be tailored and combined to form a more complete solution. Level 1 processes often focus on practices and techniques which are well suited for small, co-located teams.
- APMM Level 2: Disciplined Agile Delivery. Level 2 agile processes, including OpenUP and Dynamic System Development Method (DSDM), address the full system delivery life cycle. Level 2 agile processes temper the self organization of level 1 processes with appropriate governance and the value-driven aspect of the agile lifecycle with risk mitigation. Note that it is common to combine practices from several level 1 agile processes to form a level 2 process.
- APMM Level 3: Agility at Scale. Level 3 agile processes are level 2 processes where one or more scaling factors -- team size, geographic distribution, regulatory compliance, organization distribution, application complexity, or enterprise discipline -- are applicable. At level 3 new practices may be adopted or existing practices may be tailored to address the scaling complexities faced by the team.
The goal of this column is to show how APMM can be applied to provide context to your adoption of agile practices, in this case product backlogs. The mainstream level 1 advice for product backlogs is that it is a prioritized stack of requirements which is owned by your product owner. The product owner represents the system stakeholders and is responsible for prioritizing the requirements, making decisions in a timely manner, and providing information in a timely manner. The requirements are often captured using inclusive tools such as index cards or sticky notes, although some teams will use electronic tools such as spreadsheets, home-grown databases, or even a requirements management tool. These requirements are often user stories, but could be screen sketches, features, use cases, or usage scenarios to name a few options. Furthermore, the requirements are often captured simply. For example a user story is a single statement in the form "As a [ROLE] I want to [DO SOMETHING] to [ACCOMPLISH SOMETHING]". Clearly this isn't a lot of information, but we don't need a lot of information because agilists take an evolutionary approach where the details are gathered just in time (JIT) -- the initial high-level requirement is captured up front via requirements envisioning and the details later in the project via iteration modeling or model storming.
For a small co-located team, this works incredibly well. Index cards are easy to work with, they can be shared easily, updated easily, and sorted easily -- you can literally have a stack of requirements. If you're following the practice of active stakeholder participation, and you should, your product owner will be readily available to answer questions and to explore the details with the team on a JIT basis. This approach sounds so idyllic you can almost hear the birds singing in the background.
APMM level 2 brings some realism to the picture. First, implementing requirements isn't the only thing that developers do on project teams. They also fix defects, review the work of other teams, go on vacation, go on training, and so on. All of these things are important and need to be planned for. There are several level 1 strategies for doing so, such as reserving overhead time each iteration or having a one or more other backlogs for these things. But, when you see teams reserving 20-30% overhead each iteration, or spending a lot of effort merging their stacks, you know it's not working out well. A potentially simpler strategy is to have a single stack of work items where a work item could be a requirement, a defect, and so on.
Although it's not immediately intuitive, your product owner should still be responsible for prioritizing work items, even the non-requirement ones. The product owner represents all stakeholders of the system that you're working on, not just end users. The stakeholders include senior management, operations and support staff, other development teams of systems that integrate with yours, enterprise architects, regulatory auditors, financial people, and of course end users (to name a few). The stakeholders of any system are diverse group of people with a wide range of needs that should be prioritized appropriately. Therefore, it makes sense that end user requirements, enterprise-level business requirements, architectural constraints, defects reported via your support staff, work performed to support other development teams, and so on should be managed by the product owner.
When you start to take the full needs of a software development team into consideration, particularly when you have to start taking some of the real-world complexities that they face, a prioritized backlog captured on index cards starts to fall down. At IBM we've gone beyond the level 1 rhetoric of the mainstream agile community and actually invested time in developing tooling and practices for successfully supporting work item lists. Products like Rational Team Concert (RTC) implement work item lists, more importantly RTC supports distributed as well as co-located development teams, and you can check it out free of charge at www.jazz.net. Work items lists have been an important aspect of the Open Unified Process (OpenUP) for several years now, and you can download OpenUP free of charge.
APMM level 3 explicitly addresses the complexities inherent with scaling agile software development practices, including product backlogs. For any given practice it is unlikely that you'll have a single tailoring of a practice because there are several scaling factors -- depending on which scaling factors are applicable for your situation you'll tailor the practice accordingly. For example, if your team is large or distributed you'll more than likely organize it into a collection of subteams. Each of these subteams should have their own product owners and their own work item list, and the product owners will have to negotiate dependent priorities between them (regardless of the wishful thinking of the mainstream agile community, many requirements/work items do in fact have dependencies between them, something that becomes very apparent at scale). The implication is that your process and tooling must reflect the realities of managing and maintaining multiple work items stacks.
Teams which must conform to regulations such as the Food and Drug Administration (FDA)'s CFR Part 11 will often need to track the work that they need to do. The implication is that in these situations not only do you need to manage your work item list you also need to track doing so. The solution is to either use tools sophisticated enough to do so automatically (RTC does this) or must manually do so in addition to their "real work".
Teams which are organizationally distributed, for example multiple divisions of the same company are working together, perhaps a system integrator (SI) is involved, or perhaps individual contractors/consultants are involved, can increase the complexity of managing your work item list. For example, some people might have restricted access to the work item list. For example, people who work for organization A can only look at work items assigned to them but not the work items assigned to organization B.
In general, there are several important implications to the APMM for agile teams. First, don't apply level 1 strategies if find yourself at level 3. I love using inclusive tools such as index cards and whiteboards for modeling, I seem to remember writing a couple of books on this very subject, but they don't work so well when some of your team mates are on the other side of the horizon. The point is that you need to apply the right techniques and tools for the situation. Second, conversely, don't take a level 3 approach when a level 1 or 2 approach will do. In the past too many organizations ran into trouble because they wanted a "one size fits all" process which addressed all possible situations and thereby bogged most teams down with onerous processes. Third, use the APMM to provide some context to the agile messaging that you're subjected to on a regular basis, and hopefully help to bring some realism to the conversation. When you step back and think about it, APMM level 2 really is the bare minimum that you should be thinking about if you're on an agile team, regardless of what you're being taught in the two-day certification courses which continue to plague our industry.
Scaling On-Site Customer describes strategies for product owners and on-site customers which find themselves involved with large teams.
Scaling Scrum overviews several strategies for apply Scrum in complex situations which you may not have been taught on your two-day Scrum Master certification course.
Complex Requirements on an Agile Project overviews many of the requirements management issues faced by agile teams at scale.
Agile CMMI: Complimentary or Oxymoronic? describes how agile and CMMI can be used together effectively in practice.
Active Stakeholder Participation describes why you need regular stakeholder participation on your software development projects and how they can be actively involved.
Inclusive Modeling describes inclusive strategies and tools which support active stakeholder participation in modeling activities on system develop projects.
The Open Unified Process (OpenUP) is an APMM level 2 agile process.
The site for Rational Team Concert (RTC), and the Jazz technology platform and Jazz community in general, is found at www.jazz.net.
For more information about agile approaches to requirements, read www.agilemodeling.com/essays/agileRequirements.htm.
The Agile Process Maturity Model (APMM) is described here.
My Agility@Scale blog is posted here where I discuss strategies for adopting and applying agile strategies in complex environments.