Dr. Dobb's | Green Threads | December 12, 2006

IBM's "green threads" project is an effort to drive better interaction and integration across products and brands.

When my wife and I bought our first house, the home inspector reported a "slight suspicion" that the house might have been used in the past as a marijuana growing house. It hadn't. But the result of that comment was that I spent several weeks managing a complex flow of documents between lawyers, banks, insurance companies, police, and realtors—all of whom were ostensibly working for me, but none of whom were willing or able to talk to each other. Even within the same office, different people didn't seem to know what others were doing. If someone had come to my door offering an integrated end-to-end solution to home buying, he could have named his price.

Many businesses face this situation every day. They clearly know what their goals are, but much of their time is spent managing the friction between vendors and tools that are supposed to help them solve problems. Software development is no exception. As an industry, we are skilled at innovating new products to address new problems; however, customers almost never use a product in isolation, despite what some product teams may believe. The reality is that across their businesses, customers use a variety of products in a variety of roles. Work is handed off from one business unit to another, from one specialty to another, and in the absence of anything better, their integration tool of choice is copy-and-paste. Part of the solution to this problem lies in support for open standards, though that push really has more to do with how competing vendors of the same technology agree to interoperate. However, while necessary, standards are not sufficient. Even when there's an agreed-to set of standards—indeed, even when products originate from the same vendor—customer attempts to get a set of products from across their business to work together often fall short.

In this article, I describe IBM's "green threads" project, an ongoing internal effort to drive better interaction and integration across products and brands so that customers can focus on their goals, not their software. Our aim is to foster discussion in the development-process community and assist others in evaluating and implementing what we feel is an essential quality process for any software organization.

Green Threads: A Brief History

A few years ago, IBM's Software Group came across the idea of "red threads" in a company that it had acquired. Basically, a red thread is a trail of blood—a customer path through a product that is commonly traveled, and painfully broken, requiring many workarounds and support interventions. These red threads would be documented as a result of customer engagements, and fed into the development cycle to be considered for future improvements. On its own, this concept is not revolutionary. Most companies have some way of rolling customer pain-point information into development cycles. But it formed the basis of a much more ambitious project: We imagined new, refocused threads that would continue to document the real customer workflow across products, but which would elaborate on how things should work instead of documenting the problems with the current product set. Instead of being reactive and perpetually lagging behind the product cycle, they would become key drivers of new development and track the evolving demands of customers. In light of their rebirth and positive new message, they were given a makeover and a snappy new name—"green threads."

Green Threads: Anatomy of a Project

Green threads are tools for improving end-to-end product quality by using customer scenarios to clarify product priorities and drive new legitimate forums for cross-product discussion. At their core are real-world, end-to-end paths through multiple products and multiple brands, phrased in terms of the specific goal customers are trying to accomplish or situations they are trying to handle. It is essential that the green threads be driven by actual customer reality because the interproduct space is one with which few people in a development organization have experience. Likewise, it must follow the customer scenario from end to end because real-world workflows don't tend to match product boundaries. The temptation to duck the unpleasant complications at handover points in a scenario can be powerful. For example, how does an IBM WebSphere Business Modeler project get handed to the Rational Software Architect user? How do web services built in WebSphere Integration Developer get traced back to business objectives in Rational Portfolio Manager? These issues are often complex, poorly understood, and require a divergent set of skills. Unfortunately, they are also where customers report significant frustrations.

Green threads are not intended to be exhaustive. By design, the green threads team keeps the number of threads small; ideally, three or so should be the maximum. This restriction is important because it:

Trying to emphasize everything is equivalent to emphasizing nothing, so the importance of the initial identification and prioritization really can't be overstated. As a result, this first phase of the process—which amounts to little more than listing the chosen threads and their brief descriptions on a single slide—is the hardest part of the green thread process and one that requires strong leadership. This leadership is best provided jointly by a senior engineering lead and a senior customer-facing product or portfolio manager. The collaboration of business and technology experts here is essential to ensure the right threads are chosen, to provide credibility and authority for the work that will be undertaken as a result, and to build up a virtual team around each thread that can develop its detail and drive the resulting improvements.

Once the list of threads has been identified, the next step (see Figure 1) is elaboration: the detailing and documenting of the green thread at a fine level of granularity. When complete, this step can produce documents several hundred pages in length that detail from beginning to end the steps taken, the tools used, and the information necessary at each step. To further this aim, the documents tend to make extensive use of screenshots and other supplemental information about the product's behavior. While the work product of this stage is vastly larger than in the initial identification stage, the work is easier for three reasons:

With the subject-matter experts identified, individual teams can compile the backing documentation for their portions of a green thread in parallel and within a relatively short amount of time; see Figure 2.


Identify & Prioritize	Jan 2—Jan 25	Three green threads are identified; alignment of threads across products reviewed; priorities are agreed.
Detail & Document	Jan 25—Feb 9	For each thread, key workflow steps are identified and elaborated. These structure the face to face walkthroughs and help identify key players.
Scenario Walkthrough	Products A, B, C: Feb 20—21 Products C, D, E:Feb 26—27 Products E, F: Mar 1 Product G: Mar 8	Face-to-face walkthroughs for each product are held with the key technical leads (plus support teams like usability, technical writing, and test). Focus is on optimizing proposed workflows and identifying gaps.
Finalize Documents	Apr 27	Issues and discussion are captured; workflow details updated if necessary.
Code Rewrites & Milestone Assessments	May—Dec	Responsibility for green threads handed over to the product teams; they drive through the rest of the development cycle.

Figure 2: Sample green threads schedule for a 12-month product cycle.

The detailed document provides the structure and background for the walkthrough exercise and for the eventual recommendations, but it shouldn't be used as a script. The point of the walkthrough phase is to get the technical leads describing their portions of the green thread in turn, to drive discussion and uncover new problems; it can start with nothing more than a whiteboard and a blank piece of paper. From a scheduling perspective, the walkthroughs can become a painful endeavor because it is important that all the key technical leads are physically present, and they typically take two days to complete. The motivation for this preference, though, is that in addition to serving as a validation of the compiled document, the walkthrough creates a legitimate and concrete forum for discussion and cross talk between products. Having people physically present and able to dive quickly into certain issues and collaborate on real, implementable solutions adds immensely to the quality of the eventual recommendations, as well as significantly increasing buy-in among the technical leadership to follow issues through to resolution. This environment is often the first opportunity that technical leads have to see their products in the context of complex cross-product interactions, and with sufficient detail to allow discussion of possible solutions. It quickly becomes a fertile environment for positive change and is the first place that the tangible benefits of the process become clear.

The result of the first three stages is a mountain of documentation and feedback, and a better understanding on the part of technical leads about the extent of (and problems with) their product's interactions. Translating this information and goodwill into action is the meat and potatoes of the green threads effort. The first step, after the walkthroughs are complete, is to capture and distill the set of issues discovered in order to complement the green threads document. This should be made available to all participants in some kind of collaborative environment (Lotus Notes database, wiki, and the like), so that further discussion can occur around the recommendations for resolving each issue. Once compiled, there are three chief consumers of this information:

Making Green Threads Work for You

Investment in a green threads program is not a one-time thing—it's a culture change. To make real progress on product integration, the threads should be updated regularly to reflect new product releases and emerging technologies. The green threads team should be constantly assessing new product milestones to look for issues that have been resolved and new ones that might have been created. Not only does this iteration drive future progress, but it also provides necessary validation of green threads work. A green threads effort is also not without its share of organizational difficulties. By definition, cross-product interactions are rarely owned by any single development team and building the necessary support to get changes into your company's code will inevitably run into the question, "What's in it for me?" Indeed, it takes a special kind of team to agree to work for the greater good of the customer and the company as a whole when doing so means time and money spent with no obvious, immediate, or direct return for the local team. Really, the message here is, "Think globally, act locally." Like the environmental movement that coined the phrase, software integration raises the kinds of problems that require individuals and groups to make local changes and commitments that collectively result in significant global improvements.

The returns, direct or otherwise, do exist. Green threads create a rare forum for technical leaders to approach design and development from a customer perspective instead of an architectural one and to understand their products' contributions to the overall picture. The tactical recommendations they make measurably improve product quality and increase customer satisfaction. The contribution they can offer to strategic vision is unique, customer-centric, and critically, brand- and product-neutral. Even the documents themselves are valuable assets; we have found within IBM that green threads documents find their way into development and usability test plans, product documentation, and sales materials because their cross-product focus is one rarely found through other channels. Finally, they create, implicitly, a team within your organization that is well versed with a broad product set, deeply connected to development leadership, and able to comment credibly on customer issues and expectations.

It's not difficult to get people to agree that cross-product interactions are broken and to identify the root causes. It is far more challenging to find ways to introduce concrete change into the development process to improve the situation. The IBM green threads process is still relatively young and we continue to revise and update it to make it more efficient, more credible, and more effective. In some sense though, it is already a success. Development leaders are including green threads in their product planning and brand executives are including them in their strategic messages. With each iteration, if we continue to do our jobs right, the real beneficiaries will be our customers.