Many organizations are outsourcing software development projects to reduce their costs. Sounds like a good idea, but, despite its benefits, outsourcing carries significant risks. The good news? Though you may not be aware of them, you can explore significantly less risky options than traditional outsourcing. This month, I examine the challenges of traditional outsourcing, and investigate some more effective alternatives.
The traditional approach to outsourcing development projects typically follows a single strategy. You begin by negotiating a contract, which may be based on previously defined requirements or may include the effort of gathering, documenting and validating fresh requirements. The outsourcer then works to your requirements, typically off-site at its location, to develop the system. Regular communication via e-mail and teleconferencing helps resolve potential misunderstandings. Regular site visits by senior outsourcers occur, often in conjunction with reviews of critical documents and/or even to demo portions of the system, to keep you up to date and hopefully to sell you more services. The system is eventually delivered and taken over by your organization for operation and maintenance—or, if you prefer, the outsourcer will perform these lucrative activities for you.
Sounds ideal—not only is your system developed to your specification, it’s often done at a fraction of the cost of in-house work. Your fully loaded costs may be between $75 and $100, yet because outsourcers do the work in developing nations, they can charge $25 an hour for the same or usually better-quality work. What’s not to like? A heck of a lot.
All Is Not as It Seems
In the flip side of its advantages, outsourcing creates obvious problems. First,
like all of us, the outsourcer can fail. In this event, although you may have
a contract in place, you’ve still lost valuable time. Dun and Bradstreet
estimate that between 20 and 25 percent of all outsourcing efforts fail within
the first two years. However, that range is almost respectable when you compare
it with the Chaos Report’s (www.standishgroup.com)
widely publicized statistic that 65 percent of all large projects fail industry-wide.
Second, you become dependent on the outsourcer company. What happens when it
goes out of business? What happens if it decides to exit the outsourcing business?
What happens if it decides to raise its fees? Never forget that contracts can
always be renegotiated. Third, what if you want to insource the application
at some point? You’ll have lost both the development and the domain experience,
making it incredibly expensive to move away from outsourcing.
Now for the not-so-obvious challenges. Perhaps the communication barrier is the most debilitating problem. When was the last time you built a mission-critical business application that didn’t require significant input from the business and didn’t induce major changes in their business process? Is it realistic that people half a world away could accomplish this? The greater the business focus of your application, the riskier it is to outsource it. As Alistair Cockburn points out in his groundbreaking book, Agile Software Development (Addison-Wesley, 2002), face-to-face communication is by far the most effective way to work together—a difficult task for people separated by thousands of miles.
Document Dependency
In an attempt to bridge the communication gap, the outsourcer often follows
documentation-heavy processes, requiring you to review and sign-off each segment
before the next stage of work proceeds. That sounds great, but most of us need
software development, not documentation development. Paying $25 an hour to get
buried under paperwork isn’t such a great deal. Cockburn argues that documentation
is your worst form of communication; if this is so, document-dependent outsourcing
actually increases your project risk. Yes, a 500-page architecture document
may appear impressive, but will you actually read it thoroughly? Have you accounted
for the additional costs? Can you review the documents in a timely manner, or
will you put the project timeline at risk instead?
To make the communication challenges worse, the outsourcers may not know your domain, may not speak your native language, and certainly don’t know the exact nature of your organization’s culture. Furthermore, the outsourcing company doesn’t have an understanding of your existing technical environment—a situation that adds integration risk to your project. You may need to invest significant resources to document your existing environment sufficiently—an effort that will be billed at much higher rate than $25 an hour.
Specified Disasters
Worse yet, success is often defined as building the system to the specification,
an activity that most outsourcers focus on in their marketing literature. What
makes you think you can specify the system properly, that your environment won’t
change during the development effort, or that your priorities won’t change?
Creating software that conforms to a detailed specification sounds like a great
idea, but I’d rather make software that meets my actual needs. The near-serial,
big-design up-front (BDUF) approach favored by most outsourcers is really for
their benefit, not yours, because it helps them to restrict change. Out- or
insourced, this approach runs the significant risk that you’ll build the
wrong thing. All modern software development methodologies explicitly recommend
an iterative and incremental approach, not a serial one. Most outsourcers will
have a change management process that purports to support new and modified requirements—but
once again, this process often does more to protect them (and not benefit you)
by increasing the number of steps you have to take to change specifications.
Motivation Matters
In the matter of motivation, outsourcers are more than miles away. Like any
company, they aim to make a profit from you and to increase that profit over
time, whereas your goal is to have a system that meets your needs built in a
timely and cost-efficient manner. Motivation is important. For example, if the
project gets into trouble, do you think that the outsourcer will tell you so
you can cut your losses early, or will it cover up the problem and continue
billing? The outsourcer is also much more likely to sell you on the benefits
of extensive documentation—something they can bill you for. To the outsourcer,
you’re an account, not a friend, nor even a partner.
One benefit of outsourcers is that they may in fact be better at developing software than you are. In this case, it may be more effective to discover what they’re doing differently—perhaps by hiring a couple of their staff members to whip you into shape. Or maybe you could read the same $40 books that they have, and act on that advice.
Often overlooked in the outsourcing cost-benefit analysis is the fact that you still have to manage the project—you can’t simply “fire and forget” by handing the outsourcer the requirements and waiting for it to produce a system. Because of the complexities of coordinating various groups, it’s much more difficult to manage an outsourced project than an in-house one. If you can’t successfully manage an in-house project, your chances of managing an outsourced project are comparable to a snowball’s in hell.
The Agile Option
Instead of outsourcing your project, develop it in-house with an agile development
team, or outsource it to a local company with its own agile team. If you can
develop the same system with a team one fifth the size—a reasonable goal
with agile techniques—you need only to keep your average cost below $125
an hour to be cost-competitive with outsourcers in the developing world.
There are risks to this approach. First, because you’re trying something new, you run the likelihood of running aground. You can alleviate this risk by hiring one or two experienced team members, something that many organizations are successfully doing. Second, your existing organization may hamper your agile efforts—all it takes is one support group in your organization to insist on a serial BDUF approach, and you’ve got a serious problem. However, this problem can be easily solved by educating everyone involved, constantly communicating your agile successes and enlisting the support of senior management. Third, you’ll need active stakeholder participation, including support for colocation with the developers. If your stakeholders want the system, and you effectively explain why you must have their support, it’s actually fairly easy to get sufficient participation.
An agile approach to development is a viable option. In its infancy, you had to take it on faith that agile methods such as Extreme Programming (XP) and Feature Driven Development (FDD) worked. Today, however, supportive evidence is trickling in. Michele Marchesi, Giancarlo Succi, Don Wells and Laurie Williams’ Extreme Programming Perspectives (Addison-Wesley, 2002) shares several case studies and initial results from academic research; Jim Highsmith’s Agile Software Development Ecosystems (Addison-Wesley, 2002) summarizes and explores several successful agile development projects; and Laurie Williams and Robert Kessler’s Pair Programming Illuminated (Addison-Wesley, 2002) presents the results of detailed research studies into pair programming. (For more agile evidence, see my essay, “Answering the ‘Where Is the Proof that Agile Methods Work?’ Question,” at www.agilemodeling.com/essays/proof.htm.)
What Should You Do?
If you insist on outsourcing a software development project, do so with your
eyes open. Traditional outsourcing carries with it several systemic problems,
making it far riskier than it appears. Outsourcers often provide statistics
that show very good success rates, but their definition of success probably
isn’t yours. Outsourcers focus on producing software to specification,
but specifications change—and everyone would rather have software that
meets their actual needs. Furthermore, as the saying goes, “There are lies,
big lies and then there are statistics.” Research by DiamondCluster International
(www.diamondcluster.com)
reveals that 70 percent of surveyed companies had prematurely terminated at
least one outsourcing agreement. I suspect that this had something to do with
outsourcing’s innate problems.
How can you improve your chances of success when outsourcing a project? First, strengthen communication by designating a group of senior developers to travel between sites. These developers will first work with your business stakeholders to understand their requirements; then, they’ll work with the outsourcer’s developers to ensure that they understand those requirements. Second, make sure the outsourcer delivers working software—not reams of documentation—on a regular, perhaps weekly, basis. Third, be prepared to actively manage the project to ensure that it keeps on track.
Many people believe that traditional outsourcing has a bright future, but I suspect that time will prove them wrong. International outsourcing companies’ competitive advantage centers on their ability to supply many good workers at cheap rates. Agile development enables you to develop the same systems with much smaller, in-house teams, arguably at much less risk. My prediction is that the outsourcing market will shrink dramatically as companies discover that $25 an hour is often far more expensive than $125 an hour.
I’d like to thank Larry Brunelle, Alex Jouravlev, Jonathan Kern, Pavel Konopelko, Patrick Logan, Ashley McNeile, C. Keith Ray, Oleksiy Skrypnyk, Jason Smith and Gabriel Tanase for their input (www.agilemodeling.com/feedback.htm).
Scott Ambler is a senior consultant with Ronin International Inc. His latest book is The Elements of UML Style (Cambridge University Press, 2002).
