Many organizations are outsourcing software development projects to reduce their costs. Sounds like a good idea, but, despite its benefits, outsourcing carries significant risks. The good news? Though you may not be aware of them, you can explore significantly less risky options than traditional outsourcing. This month, I examine the challenges of traditional outsourcing, and investigate some more effective alternatives.
The traditional approach to outsourcing development projects typically follows a single strategy. You begin by negotiating a contract, which may be based on previously defined requirements or may include the effort of gathering, documenting and validating fresh requirements. The outsourcer then works to your requirements, typically off-site at its location, to develop the system. Regular communication via e-mail and teleconferencing helps resolve potential misunderstandings. Regular site visits by senior outsourcers occur, often in conjunction with reviews of critical documents and/or even to demo portions of the system, to keep you up to date and hopefully to sell you more services. The system is eventually delivered and taken over by your organization for operation and maintenanceor, if you prefer, the outsourcer will perform these lucrative activities for you.
Sounds idealnot only is your system developed to your specification, its often done at a fraction of the cost of in-house work. Your fully loaded costs may be between $75 and $100, yet because outsourcers do the work in developing nations, they can charge $25 an hour for the same or usually better-quality work. Whats not to like? A heck of a lot.
All Is Not as It Seems
In the flip side of its advantages, outsourcing creates obvious problems. First,
like all of us, the outsourcer can fail. In this event, although you may have
a contract in place, youve still lost valuable time. Dun and Bradstreet
estimate that between 20 and 25 percent of all outsourcing efforts fail within
the first two years. However, that range is almost respectable when you compare
it with the Chaos Reports (www.standishgroup.com)
widely publicized statistic that 65 percent of all large projects fail industry-wide.
Second, you become dependent on the outsourcer company. What happens when it
goes out of business? What happens if it decides to exit the outsourcing business?
What happens if it decides to raise its fees? Never forget that contracts can
always be renegotiated. Third, what if you want to insource the application
at some point? Youll have lost both the development and the domain experience,
making it incredibly expensive to move away from outsourcing.
Now for the not-so-obvious challenges. Perhaps the communication barrier is the most debilitating problem. When was the last time you built a mission-critical business application that didnt require significant input from the business and didnt induce major changes in their business process? Is it realistic that people half a world away could accomplish this? The greater the business focus of your application, the riskier it is to outsource it. As Alistair Cockburn points out in his groundbreaking book, Agile Software Development (Addison-Wesley, 2002), face-to-face communication is by far the most effective way to work togethera difficult task for people separated by thousands of miles.
Document Dependency
In an attempt to bridge the communication gap, the outsourcer often follows
documentation-heavy processes, requiring you to review and sign-off each segment
before the next stage of work proceeds. That sounds great, but most of us need
software development, not documentation development. Paying $25 an hour to get
buried under paperwork isnt such a great deal. Cockburn argues that documentation
is your worst form of communication; if this is so, document-dependent outsourcing
actually increases your project risk. Yes, a 500-page architecture document
may appear impressive, but will you actually read it thoroughly? Have you accounted
for the additional costs? Can you review the documents in a timely manner, or
will you put the project timeline at risk instead?
To make the communication challenges worse, the outsourcers may not know your domain, may not speak your native language, and certainly dont know the exact nature of your organizations culture. Furthermore, the outsourcing company doesnt have an understanding of your existing technical environmenta situation that adds integration risk to your project. You may need to invest significant resources to document your existing environment sufficientlyan effort that will be billed at much higher rate than $25 an hour.
Specified Disasters
Worse yet, success is often defined as building the system to the specification,
an activity that most outsourcers focus on in their marketing literature. What
makes you think you can specify the system properly, that your environment wont
change during the development effort, or that your priorities wont change?
Creating software that conforms to a detailed specification sounds like a great
idea, but Id rather make software that meets my actual needs. The near-serial,
big-design up-front (BDUF) approach favored by most outsourcers is really for
their benefit, not yours, because it helps them to restrict change. Out- or
insourced, this approach runs the significant risk that youll build the
wrong thing. All modern software development methodologies explicitly recommend
an iterative and incremental approach, not a serial one. Most outsourcers will
have a change management process that purports to support new and modified requirementsbut
once again, this process often does more to protect them (and not benefit you)
by increasing the number of steps you have to take to change specifications.
Motivation Matters
In the matter of motivation, outsourcers are more than miles away. Like any
company, they aim to make a profit from you and to increase that profit over
time, whereas your goal is to have a system that meets your needs built in a
timely and cost-efficient manner. Motivation is important. For example, if the
project gets into trouble, do you think that the outsourcer will tell you so
you can cut your losses early, or will it cover up the problem and continue
billing? The outsourcer is also much more likely to sell you on the benefits
of extensive documentationsomething they can bill you for. To the outsourcer,
youre an account, not a friend, nor even a partner.
One benefit of outsourcers is that they may in fact be better at developing software than you are. In this case, it may be more effective to discover what theyre doing differentlyperhaps by hiring a couple of their staff members to whip you into shape. Or maybe you could read the same $40 books that they have, and act on that advice.
Often overlooked in the outsourcing cost-benefit analysis is the fact that you still have to manage the projectyou cant simply fire and forget by handing the outsourcer the requirements and waiting for it to produce a system. Because of the complexities of coordinating various groups, its much more difficult to manage an outsourced project than an in-house one. If you cant successfully manage an in-house project, your chances of managing an outsourced project are comparable to a snowballs in hell.
The Agile Option
Instead of outsourcing your project, develop it in-house with an agile development
team, or outsource it to a local company with its own agile team. If you can
develop the same system with a team one fifth the sizea reasonable goal
with agile techniquesyou need only to keep your average cost below $125
an hour to be cost-competitive with outsourcers in the developing world.
There are risks to this approach. First, because youre trying something new, you run the likelihood of running aground. You can alleviate this risk by hiring one or two experienced team members, something that many organizations are successfully doing. Second, your existing organization may hamper your agile effortsall it takes is one support group in your organization to insist on a serial BDUF approach, and youve got a serious problem. However, this problem can be easily solved by educating everyone involved, constantly communicating your agile successes and enlisting the support of senior management. Third, youll need active stakeholder participation, including support for colocation with the developers. If your stakeholders want the system, and you effectively explain why you must have their support, its actually fairly easy to get sufficient participation.
An agile approach to development is a viable option. In its infancy, you had to take it on faith that agile methods such as Extreme Programming (XP) and Feature Driven Development (FDD) worked. Today, however, supportive evidence is trickling in. Michele Marchesi, Giancarlo Succi, Don Wells and Laurie Williams Extreme Programming Perspectives (Addison-Wesley, 2002) shares several case studies and initial results from academic research; Jim Highsmiths Agile Software Development Ecosystems (Addison-Wesley, 2002) summarizes and explores several successful agile development projects; and Laurie Williams and Robert Kesslers Pair Programming Illuminated (Addison-Wesley, 2002) presents the results of detailed research studies into pair programming. (For more agile evidence, see my essay, Answering the Where Is the Proof that Agile Methods Work? Question, at www.agilemodeling.com/essays/proof.htm.)
What Should You Do?
If you insist on outsourcing a software development project, do so with your
eyes open. Traditional outsourcing carries with it several systemic problems,
making it far riskier than it appears. Outsourcers often provide statistics
that show very good success rates, but their definition of success probably
isnt yours. Outsourcers focus on producing software to specification,
but specifications changeand everyone would rather have software that
meets their actual needs. Furthermore, as the saying goes, There are lies,
big lies and then there are statistics. Research by DiamondCluster International
(www.diamondcluster.com)
reveals that 70 percent of surveyed companies had prematurely terminated at
least one outsourcing agreement. I suspect that this had something to do with
outsourcings innate problems.
How can you improve your chances of success when outsourcing a project? First, strengthen communication by designating a group of senior developers to travel between sites. These developers will first work with your business stakeholders to understand their requirements; then, theyll work with the outsourcers developers to ensure that they understand those requirements. Second, make sure the outsourcer delivers working softwarenot reams of documentationon a regular, perhaps weekly, basis. Third, be prepared to actively manage the project to ensure that it keeps on track.
Many people believe that traditional outsourcing has a bright future, but I suspect that time will prove them wrong. International outsourcing companies competitive advantage centers on their ability to supply many good workers at cheap rates. Agile development enables you to develop the same systems with much smaller, in-house teams, arguably at much less risk. My prediction is that the outsourcing market will shrink dramatically as companies discover that $25 an hour is often far more expensive than $125 an hour.
Id like to thank Larry Brunelle, Alex Jouravlev, Jonathan Kern, Pavel Konopelko, Patrick Logan, Ashley McNeile, C. Keith Ray, Oleksiy Skrypnyk, Jason Smith and Gabriel Tanase for their input (www.agilemodeling.com/feedback.htm).
Scott Ambler is a senior consultant with Ronin International Inc. His latest book is The Elements of UML Style (Cambridge University Press, 2002).