Dr. Dobb's is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Channels ▼


Multi-Stage Continuous Integration

Damon Poole is Founder and CTO of AccuRev, a provider of process-centric software solutions to improve Agile, geographically distributed and parallel development. Damon's Agile Development Thoughts blog is at damonpoole.blogspot.com.

The main goal of software is to automate and simplify what would otherwise be accomplished using a manual process. This gives users of the software leverage to do more with less. Instead of balancing our checkbook by hand, we can use Quicken to do it faster and more accurately. Instead of maintaining the records of millions of people's financial transactions with paper and pencil, banks use mainframes.

Whatever industry you work in, what do you think about when you are creating new software? Don't you think about how it scales to meet your customers' needs as they grow with high availability? Even if you don't achieve that on the first try, aren't you still thinking about it and striving to achieve it? You know that to do it, you need to make the right technology and architectural choices. Over time, you may need to change some of those choices to keep pace with competitors. Even if your current needs are modest, software development architecture has evolved principles, patterns, and technologies to allow software to scale from a single user to thousands or even millions of users on multiple platforms at multiple locations with 99.999 percent uptime.

On the other hand, as an industry, we don't seem to be very good at scaling up the process of software development itself. Even when what needs to be built is primarily made up of well-understood tasks and requires little discovery, it seems that when the size of the effort goes beyond 5-10 people, things start to break down pretty rapidly. Adding more people usually doesn't scale in the same way that adding more hardware does.

This stems from the belief that software automates a well-defined problem but that the process of software development is a creative endeavor and cannot benefit from the same kind of thinking that goes into designing software. However, after spending many years involved in thinking about how to improve the process of software development, experience has convinced me that a great deal of the process of developing software can be treated as infrastructure. This infrastructure supports the creative parts of the software development process, but can be clearly delineated from it.

By thinking about the infrastructure of your process in the same way as you would think about any manual process that requires automation, you can leverage your skills as a developer and apply them in a new way. The process of developing software is, in effect, an algorithm implemented with various technologies. You can think of your team, the technologies you use, and your development methodology as software.

What technologies are you using? What is the architecture of your organization? Will it scale from its current size to double its size? Will it scale seamlessly to include new teams in new locations? What will happen if you acquire a company?

When creating software, you want to design it so that it is flexible and adapts to new circumstances. The same should be true for your development organization. In this article, I apply this way of thinking to the problem of scaling continuous integration to development efforts of more than 10 people.

Continuous Integration

In a typical project, the integration phase is towards the end and is the source of a great deal of problems and delays. One practice that has emerged to address this problem is continuous integration. The basic idea is that if getting build and test results on a regular basis is a good idea, getting build and test results on every change is even better.

With continuous integration, all work is integrated into a single codeline as frequently as possible. Every check-in automatically triggers a build and often a subsequent run of the test suite. This provides instant feedback about problems to developers and helps to keep the code base free of build and test failures. It also reduces the integration headaches just prior to release.

I'm a big fan of continuous integration. I've used it and recommended it for many years with great success. But it has a dark side as well. The larger and/or more complex the project, the higher the chance that it devolves into what I call "Continuous Noise." In this case, you get notified every 10 minutes or so (depending on how much building and testing is going on) that the build and/or test suite is still failing. It may be for a different reason every time, but it doesn't matter. It is tough to make progress when the mainline is unstable most of the time. This problem is not caused by CI, but it is exposed by CI.

The question is, what is a good way to structure this integration so that it will scale smoothly as you add more people to the equation? A good starting place is to look around for a pattern to follow. What are some similar situations? I have found that everything your organization needs to do in order to produce the best possible development organization can be entirely derived from the patterns and practices at the individual level. It makes it much easier to understand and much more likely that it will be successfully followed.

Related Reading

More Insights

Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.