Channels ▼
RSS

Web Development

Extending Continuous Integration Into ALM


Chaining Processes

The Staged CI approach results in multiple stages where each stage is a different build type. As such, each stage performs a build as well as one or more secondary processes. A more process-oriented (as opposed to build-centric) approach draws sharp boundaries between the processes, so that there are no overlaps. One process retrieves the source code from source control, compiles it, packages it; in other words, "builds" the software. A second process runs the quick tests. Additional processes may provide even more functionality. This separation in processes offers greater efficiency and allows the same source code to undergo progressively more exhaustive testing.

The idea is that all of these separate processes are executed in a chain. First, the build process is invoked. Once the build process completes successfully, the next process (Quick Tests in Figure 3) is invoked. It is at this point that we run into our first complication. The secondary processes, such as the Quick Tests process, are not build types and do not produce their own artifacts to be tested. This separation between the build and the test processes is the key to the Chained Processes approach and the main differentiator between it and the Staged CI approach. And since the secondary processes do not produce their own artifacts for testing, they need to get the artifacts externally. The artifacts are produced by the Build process. And typically, when the Build process runs, it places the artifacts it produces in a well-known location. This can be an agreed-upon directory on the filesystem, an SCM, or an artifact repository. The secondary processes need to obtain the artifacts from the well-known and agreed-upon location.

Figure 3: Chained Processes.

This type of artifact passing is trickier than it sounds. It is easiest to have the artifacts from the latest build overwrite the artifacts of any previous build. The alternative to having a separate location for the artifacts of each build requires that secondary processes be able to locate their intended artifacts. A simple naming convention where the artifacts of a build are stored in a directory with the build number as its name would require that each secondary process be passed to the build number so that it can find the artifacts. While this is not difficult to do, it is another detail to keep in mind.

Now that we know how the secondary process (such as Quick Tests) is going to locate the build artifacts, we can continue the walk-through. When the Quick Tests is invoked after the completion of the Build process, it retrieves the build artifacts and runs the quick test on them. These results can then be communicated to the development team in order to provide the fast feedback required by continuous integration.

After a successful execution of the Quick Tests process, we would like to run the Deploy to QA process. However, if you recall from our discussion of Staged CI, a limit in the available hardware resources may mean that you can only run a single combination of the Deploy to QA and Functional Tests processes at a time. The scheduler used to schedule the execution of these secondary processes should be robust enough to provide the desired behavior.

This approach does a good job of facilitating a common Build process that provides both the steady CI feedback and feeds longer running processes like tests. One of the benefits of this approach is that downstream processes always have a build that successfully completed all preceding processes. To illustrate this point, consider that at the time we invoke the Deploy to QA process D1 in Figure 4, the latest run of the Quick Tests process is Q2 corresponding to build B3. But since Q2 has failed, and we would like our execution of the Deploy to QA process to deploy the artifacts of the latest build without any detected problems, the D1 process is linked to Q1 and B1 instead of the latest available Q2.

Figure 4: Build and Release Pipeline.

The Chained Processes approach does present the challenge of traceability. As each process stands alone, the linkage between successive process executions is based on parameters being passed in. If we want to determine the build and code responsible for a failed Quick Tests execution, we need to get the build number that was passed into our process execution as a parameter. Process-oriented systems built around this approach typically do not provide any built-in traceability mechanism for navigating the process chains. Likewise, process-based systems typically do not provide any built-in way of passing artifacts from one process to another in a traceable manner.


Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.
 
Dr. Dobb's TV