Build and Release Pipelines
While the Build and Release Pipelines approach is similar to the Chained Processes approach in that both rely on non-overlapping processes, the difference comes down to the central concept. The Build and Release Pipelines approach uses the pipeline as the primary concept, while it is the process that is the central concept in the Chained Process approach. In the Build and Release Pipelines approach, the processes are executed within the context of a pipeline.
As with a simple continuous integration system, a commit to the SCM can trigger the build process after the quiet period is met. At this point, a new pipeline is created and the invoked build process runs within the scope of the pipeline. In this approach, all processes are executed within the scope of a pipeline. The artifacts produced by the build process are stored within the context of the pipeline. It is the pipeline that provides traceability to the artifacts (meaning that the artifacts are traceable to the pipeline and via the pipeline to the build process invocation that created them).
A successful completion of the build process invokes the Quick Tests process within the context of the same pipeline. The Quick Tests process has no problem locating the build artifacts to be tested since they are available from the pipeline context; it is the pipeline that is responsible for storage and retrieval of all artifacts and other data related to the pipeline. The results of the Quick Tests process can be communicated to the members of the development team to provide the fast feedback required by continuous integration.
As in the Chained Processes approach, the process scheduler should be responsible for ensuring that the hardware is not saturated by too many concurrent instances of the Deploy to QA process. One common strategy employed by schedulers to ensure this is to combine requests for the Deploy to QA process. If two separate pipelines made requests for the Deploy to QA process but only a single process could run, the scheduler would honor the most recent request and discard the older one. But any such scheduler needs to be robust enough to provide all desired functionality. For example, the scheduler should probably not discard process requests made by users manually.
As with the Chained Processes approach, the current approach ensures that downstream processes always have a pipeline that successfully completed all preceding processes. For example, if at the time of invocation of the Deploy to QA process, the latest pipeline contained Build B3 and Quick Tests Q2, the Deploy to QA process would run within the context of the latest pipeline to successfully pass both preceding processes, the pipeline containing B1 and Q1.
The Build and Release Pipeline approach brings us the efficiency and flexibility strengths of the Chained Processes approach and the traceability advantages of the Staged CI approach. There is more complexity in this approach; however, this complexity can be safely hidden in a system or tool so that the end result is easier and more intuitive to use.
Given that the more tests we run, the clearer our picture of quality is and the sooner we get the feedback the more valuable it is, there are plenty of reasons to extend automation beyond the development team. With approaches I've discussed here, we've seen QA and even release management teams derive the same benefits that development teams are getting. I hope that some of the ideas in this article will allow you to obtain similar results.