Kurt Keutzer is currently a principal investigator in Berkeley's Universal Parallel Computing Research Center. Tim Mattson is a Principal Engineer in Intel's Visual Applications Research Laboratory.
The trend has been well established -- parallel processors will dominate most, if not every, niche of computing. Ideally, this transition would be driven by the needs of software. Scalable software would demand scalable hardware and that would drive CPUs to add cores. But software demands are not driving parallelism. The motivation for parallelism comes from the inability of integrated circuit designers to deliver steadily increasing frequency gains without pushing power dissipation to unsustainable levels. Thus, we have a dangerous mismatch: the semiconductor industry is banking its future on parallel microprocessors, while the software industry is still searching for an effective solution to the parallel programming problem.
The parallel programming problem is not new. It has been an active area of research for the last three decades, and we can learn a great deal from what has not worked in the past.
- Automatic parallelism. Compilers can speculate, prefetch data, and reorder instructions to balance the load among the components of a system. However, they cannot look at a serial algorithm and create a different algorithm better suited for parallel execution.
- New languages. Hundreds of new parallel languages and programming environments have been created over the last few decades. Many of them are excellent and provide high-level abstractions that simplify the expression of parallel algorithms. However, these languages have not dramatically grown the pool of parallel programmers. The fact is, in the one community with a long tradition of parallel computing (high-performance computing), the old standards of MPI  and OpenMP  continue to dominate. There is no reason to believe new languages will be any more successful as we move to more general-purpose programmers; i.e., it is not the quality of our programming models that is inhibiting the adoption of parallel programming.<./li>
The central cause of the parallel programming problem is fundamental to the enterprise of programming itself. In other words, we believe that our challenges in programming parallel processors point to deeper challenges in programming software in general. We believe the only way to solve the programming problem in general is to first understand how to architect software. Thus, we feel that the way to solve the parallel programming problem is to first understand how to architect parallel software. Given a good software design grounded in solid architectural principles, a software engineer can produce high-quality and scalable software. Starting with an ill-suited sense of the architecture for a software system, however, almost always leads to failure. Therefore, it follows that the first step in addressing the parallel programming problem is to focus on software architecture. From that vantage point, we have a hope of choosing the right programming models and building the right software frameworks that will allow the general population of programmers to produce parallel software.
In this article, we describe our work on software architecture. We use the device of a pattern language to write our ideas down and put them into a systematic form that can be used by others. After we present our pattern language , we present a case study to show how these patterns can be used to understand software architecture.
Software Architecture and Design Patterns
Productive, efficient software follows from good software architecture. Hence, we need to better formalize how software is architected, and in order to do this we need a way to write down architectural ideas in a form that groups of programmers can study, debate, and come to consensus on. is systematic process has at its core the peer review process that has been instrumental in advancing scientific and engineering disciplines.
The prerequisite to this process is a systematic way to write down the design elements from which an architecture is defined. Fortunately, the software community has already reached consensus on how to write these elements down in the important work Design Patterns . Our aim is to arrive at a set of patterns whose scope encompasses the entire enterprise of software development from architectural description to detailed implementation.
Design patterns give names to solutions to recurring problems that experts in a problem-domain gradually learn and take for granted. It is the possession of this tool-bag of solutions, and the ability to easily apply these solutions, that precisely defines what it means to be an expert in a domain.
For example, consider the Dense-Linear-Algebra pattern. Experts in fields that make heavy use of linear algebra have worked out a family of solutions to these problems. These solutions have a common set of design elements that can be captured in a Dense-Linear-Algebra design pattern. We summarize the pattern in the sidebar, but it is important to know that in the full text to the pattern  there would be sample code, examples, references, invariants, and other information needed to guide a software developer interested in dense linear algebra problems.
Computational Pattern: Dense-Linear-Algebra
Solution: A computation is organized as a sequence of arithmetic expressions acting on dense arrays of data. The operations and data access patterns are well defined mathematically so data can be pre-fetched and CPUs can execute close to their theoretically allowed peak performance. Applications of this pattern typically use standard building blocks defined in terms of the dimensions of the dense arrays with vectors (BLAS level 1), matrix-vector (BLAS level 2), and matrix-matrix (BLAS level 3) operations. The Dense-Linear-Algebra pattern is just one of the many patterns a software architect might use when designing an algorithm. A full design includes high-level patterns that describe how an application is organized, mid-level patterns about specific classes of computations, and low-level patterns describing specific execution strategies. We can take this full range of patterns and organize them into a single integrated pattern language -- a web of interlocking patterns that guide a designer from the beginning of a design problem to its successful realization [6, 7].
To represent the domain of software engineering in terms of a single pattern language is a daunting undertaking. Fortunately, based on our studies of successful application software, we believe software architectures can be built up from a manageable number of design patterns. These patterns define the building blocks of all software engineering and are fundamental to the practice of architecting parallel software. Hence, an effort to propose, argue about, and finally agree on what constitutes this set of patterns is the seminal intellectual challenge of our field.