What would it look like to actively search for a theory of software design? We should emulate the worlds of physics and traditional architecture.
- We should advance theories of design, as I did in the article cited above.
- Our theories should be specific, predictive, and testable. This is a fundamental rule for science. Theories that do not make specific predictions are not falsifiable and are not part of science; they are religion.
- Competing theories are strengthened or weakened both by theoretical arguments and/or experimental evidence. The internal inconsistency of a theory might weaken it theoretically, while its parsimony might strengthen it. Experimental evidence that does not match prediction will weaken a theory (the prediction was X but in fact Y happened), while experiments that match prediction will strengthen it.
- We can use evidence both from new software projects as well as historical evidence from past projects.
All of these methods for advancing and refining theories are standard operating procedure in physics, traditional architecture, and many other disciplines. Some examples of this process within physics are string theory and biting criticisms of it from Lee Smolin and Peter Woit. Within traditional architecture see the work of Christopher Alexander, Peter Eisenman, and their famous debate.
In most fields, some scientists typically specialize in creating theories and making predictions about the world, while other scientists specialize in designing and conducting experiments to test theories. This division of labor, while not necessary, has often been helpful. Within software engineering, we have the beginnings of this specialization, with academics on one hand, and practitioners (programmers) on the other. Unfortunately, our academics often do not look to practitioners as a source of valuable experimental data, but instead try to tell programmers what they should be doing.
Within software engineering, what would evidence supporting (or refuting) a theory look like? Suppose we believe that "information hiding" is a universal software design principle. Suppose we find a substantial software system that everyone agrees is a wonderful piece of software. It has few bugs, runs faster than its competitors, is easy to modify and scale, and is actually readable by new team members. Suppose we examine the source code and see that it violates the principle of information hiding at every opportunity. Class members are all public and are referenced throughout other classes. This program would present evidence that information hiding is not such a strong design principle after all. Or, in the least, theorists advancing the case for information hiding would have to work hard to explain this apparent counter-example.
One possibility that must be considered is that there is no overall theory of software design. This point-of-view holds that different problem domains (operating systems, accounting, web interfaces, etc) are so fundamentally different that there are no general principles for good design that span all domains. This is plausible. But it should be investigated in the same way as other theories. What are some example projects that provide evidence that no single principle of software design holds consistently? Are there mushy heuristics that work instead? What are they?
Another objection that might be raised to the search for software design theory is that software is not a theoretical discipline at all. Physics, and other natural sciences, seek to discover the state of the world. They want to learn about what is. Software development, on the other hand, tries to build something; it is more akin to mechanical engineering than biology. This is true, in a strict sense. But many areas of study that are not hard sciences look for an overall theory. This includes traditional architecture, filmmaking, industrial design, and literature. These disciplines, all concerned with making sometime, have long and lively histories of debate about their underlying principles. The software world seems to lurch from fad to fad without considering the nature of what we are doing and the general ideas about how to do it well.
Note that the question of how people should go about working on software is separate, although there may be some intersection with a design theory. We can separately improve the programming languages we use, the way we organize teams on a software project, the accuracy of our time/size estimates, how we test and debug, etc. A design theory does not solve all the problems of software development. But it solves a key one.
Programmers have implicitly operated on the premise that there are no general truths about software design. Instead of starting each new project with a solid theory of design to guide us, we essentially invent our world anew each time. No wonder software development seems like a black art, and so often goes badly wrong.
What we need instead is a vigorous, active search for a small number of universal software design principles -- a theory.
As I was writing this article, this topic was attracting increased attention within the software engineering community. See The SEMAT Initiative: A Call for Action and Software Engineering Method and Theory for links to related publications and people.