Martin Fowler, chief scientist at ThoughtWorks, is also a widely respected author in the software development world, having written books on topics ranging from object-oriented design to patterns to agile development. His latest book, Domain-Specific Languages, written with ThoughtWorks CTO Rebecca Parsons, promises to aid developers in deciding when and where to best use a DSL, and how to go about building one. Fowler and Parsons recently spoke with Dr. Dobb's editor in chief Jon Erickson the nature of DSLs.
Dr. Dobb's: What exactly is a DSL?
Fowler: The definition in the book is “a computer programming language of limited expressiveness focused on a particular domain.”
Parsons: There are two key characteristics for a DSL. First, there should be an identifiable domain, whether it be a business domain, like lease accounting, a scientific domain, like rational drug design, or a technical domain, like persistence. The second is the language must be small. We don’t want these languages to grow into Turing-complete general purpose programming languages with some domain concepts thrown in for good measure. More often, we are looking at constructing programs and systems by combining one or more DSLs with one or more general purpose programming languages.
Fowler: Which ties into “limited expressiveness” in the definition. We see this as the key difference between DSLs and general-purpose languages. They aren’t just simpler languages than general-purpose languages. The limited expressiveness means that all sorts of rules change when you think of working with them ... It’s also important to note that this is the definition we use in the book. Others don’t necessarily follow that in other writings.
Dr. Dobb's: What kind of applications are well suited by a DSL solution?
Parsons: The book frequently talks about DSLs as interfaces to APIs and libraries. If the problem in question lends itself to thinking about a suite of concepts linked by actions that can be well expressed using an API, you’re likely looking at a good candidate for a DSL.
Fowler: Indeed, I like to think of DSLs as an alternative interface to a library than the usual command-query API. DSLs come into their own when you have a lot of configuring of objects in a library and those configurations and resulting object relationships are easier to visualize with a linguistic approach.
Another case is where you want to communicate detailed processing rules with domain experts. A small DSL can provide something that domain experts can read and critique. In most cases, we don’t see DSLs as something that domain experts edit directly, removing the need for programmers (although there are exceptions, such as CSS). We think that most of the time the key benefits come from domain experts reading the DSL and using that as the basis for conversations with programmers.
Dr. Dobb's: When should you avoid DSLs?
Fowler: You don’t need one if the command-query API does the job just fine.
Parsons: If there’s not really a properly stable domain language, a DSL might be premature. It’s conceivable working to design the DSL will help clarify the domain understanding, but I don’t think that would typically be true. Designing a DSL codifies certain domain assumptions, some of which may be hard to change.
Dr. Dobb's: Do developers need special skills to build and use DSLs?
Fowler: To a certain degree they do ... but they aren’t difficult skills to learn. Indeed, a large part of why we wrote this book was to remove the fear that seems to have grown up around language processing that’s made people shy away from doing DSLs.
Parsons: I think there is actually a third part to this question, regarding designing DSLs, but I’ll answer your actual question first. The skills required to implement a DSL ... are not markedly different from normal programming skills. Again, the skills requirements for dealing with simpler languages are quite different than those required to implement a full-blown programming language. The developer needs to choose whether to build an internal DSL, using a host language like Ruby, Groovy, or C#, or whether to build an external DSL by constructing a separate parser either directly or using a parser generator. Using a DSL requires constructing a script or program in the new language, likely coupling it with code in a programming language.
Now, a different question concerns the skills needed to design the DSL. Language design is an art, and even for general purpose programming languages, there is little consensus on what constitutes a good language design or how to get one. This area requires a great deal of additional attention, but the book couldn’t include everything.
Fowler: And even if we had more room, I only have a sketchy sense of the guidelines for good DSL design. If we see more activity with DSLs in the future, I hope this will become clearer.
Dr. Dobb's: Does DSL design require any special tools?
Fowler: The kinds of DSL we describe in the book, textual DSLs, don’t require any fancy tools. Internal DSLs are existing language features used in a fluent style (which admittedly can look rather funky). External DSLs are often done using a parser generator, but such tools are mature and easily available (often open source).
Parsons: While programming languages processing tools have existed for a long time, the literature on these tools has targeted people who write compilers and/or process general purpose programming languages. DSLs, as they are generally far simpler than programming languages, are easier to work with. One aim of the book is to demystify these tools.
Fowler: The great thing about textual DSLs is that to program in your DSL, you use the same tools you use for regular programming: text editors, version control, etc.
Fowler: We don’t talk too much about Language Workbenches in the book. These are sophisticated tools for constructing and editing DSLs. They have huge potential, but it’s unclear how long it will be before they are usable in the mainstream.
Dr. Dobb's: What about unique, non-standard DSLs 5 or 10 years from now? Won't they be difficult to maintain once the original developer is no longer around?
Fowler: When you’re considering the pros and cons of DSLs, it’s valuable to remember that they are nothing more than a thin veneer over a library. So many of the benefits and problems of DSLs are essentially those of libraries. We see lots of unique, non-standard libraries out there, developers have to learn their APIs and often their implementations. The question is whether using a DSL makes using that harder.
Parsons: I think we’d all accept that maintaining anyone else’s code, or even our own, after 10 years is a difficult problem. So, is it harder to maintain a DSL after that length of time? I would say no. If your development staff that takes over has a basic understanding of DSL implementation techniques and the DSL itself properly reflects the domain, the problem is no different in the DSL context. The new developer still needs to learn the domain and understand the implementation. It’s just done differently than it might have been in the past. One could argue that it should be easier, because the existence of the DSL will tend to make the business logic expressed in the DSL “stand out” in a way that business logic in code may not.
Will it always pan out this way? Nope. DSLs are no more immune to poor programming practices, the existence of technical debt, and general code atrophy as any other approach. Sorry — still no silver bullet.
Dr. Dobb's: Is letting developers have their own language really a good thing?
Parsons: As opposed to what? The tool can clearly be misused, as can every other piece of technology known to man. Letting poor, inadequately trained developers loose on a problem with insufficient guidance and time is a recipe for disaster. Your steaming pile will simply look different if DSLs are used. It won’t smell any worse.
Dr. Dobb's: Have DSL productivity gains for developers been validated or is this just another sales pitch?
Fowler: I haven’t seen validated productivity gains for anything in the software industry — even for something as generally accepted as using high-level languages rather than assembler. It all boils down to the fact that there is no way to measure output, and thus productivity. (I talk about this in Cannot Measure Productivity [a blog posted several years ago.])
Parsons: I’d say there’s compelling anecdotal evidence, but I think the idea still needs to be validated as to the degree of generality. We don’t yet really know the patterns that will emerge in the usage of DSLs.
Dr. Dobb's: Do Agile methodologies and DSLs enjoy a special relationship?
Fowler: The only link I see is that so many people in the agile community are deeply interested in the questions of good software design and communication with domain experts. But that link is just an indirect common interest of the people involved, rather than anything inherent in either technology.
Parsons: Indeed, to me there’s still an open question surrounding how DSL design works in an agile development cycle. As we described in response to an early question, there’s still a lot of thinking to do about language design for DSLs.
Dr. Dobb's: When designing and implementing a DSL, what's the number one 'gotcha' developers need to watch for?
Fowler: The biggest danger for DSLs, particularly external DSLs, is the gradual accretion of features that turns it into a general-purpose language. The greatest strength of a DSL is its limited expressiveness and narrow focus, so you have to be vigilant to keep it small.
Parsons: For DSLs to provide the most utility the language needs to reflect the domain, including the vocabulary of the domain and what constitutes “normal” in the domain. Things that are normal in the domain should be easy and natural to express in the language. The language must reflect the needs of the domain users, not the needs of the programmers (unless the users happen to also be programmers), even though programmers might still be actually writing the code.
Another significant issue is that of backward compatibility. Languages evolve, as the domains they represent also evolve. There’s an interesting trade-off each time the language needs to change regarding the cost of changing existing scripts relative to the cost of creating a less domain-friendly language.