UnixReview.com
May 2006
Regular Expressions: High-level languages are configuration languages
by Cameron Laird and Kathryn SoraizIn the March installment of Regular Expressions, we argued that your programs should be configurable, especially if they work in the parts of the real world that connect to physical devices, are incompletely specified, or are parts of larger projects — and, yes, feel free to note that nearly all programs are that way. Now it's time to look at implementations that make this real.
Flexible automation"Configuration", for this purpose, emphasizes textual interfaces. Consider a concrete example — a Web browser. You launch your favorite Web browser and navigate to UnixReview.com. You can point-and-click your way here without having to specify a default font, whether JavaScript should be enabled or disabled, that you want to rely on ISO Latin I to interpret characters, or so on. The browser more-or-less sensibly defaults to reasonable values for these settings.
At the same time as the application embeds reasonable values for these variables, it also provides a way to update them at run-time — typically GUI selections in a Preferences or similar menu. As developers and administrators, though, we emphasize processes that can be automated, so our configuration will be through command-line arguments, startup files, or other textual means.
A few computing generations ago, enhancement of an application with this sort of run-time flexibility was a major project; programmers often coded application-specific parsers for their configuration files or even to interpret command-line arguments. Do not fall into that trap! One of the battles we don't need to fight again is configuration parsing; rather than defining a special-purpose syntax for your configuration and turning to lex and yacc or other difficult means to parse it, use the parsing that's available for free in your development environment.
You could, for example, have configuration files that look like:
[server_port] <- 3518 [x-origin] <- 39 [x-units] <- meters ...and so on. Even if you did so, your configuration language would probably be simple enough to take only a few lines of a high-level language's regular expressions and character-splitting to analyze. If you use your favorite language's run-time built-ins, though, you can probably have a parser with just a command or two. Suppose, to extend this example, you are coding in Python. The standard Python distribution offers a ConfigParser module that interprets a language very close to that of Windows INI files. In such a syntax, your users would replace the sequence above with something like:
; This file has the name "configuration.txt". [basic] server_port=3518 x-origin=39 x-units=meters ...and interpret it with:
import ConfigParser
# ConfigParser interprets arguments passed
# to this constructor as default values.
configuration = ConfigParser.ConfigParser()
configuration = add_section("basic")
configuration.readfp(open("configuration.txt", "r"))
start_server(configuration.get("basic", "server_port")
...
Use of a standard building block like ConfigParser gives sensible conventions for line-continuation, comments, and so on — all "for free".
An earlier role for scripting languages
Keep in mind this isn't merely "late binding for the sake of late binding", a frivolous functionality. Our experience is that run-time configurability pays off quite satisfyingly.
How can we have such strong feelings about configuration files while claiming to work in "agile" situations? Doesn't agile programming prescribe minimalism in meeting requirements?
Yes, it does, but one of the most consistent results of our projects is that difficult developments require flexibility. Many written specifications say something like, "The timeout period for reporting loss of an external device shall be ten seconds." This invariably turns out to mean, "You'll need to perform trials with the timeout period set at one, three, eight, ten, fourteen, and twenty seconds (at least), and report back on the observations." At that point, you're far better off with a single definite, quality-assured, program instance, along with a range of command-line or configuration file inputs, than to have to make a new executable for each trial.
And as low-cost as modules like ConfigParser are — available in standard distributions at no charge, usable at the expense of only a couple of additional lines of code — most scripting languages offer an even more immediate alternative.
A couple of decades ago, the principal "ecological niche" for scripting languages was for extending or configuring larger applications. All it takes to regain that capability is to define the language of configuration as your language of implementation (or perhaps a subset)!
For this example, suppose the same application as before, but that it's implemented in Tcl. In this case, a configuration file might look like:
set server_port 3518 set x-origin 39 set x-units meters ...All it takes to configure the application is then the single Tcl command:
source config.txt
In production code, that simple source, or its equivalent in other languages, generally needs to be guarded because it makes the application too configurable (that is, insecure). The possibility of that minor rework, generally near the end of a project, is one we're willing to bear, in comparison to all the benefits we consistently experience from the first day we make any of our applications configurable in a systematic way.
Remember, then:
- Configurability pays off during development.
- It's much easier than you might have learned 10 years ago.
- Define a configuration language that's easy to parse.
- For immediate results, use the implementation language itself as a configuration language.
- Over the long term, learn about your development environment's built-in parsing libraries.
- Configuration is an issue for security and testing — but one you can manage, without crisis or confusion.
Kathryn and Cameron run their own consultancy, Phaseit, Inc., specializing in high-reliability and high-performance applications managed by high-level languages. Among other professional activities, they have published Regular Expressions for seven years, mostly on a monthly schedule.