October 2003
Regular Expressions: Fit makes for good tests
by Cameron Laird and Kathryn Soraiz
Fit is a clever testing framework Ward Cunningham invented to support the acceptance tests that Extreme Programming (XP) needs. Cunningham also invented the WikiWikiWeb, or Wiki, among many other projects.
XP is a development "methodology", "a deliberate and disciplined approach" that, in the case of XP, "emphasizes customer involvement and promotes team work" through "its simple rules and practices". XP is a frequent topic for UnixReview.com articles, including such recent book reviews as Extreme Programming.
A particular focus of XP is testing. XP requires testing, of all kinds, from the first day of a development project. Among other things, this makes XP's success unusually dependent on the cost and results of testing tasks; if testing is expensive or fragile, XP loses any advantages it has.
Practical testing
That logic is a prominent part of XP's background, in fact. XP advocates generally have a lot to say about how testing isn't as tedious or impractical as many programmers seem to believe.
 |
|
Figure 1
|
Furthermore, you don't even have to implement with Java to benefit from Fit. Although much of the XP literature addresses only Java and C++, most of its ideas apply quite naturally to the high-level languages that interest "Regular Expressions"'s readers. The Fit homepage points to Fit frameworks for several other languages.
This month, we want to direct attention to Dave Thomas' Ruby-based Fit. Thomas is, among other distinctions, co-founder of the Pragmatic Programmer series of books on "Agile methods", related to XP. His distribution unpacks easily and correctly reproduces the Fit model.
While the most natural way to work with Fit is in terms of such Web-based tables as Figure 1, it's intelligently modularized, so its pieces are also available for command-line inspection and automation. Thomas provides a README thats section on "Immediate Gratification" focuses on Web manifestations of Fit. To start using it, you need to:
tar xf fit.tar cd fit ruby process_examples.rbThis exercises example test specifications and creates:
fit/examples/op/Arithmetic.html fit/examples/op/Calculator.html fit/examples/op/Division.html fit/examples/op/DumbCounter.htmlwhich summarize the results in reports, such as:
...
Fit::Summary (source)
counts 27 right, 2 wrong, 0 ignored, 2 exceptions
input file inline
input update Sun Oct 19 15:38:49 EST 2003
output file inline
run date Mon Oct 20 11:15:41 EST 2003
...
State of the project
Do you see what this gives you? The Web interface makes it practical for an end user, or a least an end user's advocate, to code acceptance tests directly. At the same time, the test runs can feed directly into a whole range of developer-side automations, including nightly regression assurances. All this is packaged in a lightweight Wiki-like bundle that's easy to deploy and nearly maintenance free.
As exciting as Fit is for several leaders in testing circles, it remains a rather juvenile project. The Fit Wiki hasn't grown up enough yet to host the current versions of all the software it appears to have. If you'd like to work with the Ruby version, for example, you're better off getting a copy from our Phaseit, Inc., site. We still haven't figured out satisfying ways to handle testing of exceptions for situations that arise for our customers. We also don't see a good pattern for application of Fit to "sequence-dependent" validations we need, where doing operations:
A then B then C then D yields Rbut:
A then C then B then D yields S
On the other hand, Fit's "nimbleness" and portability make for the easy deployment and maintenance that's proven so valuable to Wikis. Fit's also capable enough that to have already inspired an offshoot, Robert C. Martin's Fitnesse, which strengthens the integration of Fit's Wiki and testing aspects.
What challenges does testing present for you? "Regular Expressions" will return to testing topics several times over the coming year; meanwhile, we'd love to hear about your experience with Fit and other testing frameworks.
Kathryn and Cameron run their own consultancy, Phaseit, Inc., specializing in high-reliability and high-performance applications managed by high-level languages. Join them each month as their "Regular Expressions" column explores issues and opportunities that arise in practical application development with scripting languages.