Regular Expressions: Fit makes for good tests

Fit is a clever testing framework Ward Cunningham invented to support the acceptance tests that Extreme Programming (XP) needs. Cunningham also invented the WikiWikiWeb, or Wiki, among many other projects.

October 27, 2003
URL:http://www.drdobbs.com/regular-expressions-fit-makes-for-good-t/199102579

October 2003

Regular Expressions: Fit makes for good tests

by Cameron Laird and Kathryn Soraiz

XP is a development "methodology", "a deliberate and disciplined approach" that, in the case of XP, "emphasizes customer involvement and promotes team work" through "its simple rules and practices". XP is a frequent topic for UnixReview.com articles, including such recent book reviews as Extreme Programming.

A particular focus of XP is testing. XP requires testing, of all kinds, from the first day of a development project. Among other things, this makes XP's success unusually dependent on the cost and results of testing tasks; if testing is expensive or fragile, XP loses any advantages it has.

Practical testing

That logic is a prominent part of XP's background, in fact. XP advocates generally have a lot to say about how testing isn't as tedious or impractical as many programmers seem to believe.

Figure 1

Fit demonstrates that. As Cunningham writes, Fit "is about tests that people can read", such as the example reproduced in Figure 1. The novelty here is that, with Fit, you maintain and extend these tests by filling out simple Web tables. While the implementation of the arithmetic of this example is in Java, management of a test suite doesn't demand knowledge of Java — everything shows up as readable tables like this one. End users can leverage HTML editors to specify and execute individual tests.

Furthermore, you don't even have to implement with Java to benefit from Fit. Although much of the XP literature addresses only Java and C++, most of its ideas apply quite naturally to the high-level languages that interest "Regular Expressions"'s readers. The Fit homepage points to Fit frameworks for several other languages.

This month, we want to direct attention to Dave Thomas' Ruby-based Fit. Thomas is, among other distinctions, co-founder of the Pragmatic Programmer series of books on "Agile methods", related to XP. His distribution unpacks easily and correctly reproduces the Fit model.

While the most natural way to work with Fit is in terms of such Web-based tables as Figure 1, it's intelligently modularized, so its pieces are also available for command-line inspection and automation. Thomas provides a README thats section on "Immediate Gratification" focuses on Web manifestations of Fit. To start using it, you need to:

  tar xf fit.tar
  cd fit
  ruby process_examples.rb

This exercises example test specifications and creates:

  fit/examples/op/Arithmetic.html
  fit/examples/op/Calculator.html
  fit/examples/op/Division.html
  fit/examples/op/DumbCounter.html

which summarize the results in reports, such as:

      ...
  Fit::Summary (source)
  counts       27 right, 2 wrong, 0 ignored, 2 exceptions
  input file   inline
  input update Sun Oct 19 15:38:49 EST 2003
  output file  inline
  run date     Mon Oct 20 11:15:41 EST 2003
     ...

State of the project Do you see what this gives you? The Web interface makes it practical for an end user, or a least an end user's advocate, to code acceptance tests directly. At the same time, the test runs can feed directly into a whole range of developer-side automations, including nightly regression assurances. All this is packaged in a lightweight Wiki-like bundle that's easy to deploy and nearly maintenance free. As exciting as Fit is for several leaders in testing circles, it remains a rather juvenile project. The Fit Wiki hasn't grown up enough yet to host the current versions of all the software it appears to have. If you'd like to work with the Ruby version, for example, you're better off getting a copy from our Phaseit, Inc., site. We still haven't figured out satisfying ways to handle testing of exceptions for situations that arise for our customers. We also don't see a good pattern for application of Fit to "sequence-dependent" validations we need, where doing operations:

  A then B then C then D        yields      R

but:

  A then C then B then D        yields      S

On the other hand, Fit's "nimbleness" and portability make for the easy deployment and maintenance that's proven so valuable to Wikis. Fit's also capable enough that to have already inspired an offshoot, Robert C. Martin's Fitnesse, which strengthens the integration of Fit's Wiki and testing aspects.

What challenges does testing present for you? "Regular Expressions" will return to testing topics several times over the coming year; meanwhile, we'd love to hear about your experience with Fit and other testing frameworks.

Kathryn and Cameron run their own consultancy, Phaseit, Inc., specializing in high-reliability and high-performance applications managed by high-level languages. Join them each month as their "Regular Expressions" column explores issues and opportunities that arise in practical application development with scripting languages.