"A crash is when your competitor's program dies. When your program dies, it is an 'idiosyncrasy.' Frequently, crashes are followed with a message like 'ID 02.' 'ID' is an abbreviation for idiosyncrasy and the number that follows indicates how many more months of testing the product should have had." — Guy Kawasaki
Testing is the cornerstone of robust software development, and Perl supports a flexible and powerful testing capability. In this article, I examine the testing mechanisms available to Perl programmers, focusing on the Extreme Programming (XP) Test First principle. Test First turns the traditional waterfall methodology on its head by saying that test writing precedes not only coding but even design, and that tests come straight out of requirements. In fact, the tests are a formal expression of the requirements.
This approach makes enormous sense. A requirement is not a real requirement unless it is testable. "The product shall be user friendly" is not testable, while "the system shall process 100 record updates within 500ms" is. So why interpose a huge hiatus of design and implementation between writing the requirements and finding out whether they're testable? Moreover, a full-fledged regression and unit test suite provides the safety net that lets you perform feats of derring-do on the high wire of development, safe in the knowledge that at every step you can instantly check whether the code passes the tests no matter what massive changes you may have just wrought upon it.
Clearly, the developers of Perl have embraced the XP testing philosophy. Between 1997 and 2004, the number of tests for the core Perl distribution itself mushroomed from 5000 to 70,000.
Simple to Get Started
Being lazy means never having to say you've got carpal tunnel syndrome. The test module designers applied the principle of Huffman coding to the API; that is, the most common functions you call should have the shortest names to optimize typing and reading. The most common test function is simply called ok(), which is vastly more appealing than, say, TestSocketListenerConnectionFailure(). If you're impatient (another fundamental Perl virtue), you can write a test just by creating a program containing. For instance:
use Test::Simple tests => 1; ok(sqrt(16) == 4, "sqrt(16) works");
As this shows, the ok() function is exported by the Test::Simple module that comes with Perl (since Perl 5.8.0, and before that in CPAN) and so is close to hand.
All that the ok() function does is test its first argument and print out "ok" or "not ok", depending on whether it is true. You're probably thinking, "I hardly need anyone to write a module to do that." But from such little test acorns do mighty regression suite oaks sprout. The ok() function also outputs the test label in the second argument and a running count of tests it has run. This is where the tests argument to Test::Simple comes in--it specifies how many tests we intend to run, so that Test::Simple can warn you if it sees a different number of calls to ok(). The output of a typical test program run looks like Listing One.
Listing One
1..12 ok 1 - regex search ok 2 - regex search ok 3 - regex search ok 4 - regex store lives ok 5 - regex store ok 6 - Store non-matching regex croaks ok 7 - Keys on complex store ok 8 - Hash on complex store ok 9 - lol regex store lives ok 10 - lol regex store ok 11 - Chained assignment ok 12 - Multiple match croaks
The first line shows the range of tests expected to be run; the significance of this will be seen later. Any error output from the program is passed through.
All this serves to show that the testing of Perl programs fits the same philosophy as programming Perl: Simple tasks should be simple to achieve. In practice, most testing uses the Test::More module, which exports a number of more useful functions, including:
is($expr, $value, $tag)same asok($expr eq $value, $tag).like($expr, $regex, $tag, same asok($expr =~ $regex, $tag).is_deeply($struct1, $struct2, $tag), compares arbitrarily deep structures.isa_ok($object, $class), verifies that an object is a member of a given class.use_ok($module, @imports), test that a module can be loaded
Note again the Huffman principle on names of commonly used functions. Even though several of the functions are syntactic sugar for variations on ok() (and all of them end up calling ok() anyway), any improvement in readability justifies a new function so that testing can be made as painless as possible. (There's even an isnt() function to avoid having to put a negation operator in an is() call, and, in true Perl tradition, there's an isn't() alias.)
Test::More lets you designate blocks of tests as conditional in two different ways. By naming a block SKIP, you can then call the skip() function inside the block if you want to skip the tests in it:
SKIP: {
eval { require Acme::Garden;
Acme::Garden->export;
};
$@ and skip "Don't have Acme::Garden installed", 2;
is(pull_weed(), "kudzu", "It's taking over!");
is(seed_depth("gladiolus"), 7, "Flower module ok");
}
The other way of marking tests as conditional is to put them in a block named "TODO". Failing tests in such a block will be specially annotated so as not to perturb aggregate statistics reported by Test::Harness.
The Perl test mechanism is not coupled to source code (Perl 5 does not lend itself easily to reflective tools; this will be remedied in Perl 6.) On the other hand, its simplicity allows it to be used to write unit tests, integration tests, regression tests, or acceptance tests.
Test::Simple is itself built on a module, Test::Builder, which is designed for reuse by modules providing testing functionality. The introduction of Test::Builder spurred an explosion in the creation of testing modules; there are currently over 250 such modules on CPAN in nearly 100 distributions. One of the more useful ones is Test::Exception, which verifies that code dies or fails to die when it should. You can also find Test::Pod, which vets the syntax of inline documentation written in POD ("Plain Old Documentation").
