Natural Language Acceptance Testing with Cucumber
Cucumber is a testing framework for writing high-level natural language descriptions of software functionality. Translated to programmer speak this means that it's a compelling tool with which to write integration tests, and can be a big part of your Behavior Driven Development (BDD) strategy...
We all know that good integration / acceptance tests are written much like storycarded user stories are, and in Cucumber they take the following form. You declare a particular feature, followed by a number of scenarios that demonstrate particular behavior of that feature. Here's a sample:
Feature: User Registration
In order to keep track of my bookmarks
As a user
I want to have an account
Scenario: Sign up with invite code
Given That I do not have a user account
And I have a valid invite code "abc123"
When I go to the signup page
And I fill in "Username" with "MyName"
And I fill in "Password" with "Password"
And I fill in "Password Confirmation" with "Password"
And I fill in "Invitation Code" with "abc123"
And I press "Signup"
Then I should see "Thank you for signing up"
Note the use of the Given / When / Then template here. It's used repeatedly in Cucumber tests, asserting what we know about what the user wants / has, what actions they take, and what the expected outcome is. How these English-language steps are taken and converted into runnable test cases is the magic of Cucumber and its Gherkin domain specific language (Gherkin supports many langauges in addition to English too). Cucumber contains a number of step definitions, and when combined with the powerful WebRat web integration testing package, you'll find that it natively understands many commands used to visit particular pages, traverse the DOM, and fill in and submit forms.
Anyway, if Cucumber isn't sure how to interpret a particular step, the runner process will tell you that the step is undefined and prompt you to define those steps yourself. For instance, in the example above, Cucumber would have no way of knowing what a "valid invite code" is, so you would need to define that. The step definition might look something like this
Given /^I have a valid invite code \"(.+)\"$/ do |code|
Invitation.create!(:code => code)
In true BDD fashion, you're encouraged to implement the behavior specifications first, and continue to implement behavior (and the appropriate step definitions) until the criteria established with your stakeholders is satisfied.
The difference between using Cucumber for integration tests and using something like WebRat straight-up is largely syntactical. But since the ultimate goal of Cucumber (and of BDD in general) is to communicate the behavior of your system to everyone involved in a project, this is a pretty important distinction. Sometimes your stakeholders won't be folks with a technical skillset, and in those cases Cucumber's natural language stories present an easy way for everyone to make sure they're on the same page regarding overall system functionality
Do Cucumber acceptance tests take extra time to write? Absolutely. But the reward that you get in terms of lowering the communication barrier with your customers could definitely be worth the effort. Next time you start a project, start with writing proper integration tests, with Cucumber, in English. Review them with your clients. Make sure that the stories make sense to them. Maybe even get them to participate in the spec writing process itself.
Then, simply implement the steps and iteratively develop your software to satisfy them, piece by piece, until all the known specifications pass. It sounds simple, and it is -- but it's also a very powerful approach. I know I've been tackling software projects in a vein similar to this for quite some time, but, toying with Cucumber just now I can see how this extra layer of abstraction can make the functionality specification process even easier, while at the same time helping to ensure that there aren't any communication problems with the clients or stakeholders.
Make note that Cucumber is _not_ a replacement for a unit testing framework, but rather a higher-level tool for constructing and testing scenarios, including customer acceptance tests. It works hand-in-hand with RSpec, Test/Unit, or whatever your unit testing framework of choice is. I should also probably say that, although Cucumber is implemented in Ruby, it can certainly be used to test (at the web integration level) web-based software written in other languages!
Anyway, that's it for me. For more information, make sure to check out the official Cucumber website which is packed with helpful examples. Ryan Bates of Railscasts fame has also put together a couple short screencasts (155, 159) that were really helpful to me when researching the topic.