Channels ▼

Nick Plante

Dr. Dobb's Bloggers

Natural Language Acceptance Testing with Cucumber

June 26, 2009

Cucumber is a testing framework for writing high-level natural language descriptions of software functionality. Translated to programmer speak this means that it's a compelling tool with which to write integration tests, and can be a big part of your Behavior Driven Development (BDD) strategy...

We all know that good integration / acceptance tests are written much like storycarded user stories are, and in Cucumber they take the following form. You declare a particular feature, followed by a number of scenarios that demonstrate particular behavior of that feature. Here's a sample:

    Feature: User Registration
      In order to keep track of my bookmarks
      As a user
      I want to have an account

      Scenario: Sign up with invite code
        Given That I do not have a user account
        And I have a valid invite code "abc123"
        When I go to the signup page
        And I fill in "Username" with "MyName"
        And I fill in "Password" with "Password"
        And I fill in "Password Confirmation" with "Password"
        And I fill in "Invitation Code" with "abc123"
        And I press "Signup"
        Then I should see "Thank you for signing up"

Note the use of the Given / When / Then template here. It's used repeatedly in Cucumber tests, asserting what we know about what the user wants / has, what actions they take, and what the expected outcome is. How these English-language steps are taken and converted into runnable test cases is the magic of Cucumber and its Gherkin domain specific language (Gherkin supports many langauges in addition to English too). Cucumber contains a number of step definitions, and when combined with the powerful WebRat web integration testing package, you'll find that it natively understands many commands used to visit particular pages, traverse the DOM, and fill in and submit forms.

Anyway, if Cucumber isn't sure how to interpret a particular step, the runner process will tell you that the step is undefined and prompt you to define those steps yourself. For instance, in the example above, Cucumber would have no way of knowing what a "valid invite code" is, so you would need to define that. The step definition might look something like this

    Given /^I have a valid invite code \"(.+)\"$/ do |code|
      Invitation.create!(:code => code)
    end

In true BDD fashion, you're encouraged to implement the behavior specifications first, and continue to implement behavior (and the appropriate step definitions) until the criteria established with your stakeholders is satisfied.

The difference between using Cucumber for integration tests and using something like WebRat straight-up is largely syntactical. But since the ultimate goal of Cucumber (and of BDD in general) is to communicate the behavior of your system to everyone involved in a project, this is a pretty important distinction. Sometimes your stakeholders won't be folks with a technical skillset, and in those cases Cucumber's natural language stories present an easy way for everyone to make sure they're on the same page regarding overall system functionality

Do Cucumber acceptance tests take extra time to write? Absolutely. But the reward that you get in terms of lowering the communication barrier with your customers could definitely be worth the effort. Next time you start a project, start with writing proper integration tests, with Cucumber, in English. Review them with your clients. Make sure that the stories make sense to them. Maybe even get them to participate in the spec writing process itself.

Then, simply implement the steps and iteratively develop your software to satisfy them, piece by piece, until all the known specifications pass. It sounds simple, and it is -- but it's also a very powerful approach. I know I've been tackling software projects in a vein similar to this for quite some time, but, toying with Cucumber just now I can see how this extra layer of abstraction can make the functionality specification process even easier, while at the same time helping to ensure that there aren't any communication problems with the clients or stakeholders.

Make note that Cucumber is _not_ a replacement for a unit testing framework, but rather a higher-level tool for constructing and testing scenarios, including customer acceptance tests. It works hand-in-hand with RSpec, Test/Unit, or whatever your unit testing framework of choice is. I should also probably say that, although Cucumber is implemented in Ruby, it can certainly be used to test (at the web integration level) web-based software written in other languages!

Anyway, that's it for me. For more information, make sure to check out the official Cucumber website which is packed with helpful examples. Ryan Bates of Railscasts fame has also put together a couple short screencasts (155, 159) that were really helpful to me when researching the topic.

Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.
 


Video