Dr. Dobb's is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Channels ▼

Pablo Santos

Dr. Dobb's Bloggers

Beyond automated testing

May 30, 2009

I've been always a big fan of automated testing. I pushed my old team at a local software business company to create hours of Rational Robot (http://www-01.ibm.com/software/awdtools/tester/robot/index.html) GUI tests. It was very good since they were not using any kind of automated testing before, and it actually played its magic: hundreds of business scenarios were covered, starting from the core ones, the one that shouldn't ever fail, so show-stopper errors rapidly started to vanish.

First steps are so easy...

It's always great to automate at projects were they don't have any previous experience since a small effort returns a huge improvement. The team won't probably ever climb the quality ladder as fast as they do when they introduce serious testing, issue tracking and version control... once the basics are covered the next stairs become much harder.

Testing is all about project management

When you talk about testing, automated or any other kind, everyone starts thinking about how product quality will raise, how it will affect the way customers perceive the product and how it will affect sales.

But for me, as a project manager who wants his team to survive the current project and hopefully a few more, it will just bring calm to the team.

Let me explain: yes, I do care about product quality, sales, reducing the bug count and so on, but what I like about testing is that it allows developers to do their job as they like to do: without urgent rushes and unexpected crisis.

Automated testing won't cover everything (unless you enjoy the kind of unlimited resources I never had) but it will avoid priority one bugs, the ones that stop your daily tasks, they ones you have to fix immediately to deliver a new release, the ones altering your schedule while you see how your plan vanishes.

When a show-stopper is found, especially by a customer, all the team will rush to solve it, it's mandatory, is a big priority. So it won't matter how good your SCRUM sprint was planned, or how detailed your wall-wide Project Gantt chart is: you'll have to switch to abandon the plan and fix the dammed bug.

Automated tests will prevent this to happen so you and your team will be able to focus on what you said you were going to do, so everyone around will feel better, and upper management will feel you've things under control. And all started with some small xUnit tests and hopefully a number of UI automated tests.

Warning: developer way of thinking

But the story does not always have a happy ending: if the tests are created by developers, chances are the whole test suite will be covering canonical scenarios only. I mean, if you've a command accepting three parameters, chances are developers will test the weirdest combinations of the right parameters triggering the most complex possible calculation, but won't create a test to check how the command breaks if a wrong number of parameters or illegal input is entered.

We've a big test suite composed of unit tests, automated command line based tests and UI ones, but we still miss exception scenarios sometimes.

So, it seems extremely positive to include real testers (I mean, normally not being tied to the active development, although the real structure and background will vary, as exceptionally explained here: Agile Testing: A Practical Guide for Testers and Agile Teams) who can bring fresh air and a user point of view to the project.

In our case, we're actively looking to enhance our current automated system with complementary manual testing.

Related Reading

More Insights

Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.