Channels ▼


The Trouble with Software QC

Colin is the founder and CEO of The Original Software Group (, which develops tools to automate and simplify software testing. Colin can be reached by e-mail at [email protected]

Remarkably, many IT organizations still use sheer manpower to find defects in software, rarely employing methodologies within which to work and measure effectiveness. Because most IT shops race to press new code into service, end users often bear the burden of testing. Let's face it, having end users report problems to an overloaded help desk is not the best way to achieve the goal of releasing defect-free code. Unsurprisingly, a reactive development/QA environment can exacerbate damage when application performance plummets, or when programs fail all together.

The ethos of quality is sweeping into the IT space, as it did in manufacturing a few decades ago. Drivers of this movement include the need for improved software development productivity, higher end-user expectations, reduced time to market, the need to manage increasingly complex software, and the business-criticality of software. And with SOX, HIPAA, and other federal legislation aimed at compliance, the government is leaning on organizations to improve systems and be more responsible.

While awareness is growing, recent research by Forrester shows that quality consciousness exists more in the mind and is less apparent on the ground. Although dictums for quality infuse annual corporate reports, some companies are having trouble putting their money where their mantras are these days. Top-level IT managers say they feel disconnected from end users who often struggle with new systems. When asked, IT managers admit that testing doesn't end when an application has been repeatedly pushed to failure and then fixed. Rather, testing ends when software works for the first time on the first test. And strangely, it also ends when the IT department bumps up against a deadline set by some other department.

However, once the will exists, IT organizations can take steps to improve. Solid procedures, metrics, and tools can ensure that software works predictably even under extreme circumstances. A formalized QA regimen used at several intervals in the development cycle improves any development shop's output and helps IT better align itself with the quality-based objectives of the organization it serves.

A testing methodology I find useful for development is the Software Testing Maturity Model, also referred to as "S-TMM" ( pub/sqp/past/vol1_issue4/sqpv1i4burnstein.pdf). S-TMM is one of several testing models that share a common conceptual heritage with the "Software Capability Maturity Model" (S-CMM) that evolved at the Software Engineering Institute at Carnegie Mellon University. S-TMM can be used in conjunction with the current iteration of S-CMM flavor, called "Capability Maturity Model Integration" (CMMI), or it can stand by itself.

S-TMM serves the objectives of IT departments better than many of the other quality-control models because it is easy for IT organizations to embrace. It also lays out a strategy for self assessment; guiding IT departments though a series of stages that gradually evolves the development process over time.

The S-TMM identifies five levels of maturity. All IT organizations hover at one of these levels and those who register at the lower levels of testing maturity can benefit by moving to a higher plane. Experience with the S-TMM shows that high-maturity organizations achieve great success and constantly improve their software development process, while low-maturity organizations tend to wade in mayhem.

The five levels of maturity identified by the S-TMM are:

  • Level 1. Initial, no structure or method.
  • Level 2. Phase Definition.
  • Level 3. Integration.
  • Level 4. Management and Measurement.
  • Level 5. Optimization/Defect Prevention and Quality Control.

Why should you adopt S-TMM? The difference between where you believe you are in terms of testing maturity and where you really are can be best clarified with S-TMM. Who knows, you may be surprised by what you see in the mirror. Once you've established a baseline for actual maturity, you can also use S-TMM as a guidebook for improvement. S-TMM will guide you through a process to improve the software produced by your IT organization, and it will elevate the level of professionalism within the organization. The credibility that documented success garners also helps secure high-level support of subsequent projects.

Related Reading

More Insights

Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.