Channels ▼
RSS

Testing

The Shrinking Role of QA


The Industrial Revolution began about 250 years ago and machines started to take over for human labor in factories, fields, and mines. This led to major economic growth, but machines started to replace the average worker, who could not get another job or acquire a new skill, which hurt many people. The similarity of the situation QA testing finds itself in currently is uncanny. QA was a savior during the Internet boom — when budgets were virtually unlimited and software was being produced at an exponential rate. However, as the economy bottomed out, budgets deflated, Agile development became more widespread, and automated testing began taking over. Just as some forms of manual labor became obsolete over time due to the Industrial Revolution, manual QA testing finds itself facing the same predicament.

Let's take a step back and examine how things used to work. Waterfall has been the method of choice for many software development teams since the 1950s. This approach allowed for developers to design everything upfront, then focus on their code, pass it on to QA for testing, and get it back with a list of bugs to fix. Over time, systems were upgraded, development methods got better and faster, and we found ourselves producing so much software at such a rate that we needed more testers. More testers were hired and developers got lazier knowing that QA would be there to catch their mistakes.

In February 2001, as the dot-com bubble was bursting, the Agile Manifesto was published and a new way of developer thinking started to emerge. Agile development methodologies breathed new life into the developer world, where adapting to ever-changing situations and rapidly deploying working software became the focus of development teams. Agile is more involved in each working part of a team and the code, with a special focus on developer testing rather than QA testing. As Agile continues to become more widespread and effective, QA is needed less, and that means it has one foot out the door.

More QA, More Problems

Enterprise software development is an expensive and complex endeavor. When using the Waterfall model, it is common to find that the original product goals are not reached — at which point, management typically makes one of three choices:

  • Add Budget: More money can be thrown at a project. This might make it possible to complete the project on time, taking into account the law of diminishing returns. But this is not a preferred scenario for management.
  • Trim Features: Neither developers nor management are keen on giving customers less than they are paying for. This generally is not an option for many companies.
  • Lower Quality: Quality is expensive, and thus is often the first thing to be cut back.

What we are left with is a sense of imbalance. Developers create code that lacks the necessary quality, while at the same time, management is cutting down on QA. There is an inherent flaw here. This is where Agile fundamentals can come into play.

It's An Agile Thing

After the dot-com bubble burst and the economy was gradually working its way back, companies knew that they needed to create good software without the huge costs. Fortunately, Agile development is about developers testing their own code. While QA can still test fringe or flow cases, developers recognize the need to take responsibility for their code. One of the pillars of Agile development is delivery of working software. Some Agile methodologies include TDD and unit testing performed by the developers. Unit testing is about developers doing their part to make the whole better with the benefit of continuous, instant feedback that flushes bugs out quickly and cheaply. Without good unit testing coverage, it is very hard to stay Agile because, as a design continuously changes, the unit tests can flag problematic consequences of the rolling changes.

Unit Testing: Possible QA killer

It has been shown that unit testing can properly check 90% of code and, unlike QA Automation tools, unit tests that are built correctly evolve with your codebase. As a platform for testing code (and creating abstract design), unit testing costs less and gets better quality software out the door quicker than competing approaches that rely on QA to find all the defects. And, as mentioned previously, automated unit testing and TDD enable developers to adapt their code to new features and other changes on the fly. When combined with other Agile techniques, these approaches rely on delivering working software every day. The question is what role, if any, QA departments will be left to handle?

At many companies, when working software passes UATs, it is good enough for deployment for internal use. This leaves a rapidly shrinking role for QA departments. Obviously, companies with products for external consumption or with tight safety or regulatory requirements will need extensive support from QA and ancillary testers, but increasingly, those might well be the last redoubts of the formerly universal QA departments. And even their fate will hinge upon how much testing developers take on, which itself is a function of how much developers can automate the testing of their own code.


Eli Lopian is the CEO of Typemock, the Unit Testing Company. A well-known figure in the agile and test-driven-development arenas, Eli has over 17 years of R&D experience at large companies such as AMDOCS (NYSE:DOX) and Digital Equipment Corporation (DEC). Andrew Binstock is the editor in chief of Dr. Dobb's.


Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.
 

Comments:

ubm_techweb_disqus_sso_-5e13180b23a2131efd9fdc15674ef8e9
2012-08-23T20:05:45

This is a road map to increasing maintenance costs.


Permalink
ubm_techweb_disqus_sso_-06f04ec49ca64a54ecc91375a128f950
2012-08-03T17:48:03

I have issues with that article. It is highly flawed, I think the author has just a little bit of bias and has a serious conflict of interest.

The first flaw is that the author seems to equate 'QA' as being nothing more than the traditional QC 'testing' phase of waterfall. If you changed just about every instance of 'QA' to 'tester/testing' in what he wrote, it might be a little closer to the mark. Yes, there is less need for 'testers' who go through a static set of test cases created by other more senior testers, and attempt to somehow 'test in quality' during a testing phase. That is very much reduced in an agile development environment. And that is not news, just read something like the book 'Agile Testing' (1/9/2009 Crispin & Gregory) and you will see that hey guess what, us QA folks already know this.

Secondly promoting TDD as a way to be able to get rid of your QA/Testers is just bunk. Not that TDD is not a great thing, but it is not some Agile Silver Bullet and it does not replace the value provided by having a skilled QA person on your team. It's not an us verses them thing either, as a tester I LOVE it when my team does TDD because it means I spend less time having to clear out stupid test blocking basic functionality bugs, and can get to finding the much more challenging and ugly bugs deeper in the system. It also means my devs are less likely to break things when they refactor or add new features.

TDD is but one practice to help developers better understand what they are building, know when they are done, and be able to refactor with confidence. But the tests it creates operate at only one level. Unit tests tell you your code works right, they do not tell you if it is doing the right work. You still want and need to look at integration, service and api interfaces, acceptance, usability, and a host of other things, none of which can really be accomplished via unit tests.

The 'test' role changes in agile, but is still very much needed. As agile itself continues to 'inspect and adapt' we are seeing new processes such as Spec by Example, ATDD, BDD, etc which all work best when skilled QA professionals are involved. 'testers' are now embedded within teams, working ahead, alongside, and after developers to define acceptance criteria, drive ambiguity out of stories, create test automation, do manual and exploratory testing, and also promote practices such as TDD, Peer Code Review, Static analysis, CI, etc etc which may not directly involve testers, but help to prevent or detect issues as early as possible.

That said, if you were just a 'checker' type tester, waiting for someone to 'throw code over the wall' to your testing team, you need to learn a lot more about test theory and true Quality.Assurance to work with an Agile team.

Agile needs MORE QA not less.. because you have to deliver high quality working software with each iteration. The Agile community knows this, and in fact a little less than two years go, the Gordon Pask Award (for recent contributions to Agile practice which deserve to be more widespread) was awarded to a pair of QA folks (Liz Keogh & Elizabeth Hendrickson). http://www.agilealliance.org/p...

Quality processes are like a multi-course dinner, each part serves it's purpose and brings value.. and just because someone can deliver you a killer TDD steak, does not mean you toss out the appetizer, beverages, soup, salad, and dessert and try to get by on a great steak alone.


Permalink
ubm_techweb_disqus_sso_-d920be14c2f4808f420323acda37884f
2012-08-01T07:26:46

QA shape may change but the assurance of code written will always be done by a resource, which will need time and cost now you can call that resource as QA resource or any XYZ name


Permalink
ubm_techweb_disqus_sso_-d920be14c2f4808f420323acda37884f
2012-08-01T07:18:07

After reading this post one thing which is for sure that the writer is missing the basic principal of QA and that is "If the QA person writes the code (Even Unit Test), Who will QA it". In my opinion and working with all the Software development models as a QA engineer in Agile you need more better QA engineers. As, if you need the project and product to run and want more projects you need them to be at the top class quality. I will say QA role is flourishing and need in market from better QA to Best is increased as the time lines are shrinking.
One more comment on the statement "It has been shown that unit testing can properly check 90% of code" i will like to know how much time it needs to implement and manage these test and who assures their reliability?

The shape of QA may Change but the assurance of implementation will always be done by some resource which will consume time and cost. Now you can call that resource as a QA resource or any XYZ


Permalink
ubm_techweb_disqus_sso_-f06d2d26c6ba0cf431015c5595f2c126
2012-07-31T23:39:36

Unit testing of anything, whether "Agile"ly produced or not, can only ever test the quality of the "grain" of system business function that it encompasses. Perfectly tested units can still deliver dysfunctional business systems..... "its just what we asked for but its not what we want".

QA is one of the more significant components in the systematic avoidance of that trap!


Permalink
ubm_techweb_disqus_sso_-60412f78602a9aef9edcb0bf64a04863
2012-07-31T15:47:32

As a designer, I do agree with the fact that unit-testing falls within the realm of the feature implementer, however our QA test team is still, and will remain, invaluable. The "real" testing comes with inter op testing with other components of the system or other nodes in the network. Unit testing is only a piece of the entire test process. I reject the hypothesis that Agile is a QA killer.


Permalink
ubm_techweb_disqus_sso_-ff0ca50dfc80740db7ff5e957a5c1dbf
2012-07-30T19:12:29

I spent a decade in QA (initially acting more like QC, but we morphed the job over time). The real meat of our job was attending design and code reviews. Wearing the users hat and trying to keep the developers from falling into group-think by asking those awful "what-if" questions was our major contribution. More unit testing is usually good, but the real case for QA lies up front, not in the testing after development (fun as that is). Any fool who thinks only unit testing is needed is just that: a fool.


Permalink
ubm_techweb_disqus_sso_-99125c3aab40614e0c172d544d335fc8
2012-07-29T05:31:40

Taking a page from the Industrial revolution, and how the focus of quality evolved in the context of mass produced products, People confuse Quality Assurance (QA) with Quality Control (QC). They are two very different things. QC is in essence testing. You can never "test" quality into any product. This is where QA comes in, and is meant to address all the issues and risks being written about, on challanges to creating "high quality" products. There will always be a Cost of Quality. Cost of Quality is effort to create a high quality product. Truth is, the effort does not change in an Agile project v/s a standard waterfall approach, Agile just uses the effort in more efficient activities that result in better quality products. Some of the well thought out steps to get it "all together" are in various bodies of knowlegde embedded into ISO, CMMi, LEAN, SIX SIGMA etc.. and these models understand the inherent limitations of human beings and team work and hence package the experience as a model to adopt to avoid all the risks we know will create poor quality products. In the end, to me agile means, do what "best" works for you. If that is in the end the truth to create high quality products, then quickly figure out what works for you, decide on it and agree to follow the rules. Improve the rules with each sprint or iteration. This IS, the essence of QA, and hence this role will always be required. QA will look for ways to eliminate QC, as it seeks optimization of cost. Let us not mix QA with QC


Permalink
ubm_techweb_disqus_sso_-41836e89a72c4bede3540193901f5b35
2012-07-26T10:56:07

Biased, short sighted and foolish. Unit testing is a good and necessary starting point for testing, but it is only the beginning. Unit testing does not find integration errors, just ask the big banks. Nor does it ensure the promised features are delivered, as we learned during the limp wrist-ed "Trust but verify" approach taken by our governments for the past few years. And, it is in no way helping the quality of the user experience.

We build ever larger systems where code quality is the integrity of the basic building blocks, it must be there or the systems we build will fail. But, a badly designed or improperly constructed system is going to fail for different reasons. Testing and quality assurance are our best weapons against these types of failures, certainly far better than denial and myopia.


Permalink
ubm_techweb_disqus_sso_-6099b4b2f819f1591e37f3381e3420bc
2012-07-26T07:30:31

The unit tests that developers can and should develop do not eliminate the need for

- test framework development and maintenance
- scenario/story testing
- usability testing
- performance/load testing
- scalability testing
- longevity (resource leak) testing
- regression suite maintenance
- code coverage metrics

The cost (including brand/reputation damage) of allowing a defect to reach users is enormous, and growing. This article's claim that unit testing eliminates much of the need for QA is ludicrous.


Permalink
ubm_techweb_disqus_sso_-5c4c554d692c22fcef4e317c2f9b5a59
2012-07-26T00:37:00

It seems like what you call "QA" we call Integration & Test. I'm not sure the objectives of the testing you speak of are the same. Code and Unit Test (CUT) is not the same as I&T testing which brings all the pieces together and ensures the system works as a whole. Neither CUT nor I&T are Quality Assurance (QA). QA is concerned with ensuring the testing conducted is adequate and that proper processes were followed in conducting both the SW development as well as the testing. I've spent the majority of my 33 year career as a software developer and system engineer trying to avoid QA's rigor, but have always recognized their role.


Permalink

2012-07-25T20:38:35

I don't think it is as much developers doing 'more testing', as I don't know any developers who "got lazy" thinking QA would test it, since doing so would have quickly gotten them downgraded in assessment and reviews for being sloppy in coding and not doing basic unit testing.

What has changed is more companies shipping software as 'Beta' quality, and being willing to ship buggier software to customers, because they know the customers will still continue to buy.

At least, 3 large software vendors, 2, recent -- both of them being backup software companies -- Genie, and Acronis had bugs so severe in them that it made their products unusable -- the latter so severe that I got a refund of downloaded software, the former was just so slow that it would have taken over a week to restore my data in the event of a data loss.

Nevertheless, both companies continue to try to sell me newer versions of SW... because full and incremental backup SW is not included for Windows 7 (full image of sysdisk only, OR no-executable backups of other disks)...either way, backups are incomplete (assuming the backup program will even run (which it doesn't, on my machine)... Customers are willing to buy *crap*. So why bother spending money on QA when the customers will do it for you?


Permalink
ubm_techweb_disqus_sso_-fa0e6417d5b8c7eac9d9d612c1eb8940
2012-07-25T19:16:56

So, to summarize previous comments, can we assume that this article is just an attempt in slightly covered up and sprinkled with 'agile'-buzzwords advertisement for the unit test product company whose CEO is the author of the article?


Permalink
ubm_techweb_disqus_sso_-8cd7880ec96977226fe55f87da548c96
2012-07-25T17:22:10

Unit tests are great for catching the bugs that you expect.

But doing away with QA means that the bugs you don't expect will not be caught. Which is fine if your user base doesn't expect finished software when you deliver, and is willing to perform unpaid test work.


Permalink
NowTourist
2012-07-25T15:51:00

"Agile development is about developers testing their own code. " -- well, I guess it's better than no testing at all, but probably not much.

It's probably a fact of the new marketplace (and the MBA mindset) that customers be responsible for QA. It's cheaper. The product isn't as good, but it is cheaper.


Permalink
ubm_techweb_disqus_sso_-c105197a07a6d943a43ed9b0cea04c7e
2012-07-25T14:33:59

As a developer, I have a long-standing rule that my testing doesn't count. Most if not all developers can be very biased on their testing, and not catch the simple errors.
I always love telling someone else to try and "break" my code. If they can, then I know what to improve and change. As a developer one can often be blinded by how it should work and not catch the extreme cases of insanity.
QA is suppose to be the last step before the customer gets the software. If the customer then breaks something they are usually less forgiving and you may have lost more....


Permalink
ubm_techweb_disqus_sso_-ee02416db8f4b6755655ea42620959cf
2012-07-25T13:58:52

I must agree with most of the previous comments. Quality cannot be something that is added, injected or checked at the end of the development process, but a component that belogs to each phase in the process. QA has no other alternatives as to become mentors of the scrum-team members, specially those acting (eventually) as internal quality contacts or quality responsibles.


Permalink
ubm_techweb_disqus_sso_-b457251c795ef1a7502b9e534a4666cb
2012-07-25T13:50:28

This guy is so out of touch! However I see he is CEO of a unit testing company, no wonder h is so biased. I am a tester myself, and I mean an agile tester! Oh software can be united tested to the wazoo, doesn't mean it is quality software. The quality of unit tests is dependent on each developer and we know developers mostly focus on the high level fruit, postive testing. Software only gets more complex, and with this complexity automated testing is a life saver and all manual testers should use this tool to their benefit, but to say manual testing is headed out the door I think is a little far fetched.


Permalink
ubm_techweb_disqus_sso_-a9629f002257a54d9b036b05244dbd2a
2012-07-25T12:16:45

I think the post forgets integration, performance, smoke and other tests that must be done for any quality system and they're done or at least guided by the QA people.


Permalink
ubm_techweb_disqus_sso_-0c53ec61a307cf811162cec7f45a6d0f
2012-07-24T17:21:55

To claim that any development team passes on fully working code to QA teams is a myth. It is well possible that code gets better each time developers get better tools, but even unit tests cannot prevent bad UX or other flawed design. The role of QA especially in Agile is morphing from checking off things on a list to becoming more involved in the design process. Therefore, QA needs to be more exposed to support (what do users want and need) as well as to the design aspect (how do we create what users want and need). QA can also take on a more 'visionary' role (what do users want and need, but users don't know yet that they want and need it).
I worked in several development teams that were not strictly following Agile. But they were also not strict waterfalls either. It was the best of both worlds, put a lot of thought in upfront and tweak as you go rather than be entirely static or always work on the fly.
In any case, the role of QA is not shrinking, if anything it is constantly morphing and growing. Not sure how the authors came to that conclusion, apparently they have neither worked in QA nor talked to anyone in QA during the past years.


Permalink
ubm_techweb_disqus_sso_-cfbb87a8026eaf7faa987314ab96d9f2
2012-07-24T17:20:20

TDD and peer programming increase the code quality, specially robustness, correctness, code readability and reusability, and this reduces the burden on QA. However unit testing has always been a programmer responsibility not QA's. QA tests the overall product quality rather than the source code quality. I think some kinds of testing like user interface testing, performance testing, documentation testing and configuration testing are done better by QA staff than programmers.


Permalink
ubm_techweb_disqus_sso_-9c052414ac9da9eb68a0f7ba526e1d76
2012-07-24T16:13:58

Yes "QA" is changing, and it always has. The idea of a static QA department, with static procedures, was always a bad idea, and if we developers are truly agile we will value people and teamwork and minimize the titles and departmental divisions. And yes, anything newish like Unit Testing will of course also fluctuate in importance depending on the context and environment and people. So your idea is OK, but statistics like "unit testing can properly check 90% of code" are suspect without a proper reference and description of context. I love unit tests, but can't often apply them very easily, because my context is highly interactive software where that pesky 10% is large indeed.


Permalink

Video