Channels ▼

Christopher Diggins

Dr. Dobb's Bloggers

Unit Testing as the Last Resort for Code Verification

November 07, 2009

I love code which is stable and rigorous, I just don't feel that unit testing is the most effective way to achieve it. 

I stumbled upon the following question posted on StackOverflow, "What's your most controversial programming opinion? ", and it made me think about my views on unit testing. 

I have mixed feelings on unit testing. I do use it often, but I find that it does not give me sufficient peace of mind, compared to other code verification techniques. The following is my list of code verification techniques in order of importance:

  1. System testing
  2. Use the compiler's type checker
  3. Use assertions 
  4. Code reviews
  5. Unit testing
First, any code verification technique is pointless without system testing. While it may seem almost obvious to include, it does happen where people get caught up in the minutiae of code verification and overlook the big picture of whether or not a system is behaving as intended. 
My preferred code verification technique is to leverage the language's type system to express assumptions, requirements, and  invariants. It is nice to know that if something compiles, then it is correct. Some examples of how this can be done are:
  • using unsigned types instead of signed types so you don't have to worry about non-negativity. Consider for example the example of using an unsigned type instead of a signed type in a square root function.
  • using id types instead of integers to identify objects. I recently ran into problems in my code, because I violated this rule. I confused the index of a vertex, with the index of an index array into the vertex array. The fix was easy: use a new VertexID type.   
  • using smart pointer classes with the appropriate semantics. E.g. null-checking pointers, deletion prevention pointers, pointers with ownership semantics, etc. 
In this bucket I also include any static contract verification tools like those found in Spec# and Eiffel.  This is because the techniques of type checking and contract verification are closely related, and both based on theorem proving techniques.
When the type system can't be used easily, I find that the next most powerful code verification technique is to use assertions to check assumptions, requirements and invariants at run-time. Assertions turn regular system testing into a much more powerful tool for uncovering design errors and potential defects. In addition assertions occur directly in the code, as a kind of documentation that is useful during code reviews.
Code reviews are another technique which I find is more powerful than unit testing, if it is done properly. In order to really leverage the power of a code review, code has to be written so that verification is as simple as possible. I will often sacrifice performance and ignore any popular idiom of the week  if it means making my code's meaning obvious and unambiguous. 
When I find unit testing to really become valuable as a tool, is when code can't be proven correct by the compiler or fully verified through assertions, and that a code review can't give a high confidence that the code is correct because of inherent complexity. 


Related Reading

More Insights

Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.