Channels ▼

Andrew Koenig

Dr. Dobb's Bloggers

Debugging by Hypothesis

December 13, 2012

Last week's article got an unusually large number of comments, so I'm going to continue this theme with another example. This time, the failing program was a compiler.

More Insights

White Papers

More >>

Reports

More >>

Webcasts

More >>

There's a technical term for people who claim that their programs don't compile because of a compiler failure rather than a bug in their own programs: arrogant. In this particular case, though, this arrogance was justified, because I would compile a program and get a bunch of error messages, and then I would compile exactly the same program again without any problems at all. Moreover, the error messages seemed to have nothing to do with my program: They would complain about invalid characters that my program most definitely did not have.

It's particularly hard to find out what is wrong when doing the same thing twice gives different results. Even worse was that the failures were rare: The spurious error messages would occur only about one time in 10. Moreover, I had nothing to do with the compiler in question, so I couldn't fix the bug even if I knew what it was. On the other hand, I needed to be able to compile my programs. What could I do?

I decided that my only chance of getting anywhere was to figure out how to reproduce the problem. If I could say to the compiler folks: "When you do X, Y, and Z, your compiler occasionally produces spurious error messages," then I might be able to get them to fix it.

This particular compiler had several phases, which were run from a shell script. As a result, the logical first step was to figure out which phase was failing. Once I had done that, I could easily change the shell script to capture a copy of the input to that phase. My hope was that I could run just that phase with that particular input, thereby provoking a failure. I would then be able to give this input file to the compiler group and tell them that their compiler occasionally, but not always, failed with this particular input.

This is probably a good time to take a step back and look at the principles behind this strategy.

  • An important early step in debugging any problem is to learn how to make it fail.
  • If you can divide a failing program into several parts (compiler phases in this case), and you can show that the failure is happening in a particular part, the problem will be easier to find.
  • If someone else is going to be fixing the problem you have found, it is important to bundle up all the information needed to produce a failure.

These principles seem straightforward enough. However, an important one is missing:

  • Once you have bundled up the information needed to produce a failure, you must verify that your information actually does produce a failure.

I would have been embarrassed indeed if I had not realized this last principle. For when I ran the failing compiler phase on the input file that I had captured from a prior failure, it worked perfectly. Every time. Apparently, something about the act of capturing the input file caused the problem to go away.

I invite readers to speculate about what might cause this state of affairs before I continue the story next week.

Related Reading






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.
 


Video