Static Analysis and Systematic Testing
Again, safety-critical developers are required to demonstrate coverage. There are of course different kinds of coverage, and the risk the code carries dictates which kind of coverage is required. In the DO-178B Standard for aviation, the riskiest code requires 100-percent modified condition/decision coverage (MCDC). The next two most risky classes require 100-percent decision coverage and statement coverage, respectively. The least risky code, such as the inflight entertainment system, has no coverage requirements at all.
Most developers are familiar with statement coverage, and anyone who has tried to test with 100-percent statement coverage knows how difficult it is to achieve. Decision coverage is more stringentit requires that all branches in the control flow are taken. You must make test cases that force all your conditional expressions to evaluate both true and false. MCDC coverage is stronger still, and means that all subexpressions in a conditional should be evaluated to both true and false. Consider what this means in terms of the number of test cases for this code snippet:
if (a || b || c) foo();
Table 1 shows the number of test cases required to achieve full coverage for each coverage criterion, and example values required of the inputs.
|Statement (1 case)||1|||||
|Decision (2 cases)||1|||||
|MCDC (4 cases)||1|||||
Clearly, achieving full coverage is nontrivial. What can make it extremely frustrating is if it is fundamentally impossible. If your program contains unreachable code, then you cannot even get full statement coverage, never mind decision or MCDC. Similarly, if your code contains redundant conditions (conditions that are either always true or always false), then you might achieve statement coverage, but not decision coverage.
A developer may spend hours trying to develop a test case before it becomes evident that it is impossible. If you know going in that this is the case, then you don't have to waste time on it.
In Figure 3, the value of variable rest (which is an unaliased automatic integer) must be at least 3 by the time you get to line 12. The decrement means it is at least 2 at the comparison, so the condition never evaluates to false. The following line 13 is also redundant by the same reasoning, and appears in a different report. This is a fairly simple example, and many humans would have spotted this in a code review because all the relevant expressions are in the same vicinity. If the bodies of the conditionals had been larger or if the conditions themselves had been embedded in a function or a macro, then the weakness would not have been as easy to spot. However, the static-analysis tool would not have been fooled.
These kinds of issues correlate well with bugs, too. Consider Figure 4, in which the column on the left shows the condition that cannot be satisfied: pb->type can never be equal to 3. This value comes from a call to get_type(); see Figure 5. The error is on line 30 and is clearly a typothe programmer missed the final X.
Bugs are a plague on our profession and bad for business. That's why advanced static-analysis tools should be in every programmer's toolbox, and not just for safety-critical systems. They help find real errors, and reduce cost.