Static source code analysis for bug finding ("static analysis" for short) is the process of detecting bugs via an automated tool that analyzes source code without executing it. The idea goes back at least to Lint, which was invented at Bell Labs in the 1970s, but static analysis has undergone a revolution in effectiveness and usability in the last decade. The initial focus of static analysis tools was on the C and C++ programming languages. Such tools are particularly necessary given C/C++'s notorious flexibility and susceptibility to low-level bugs. More recently, tools have flourished for Java and/or Web applications; these are needed because of the prevalence of easily exploitable network vulnerabilities. When using such tools, it is all too easy to deploy them in a way that looks good superficially, but misses important defects, shows many false positives, and brings the tool into disrepute. This article is a guide to the process of deploying a static analysis tool in a large organization while avoiding the worst organizational and technical pitfalls.
Leading commercial static analysis tools with which I am familiar include Coverity, Fortify (now owned by Hewlett-Packard), and Klocwork. Klocwork and Coverity both initially focused on C/C++, although they came from opposite origins: Klocwork from the telephone equipment company Nortel, and Coverity from Stanford University. Fortify's initial focus was on security for Web applications in languages such as Java and PHP. All three companies are now encroaching on each others' territories, but it remains to be seen how well they will do outside of their core competencies.
An excellent, free, but limited, academic static Java byte code analysis tool is FindBugs. Its lack of an integrated database for defect suppression makes its large-scale use difficult in sizable organizations, but its use by individual developers within the Eclipse development environment can be extremely valuable. Similarly, recent versions of Apple's Xcode and Microsoft's Visual Studio development environments contain integrated static analysis tools for C/C++. These are useful for finding relatively shallow bugs while an individual developer is writing code; their short feedback loop bypasses the difficulties of broader deployment of tools that perform deeper analysis. A longer list of tools is provided in Figure 1 (which is taken from "Magic Quadrant for Static Application Security Testing," by Gartner Inc.), albeit with a strong bias towards security and adherence to Gartner's strategy recommendations.
There is no general-purpose introductory textbook on the subject; the best general introduction is a short article by Dawson Engler, the inventor of Coverity, et al. Two of the leaders at Fortify have written an introductory textbook, but it focuses primarily on their tool and on security, and skimps on key static analysis concepts. There is a rigorous academic textbook on the more general concept of static analysis, but it preceded the revolution in static analysis for bug finding.
Getting Started: The Politics
The first question to ask before deciding to do static analysis in an organization is not what tool to buy, nor even whether you should do static analysis at all. It's "why?"
If your purpose is genuinely to help find bugs and get them fixed, then your organizational and political approach must be different from the more usual, albeit unadmitted, case: producing metrics and procedures that will make management look good. (Fixing bugs should actually be your second goal: An even higher goal is preventing bugs in the first place by making your developers learn from their mistakes. This also contraindicates outsourcing the evaluation and fixing of defects, tempting though that may be.)
Political Issues to Settle in Advance
Get buy-in from your testing/quality assurance department. They must support the project and will have authority over quality-related issues, even if they inconvenience the other stakeholders. Quality has a much smaller constituency than the schedule or the smooth running of internal procedures, but it must be the final arbiter for crucial quality-related decisions (see chapter 22 of Joel On Software for more on this topic).
Give some thought to what part of the organization, if any, should be in charge of running the tool once it's set up. If your organization has a tools team, it might seem the obvious owner, but this does need careful consideration. Static analysis for bug finding is probably not your organization's core competency, and you will need to worry about the Iron Law of Bureaucracy: Your tools team's institutional interest will be in the smooth running of the tool, not in the messy changes necessary for finding bugs. Even if you're reluctant to outsource the rest of the process, administration and configuration may be more flexible if done by external players rather than an internal team with its own interests, habits, and procedures. It may also be more productive to hire an expensive consultant for a few hours, rather than a lesser-paid internal resource full-time; an external resource may be more flexible and less prone to establishing an entrenched bureaucracy.
The conventional wisdom is that getting most developers to use a static analysis tool requires a high-ranking management champion to preach its benefits, ensure that the tool is used, and keep the focus on finding bugs and getting them fixed. The flip side to this is that any attempt to herd cats (or to get programmers to adhere to best practices) will cause a backlash. Your tool must withstand scrutiny from developers looking for excuses to stop using it.
Get buy-in from engineering that they will make time in the schedule to review and fix bugs found, even if they are disinclined to do so, which they will be once they see the first false positive. (Or even the first false false positive; more on this later.) Ensure that it's not the least-effective engineers whose time is allotted for reviewing and fixing static analysis bugs (more on this momentarily). You'll also need agreement from the security team that sales personnel will get access to your real source code; more on that is also to follow.
Smart Programmers Add More Value and Subtract Less
Handling static analysis defects is not something to economize on. Writing code is hard, finding bugs in professional code should be hard, and evaluating possible mistakes in alleged bugs is even harder. Learning to evaluate static analysis defects, even in a developer's own code, requires training and supervision. It is necessary to tread delicately around the polite pretense that the code owner is an infallible authority on the behavior of that code. Misunderstandings about the actual behavior of unsigned integers and assertions are, for instance, regrettably common in my experience.