450 Million Lines Of Scanned Software Code Can’t Be Wrong

The 2012 Coverity Scan Open Source Report arrives this month from the prominent development testing company. The report details the analysis of more than 450 million lines of software code through the firm’s own scanning service.

NOTE: This scanning service, which began as the largest public-private sector research project focused on open source software integrity, was initiated between Coverity and the U.S. Department of Homeland Security in 2006. It is now managed outright by Coverity.

Key findings this year suggest that code quality for open source software continues to mirror that of proprietary software. Defect density (defects per 1,000 lines of software code) is a commonly used measurement for software quality.

Coverity's analysis found an average defect density of .69 for open source software projects that leverage the firm's own scan service. It also found an average defect density of .68 for proprietary code developed by the firm's own enterprise customers.

Both have better quality as compared to the accepted industry standard defect density for good quality software of 1.0.

Covertity states, "As projects surpass one million lines of code, there's a direct correlation between size and quality for proprietary projects, and an inverse correlation for open source projects.

Proprietary code analyzed had an average defect density of .98 for projects between 500,000 – 1,000,000 lines of code. For projects with more than one million lines of code, defect density decreased to .66, which suggests that proprietary projects generally experience an increase in software quality as they exceed that size.

Andy Chou, cofounder and CTO for Coverity, points out that open source projects with between 500,000 – 1,000,000 lines of code, however, had an average defect density of .44, while that same figure increased to .75 for open source projects with more than one million lines of code, marking a decline in software quality as projects get larger.

"This discrepancy can be attributed to differing dynamics within open source and proprietary development teams, as well as the point at which these teams implement formalised development testing processes," said Chou.