Channels ▼
RSS

Testing

Open Software's Thousand Eyes: A Misunderstood Claim


In the days when the open-source model needed to prove itself to be taken seriously by businesses, a frequently touted benefit was that the availability of source code made security reviews possible. And through these reviews, defects could be caught and repaired quickly. This benefit is sometimes referred to as Linus's Law, even though it appears to be formulated by Eric Raymond in his seminal book The Cathedral and the Bazaar. Raymond's articulation is: "Given enough eyeballs, all bugs are shallow." Other sources attribute it directly to Linus Torvalds, formulated as: "Given a thousand eyes, all bugs are shallow."

This claim, which I'll get into in a moment, was important at the time because it directly attacked a pervasive, countervailing fear that providing the source code to software was a security risk because it enabled hackers to find ways to find exploits.

The claim of many reviews, however, was problematic from the get-go, mostly because of the inference that the uninitiated drew from it — that there would in fact be thousands of eyes looking for bugs. Even in those early days, those of us in open source knew that was not going to happen. There would not be a thousand eyes scanning the source of the Apache Web server or of Subversion looking for defects or exploitable toeholds.

At best, if defects made their presence known, users could assist developers in tracking down the offending code and signaling the problem and, sometimes, the possible solution. That is, detectable bugs would indeed become shallower due to universal access to source code.

And this is pretty much how things have worked out in open source ever since. The vast bulk of the code is written by core committers. The galaxy of additional contributors (or in most projects, the constellation) generally provide bug reports, proposed patches, and occasionally code to a desired feature.

As Marc Fleury, who headed up the JBoss project, once expressed to me: "A principal value of open source to the project is the ability of having hundreds of testers and bug reports, often accompanied by diagnostic information."

Unfortunately, this important dynamic has been misunderstood by many people, who view open source as having claimed to provide better security because those "thousand eyes" would review the code. (And unfortunately, this claim is still occasionally made on behalf of open source.)

The disappointment that such review work does not happen has been given new life due to the two recent high-profile defects: the OpenSSL bug ("Heartbleed") and "shellshock" in bash.  

While few critics directly attack the idea of open source anymore, they do assail these projects for not finding vulnerabilities and defects earlier. As I've discussed in an earlier editorial, this view asks the wrong question on projects where the developers are unpaid. As to the larger OSS community, however, the reasons for the delay are the same as they would be for all software, open or closed: No one found it, or no one tested it, or the defect was not sufficiently symptomatic to be manifest. These answers are unremarkable because they are the same, regardless of the development model.

The difficulty in castigating open source for not having fixed longstanding defects is that we have no real idea how the track record for OSS compares with closed source. While we can count defects fixed in some closed-source packages, based on the updates and data the vendors publish, we rarely have the benefit of knowing how long a given defect has remained unfixed.

It's a shame that recent high-visibility bugs cast a shadow on open source, whereas their equivalents in closed source software suffer no such effect. But in fact, part of the problem is one of expectations and it lies with a misunderstanding about what open source could deliver ("1000 eyes") versus what it would reasonably deliver (the eyes of a few core developers).

— Andrew Binstock
Editor in Chief
alb@drdobbs.com
Twitter: platypusguy
Google+


Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.