Thursday, September 25, 2014

Many eyes theory conclusively disproven

Just because a bug was found in open-source does not disprove the "many eyes" theory. Instead, it's bugs being found now that should've been found sometime in the last 25 years.

Many eyes are obviously looking at bash now, and they are finding fairly obvious problems. It's obvious that the parsing code in bash is deeply flawed, though any particular bug isn't so obvious. If many eyes had been looking at bash over the past 25 years, these bugs would've been found a long time ago.

Thus, we know that "many eyes" haven't been looking at bash.

The theory is the claim promoted by open-source advocates that "many eyes makes bugs shallow", the theory that open-source will have fewer bugs (and fewer security problems) since anyone can look at the code.

What we've seen is that, in fact, very few people ever read code, even when it's open-source. The average programmers writes 10x more code than they read. The only people where that equation is reversed are professional code auditors -- and they are hired primarily to audit closed-source code. Companies like Microsoft pay programmers to review code because reviewing code is not otherwise something programmers like to do.

From bash to OpenSSL to LZO, the evidence is clear: few eyes are looking at open-source.


DimArchon84 said...

Yes and no. I agree it's proven that few eyes aren't looking at open source consistently, but the caveat being that when something does go wrong, lots of eyes do start looking.

If Bash had been open source, the first patch may have been released and people would have patched and then... trusted things were fixed? They can't see the source so that's the best they can do.

It's open source though, so the first patch was released and every eye went to it, searching for more exploits, finding them, fixing more bugs...

DimArchon84 said...

*if Bash had been closed source

clarkmr4 said...

The 10x number is interesting. Reference?

Security Leaders Group said...

Should government cyber centers offer bug bounties for open source? Would that increase the number of eyes?

tim said...

When I read code, it's either (1) because I need to know how to interface with it; or (2) to try to learn something from it.

In either case, I'm not going to spend long trying to make sense of badly written code. If I need to interface with it, I can decide it's not possible and go find an alternative. If I'm trying to learn from it, I'll go learn from something better.

I don't read code specifically for the purpose of finding bugs. That's boring and takes time away from my projects.

Ewing Fox said...

If the "Many Eyes" argument is being called into question, what is terrifying me today is how many of my vendors (big, big companies) were completely unaware of this exploit or the ramifications. One of my vendors said, well, corp xyz is always on top of things, so I'm sure they'll let us know... I wasn't satisfied with this answer so I called the operation, pretended to be a vendor, and discovered that as of 1:00pm,(EST) no one had even heard of it. A customer, pretending to be an inside vendor, alerted an international company to the problem.

I'll take the intensely critical, lurking, trolling, snippy, RTFM'ing community of many eyes any day.

Ryan Pavlik said...

"few eyes are looking at open-source"

Few eyes are looking at bash. Bash is open source. Therefore few eyes are looking at open source. Logic much?

And, even if there are, that's more eyes than looking at closed source.

Burrito. said...

@Security Leaders Group

Any large entity (including governments) using an open source project should consider funding its development, either through the respective Foundations these projects belong to, or by directing developers paid by them to review and develop that code. It is in their interests to do so.

On the other hand, those Foundations should probably start running formalized bug bounty programs using funding they receive.

Mark Fisher said...

This doesn't mean people didn't read the code, we know that the exploit was already in use.

Greg Nation said...

> This doesn't mean people didn't read the code, we know that the exploit was already in use.

If this is true, it's an important point because it debunks the claim that the many eyes theory has been conclusively disproven. Some of the many eyes did in fact spot the bug. Those eyes just decided to exploit it rather than fix it, which is an entirely different issue altogether. The many eyes theory states that all bugs are shallow. It doesn't say anything about fixing bugs, just finding them.

Scott Lunsford said...

There already are government agency's paying bounty for open source exploits along with any other exploits. Vulnerability exploits now have too much value to "the man" when they are kept secret.

Admittedly, as a penetration tester, I am guilty of exploiting vulnerabilities and pointing them out to my client, while crossing my fingers that somehow an advisory and vendor patch doesn't follow. It's called laziness. Why keep hunting bugs when you can just keep the same ones around for 20 years?

Unknown said...

"Companies like Microsoft pay programmers to review code..."

...when it's seen as beneficial to those companies' bottom line. How often it's seen as such varies widely among different companies and within different groups of the same company.

Jim said...

Linus's Law says that if you have enough eyeballs, your bugs are shallow. It doesn't say You will have enough eyeballs. It simply says that it's nice if you do.

You can only really disprove it if you can prove the actual presence of "enough" eyeballs on the code.

There's a bit of self-fulfilling prophecy here. "Enough" is a weasel word because no matter how many eyes are on the code, if someone misses a bug... well, you clearly didn't have enough eyeballs. Right?

What we're actually proving is that we don't have enough eyeballs on the code. :D

Luca Carettoni said...

Few eyes are looking at open-source, but at least we have the option to look at src code and potentially fix bugs

Andrea Margiovanni said...

well, I totally agree.