Wednesday, May 04, 2016

Vulns are sparse, code is dense

The question posed by Bruce Schneier is whether vulnerabilities are "sparse" or "dense". If they are sparse, then finding and fixing them will improve things. If they are "dense", then all this work put into finding/disclosing/fixing them is really doing nothing to improve things -- because there are still an unending amount of unpatched bugs still to be discovered.

I propose a third option: vulns are sparse, but code is dense.

In other words, we can secure specific things, like OpenSSL and Chrome, by researching the heck out of them, finding vulns, and patching them. The vulns in those projects are sparse -- disclosing/fixing those bugs improve security.

But, the amount of code out there is enormous, considering all software in the world. And it changes fast -- adding new vulns faster than our feeble efforts at disclosing/fixing them. Finding and disclosing bugs in random software (like IoT) devices has little marginal impact on the total bugs out there.

So measured across all software, no, the secure community hasn't found any significant amount of bugs. But when looking at specific critical software, like OpenSSL and Chrome, I think we've made great strides forward.

More importantly, let's ignore the actual benefits/costs of fixing bugs for the moment. What all this effort has done is teach us about the nature of vulns. Critical software is written today in a vastly more secure manner than it was in the 1980s, 1990s, or even the 2000s. Windows, for example, is vastly more secure. Sure, others are still lagging (car makers, medical device makers, IoT), but they are quickly learning the lessons and catching up. Finding a vuln in an iPhone is hard -- so hard that hackers will earn $1 million doing it from the NSA rather than stealing your credit card info. 15 years ago, the opposite was true. The NSA didn't pay for Windows vulns because they fell out of trees, and hackers made more money from hacking your computer.

My point is this: the "are vulns sparse/dense?" puts a straight-jacket around the debate. I see it form an orthogonal point of view. Vuln disclosure helps specific software, and the overall way to create software, even while code is so "dense" that disclosure makes no dent in the total number of vulns in the universe.


ThingFish said...

Have you seen Andy Ozment's Milk or Wine paper? It would seem to back up your theory.

dizkonekdid said...

I like looking at it from an egocentric point of view or in the case of aiding someone, what does it mean to them? This is the reason why application infrastructure has not moved at a very fast pace towards new SDN or cloud applications that quickly. The risk reward is still hockey-stick.

Good article though.

Security Leaders Group said...

In some cases vulns are very dense. "In one experiment run by the Air Force, three million lines of proprietary code were scanned for vulnerabilities. They found one “software vulnerability” per eight lines of code, one “high vulnerability” per 31 lines of code, and one “critical vulnerability” per 70 lines of source code."