Freaking out over the DBIR

Many in the community are upset over the recent "Verizon DBIR" because it claims widespread exploitation of the "FREAK" vulnerability. They know this is impossible, because of the vulnerability details. But really, the problem lies in misconceptions about how "intrusion detection" (IDS) works. As a sort of expert in intrusion detection (by which, I mean the expert), I thought I'd describe what really went wrong.

First let's talk FREAK. It's a man-in-the-middle attack. In other words, you can't attack a web server remotely by sending bad data at it. Instead, you have to break into a network somewhere and install a man-in-the-middle computer. This fact alone means it cannot be the most widely exploited attack.

Second, let's talk FREAK. It works by downgrading RSA to 512-bit keys, which can be cracked by supercomputers. This fact alone means it cannot be the most widely exploited attack -- even the NSA does not have sufficient compute power to crack as many keys as the Verizon DBIR claim were cracked.

Now let's talk about how Verizon calculates when a vulnerability is responsible for an attack. They use this methodology:

1. look at a compromised system (identified by AV scanning, IoCs, etc.)
2. look at which unpatched vulnerabilities the system has (vuln scans)
3. see if the system was attacked via those vulnerabilities (IDS)

In other words, if you are vulnerable to FREAK, and the IDS tells you people attacked you with FREAK, and indeed you were compromised, then it seems only logical that they compromised you through FREAK.

This sounds like a really good methodology -- but only to stupids. (Sorry for being harsh, I've been pointing out this methodology sucks for 15 years, and am getting frustrated people still believe in it.)

Here's the problem with all data breach investigations. Systems get hacked, and we don't know why. Yet, there is enormous pressure to figure out why. Therefore, we seize on any plausible explanation. We then go through the gauntlet of logical fallacies, such as "confirmation bias", to support our conclusion. They torture the data until it produces the right results.

In the majority of breach reports I've seen, the identified source of the compromise is bogus. That's why I never believed North Korea was behind the Sony attack -- I've read too many data breach reports fingering the wrong cause. Political pressure to come up with a cause, any cause, is immense.

This specific logic, "vulnerable to X and attacked with X == breached with X" has been around with us for a long time. 15 years ago, IDS vendors integrated with vulnerability scanners to produce exactly these sorts of events. It's nonsense that never produced actionable data.

In other words, in the Verizon report, things went this direction. FIRST, they investigated a system and found IoCs (indicators that the system had been compromised). SECOND, they did the correlation between vuln/IDS. They didn't do it the other way around, because such a system produces too much false data. False data is false data. If you aren't starting with this vuln/IDS correlation, then looking for IoCs, then there is no reason to believe such correlations will be robust afterwards.

On of the reasons the data isn't robust is that IDS events do not mean what you think they mean. Most people in our industry treat them as "magic", that if an IDS triggers on a "FREAK" attack, then that's what happen.

But that's not what happened. First of all, there is the issue of false-positives, whereby the system claims a "FREAK" attack happened, when nothing related to the issue happened. Looking at various IDSs, this should be rare for FREAK, but happens for other kinds of attacks.

Then there is the issue of another level of false-positives. It's plausible, for example, that older browsers, email clients, and other systems may accidentally be downgrading to "export" ciphers simply because these are the only ciphers old and new computers have in common. Thus, you'll see a lot of "FREAK" events, where this downgrade did indeed occur, but not for malicious reasons.

In other words, this is not a truly false-positive, because the bad thing really did occur, but it is a semi-false-positive, because this was not malicious.

Then there is the problem of misunderstood events. For FREAK, both client and server must be vulnerable -- and clients reveal their vulnerability in every SSL request. Therefore, some IDSs trigger on that, telling you about vulnerable clients. The EmergingThreats rules have one called "ET POLICY FREAK Weak Export Suite From Client (CVE-2015-0204)". The key word here is "POLICY" -- it's not an attack signature but a policy signature.

But a lot of people are confused and think it's an attack. For example, this website lists it as an attack.

If somebody has these POLICY events enabled, then it will appear that their servers are under constant attack with the FREAK vulnerability, as random people around the Internet with old browsers/clients connect to their servers, regardless if the server itself is vulnerable.

Another source of semi-false-positives are vulnerability scanners, which simply scan for the vulnerability without fully exploiting/attacking the target. Again, this is a semi-false-positive, where it is correctly identified as FREAK, but incorrectly identified as an attack rather than a scan. As other critics of the Verizon report have pointed out, people have been doing Internet-wide scans for this bug. If you have a server exposed to the Internet, then it's been scanned for "FREAK". If you have internal servers, but run vulnerability scanners, they have been scanned for "FREAK". But none of these are malicious "attacks" that can be correlated according to the Verizon DBIR methodology.

Lastly, there are "real" attacks. There are no real FREAK attacks, except maybe twice in Syria when the NSA needed to compromise some SSL communications. And the NSA never does something if they can get caught. Therefore, no IDS event identifying "FREAK" has ever been a true attack.

So here's the thing. Knowing all this, we can reduce the factors in the Verizon DBIR methodology. The factor "has the system been attacked with FREAK?" can be reduced to "does the system support SSL?", because all SSL supporting systems have been attacked with FREAK, according to IDS. Furthermore, since people just apply all or none of the Microsoft patches, we don't ask "is the system vulnerable to FREAK?" so much as "has it been patched recently?".

Thus, the Verizon DBIR methodology becomes:

1. has the system been compromised?

2. has the system been patched recently?

3. does the system support SSL?

If all three answers are "yes", then it claims the system was compromised with FREAK. As you can plainly see, this is idiotic methodology.

In the case of FREAK, we already knew the right answer, and worked backward to find the flaw. But in truth, all the other vulnerabilities have the same flaw, for related reasons. The root of the problem is that people just don't understand IDS information. They, like Verizon, treat the IDS as some sort of magic black box or oracle, and never question the data.

Conclusion

An IDS is wonderfully useful tool if you pay attention to how it works and why it triggers on the things it does. It's not, however, an "intrusion detection" tool, whereby every event it produces should be acted upon as if it were an intrusion. It's not a magical system -- you really need to pay attention to the details.

Verizon didn't pay attention to the details. They simply dumped the output of an IDS inappropriately into some sort of analysis. Since the input data was garbage, no amount of manipulation and analysis would ever produce a valid result.

---

False-positives: Notice I list a range of "false-positives", from things that might trigger that have nothing to do with FREAK, to a range of things that are FREAK, but aren't attacks, and which cannot be treated as "intrusions". Such subtleties is why we can't have nice things in infosec. Everyone studies "false-positives" when studying for their CISSP examine, but truly don't understand them.

That's why when vendors claim "no false positives" they are blowing smoke. The issue is much more subtle than that.