Those who don't know the state-of-the-art are doomed to repeat it

I was reading this article about Microsoft's "Network Inspection Engine" or "NIS". It attempts to solve the problem of false-positives in IPS by using more application level protocol analysis that keeps track of protocol state, message structure, and message context.

Welcome to state-of-the-art, 1999, when I released the first IPS, BlackICE Guard (now sold as IBM Proventia). McAfee's and Palo Alto Networks' IPS products also do a lot of protocol analysis. A lot of bad products give a bad reputation to the industry, but that doesn’t mean there aren't good products.

Even something like Snort does a good job with protocol analysis these days. I say "even" because Snort has one of the worst reputation for false-positives. But that's not really the fault of the internal technology so much as the open-source nature of Snort, that contains a lot of user-contributed signatures of dubious quality, and which must serve multiple interests. Most people who download free Snort want a system tuned to be "chatty" so that they can get instant visibility to what's going on across their network. [*** see below]

In contrast, if you take a look at SourceFire's ".so signatures" for Snort (which are written in C code to do more advanced protocol analysis), you see a technology that looks exactly what I did with BlackICE/Proventia. The ones I've looked at (those dealing with MS-RPC threats) look really good. Indeed, if you don't have an MS-RPC parser and something like .so signatures(e.g. TippingPoint), then you have no adequate defense against MS-RPC attacks. There's no way to tune a signature written with a regular expression that can both block the attack without evasions as well as not have false-positives.

If you think this new "Network Inspection Engine" technology is great, then buy a box from IBM or McAfee or Palo Alto Networks or SourceFire – people who have been running that technology for much longer.

Note that I'm not recommending a specific product here, just a class of products. Except for IBM Proventia, I've never seen them in action, and I haven't seen my own product in action for the last 5 years (I quit right before IBM acquired the company). Regardless of whether you have the right technology, the products still depend upon the team writing the signatures. You can write surprisingly bad signatures with great technology.

---

By the way, every year multiple masters/doctor thesis's are written describing a new way of detecting hacker activity without signatures. They are invariably identical to some 30 year old failed solution to the problem. This is the inverse problem: those who are unaware of the failures of the past tend to repeat them.

---

Chatty: Few really understand what a "false-positive" is. Many define it as a "false-result", but the way people really use it is to mean "something I didn't want".

For example, Proventia has a signature for "SNMP_Public" that triggers whenever it sees an SNMP packet with the password "public". You should never see such packets on a well run network. the problem is that no network is strictly "well run", and such packets appear regularly.

So is "SNMP_Public" a false-positive? A better description is "chatty": it's true, but unwanted.

The way we solve that is to turn it off by default when we ship the product. Some IDS/IPS products ship that way, to be silent when you connect them to a well run network. A lot of customers don't like that, feeling that there ought to be more, so they turn on everything, then disable the chatty signatures one by one. Other products ship in chatty mode since they expect customers to go through this tuning step anyway.

The point about "chatty" is that while everyone complains about IDS false-positives, they actually don't know the cause of them. Often, it's their own decisions rather than the vendor's.