d3tm4r a.k.a Kamerazukleber
Still missing in Snort: inclusion of HTTP response codes in alerts & appropriate prioritization.
This appears to be a simple feature, but it's actually quite complex to do right. I'll explain it here. I implemented this feature back in 1998 in the "BlackICE" IPS (now sold as IBM Proventia), and it works really well.
The HTTP response code is something like "200 Ok" or "404 Not Found". If the attack was sent to a server, looking at the response from the server can indicate whether the attack succeeded or failed.
You'll notice a problem here: the response code comes after the intrusion. Should the IDS report a second event for the response code, or should it hold onto the intrusion event for a couple seconds waiting to see if there is a matching response code it can report with the intrusion?
The BlackICE solution was to do both. Each event is marked with a unique ID. The intrusion on the request side is reported first. Then, when the response code is seen, a second intrusion event is reported with the same unique ID as the initial event.
What happens is that the system "coalesces" the event. Events are held for a time on the sensor in an outgoing queue, arrive on the console and are stored in an incoming queue, and are then stored in a database. The update event chases the initial event. If it catches up in the outgoing queue, the event is coalesced on the sensor. If it catches up on the console's incoming queue, it's coalesced there. Otherwise, it's coalesced in the database. The user-interface (which is a front-end on top the database) will also coalesce events.
My point here is to point out that this simple change, depending on how you implement it, can affect the entire ecosystem. It was a major change for ISS when they bought my company and replaced their RealSecure with BlackICE and renamed it Proventia. BlackICE was highly modular, so swapping in new brains into the sensor was a trivial task. Redefining what it meant to handle events, though, took longer to integrate.
Another problem is how you report the data. BlackICE was a protocol-analysis system rather than a pattern-matching system. What that means is that, like Wireshark, it would pull out the contents of fields and report them along with the event. The number of protocol fields decoded by Snort is fixed at a small number (less than 20). The number of protocol fields reported in BlackICE events number in the thousands, and new ones appear every month. Snort defines it's fields statically, whereas BlackICE has space for arbitrary
Again, it appears that adding HTTP response codes requires a change in the entire Snort ecosystem.
Finally, there is a difficulty about what, precisely, the response codes mean. For some events, a "200 ok" means the attack succeeded. For others, "200 ok" means the attack failed. In still others, "200 ok" is inconclusive: ColdFusion always returns "200 ok" regardless of what happens. For other events, any HTTP response code means the attack failed, and a RST indicates the attack probably succeeded.
When adding HTTP signatures to BlackICE, only 20% changed their priority based on the HTTP response code. For the rest, we simply reported the HTTP response code as part of the event, and let the analyst on the console deal with it. For example, if the analyst saw a large number of "directory traversal" events, the analyst could sort by response code to see which ones succeeded and which ones failed.
Conclusion
You can actually do the basic feature today with Snort. The latest version allows rules to be triggered on HTTP response codes that can be dynamically enabled after another event fires. It's not pretty, but for specific HTTP events where you absolutely must know the response code, the technique will work. It's one of the extraordinary things about Snort: it's flexibility means that somebody might figure out how to do something that somebody else else thinks is impossible.
But to do it right (as I define "right"), you need to redesign the entire Snort ecosystem. Event coalescing and decoration, they way BlackICE/Proventia does it, is extraordinarily useful in so many ways, so I recommend that Snort make the change.
What would the change look like?
A simple solution would be to define a new type for the "extra data" field, where the content of that field will consist of 8-bit "name length" followed by a name, followed by "value length", followed by the value data, followed by yet more name/value pairs. This would fit neatly into the Snort output without disturbing the downstream ecosystem.
The next step would be to start decorating events. For example, for those protocols that have a "username", like SMB or FTP or HTTP or MSRPC, parse that username, save it per TCP connection, and when events are reported, include the "username" in this extra data.
Now that you have arbitrary name=value pairs in events, think of ways to use them. For example, consider the following:
alert tcp any any -> any any (msg:"Extract Foo"; content: "foo="; pcre:"foo=(?P=<BAR>[A-Z]*)";)
What this arcane magic means is search for the pattern "foo=" in a TCP stream, then extract the following upper-case text, then report that text with the name/value pair BAR.
Ok, now that you're producing events with name/value pairs, the rest of the ecosystem has to catch up. This largely means a massive change to the database schemas to add a table with the columns "EVENTID=int, NAME=varchar, VALUE=blob".
In the user interface, you now need to allow people to "inspect events" to view any name/value pairs attached to it. You also need to be able to display table of events, showing the different values of the name/value pairs. You also need to be able to search for events based upon the name/value pairs, such as:
SELECT * FROM events WHERE event_id IN (SELECT event_id FROM namevalue WHERE name='BAR');
I'd love to hear you thoughts on why this helps the analyst. The response code comes after the attack has happened and in the case of success can be made to be any code desired. In the case of failure it can't be trusted as such.
ReplyDeleteIt is much the same as relying on X-forwarded-for as an indicator of anything other than a proxy was potentially used.
I understand the practical aspects and that necessary compromises are made for efficiency but the same effect of capturing and making available the response is possible by using the tag: keyword without breaking the analyst mindset.
Response code is an early sign that something wrong is happening with the web app. They are very common in SQL Injection attempts when the attacker is trying different variations of attack string before finally finding the right one. Typically a WAF has a default rule to alert on suspicious response code, it can be correlated with other elements like a deviation from usual input type on an HTTP parameter.
ReplyDeleteThere is never a single source of information to point at all attacks, this one is just an additional layer just like everything else in the security technologies.