Monday, April 28, 2008

Racing to Zero

This article claims that AV vendors are upset about an anti-AV competition during DefCon. They claim that it provides too much help to hackers.

Yet, such contests also help customers. The 'protectors" (product vendors) have big marketing budgets to tell us their side of the story about how good they are protecting us from 'hackers'. The 'hackers' have neither the budget nor the inclination to tell us how good they are at breaking past the 'protectors'. What we are hearing is the sound of one hand clapping. We only get one side of the story.

This contest will tell us the other side of the story. That's what conferences like DefCon, BlackHat Briefings, and CanSecWest are for - they are the counterweight to product conferences like RSA. Educating the consumer unfortunately means hackers often get educated to.

The educating needed here is that the mainstream anti-virus technologies are easily evaded, and that such evasion happens a lot, especially against high-value targets like financial institutions and government organizations. I see it when I talk to customers, but there is no reliable statistics on the matter. Anti-virus vendors publish tests "proving" a 99% detection rate, when no such detection rate happens in the real world.

There are niche technologies that can improve this situation. However, customers aren't demanding them, so mainstream vendors don't invest in them, and the niche products don't get traction. The more the inherent deficiencies with anti-virus come to light, the more these technologies will find their way to the market.

This open-letter signed but some respected people in the field asserts the principle: "It is not necessary and it is not useful to write computer viruses to learn how to protect against them". This is absolutely true. However, that doesn't apply to customers. Often, the best way to test an anti-virus product is to create your own virus. When I was building such products, I felt no need to create viruses in order to develop defenses. Now that I'm hired to evaluate products, I have already built my own viruses to evaluate how anti-virus products work, and whether they live up to their claims.

I would hope that the contest organizers take this into account. While judging how well hackers defeat anti-virus products, I would hope that they likewise give an award to the anti-virus that is best at defeating the hackers.


Andrew said...

I certainly am not siding with the A/V vendors on this one. I have no issue with a structure, public testing of any security technology. Elsewhere, I've seen commentary on this topic that the A/V industry should embrace this as it will show them what the bad guys are up to and help them with better defenses. I take exception with the idea that this is of much value in advancing the state of the industry in defense. I would suggest that most hardcore blackhats, those who are using unknown flaws for financial gain, would not bother to reveal their knowledge in a gaming setting such as this. You may get a lot of very clever greyhats, and perhaps some whitehats, who know how to easily subvert A/V, but I'm not sure any earth-shaking attacks will come out of this game. It will be interesting to watch the marketing folks hop around though.

Robert Graham said...

I take exception with the idea that this is of much value in advancing the state of the industry in defense

You are right in that the vendors won't learn any useful information from this.

Customers will, however.

decius said...

I happen to think that open letter is facially absurd. You obviously have to understand how computer viruses work in order to understand how to detect them. If you have no experience with computer viruses, writing one would obviously help you understand how they work. Therefore, writing a computer virus can obviously be useful in learning how to protect against them. This is a simple logical fact. It remains true no matter how many well respected people you get to line up on the other side.

Are there negative consequences associated with teaching students to write computer viruses in a college classroom? Could be. For example, someone who might never have thought to write a virus might decide to do something malicious as a result of participating in that class. There is at least enough to this for a discussion. But even if good arguments exist, this is a bad one. Refusal to support this letter's fallacy is not akin to agreeing that virus writing ought to be taught in a classroom.

Furthermore, I think its counterproductive to cling to a logical fallacy as a substitute for actually giving a question like this due analysis. I think often technical people in computer security have a hard time thinking about the social or moral context around their work. They let emotional convictions about what people should and should not do and who they think is bad or good cloud their judgment about right and wrong. The result is that they constantly advocate that the lines be drawn in the wrong places, as this letter does.