Can I drop a pacemaker 0day at DefCon that is capable of killing people?
The problem is that manufacturers are 20 years behind in terms of computer "security". They don't just have vulnerabilities, they have obvious vulnerabilities. That means not only can these devices be hacked, they can be easily be hacked by teenagers. Vendors do something like put a secret backdoor password in a device believing nobody is smart enough to find it -- then a kid finds it in under a minute using a simple program like "strings".
Telling vendors about the problem rarely helps because vendors don't care. If they cared at all, they wouldn't have been putting the vulnerabilities in their product to begin with. 30% of such products have easily discovered backdoors, which is something they should already care about, so telling them you've discovered they are one of the 30% won't help.
Historically, we've dealt with vendor unresponsiveness through the process of "full disclosure". If a vendor was unresponsive after we gave them a chance to first fix the bug, we simply published the bug ("drop 0day"), either on a mailing list, or during a talk at a hacker convention like DefCon. Only after full disclosure does the company take the problem seriously and fix it.
This process has worked well. If we look at the evolution of products from Windows to Chrome, the threat of 0day has caused them to vastly improve their products. Moreover, now they court 0day: Google pays you a bounty for Chrome 0day, with no strings attached on how you might also maliciously use it.
So let's say I've found a pacemaker with an obvious BlueTooth backdoor that allows me to kill a person, and a year after notifying the vendor, they still ignore the problem, continuing to ship vulnerable pacemakers to customers. What should I do? If I do nothing, more and more such pacemakers will ship, endangering more lives. If I disclose the bug, then hackers may use it to kill some people.
The problem is that dropping a pacemaker 0day is so horrific that most people would readily agree it should be outlawed. But, at the same time, without the threat of 0day, vendors will ignore the problem.
This is the question for groups that defend "coder's rights", like the EFF. Will they really defend coders in this hypothetical scenario, declaring that releasing code 0day code is free speech that reveals problems of public concern? Or will they agree that such code should be suppressed in the name of public safety?
I ask this question because right now they are avoiding the issue, because whichever stance they take will anger a lot of people. This paper from the EFF on the issue seems to support disclosing 0days, but only in the abstract, not in the concrete scenario that I support. The EFF has a history of backing away from previous principles when they become unpopular. For example, they once fought against regulating the Internet as a public utility, now they fight for it in the name of net neutrality. Another example is selling 0days to the government, which the EFF criticizes. I doubt if the EFF will continue to support disclosing 0days when they can kill people. The first time a child dies due to a car crash caused by a hacker, every organization is going to run from "coder's rights".
By the way, it should be clear in the above post on which side of this question I stand: for coder's rights.
Update: Here's another scenario. In Twitter discussions, people have said that the remedy for unresponsive vendors is to contact an organization like ICS-CERT, the DHS organization responsible for "control systems". That doesn't work, because ICS-CERT is itself a political, unresponsive organization.
The ICS-CERT doesn't label "default passwords" as a "vulnerability", despite the fact that it's a leading cause of hacks, and a common feature of exploit kits. They claim that it's the user's responsibility to change the password, and not the fault of the vendor if they don't.
Yet, disclosing default passwords is one of the things that vendors try to suppress. When a researcher reveals a default password in a control system, and a hacker exploits it to cause a power outage, it's the researcher who will get blamed for revealing information that was not-a-vulnerability.
I say this because I was personally threatened by the FBI to suppress something that was not-a-vulnerability, yet which they claimed would hurt national security if I revealed it to Chinese hackers.
Again, the only thing that causes change is full disclosure. Everything else allows politics to suppress information vital to public safety.
Update: Some have suggested it's that moral and legal are two different arguments, that someone can call full disclosure immoral without necessarily arguing that it should be illegal.
That's not true. That's like saying that speech is immoral when Nazi's do it. It isn't -- the content may be vile, but the act of speaking never immoral.
The "moral but legal" argument is too subtle for politics, you really have to pick one or the other. We saw that happen with the EFF. They originally championed the idea that the Internet should not be regulated. They, they championed the idea of net neutrality -- which is Internet regulation. They original claimed there was no paradox, because they were saying merely that net neutrality was moral not that it should be law. Now they've discarded that charade, and are actively lobbying congress to make net neutrality law.
Sure, sometimes some full disclosure will result in bad results, but more often, those with political power will seek to suppress vital information with reasons that sound good at the time, like "think of the children!". We need to firmly defend full disclosure as free speech, in all circumstances.
Update: Some have suggested that instead of disclosing details, a researcher can inform the media.
This has been tried. It doesn't work. Vendors have more influence on the media than researchers.
We say this happen in the Apple WiFi fiasco. It was an obvious bug (SSID's longer than 97 bytes), but at the time Apple kernel exploitation wasn't widely known. Therefore, the researchers tried to avoid damaging Apple by not disclosing the full exploit. Thus, people could know about the bug without people being able to exploit it.
This didn't work. Apple's marketing department claimed the entire thing was fake. They did later fix the bug -- claiming it was something they found unrelated to the "fake" vulns from the researchers.
Another example was two years ago when researchers described bugs in airplane control systems. The FAA said the vulns were fake, and the press took the FAA's line on the problem.
The history of "going to the media" has demonstrated that only full-disclosure works.