Friday, May 15, 2015

Those expressing moral outrage probably can't do math

Many are discussing the FBI document where Chris Roberts ("the airplane hacker") claimed to an FBI agent that at one point, he hacked the plane's controls and caused the plane to climb sideways. The discussion hasn't elevated itself above the level of anti-vaxxers.

It's almost certain that the FBI's account of events is not accurate. The technical details are garbled in the affidavit. The FBI is notorious for hearing what they want to hear from a subject, which is why for years their policy has been to forbid recording devices during interrogations. If they need Roberts to have said "I hacked a plane" in order to get a search warrant, then that's what their notes will say. It's like cops who will yank the collar of a drug sniffing dog in order to "trigger" on drugs so that they have an excuse to search the car.

Also, security researchers are notorious for being misunderstood. Whenever we make innocent statements about what we "could" do, others often interpret this either as a threat or a statement of what we already have done.

Assuming this scenario is true, that Roberts did indeed control the plane briefly, many claim that this is especially reprehensible because it endangered lives. That's the wrong way of thinking about it. Yes, it would be wrong because it means accessing computers without permission, but the "endangered lives" component doesn't necessarily make things worse.

Many operate under the principle that you can't put a price on a human life. That is false, provably so. If you take your children with you to the store, instead of paying the neighbor $10 to babysit them, then you've implicitly put a price on your children's lives. Traffic accidents near the home are the leading cause of death for children. Driving to the store is a vastly more dangerous than leaving the kids at home, so you've priced that danger around $10.

Likewise, society has limited resources. Every dollar spent on airline safety has to come from somewhere, such as from AIDS research. With current spending, society is effectively saying that airline passenger lives are worth more than AIDS victims.

Does pentesting an airplane put passenger lives in danger? Maybe. But then so does leaving airplane vulnerabilities untested, which is the current approach. I don't know which one is worse -- but I do know that your argument is wrong when you claim that endangering planes is unthinkable. It is thinkable, and we should be thinking about it. We should be doing the math to measure the risk, pricing each of the alternatives.

It's like whistleblowers. The intelligence community hides illegal mass surveillance programs from the American public because it would be unthinkable to endanger people's lives. The reality is that the danger from the programs is worse, and when revealed by whistleblowers, nothing bad happens.

The same is true here. Airlines assure us that planes are safe and cannot be hacked -- while simultaneously saying it's too dangerous for us to try hacking them. Both claims cannot be true, so we know something fishy is going on. The only way to pierce this bubble and find out the truth is to do something the airlines don't want, such as whistleblowing or live pentesting.

The systems are built to be reset and manually overridden in-flight. Hacking past the entertainment system to prove one could control the airplane introduces only a tiny danger to the lives of those on-board. Conversely, the current "security through obscurity" stance of the airlines and FAA is an enormous danger. Deliberately crashing a plane just to prove it's possible would of course be unthinkable. But, running a tiny risk of crashing the plane, in order to prove it's possible, probably will harm nobody. If never having a plane crash due to hacking is your goal, then a live test on a plane during flight is a better way of doing this than the current official polices of keeping everything secret. The supposed "unthinkable" option of live pentest is still (probably) less dangerous than the "thinkable" options.

I'm not advocating anyone do it, of course. There are still better options, such as hacking the system once the plane is on the ground. My point is only that it's not an unthinkable danger. Those claiming it is haven't measure the dangers and alternatives.

The same is true of all security research. Those outside the industry believe in security-through-obscurity, that if only they can keep details hidden and pentesters away from computers, then they will be safe. We inside the community believe the opposite, in Kerckhoff's Principle of openness, and that the only trustworthy systems are those which have been thoroughly attacked by pentesters. There is a short term cost of releasing vulns in Adobe Flash, because hackers will use them. But the long term benefit is that this leads to a more secure Flash, and better alternatives like HTML5. If you can't hack planes in-flight, then what you are effectively saying is that our believe in Kerckhoff's Principle is wrong.

Each year, people die (or get permanently damaged) from vaccines. But we do vaccines anyway because we are rational creatures who can do math, and can see that the benefits of vaccines are a million to one times greater than the dangers. We look down on the anti-vaxxers who rely upon "herd immunity" and the fact the rest of us put our children through danger in order to protect their own. We should apply that same rationality to airline safety. If you think pentesting live airplanes is unthinkable, then you should similarly be able to do math and prove it, rather than rely upon irrational moral outrage.

I'm not arguing hacking airplanes mid-flight is a good idea. I'm simply pointing out it's a matter of math, not outrage.


Sean said...

A lot of security researchers talk about what they "could" do before knowing they can't. I find his story hard to believe.

In any case, they say cockpit doors are secure and safe while in fight too, but airlines don't want you messing with those either. Seems fishy - would you say the only way to know would be to live pentest it?? seriously??

John S said...

OK, then shell out the money to guy your own Boeing, and hack the shit out of it.

Saying it's ok because it "probably" won't get everyone killed is extremely egotistical and entitled. It's not your place to make that decision, and if you truly think it is, then you need to be as far from the decision making process as possible.

This blog is a troll.

Paulo Almeida said...

"Does pentesting an airplane put passenger lives in danger? Maybe. But then so does leaving airplane vulnerabilities untested"

That's a false dichotomy, as you show yourself when you later add:

"There are still better options, such as hacking the system once the plane is on the ground."

So why is the outrage unjustified? I'm not an expert, but after reading your blog post, it seems to me that after doing the math, the outrage is even more justified.

There was a similar discussion in Portugal, a few years ago, when someone did die because a consumers association did some fake phone calls to the natioal emergency number, to assess its efficiency, and an ambulance was diverted from a real emergency by one of those fake calls.

I'm not one for easy outrage, and I agree with you that "priceless" is a foggy notion (perhaps utopic would be the word), but I do think that human life has a very high value and it should be endangered only when there are no feasible alternatives.

I also think your example with the babysitter is a blatant appeal to emotion, by putting children in the equation, because the real issue there is that even if babysitting is a lot safer than taking children to the store, the probability of death is extremely low in both cases.