Ultron didn't save the world

The movie Avengers: Age of Ultron has a message for us in cybersec: In our desire to save the world, we are likely to destroy it.

Tony Stark builds "Ultron" to save the world, to bring peace in our time. As a cybernetic creation, Ultron takes this literally, and decides the best way to bring peace is to kill all humans.

The problem, as demonstrated by the movie, isn't that there was a bug in Stark's code. It was the hubris thinking that Stark could protect everyone. Inevitably, protecting everyone meant ruling everyone, bringing peace by force. It's the same hubris behind the USA's effort to bring peace to Iraq and Afghanistan.

I mention this because in the cybersecurity industry, there are many who propose to bring security through force. They want government to impose liability on software makers, dictate how they write code, and punish them for doing things wrong.

This sounds reasonable. After all, nobody wants medical equipment like pacemakers to be hacked, or cars to crash, or airplanes to fall out of the air. But here's the thing: it's a tradeoff. Computer controlled planes and cars can save lives by taking fallible humans out of the equation. Computer controlled devices potential to vastly improve health, whether it's Apple Watches monitoring your heart, pacemakers, insulin pumps, or sensors embedded in the body. While these devices can be hacked, the practical reality is that if an evildoer wanted to kill people, bombs and bullets are still easier than hacking medical devices. Standards and liability, on the other hand, will chill innovations -- innovations that save lives. The fallacy of asserting authority to bring security means killing people because the innovation that would've saved them was delayed because developers worried too much about hackers.

The cybersecurity industry is weird. We are the first to point out the hollow rhetoric of the surveillance and police state. Yet, we are the first to become totalitarian when we think it's going to be us who will be in control. No, we should learn from Tony Stark: even when it's us "good guys" who are running the show, we should still resists the urge to impose our authority by force. The tradeoffs from the security we demand is often worse than the hackers it would stop.