Sunday, November 08, 2009

Brazil outage NOT caused by hackers


I just got through watching the CBS 60 Minutes special on cyberhackers, where they claim that major power outages in Brazil (in 2005 and 2007) were caused by hackers. This is unlikely to be true.

Hackers are like witches in Salem in the 1600s. When crops failed, people blamed it on the witches, who were burned at the stake. These people believed they were acting intelligently. The witches were convicted in “fair” trials, with “proof beyond a reasonable doubt”. For example, victims would testify how the accused witch would curse them, or give them the Evil Eye. Why would they lie about being cursed?

Now, when computers fail, people are immediately suspicious of hackers.

We know the CBS story is bogus. CBS news did not investigate the evidence. They instead cite “half a dozen sources” in the US intelligence community. However, these sources themselves did not investigate the evidence: they are simply confirming that they heard the rumor from people in the Brazilian government. Those government officials likewise did not investigate the evidence, they are likewise just passing on rumors.

CBS news didn't track this down. They didn't attempt to contact anybody in Brazil. They did not contact anybody at “Furnas Centrais Elétricas”, the company responsible maintaining those transmissions lines. They didn't even do a simple Google search, which would tell them that the company claimed at the time that the 2007 outage was caused by dust and soot from local forest fires (which, apparently, is a common problem in power transmission).

Most rumors of hacker infiltrations are false. If you investigate computers in any large organization hard enough, you'll find malware. This doesn't mean hackers have broken in, because most viruses are not under control of the hacker who launched them. Also, things get on computers that trigger deep scans from anti-virus scanners that are not necessarily malicious malware. This malware becomes a distraction to finding the true cause of what happened. Thus, when investigating a power outage, finding malware on computers doesn't mean hackers caused the outage.

Several years ago, I was doing a security assessment in a foreign country (not US, not Brazil). The customer told me a story they had personally been involved in. There had been an incident where hackers claimed to have come in via the Internet and turned off the power in several cities, and were demanding ransom money. On further investigation, however, it turned out to be an inside job. The outage was caused by one of the employees who worked on the main control console. The guy had simply flipped a switch, turning off the power. The guy, and his accomplice, were arrested, tried, convicted, and sent to jail. No “hacking” was involved.

This story sounds suspiciously like the story CIA agent Tom Donahue gave at a security conference a couple years ago. The difference is that his story stops at the point where hacker demand extortion money. Well, what happened next? Was the money paid? Or were the hackers caught? Donahue doesn't say. Like the CBS story about Brazil, we are given no details, we are expected to trust them. I doubt that Donahue was telling the truth, that anybody really investigated the evidence. I think he was just passing on rumors.

So why is CBS passing on these rumors? The answer is the same as the witch trials in the 1600s. The people who were accused were usually in some sort of conflict with their neighbors. Accusing them of witchcraft and testifying to being “hexed” was one way of resolving the conflict. The same is true of these cybersecurity stories: people in government want more control over the Internet. Different departments are fighting amongst themselves for that control (such as the NSA vs. the DHS), and all are fighting for more legal control against the private sector.

The CBS story is obvious government propaganda. All their sources are from the government, from people who stand to gain from increased government control over the Internet. For example, it says that the US power grid is insecure, and claims that the reason it's insecure is because it's not regulated by the government. That's not a reason. The federal government's computers are even less secure than the power grid – there is no reason to think that Congress can secure the power grid if they can't secure their own computers. Conversely, all the energy companies belong to the “National Energy Regulatory Commission” or “NERC”, which is does indeed regulate the cybersecurity of the power grid. The reason the CBS story exists is because somebody else, such as the DHS or NSA, wants to take control away from the NERC. That's why you have such a one-sided story from CBS – they never talked to anybody at NERC, or any of the power companies.

As a pentester, I know that our power grid is insecure. I've done security assessments at power companies. I know I can hack in from the Internet and cause power outages. However, government regulation isn't the answer. Cybersecurity regulation has proven itself to be a cure worse than the disease. It drives up the costs without doing anything significant to reduce the threat. For example, we just got through doing a pentest at a company that was paranoid about following all the regulations (HIPAA, SOX, PCI, etc.), yet we were able to break in easily with SQL injection bugs and the same vulnerability that led to Conficker. It was one of the most secure companies we've seen, but all these regulations had become a distraction to an otherwise talented security team.

There is a risk. Hackers will eventually cause a major power outage. In the grand scheme of things, though, it's not a big deal. Major power outages from accidental mistakes will always be a bigger threat. Nation states blowing up power lines (with bombs) will always be a bigger threat. Bad government regulation of the power grid will always be a bigger threat. The CBS piece is just propaganda.

UPDATE: Wired ThreatLevel confirms it was soot, not hackers. They did something radical: journalistic investigation. CBS, take note about how journalism should be done.

UPDATE FROM TWITTER: jack_daniel All you 60 Minutes naysayers are missing the point: it was CYBER-soot on the insulators.

13 comments:

Roland Dobbins said...

Spot-on - they want the control and the money, let's not forget that, heh.

Another way to evaluate the validity of claims in this arena is employment of the appellation 'cyber-' in a non-sarcastic manner. Anyone who uses 'cyber-' seriously is likely to lack clue, in my experience.

Mike said...

"Hackers are like witches in Salem in the 1600s. When crops failed, people blamed it on the witches, who were burned at the stake."

Actually, those who died were hanged rather than burnt. (And some also died awaiting trial.)

I don't recall that there had been a crop failure, either.

However, the crops were quite possibly of importance, inasmuch as it's been suggested that damp weather had caused ergot fungus to grow on the rye. This would have led to illness and strange behaviour in those who ate it. People who lacked an understanding of the causality involved might well have attributed that to human agency (in the form of witchcraft). After that, hysteria could take over - specially as Mather, whose head was cooler, was away in England at the time.

So you're historically inaccurate. But I suppose the basic point that people who lack understanding see human agency where it's not is a valid one. On the other hand, I'm not sure that that's quite what you are saying. ...

Interestingly, the anthropologist E. E. Evans-Pritchard famously found that the Azande actually lacked a concept of the accidental. It's since been said that that's not unusual among primitives. Perhaps to see agency in events is "natural" to us as humans, but the more we know the more that gets pushed back.

DK said...

As a pentester, I know that our power grid is insecure. I've done security assessments at power companies. I know I can hack in from the Internet and cause power outages.

That is a bold statement to make. I believe it to be possible, don't get me wrong. But since you claim to have inside knowledge, maybe you can elaborate on how you would accomplish this (since others like 60 minutes have it wrong).

Robert Graham said...

maybe you can elaborate on how you would accomplish this

Power companies believe their "control" networks are wholly disconnected from the Internet. This is never true. We consistently break into the "business" network with a typical SQL injection or WiFi issue, then scan the network until we find some sort of dual-homed machine (often some old Sun machine that hasn't been patched in a decade), and from there hop into the control network. Control networks are rarely patched -- in fact, they rarely have any authentication at all, so they are wide open.

There are also indirect ways to cause outages. For example, we can hack a billing server to convince it that a downstream customer hasn't paid. Or, social engineering works to convince somebody to throw a switch.

Rob said...

Considering what the fact that these power companies run software from the mid-90s, why is it a stretch to think that hackers brought down the power in Brazil?

I know that CBS is not exactly a "pillar of truth", but this isn't exactly tin foil stuff.

The NSA (and some obscure def contractors) have been working on this for sometime...and yes, this is really happening.

Robert Graham said...

why is it a stretch to think that hackers brought down the power in Brazil?

It's not a stretch. Neither is it a stretch to believe Al Qaeda was responsible. The point is, there's no evidence. The CBS report is clearly bad reporting that does not double-check its anonymous sources. Look up "jorunalistic ethics" and "anonymous sources" on Wikipedia to understand the underlying problem.

The fact is that hackers are less of a threat to our power grid than accidental outages or physical bombs. Yet, people are more afraid of hackers because they fear most what they least understand. The CBS report played upon those fears.

George said...

This is really a great post Robert. Your point that there is no evidence is well taken.

The only thing I would disagree with is your assertion that accidental outages and bombs are more dangerous. I think up to this point, that is correct. However, isn't hacking a system far easier and less risky than trying to plant a bomb somewhere?

Moreover, what happens when we have a smart grid that is more oversubscribed by relying on a system that would balance the load such that major appliances don't all turn on at the same time? What happens if a hacker convinces the appliances to all turn on at the same time and overload the grid to burn parts of the infrastructure down? What happens if they wait till a really hot day to do this so that thousands of people (mostly elderly) die from the heat?

I think the real risk in these bogus stories is that it's like crying wolf. People get sick of it and they stop listening to your advice on the need to lock down the system. They get complacent and think that no one will bother.

Matthew Wollenweber said...

Great post. I agree with your initial assessment. I'd differ with you in regards to government regulation and the potential threat of hackers.

I'd add a few points:

1. Nation states have likely compromised power systems already. Wouldn't you? But them doing anything is unlikely as most nations consider any cyber warfare an act of war and others equate it to WMD. I think this is risk that one must just accept - besides, what else could you do?

2. Non-state actors could liekly compromise the power grid, but they have limited reasons to do so. Estimates to compromise hardened targets by developing custom exploits and implant software are in the $1-2M range. Those numbers are largely made up, but I've seen several estimates by people who write such things agree on the ballpark. That's a lot of initial investment for a profit driven criminal endevour or a "just for fun" hacking experience.

3. You alluded to the real problem - the fragility of the electrical grid. Hacking one power plant, scada system, whatever shouldn't have the potential to seriously mangle the system. But "cascading failure" seems to be the accepted norm. I'm not an electrical engineer but investing in a robust system would seem most prudent to me.

Matthew Wollenweber said...

Blogger DK said...

As a pentester, I know that our power grid is insecure. I've done security assessments at power companies. I know I can hack in from the Internet and cause power outages.

That is a bold statement to make. I believe it to be possible, don't get me wrong. But since you claim to have inside knowledge, maybe you can elaborate on how you would accomplish this (since others like 60 minutes have it wrong).


DK - I've done assessments of power companies as well. I was able to compromise a system that managed fuel levels. It's largely speculation as to what would happen if one were to adjust the level to cause the tanks to go empty or to cause a spill, but it's one of those screenshots where the customer suddenly becomes very pale.

The other example I give in regards to the power grid is nuclear plants. I've been told by several nuclear engineers, that one requires a license modification to drill most holes in the walls of nuclear power plants. If you want to network two machines on different sides of a wall without dealing with the government, then an expedient solution is to connect them with wifi. I'm sure there's some sanity checks on what they systems do, but in every network I've been on systems usually have more access than administrators understand.

fabricio_machado said...

Would they doing some tests???

http://www.cbsnews.com/stories/2009/11/10/world/main5607148.shtml?tag=stack

:-D

Denny said...

Robert, look this: Massive blackout leaves Brazil on edge (11/11/2009).

"A massive blackout plunged tens of millions in Brazil's largest cities into darkness, sparking major disruptions, fears of crime and energy supply concerns Wednesday for the newly named Olympic hosts."

See more: http://news.yahoo.com/s/afp/20091111/wl_afp/brazilenergyblackout.

aaron said...

"""The fact is that hackers are less of a threat to our power grid than accidental outages or physical bombs. Yet, people are more afraid of hackers because they fear most what they least understand. The CBS report played upon those fears."""

_Excellent_ point that so few seem to understand/see. TV news is often about fears, with the solution (or lack of) just after the next commericla break. So many people don't seem to simply be rational. It's about accepted (but hopefully minimized) risk.. it's just a lack of asking questions (or maybe lack of thinking itself). Thanks for (yet again) helping to bring the truth to light.

Cappella said...

Actually, for all we know, CBS just potentially launched a cyberwar: A war based on disseminating misleading information. Under this circumstance, tricking attackers to think that Brazil's network is that vulnerable. And everyone starts attacking them, thus making it really happened.

The rules did not say that cyberattack first launch has to be from network probing. :P