Hacker exposes global warming researcher (Climategate)

Hackers broke in and revealed the private e-mails of Phil Jones (NYTimes, BBC ), a famous climatologist. This is going to be one of the most politically relevant hacks of the last few years. When hackers broke into Sarah Palin's e-mails during the presidential campaign, they failed to find any interesting dirt. Phil Jones' e-mails, though, are full of dirt. There's no proof of a "conspiracy" or "cover-up", but a lot of the e-mails look bad for Jones and some of his fellow researchers.

As a cybersecurity expert and a climate skeptic, I thought I'd give some background on what happened.

THE CLIMATE DEBATE

Climate skeptics accept the fact that CO2 is a major greenhouse gas and that mankind has produced a lot more of it. However, the effect is logarithmic, suffering decreasing marginal returns, which means that even when we double CO2 in the atmosphere around the year 2100, by itself, CO2 will only increase temperatures by 1 degree. Global warming alarmism is based on the idea that the atmosphere is unstable, with reinforcing feedbacks, and "sensitive" to changes. Warmer air holds more water vapor which holds in more heat, which in turn warms the air further. Ice reflects heat, and when some melts do to slight warming, it exposed rock which absorbs heat, causing even more warming.

That's the debate in a nutshell. Alarmists think climate sensitivity is large, skeptics think it's small.

There are two major proofs for alarmism: computer models and historic reconstructions.

Computer models attempt to reproduce the entire climate. They are hugely complex, trying to incorporate everything we know about the climate. Models show warmer air holding more water vapor causing a positive feedback. Models require supercomputers to run. It is experimentation with computer models that "proves" climate sensitivity is large. Climate skeptics think computer models are bogus, that they do not replace experimental evidence, and that the billions of dollars spent experimenting with models would be better spent experimenting with the atmosphere. I'm a skeptic because I understand computer models, how they can be deceptive, and that the IPCC's reliance on them is unwarranted.

Historic reconstructions try to figure out what the temperature has been in the past compared to the present. They try to answer the question whether the current warming is normal or unprecedented. During the "Medieval Warming Period" a thousand years ago, Greenland was green, and Europe was warmer than it is now. Was that a localized climate phenomenon, or was it global? Skeptics think it was global, alarmists think it was local, and therefore the planet is warmer now than any time in the last several thousand years.

Historic temperatures are constructed from a wide variety of sources, such as ratios of isotopes, widths of tree rings, contents of sediments, and so on. Climatologists rely upon statistics to reconstruct useful information from what is otherwise a chaotic jumble of data.

The problem with these reconstructions is that the authors do not release their data to the public. Skeptics can't review the raw data or methodology and challenge the results. This is not unique to climate science, other scientific disciplines have similar data sharing problems. Collecting the data, such as measuring the rings of thousands of trees, takes a lot of work, and a scientist can make many discoveries from the data set. They only want to release the data after they've spent a few years writing papers based on it. Another problem is that it takes a lot of work to archive and maintain the data for critics who want to challenge it years later.

So, the lack of reproducibility doesn't mean there's a conspiracy or malfeasance, but it does mean that the science isn't "settled" (as Al Gore claims). It cannot be settled until critics have had a chance to rebut the claims. The lack of reproducibility gives the skeptics too much credibility (if they are wrong), or not enough credibility (if they are right).

At the center of the reproducibility debate is Stephen McIntyre and his website ClimateAudit. McIntyre documents his struggles to obtain the raw data and reproduce the results of famous temperature reconstructions. When he finally gets his hands on the data, often years later, he's often successful at finding important flaws.

The targets of McIntyre's campaign have their own website, RealClimate. They stress that their website is by "climate researchers", and important point because a lot of critics are not (McIntyre is a statistician).

Phil Jones, the guy whose e-mails were hacked, is part of the RealClimate crowd, and the target of McIntyre's attempts to reproduce results. Phil Jones runs that UK's University of East Anglia Climate Research Unit (CRU). CRU maintains global temperature records from the present going back to the 1800s. It also holds the data for a lot of reconstructions, especially the reconstructions used by the UN's IPCC (Intergovernmental Panel on Climate Change). McIntyre wants to challenge the IPCC's conclusions, but he can't, because Phil Jones refuses to release the data.

THE HACK

Somebody(s), we don't know who, stole a thousand confidential e-mails of Phil Jones, head the CRU. They also stole some important raw data used in many climate research papers. They put it in a 61-megabyte ZIP file and posted it to an anonymous FTP server in Russia. The posting was accompanied with the note:

We feel that climate science is, in the current situation, too important to be kept under wraps.
We hereby release a random selection of correspondence, code, and documents

All evidence points to this as being genuine. Phil Jones has admitted that one of pieces of dirt, about "hiding" a warming trend, is indeed genuine, although misinterpreted and out of context.

CRU has canceled everyone's passwords, forcing everyone to choose new passwords. This hints they have logs showing a specific account accessing the data (possibly Jones' own account).

Nobody knows how it happened. We are unlikely to figure out how the hack occurred unless we discover who did it.

It's a roughly 80% chance it was done by some sort of "insider", by somebody who has at least partial access to one of the internal computers. There is only a 20% chance it was done by an outside hacker breaking in. (This is my gut feel as a security researcher, there is no robustness in this estimate). Universities are more easily hacked by outsiders than most networks, ironically because Universities have a culture of sharing data.

UPDATE: A wag suggested this: Phil Jones and crew were likely logging onto their accounts using an open wifi at a climate conference. If you wanted to break into climatologist's e-mail, that'd be the easiest way to do it.

UPDATE: The hacker used open proxies to post the content, hiding his/her IP address.

UPDATE: Phil Jones is quoted as saying "It was a hacker. We were aware of this about three or four days ago". He's referring to the first attempt by the hacker to post the data to RealClimate.

The data is oddly specific. Only Phil Jones e-mails were copied, and a lot of the data that was hacked is specific to certain climate controversies. If it wasn't an insider, it was certainly somebody familiar with the central debate about reproducibility of climate reconstructions.

The fact that they posted the data to an anonymous FTP site in Russia also points to somebody who is active in the hacking community. This narrows things down. I suspect that at the end of the data, they'll find some sort of computer administrator working for CRU.

This hack is not simply about global warming, but the ethics of hacking.

This is similar to the Palin hack. If you'll remember, Alaska had rules that all e-mails must be archived, for the purposes of making government transparent. Palin conducted official business through her Yahoo account, evading these rules. It wasn't done maliciously, the hack actually proved there was nothing being covered up. Yet, it was still a violation of the rules.

In much the same way, Phil Jones is hiding data. It's bad science, it's bad politics. Again, there is no conspiracy or cover up here. Jones passionately hates McIntyre, he doesn't like skeptics, and he doesn't want to go through a lot of work to help skeptics. (Actually, I feel for Jones: we often come across virus samples, we send them around to people we like who ask, but we are too lazy to make them more widely available).

I think hackers do the world a favor by making things publicly available that should have been available in the first place. I believe "transparency" is fundamental to our political system, and if they politicians fail to be transparent, hackers should force the issue.

On the other hand, I'd like to see the hacker come forward publicly and admit the deed. It's a bad principle for hackers to decide for themselves when it's right or wrong to hack. This because hackers always have a justification for their hacks, only rarely are they going to find that others agree with them. It's like the question whether you'd kill Hitler before he could cause WW II and the Holocaust. My answer is that I would - but I'd expect to be convicted of murder and sent to jail. It's like how Henry David Thoreau practiced civil disobedience: he refused to pay taxes, and expected to go to jail (and was annoyed when his friends released him from jail by paying his taxes). People who hide from the government in order to avoid taxes are douchebags, people who stand up for principles are heroes.

UPDATE: It's the same ethics as "whistleblowers". If this "hack" was an insider job, the hacker might be protected by UK's whistleblowing laws.

UPDATE: How is this different than Gary McKinnon, famous British autistic hacker fighting extradition to the US? He broke into NASA because he believed they were hiding evidence of UFOs. The big difference is that UFOs are a wackjob conspiracy theory, but the inability of critics to reproduce studies used by the IPCC and government policy makers is a valid concern. The CRU hacker did not claim there was a cover-up or conspiracy, but simply pointed out that the data should be public.

SOME OF THE E-MAILS

This page from Climate Examiner has a long list of some of the damning e-mails. I thought I'd list just a few here from Jones to give a taste of what's happening.

"I've just completed Mike's Nature trick of adding in the real temps to each series for the last 20 years (ie from 1981 onwards) amd from 1961 for Keith's to hide the decline."

Jones is correct that this is taken out of context, and that it doesn't mean what non-climatologists thinks it means. It's not a smoking gun, ignore it (although ClimateAudit has a technical criticism of this e-mail).

UPDATE: As Cardinal Richelieu said, :If you give me six lines written by the hand of the most honest of men, I will find something in them which will hang him". The above e-mail, and a lot of others in the archive, are a good example of this principle.

"I can't see either of these papers being in the next IPCC report. K and I will keep them out somehow - even if we have to redefine what the peer-review literature is !"

One of the complaints of skeptics is that the peer-review process, by either journals or the UN's IPCC, is stacked against them. Skeptics claim that papers are accepted or rejected for political reasons, not technical reasons. This could be a conspiracy theory by skeptics, but the above e-mail shows it has at least some partial merit.

UPDATE: Specifically, they claim that they don't need to make data available for others to reproduce their work, because peer-review has already reviewed it. However, as these e-mails show, it's largely their friends who do the peer-reviewing.

"Mike, Can you delete any emails you may have had with Keith re AR4 ? Keith will do likewise. He's not in at the moment - minor family crisis. Can you also email Gene and get him to do the same? I don't have his new email address. We will be getting Caspar to do likewise."

This demonstrates that Jones is acting in bad faith, actively trying to delete information that might be revealed via a FOI (Freedom of Information) request. This is a counterexample of the Richelieu quote above, there's no way to misinterpret this e-mail.

REBUTTAL

RealClimate has a post about this issue. Except for responses directly from Phil Jones and CRU themselves, this would be the most definitive rebuttal. Their comment is fair:

More interesting is what is not contained in the emails. There is no evidence of any worldwide conspiracy, no mention of George Soros nefariously funding climate research, no grand plan to 'get rid of the MWP', no admission that global warming is a hoax, no evidence of the falsifying of data, and no ‘marching orders’ from our socialist/communist/vegetarian overlords. The truly paranoid will put this down to the hackers also being in on the plot though.

UPDATE: There's no hoax, but the e-mail prove these guys collaborate to prevent anybody else from reproducing and challenging their results.

UPDATE: RealClimate makes claims of a global right wing conspiracy to attack global warming with much less evidence of malfeasance than is shown in these e-mails.