Monday, March 05, 2012

The Ruby/GitHub hack: translated

(Update: this post is not obsolete, as Ruby-on-Rails is actually going to fix the problem, and this humorous post explains the controversy better than my post).

For those who don’t speak Russian or Ruby-on-Rails, I thought I’d translate the recent GitHub hack controversy.

The underlying issue is an “Insecure Direct Object Reference”, #4 on the OWASP Top 10 list of most important web-application vulnerabilities. It means that that a hacker can change what's in the website database without having permission.

The cause of the GitHub hack was the toolkit/programming-language they used to build their site: Ruby-on-Rails. Ruby-on-Rails supports a feature known as “mass assignment”. It means that hackers can add parameters to webrequests that don’t belong there, and they’ll be assigned to the object along with the normal parameters. In other words, your account might have a flag otherwise invisible to you like “admin=false”. By including “user[admin]=true” in an otherwise innocent web request, you can give yourself administrator privileges and take over the site.

This “mass assignment” feature is pretty toxic, at least as far as cybersecurity goes. It’s one of those big features that make Ruby-on-Rails so easy to use in order to build websites. Fixing the security problem therefore makes it harder to program websites in Ruby-on-Rails, and would instantly break most existing websites built using the toolkit.

That’s why when Russian programmer Egor Homakov reported the “bug”, it was rejected. The argument was that this is behaving by design, and that while it was dangerous, it was prominently described in the “Ruby On Rails Security Guide”. There is a solution for programmers: “blacklist" values that should not be mass assignable, such as marking “admin” as something that causes a mass assignment to fail. In other words, it’s the owner’s responsibility to not shoot themselves in the foot, not the gun manufacturer’s.

Any cybersecurity professional knows this is wrong, but being Russian, Homakov’s English skills aren’t so good at arguing his point. So he used his hacking skills instead, and used the bug to hack into GitHub.

GitHub and Ruby-on-Rails are independent projects that are otherwise unrelated to each other. GitHub is a popular project hosting site, and it just so happens that one of the zillions of projects it hosts is “Ruby-on-Rails”. On the other side, Ruby-on-Rails is a toolkit for creating websites, and it just so happens that one of the zillions of websites written with Ruby-on-Rails is “GitHub”.

This gave Homakov a neat opportunity. He wanted the owners of the Ruby-on-Rails project to take him seriously, so what better way to do that than hack the very site hosting Ruby-on-Rails using the very bug he was talking about? He used the bug to hack into GitHub, and gave himself administrative control over the Ruby-on-Rails project. He used this hacked authority to then write a comment into the Ruby-on-Rails source code discussing the problem.

It took no time at all for GitHub to fix that one bug, by blacklisting the hacked value so nobody can repeat the hack. But GitHub can’t be sure that there aren’t more mass assignment problems in their source code waiting to be discovered. Given any complex website written in Ruby-on-Rails, there is a good chance you can eventually find a mass assignment bug if you try hard enough.

There are a few cybersecurity lessons here.

The first lesson is that when you own a gun that accidentally fires 1% of the time you pick it up, then you’ll eventually shoot yourself in the foot. The same is true of Ruby-on-Rail’s mass assignment: you’ll eventually shoot yourself in the foot. It’s inherently dangerous, you can’t consider it for your next big project until this bug is fixed -- and since fixing this bug will break all existing Ruby-on-Rails websites, I doubt that will ever happen.

The second lesson is that “blacklisting” doesn’t work. We’ve tried blacklisting known bad characters (like ') from SQL, and found it doesn’t stop SQL injection. We’ve tried blacklisting known bad characters (like <) from HTML, and found it doesn’t stop cross site scripting. You can’t expect programmers to correctly blacklist all variables that shouldn’t be set with mass assignment, because they’ll eventually forget one, a hacker will find it, and break in. Instead, you have to “whitelist” known safe variables, where the consequence of a programmer error is simply a bug in the website, not a hack.

The third lesson is that there is no good solution to the problem. Ruby-on-Rails already has plenty of what appear to be good solutions -- that fail. If you put “ActiveRecord::Base.send(:attr_accessible, nil)” in the file “config/initializers/disable_mass_assignment.rb”, you will fix the problem the way it should be fixed. But while good for you, most programmers still won’t use it,and there will still be a pervasive problem with mass assignment. We’ve had an obvious solution to SQL injection for a decade (“parameterized queries”), but most programmers still don’t use them, so SQL injection is still the web application’s #1 problem. “Insecure direct object reference” will stay at #4, or even rise to #3, until the Ruby-on-Rails project makes a change.

The fourth lesson is understanding "ease of use". Mass assignment makes creating websites a lot easier. But it makes securing websites much harder. You lose more in security than you gain in ease-of-use of the toolkit. As the GitHub hack showed, if you are putting your site on the Internet, security isn't an option. You don't need to fix every security problem, you don't even need to fix everything in the OWASP Top 10. But if you ignore the the top 4, you are a fool.

Finally, this shows the benefit of irresponsible disclosure. Homakov possibly broke the law, or at least, behaved “unethically” according to many people’s definition. Yet, it was not malicious, and caused no harm (other than deserved harm to reputation). It was the level of “disclosure” needed to bring what I feel is necessary attention to the problem.

Update: ASP.NET has a mass-assignment problem too, just not as big as Ruby-on-Rails.


Marc said...

That's like saying: OMG I'll never program anything on Android because an app had a security issue!

FYI, the issue is not Rails', it was GitHub's. Rails allows mass assignment by default in order to make things easy, but it's the developer who has to restrict which attributes the final user can modify. So if you, as a developer, allow the final user to modify the "admin" flag, it's your fault, not the framework you're using.

Don't blame Rails for that. Inform yourself. Learn some Rails before commenting on such things, please.

Thank you.

Robert Graham said...

It is a Rails problem.

As I explained in my post, mass assignment makes it easy to create insecure sites, but makes it much harder to create a secure site.

I can remove all the safeties from a gun, and this will make the gun easier to shoot -- but it will also make the gun too dangerous to own.

The only sane implementation is to force the developer to declare which values can be mass assignable, rather than declaring which cannot be. Until that changes, Rails mass assignment bugs will be a constant source of hacks.

As for first sentence, the issue isn't that we should be concerned with all possible issues, or even all of the OWASP top 10, but you'd be a fool to ignore the OWASP Top 4. This issue is #4, and really needs to be fixed.

Cliff Wells said...

I'm sure the same crowd who uses RoR specifically because it "makes things easy" is the exact same crowd who will carefully craft their code to avoid this security hole.

That makes perfect sense. After all, making it 10% easier to write a web app while making it 90% harder to secure is a good trade-off.

Raymond Forbes said...

So, I am not sure it is fair to call this a blacklist problem. It is easy to white list fields in the model using attr_accessible. In fact, most rails tutorials and books show the use of this method. If you add attr_accessible to the model at that point none of the fields are available through mass-assignment. Anything that needs to be assigned through a form would then need to be explicitly passed to the model.

It seems like this is a really easy fix for the Rails team, if they wanted and that is to add attr_accessible to the model generator with no fields passed to it and require the developer to define what fields to include.

Robert Graham said...

Raymond Forbes: "It seems like this is a really easy fix for the Rails team"

It is really easy, fixing it just as you describe.

But they don't think it's their problem, and it would break a lot of websites -- insecure websites.

Thus, I doubt they will fix it, at least, not until the next major version (a point when people expect old code to break).

Raymond Forbes said...

the fix i mentioned should not break any sites at all. The generator is invoked whenever a new app is created. This would mean any new app would be more protected. Doesn't solve the problem for old apps, however.

Stoune said...

There is two ways to build a system security:
- All denied by default and you should enable explicitly features which you wan't. This is how the most OS(any Linux distribution, FreeBSD, Windows 2008 Server) was built. This is how any serious production system should be built.
- All enabled by default. This is how Rails is developed. And this is very sad to hear what Rails developers can't understand such basics. Rails very good product, but with such agresive denial of fault their developers and their comunity put yourself into awkward situation.

Anonymous said...

I am just wondering: is this issue lies in Python Framework "Django" like rails?

postmodern said...

You forgot to mention how the Rails core-team and the community are responding to this issue.

Rails has changed the generators to enable whitelisting of attributes by default:

Yehuda Katz (of the great Rails3 refactoring fame) also proposed a more thorough solution which sparked an interesting discussion:

Anonymous said...

to the statement "But GitHub can’t be sure that there aren’t more mass assignment problems in their source code waiting to be discovered." I say It is not an absolute, but it is the only static code analysis tool built for Rails. The GitHub folks should be using it to scan their code. It's free. It's good.

Bill Leeper said...

I keep seeing a few comments about it is the developers problem, however, this is a relatively quiet feature, and not raising it's awareness up means that there are 1000's of developers unaware of the implications.

I for one have been pretty diligent in various security aspects of rails and other languages and was unaware of the significance of this and will be more diligent in the future thanks to this article.

Unknown said...

Ruby on Rails is designed to make it simpler for Ruby on Rails developers to develop applications and website