Wednesday, March 27, 2013

Let me hadoop that for you

We are a tool using species. It started with sticks to knock down bananas. For the modern homo sapiens, it's tools like JavaScript and Hadoop.

Today, I was working on a technical problem. I came across this blogpost from 2012 that had close to the answer I was looking for, but it missed a key detail that was important to me.

Well, I follow the author on twitter, and the author follows me. So I sent him a tweet asking if he still had the data around, and whether he could analyze it again to solve my specific question. He responded with "Just let me run a quick hadoop query for you".

Minutes later, he posted the results of the query to GitHub that had exactly what I was looking for.

This all took place within 30 minutes from start to finish. It's possible because we tool using homos have opposable thumbs that can grasp the blogs, the twitters, the github, and the hadoop. I don't know how many servers @jedisct1 has at his disposal, but in theory, with a few keystrokes, he could reach into the cloud (like Amazon's service) and marshall a thousand machines equalling a million dollar super computer to answer this small question for me.

No comments: