Monday, October 24, 2011

Cloud Computing is a Commodity, not a Utility

A computing pioneer named John McCarthy (creator of the LISP language) died recently. Some are giving him credit for coming up with the idea of cloud computing in 1961, when he described "time-sharing" (the sharing of a mainframe computer) as becoming a "public utility" like electricity or water. Nothing can be further from the truth. Today's cloud computing is the opposite of the "utility computing" that he imagined. This is as absurd as saying the original Star Trek TV show, with a voice activated computer, somehow invented Siri, Apple's new voice response system for the iPhone.

The archetype of cloud computing is Amazon EC2, a network of thousands of machines that can do anything from run supercomputer simulations to serve web pages. But cloud computing is more than EC2. It's a whole range of things, from Google apps, to Apple's iCloud storage of music, to the Amazon Kindle's storage of books. I have over 300 books on my Kindle – not actually on the device, but in my account with Amazon. When I broke my Kindle by dropping it two stories, I bought a new one, and all my books were still there. That is cloud.

Sure, Amazon's EC2 sounds a lot like the time-share systems of the early 1960s, but there are important differences. The biggest difference is "how we got here". There wasn't a slow progression of huge "mainframe" computers, but a rapid change from mainframes to "personal computers" in our homes (following Moore's Law).

If other utilities had progressed at the same speed as computing, then we'd all have a small fusion reactor in our homes supplying our electricity. Your iPhone can supply, through the cell network, all of the time-sharing needs of the 1960s.

Yes, Amazon sells compute power, but the word you are looking for is not "utility" but "commodity". Here is the current Wikipedia definition of a public utility:
A public utility (usually just utility) is an organization that maintains the infrastructure for a public service (often also providing a service using that infrastructure). Public utilities are subject to forms of public control and regulation ranging from local community-based groups to state-wide government monopolies. Common arguments in favor of regulation include the desire to control market power, facilitate competition, promote investment or system expansion, or stabilize markets. … The term utilities can also refer to the set of services provided by these organizations consumed by the public: electricity, natural gas, waterand sewage.
None of the above applies to Amazon EC2. But, the following Wikipedia page on commodities sounds a lot like Amazon EC2:
It is used to describe a class of goods for which there is demand, but which is supplied without qualitative differentiation across a market. A commodity has full or partial fungibility; that is, the market treats it as equivalent or nearly so no matter who produces it. Petroleum and copper are examples of such commodities.
In other words, cloud computing is fungible commodity like oil and copper, not a utility like electricity or sewage.

From this perspective, recently deceased Dennis Ritchie (who developed C and co-developed Unix) deserves much more credit than John McCarthy. The reason Ritchie developed Unix was precisely to break the "utility" model of time-share computing up to that point, and to make computers into a "commodity". Today's cloud computers like Amazon EC2 run mostly Unix, and mostly code written in C. They run almost no code written in John McCarthy's LISP.

The personal computing and Internet revolution is a genie that escaped the "utility" bottle. Many want to put that genie back again, and regulate the Internet and computers like utilities. Their arguments always sound good, but they are deceptive. It's the old phone utilities that lobby for regulations requiring new VoIP companies to provide 911/emergency services, making VoIP much more expensive. Likewise, it's law enforcement that lobbied for laws requiring mobile phones to have GPS location tracking features again for 911/emergency calls, but which law enforcement also uses to locate criminals.

Another example is "reliability". We all get frustrated when computers fail (as BlackBerry users recently experienced). Regulators promise to improve reliability. But this comes at a cost. Reliability has decreasing marginal returns, costs quickly explode as government demands more reliability. Right now, bandwidth and cloud computing is free, but it means that sometimes when I try to sync my Kindle, it might fail for a few hours. Despite what the regulators promise, there is no such thing as a free lunch, and regulations will not simultaneously keep costs down and reliability up.

How we define the "cloud" means a lot for our future. We are putting more and more of our "stuff" in the cloud, which special interests want to regulate, control, and monitor. We will lose our freedom unless we fight to keep it. Unless we fight to keep the cloud a "commodity", it will indeed start to look like an Orwellian "utility".

The full quote from McCarthy speaking at the MIT Centennial in 1961:
"If computers of the kind I have advocated become the computers of the future, then computing may someday be organized as a public utility just as the telephone system is a public utility... The computer utility could become the basis of a new and important industry."


Sam Johnston said...

Thanks for your interest in my article — I'll concede that he may be considered [one of] the father[s] of cloud computing, but the only other contendor I'd be considering is Douglas Parkhill who expanded the idea and released "The Challenge of the Computer Utility" some 5 years later. This was truly revolutionary thinking, a full half century before its time.

As for your claim that "Cloud Computing is a Commodity, not a Utility", you're mixing issues. Cloud is the delivery of IT, which was traditionally delivered as a product, as a [utility] service — albeit not necessarily a *public* utility. It so happens that the consumerisation and commoditisation of computing has facilitated its delivery at scale using commodity components, but nothing stops me doing it today (albeit uncompetitively) using mainframe technology.

There are a number of other questionable claims (such as the suggestion that utility requires regulation) but given the underlying premise is flawed I'm not going to get into them.

Robert Graham said...

The term "public" utility means regulation, or you would use just "utility" alone. McCarthy uses "public", and he meant "government regulated monopoly" even every sense of the term.

Sam Johnston said...

Did he say he meant "government regulated monopoly" or did you assume that's what he meant? The term "public utility" can refer to a service provided *by* a public body (e.g. government), either directly or indirectly, or to services provided *to* the public — with a shift towards the latter in recent years due to deregulation and privatisation.

In any case the definition of "public utility" (according to Google's dictionary) is "an organization supplying a community with electricity, gas, water, or sewerage". Nothing there about governments *or* regulation.

EscaperPaper said...

I always though the utility comparison was a very handy but ultimately flawed analogy; the only way it ever made a lot of sense to me was as a way to help folks understand the concept of always-there, ready to consume, metered compute infrastructure.

Helps to bridge the gap between "What do mean, that is MY server, mine, mine, mine" and "It's kind of like but a hell of a lot more useful". But utility falls apart once you get past the rough comparison. There's way too much individuation in how it's consumed, which Rob has very neatly illustrated. Timesharing was utility computing; running bazillions of VMs on a very fancy grid is a different animal. I don't have to punch in a password to the electric co. to turn on the lights in my house.

Commiditization of compute via cloud services is definitely well under way, and we're seeing that from the actually racks on up to complex IT services. But that, I'll also quibble with, since server makers, data centers and hosting providers had essentially already done that for many intents and purposes. Dell was selling a hell of a lot of cookie-cutter servers way before AWS came along.

What's changing now is the way infrastructure gets provisioned by services like Amazon, things the user never, ever sees but are seismic to the data center world. That's a new and vitally important part of IT commoditization that is 100% driven by cloud.

And all that said, it's not a stretch at all to credit both of your points of view; cloud computing has enabled the delivery(and the provisioning) or IT as a commodity- but that delivery is on a utility basis.

I don't turn a tap to get pork belly on demand or provision corn futures to run my business, e.g. but I do use (hypothetically) AWS in exactly that fashion. If I'm a software startup , AWS is probably more important to my business than electricity or running water, at this point. A new breed altogether, I'd say.:)

Robert Graham said...

In any case the definition of "public utility" (according to Google's dictionary) is "an organization supplying a community with electricity, gas, water, or sewerage". Nothing there about governments *or* regulation.

If that is your definition of "utility", then everything is a utility, including Starbucks, which supplies coffee to the community.

When you limit your definition of "utility" so that it doesn't include "Starbucks", then it no longer includes "cloud" either. That's because cloud is a commodity.

You could try "metered". But cloud services aren't really "metered" any more than coffee is "metered" by cup size, or tanning salon time is "metered", or gasoline is "metered".

You might try "reliability". But then you get back to government regulation again. Government enforces a one-size-fits-all policy of high-price/high-reliability on utility providers, whereas cloud providers provide a ranging reliability. The $7/month virtual private servers don't provide much reliability, but at insanely low prices.

Sam Johnston said...

Coffee is a product.

Robert Graham said...

SamJ: Coffee is a product.

Coffee is a commodity, listed on the Wikipedia page defining "commodity". It's a fungible item: if you don't buy it at Starbucks, you can buy some at a different store.

Robert Graham said...

Although to be fair, Starbucks "coffee" is more than just the underlying coffee, but the entire morning cafe experience, at which point, I would grant you, it's a "product".

Unknown said...

I agree with Robert Graham.