Amazon, Google, and Microsoft are going up against traditional infrastructure makers like IBM and HP as businesses move their most important work to cloud computing, profoundly changing how companies buy computer technology
Amazon.com's (AMZN) squat Seattle headquarters looks nothing like the country club affairs found in Silicon Valley. There are no free soft drinks or volleyball courts. The light fixtures hanging from the ceiling in the reception area aren't fixtures at all but rather collections of extension cords fitted with bulbs. The receptionists lack computerized systems for registering guests. They simply write down visitors' names on a piece of paper. Such is low-margin life in online retail, where Wal-Mart (WMT) stands at the ready, waiting to take away your extension cords.
Most people recognize this Amazon: Jeff Bezos's hyperproficient Borders-killer; one of the few dot-com initial public offerings that didn't end up a punch line; fount of millions of smiling cardboard boxes bearing everything from dildos to diapers. Sitting in a sparsely decorated employee cafeteria, Andy Jassy pitches a newer if equally thrifty side of the e-tailing giant. Although all shoppers are welcome, this Amazon, he explains, is for business customers and isn't well marked on the home page. It's called Amazon Web Services, or AWS. As senior vice-president, Jassy heads up AWS, which rents out computing power for pennies an hour. "This completely levels the playing field," Jassy boasts.
AWS makes it possible for anyone with an Internet connection and a credit card to access the same kind of world-class computing systems that Amazon uses to run its $34 billion-a-year retail operation. "This will be a very high-volume, relatively low-margin business," Jassy says, sipping his third Diet Coke of the morning. He says he'd like to curb his 12-can-a-day habit, but why bother? If ever there were a time to get hopped up at Amazon, it's now.
AWS is growing like crazy. Although he won't cite exact numbers, Jassy claims "hundreds of thousands of customers" already use the service, and analysts at UBS (UBS) estimate Amazon will do about $750 million of business on AWS this year. In fact, a whole generation of Internet companies couldn't exist without it. Netflix's (NFLX) movie-streaming empire runs on it; Zynga, the social gaming company, uses it to handle sudden spikes in usage. AWS has become such a fact of life for Silicon Valley startups that venture capitalists actually hand out Amazon gift cards to entrepreneurs. Keeping up with the demand requires frantic expansion: Each day, Jassy's operation adds enough computing muscle to power one whole Amazon.com circa 2000, when it was a $2.8 billion business.
The physical expansion of all that data takes place in Amazon's huge, specially designed buildings—the biggest can reach 700,000 square feet, or the equivalent of roughly 16 football fields. These interconnected facilities, scattered all over the world, are where AWS conducts its business: cloud computing. The "cloud" refers to the amorphous, out-of-sight, out-of-mind mess of computer tasks that happen on someone else's equipment. For the past five years or so the cloud has been hyped by companies to mean anything that happens on the Web, which is how "cloud computing" came to rival "social networking" in overuse.
Right now, the cloud is small: It represented about 5 percent of the $1.5 trillion in corporate information technology spending last year, according to industry data supplied by International Data Corp. and Gartner (IT). Yet the phenomenon of businesses moving their most important and innovative work into the cloud is real and is profoundly changing how companies buy computer technology.
One other thing about the cloud: It's turbulent. It's getting to be war up there.
The battle roughly breaks down to two sides. On one is Amazon and two other rising superpowers of the cloud, Microsoft (MSFT) and Google (GOOG). That's right, the company that once touted itself as "The World's Largest Bookstore" sits alongside a search company and PC software giant as a leader in new business technology. These three companies are hardly allies, of course, but each poses a threat to traditional infrastructure makers such as AT&T (T), EMC (EMC), Hewlett-Packard (HPQ), IBM (IBM), Oracle (ORCL), and Verizon (VZ).
Those giants collectively control the other 95 percent of the market and sit on vast cash reserves earned by steering the creation of data centers inside large corporations. They specialize in selling and servicing high-margin products and reap the consulting fees that arrive from helping companies manage their equipment. These global powerhouses might seem invulnerable, but they must take Amazon, Google, and Microsoft seriously. The cloud war is only partly about next quarter's market share; it's also a culture clash.
Amazon, Google, and Microsoft have attracted a generation of converts to their message that technology can be fun again. They are the cloudpeople. This species has a vision of freedom and excitement. They see a world of innovation, where the digital engine rooms of the economy transform from unwieldy assemblies of parts into smooth-running dynamos. The cloud, they believe, can unleash the same kind of creative ferment going on in consumer tech, where smartphones and tablets have captured people's imaginations.
"Things are downright Darwinian right now," says Mike Olson, the chief executive officer of Cloudera, a startup that specializes in data analytics software. "There hasn't been this type of Cambrian explosion in corporate technology in 20 years."
On a January evening in Mountain View, Calif., about 80 programmers cram into an unadorned conference room, helping themselves to tables full of pizza and Blue Moon beer. The crowd is mostly male, mid-40s. There are a lot of T-shirts, gray hair, and ponytails. These are rank-and-file members of the Silicon Valley Cloud Computing Group, and they're here to learn.
At around 6:50 p.m., the attendees drift to the rows of tables and chairs where they've set up their laptops for the evening. At 7 p.m. sharp, a young, skinny guy at the front of the room calls for attention. Sebastian Stadil, the 26-year-old founder of the club, declares the meeting in session and cedes the floor to the guest speaker, Shashank Tiwari, a local cloud developer. The topic of tonight's three-hour presentation: working with the "sloppy quorums," "gossip-based membership protocol," and other fine points of a database technology invented at Facebook called Cassandra.
Stadil runs these night classes in his spare time. He's a dedicated cloudperson with his own software startup and feels it's his duty to educate his peers about the coming era. The admission fee is $20 a session, which covers the pizzas and beer. (The conference room, in the offices of prominent Bay Area law firm Fenwick & West, he finagles for free.) Attendees come to network and hear experts opine on Voldemort, Hadoop, Puppet, Chef, and other playfully named types of cloudware. "When the meetings started two years ago, it was mostly people in their twenties," Stadil says.
Since then, as cloud technology has crept through corporate IT departments, the crowds have gotten older. Much of the work done by people who managed computer systems is getting automated, which is forcing managers and engineers to upgrade their skills. Also driving the urge to learn: New IT tasks tend to be more intellectually demanding and nuanced than those of previous eras. Companies that used to dig through six months of customer data to tease out sales trends—"what's the average selling price" and "who's the biggest customer"—now use tools built by Google and other Web specialists to plow through 10 years' worth or more. And they're asking such computer-intensive questions as: What do people like? Why do they like it? What's the optimal coupon to send to their smartphones as they shop for it in aisle 6 at ShopRite? As Stadil puts it, "Companies don't need the guy who is good with a shovel anymore. They need a guy who is good with a bulldozer."
After the evening's talk, some of the old-timers grouse that cloud computing is just a new name for an old idea. It's true that technology types have long used a little picture of a cloud as whiteboard shorthand for "the Internet," or "the ethereal place somewhere on the network where work gets done." What Stadil's classes are about, though, is something more specific, and that message got through loud and clear to the other attendees. Some of them almost sound like true cloudpeople. "This time things really are different," says Robert Harker, a veteran systems administrator.
What's different this time—as compared with the rise of the mainframe or the PC—is scale. As the consumer Web exploded, the global mass of computer data went supernova. This year, according to IDC, the world's digital universe will reach 1.2 zettabytes, or 1.2 quadrillion megabytes. If you take every word ever written in every language, it's about 20,000 times that.
To cope with the onslaught, Web companies have built dozens of megadata centers—often at $500 million a pop. Some of the players—and here we return to Amazon, Google, and Microsoft—pioneered many of the techniques needed to deal with hundreds of millions of people trying to access unprecedented volumes of information. These companies built software that spreads data across the globe, automated the command and control of tens of thousands of servers, and refined data-center designs to be more energy-efficient. They had to out-innovate the big guys and do it on the cheap because their customers were consumers who weren't locked in with fat contracts and dedicated account reps. The next logical step for the cloudpeople: Sell the capabilities of their new supersystems to customers who would pay—other businesses.
In the early 2000s "software as a service" companies such as Salesforce.com (CRM) pioneered the model of renting niche applications to customers by the month. Hosting companies also offered to run companies' software in their far-off computing centers for monthly and yearly fees. But it was the Web companies, led by Amazon, that went a step further and sold these types of services on a utility basis. A company could fire up hundreds or thousands of computers on the fly and turn them off when the work was done. Instead of monthly or yearly contracts, customers pay only for what they use in computing cycles, bandwidth, and storage. "Pay-by-the-drink pricing seemed natural to us," Jassy says.
The most obvious appeal of such cloud services, of course, is the potential to save money. Business buyers have gone through decades of technology transitions, tacking new hardware and software onto the old. The average corporate IT department has to deal with the dreaded 70/30 rule where they spend about 70 percent of their technology budgets just trying to keep this jumble of products running, leaving only 30 percent to chase new ideas. Chief information officers have to buy equipment by the ton to meet spikes in computing demand or prepare for disasters, and then watch as that gear sits idle most of the time. While companies struggle to deal with this mess, they've become overwhelmed by the influx of data in the Internet Age. The promise of the cloud is shoving the costs of dealing with all that off the bottom line.
The cloudpeople suggest a daring complement to the cost-cutting: Adopt the rapid-prototyping, beta-testing lifestyle of the new era. Relinquish control of your technology infrastructure—you don't need it anymore. Let employees toss out new services and see what sticks. Innovate with impunity. "In the cloud," says Tony Scott, CIO of Microsoft, "there's no penalty for guessing wrong."
"This stuff finally works," says Andy Bechtolsheim. "If you're a startup, you would never build a data center again." Bechtolsheim co-founded Sun Microsystems and built the company's first computer workstation, which itself set off a revolution of lower-cost systems for businesses in the 1980s. He also made billions cutting some of the first checks to Google and VMware (VMW), the big enterprise-software maker. As Bechtolsheim sees it, the cloud is a foregone conclusion for new companies such as his latest venture, Arista Networks. The only hardware of any significance on the premises of Arista are the high-speed switches it sells to companies building large-scale cloud computing systems. Everything else—e-mail, order entry, customer management, Web software—sits on someone else's hardware and is someone else's problem.
Or take the case of Etsy, an online marketplace for handmade goods. It rents hundreds of Amazon computers every night to analyze data from the 1 billion monthly views of its website. When Etsy's engineers get to work in the morning, they have a wealth of data showing what types of clothes, furniture, and jewelry appeal to what types of people. Etsy has used this information to create product recommendation systems that let people rank their interest in a series of products and then end up with a list of 100 or 200 they might like. Consumers can also grant Etsy permission to trawl through their Facebook accounts and find products friends might like as gifts. If, for example, Etsy sees a person "liking" something related to Michael Jackson, it will offer up some Michael Jackson dolls or vintage clothes. Jason Davis, the lead scientist at Etsy, loves that he's got his hands on technical tools that used to be available only to retailers the size of Gap (GPS) or Ikea. "This wouldn't have been possible five years ago," he says.
Five years ago that flexibility eluded big companies, too. For all their resources, large institutions tend not to encourage technological initiative. The classic chain of events: When someone in an IT department wants to try out a new idea, that person has to justify the expense in budget approvals and PowerPoint supplications, and then wait weeks, if lucky, for the gear to arrive and start working properly. Heaven forbid someone in marketing or sales wants to try out a new idea. That means more approvals, justifications, and delays.
It was this type of tech-hungry underling who drove the initial interest in the cloud. Unbeknownst to their bosses, software developers were running off to cloud services—in particular, Amazon's—to test out ideas, often using their own credit cards to pay for work. For the first time, cube dwellers with ambition had the freedom to tinker on serious computing systems without much overhead. When something clicked, the developers could then take it to their bosses.
Just such a thing happened at Northrop Grumman (NOC), where a research and development team used Amazon's cloud service to develop an advanced cybersecurity system. The team rented Amazon's computers for less than a day to train machine-learning algorithms on more than 1.3 million files. When Justin Kessler, an applied mathematician at Northrop Grumman Information Systems, showed the results of the project to his managers, they were struck by the low price tag. "It was eye-opening, and they were thrilled at how efficiently we solved the problem on someone else's infrastructure," he says.
Other global corporations such as Bechtel, Eli Lilly (LLY), and Pfizer (PFE) have experienced this ground-up push. So, too, has InterContinental Hotels Group, which manages 650,000 rooms at 4,500 hotels worldwide. IHG has started a project to revamp its technological lifeblood, its central reservation system. The current version relies on decades-old mainframe technology; the new one is coming to life on Amazon's cloud. Bryson Koehler, a senior vice-president at IHG, says the company considered keeping the software in-house, just on upgraded hardware. "But this is a six-year journey," he says. "The last thing I want to do is move over to current technology and then be outdated by the time we finish the project." Koehler expects that pooling all of the company's data into the cloud will let IHG deliver inform