Early experiments in cloud computing

Cloud computing has many faces, some just beginning to take shape. But at the New York Times and Nasdaq, first steps into on-demand infrastructure show promise

 

The Times didn't coordinate the job with Amazon — someone in IT just signed up for the service on the Web using a credit card, then began uploading the data. "After about 3TB, we got an e-mail [from Amazon.com] to ask if this would be a perpetual load," recalls Derek Gottfrid, senior software architect at the Times.

Then, using Amazon.com's EC2 computing platform, the Times ran a PDF conversion app that converted that 4TB of TIFF data into 1.5TB of PDF files. Using 100 Linux computers, the job took about 24 hours. Then a coding error was discovered that required the job be rerun, adding a second day to the effort -- and increasing the tab by just $240. "It would have taken a month at our facilities, since we only had a few spare PCs," Gottfrid says. "It was cheap experimentation, and the learning curve isn't steep.

Digital Fountain, a digital-media distribution company, uses the EC2 service to deliver mobile videos over the Internet. When the company decided to launch this new offering, "we didn't want to buy our own servers and get the people to do that work," says CTO Mike Luby. So Digital Fountain now streams them from Amazon.com's EC2 servers. Because Amazon.com doesn't guarantee availability, Digital Fountain streams the video from several servers, ensuring built-in backup for its provisioning. And it can throttle the number of servers to match demand as it rises and falls, Luby notes.

Over time, Luby expects to rely on other providers in addition to Amazon.com, to ensure a geographic diversity to keep streaming times manageable, as well as to increase server density without overloading any one provider.

[ Get a complete view of the cloud in our special report.]

There's more to utility computing than Amazon.com. Sun also has its own cloud-based computing platform, Network.com. Unlike EC2, though, it's a grid, meaning it specializes in parallel processing, where a task can be broken into independent steps that a large array of processors can tackle all at once. That limits its use to applications such as rendering, data scrubbing, and image transformation. "Not everything can be thrown at the Sun grid," says Subbu Manchiraj, vice president for technology at Infosolve Technologies, a provider of data management services. But where a task can be parallelized, the benefit is huge, he said.

Infosolve has used the Sun grid for the past 18 months to scrub names and addresses, making sure they are correct (such as verifying the ZIP code and ensuring that the street addresses are properly segmented). "With Sun, we can run 2,000 processors and get the data back quickly." Plus, Infosolve is a Java shop, so its application development skills were easily tuned to the style of Sun's grid apps. That let Infosolve offer its customers a turn-key data-scrubbing service it couldn't afford to stand up itself. "It's an offsite datacenter. And we pay only for what we use," he adds.

The grid's quick scalability has meant that Infosolve doesn't need to worry about balancing customers' loads. But another factor also helps Infosolve avoid worrying about scheduling: The jobs are batched, and customers have no expectation of real-time response. Thus, if resources do run out, the customer won't ever know. Ditto if there's a failure: "We can just restart the job," Manchiraj says.

Testing out online IDEs
Less mature than the cloud infrastructure plays are the app dev and app hosting platforms provisioned over the Internet. These are intended for apps that will be delivered over the Internet and through the browser anyhow, such as online commerce and services and apps delivered to mobile and remote employees. So it's no surprise that most early adopters of these online IDEs are themselves Web-based service providers.