How I learned to stop worrying and love the private cloud

With the right mix of embedded IT experience and managerial support, private clouds can benefit business IT and users alike

It's a Monday morning, and as with most Monday mornings, things are already spiraling out of control. Today it's more than the usual fallout from server maintenance and the oddly reliable number of people who forgot their passwords over the weekend.

Accounting just called to let you know it bought a new application and "needs a server" for next week when the vendor is coming to install it. Closer inspection reveals that the new accounting package is a complicated, multiple-tier application that will need five new machines and a sizable brick of SAN space to implement. Then the head of software engineering drops by to remind you he's still waiting for the three new dev environments he requested close to a month ago.

Just when you're wondering whether your day could get any worse, the Pointy Haired Boss graces you with his presence. As usual, he has returned from the executive suite carrying an IT trade rag packed to the gills with hype and buzzwords. Flipping to a page marked by a pink Post-it, he gestures wildly and happily exclaims, "We should do this! It's all the rage! Make it happen!" He then promptly wanders off to find his next victim. This time, it's not some vague piece on SOA or porting your entire business application stack onto the iPad -- it's a big fat article about the private cloud.

The unfortunate reality of scenes like this keep Scott Adams clothed and fed. Yet this time, through some accident of fate, the PHB's suggestion may actually have some merit. Believe it or not, he may have just provided you with a way to make your Monday mornings slightly less horrifying.

Isn't virtualization enough?

To many of us, the private cloud sounds like a fancy name for something we're already doing: virtualizing the data center. That tends to be a huge piece of it, but to stop there is to miss the larger picture of what the private cloud has to offer. If you support business units that contain some level of IT expertise or regularly deal with third-party vendors who are charged with the installation of new business applications, incorporating some aspects of a private cloud into your infrastructure may save you a tremendous amount of time and, amazingly, make your users happier.

If you're already substantially virtualized (as most enterprises are today), you've seen some of the benefits of what a private cloud can offer. Resources are heavily consolidated, allowing you to draw CPU, memory, and storage from a single pool and share it among different applications -- thereby increasing resource utilization and making the data center more efficient. Likewise, scaling the capabilities of a server virtualization stack is far easier. Running out of CPU bandwidth or memory? No problem -- just add another host or drop in some more RAM. It doesn't really matter which server is eating it up; it's all coming from the same place.

The problem with simple virtualization

Virtualization can't solve all problems by itself. Let's take the servers for the accounting department as an example. First, the good part: If the new accounting application can be virtualized (which is increasingly common, thankfully), you probably don't need to spec hardware, get the capital equipment purchase approved, and wait for it to show up before you can get the ball rolling. All you need to do is allocate a chunk of your virtualized infrastructure to the new application.

Problem is, that's not as easy as it seems. First, you're probably going to spend a lot of time going back and forth with the software vendor about system requirements and translating them into VM specs -- and you're going to have to figure how much to charge back the accounting department for infrastructure. This slows down the process and contributes to the perception that you're in the way of progress. Worse, if you're not careful about how you deploy the VMs and fail to communicate effectively with the project sponsors, the accounting department may not know what sort of service level to expect. How do you know whether they're bumping up against the artificial resource limitations you've put in place? How do you handle "upgrading" them from an administrative point of view?

Also, bear in mind that when you charge for infrastructure, you'll be doing so for hardware (say, more SAN space and an additional virtualization blade) that the accounting department does not strictly "own." That can be a sticky business. You may be able to fudge this by charging back what it might have cost to deploy the application on physical servers, but then how do you deal with your own capital expenses when it comes time to upgrade the virtualization farm?

In other words, while it's true that virtualization has reduced time and effort spent on server management to a fraction of what it used to be, managing environments with high levels of turnover without losing administrative oversight can be very difficult. Ironically, this is mainly because virtualized environments are so easy to change.

A great example of this is the development environment craved by our hypothetical engineering department. The engineers would love to be able to stand up a raft of servers, test some software under different conditions, and roll it all back to the way it was without picking up the phone to call you. And in different stages of their development cycle, they might need one or two boxes, or they might need ten. How can you react to that quickly enough to keep them happy? How can you even hope to figure out what to charge them for the privilege? You could give them their own virtualization hosts to play with, but then you're stranding a bunch of top-tier performance that you could be sharing with production loads.

The solution

The more I've delved into it, the more I see the the private cloud -- or implementing cloudlike policies -- as an effective way to solve the problems incurred by virtualization. Note, however, that a private cloud doesn't have to be an all-or-nothing affair. It can be rolled out incrementally on an as-needed basis. Even though a number of hardware and software vendors imply that you need a whole rack of new hardware and an industrial-size barrel of software spaghetti to build a real private cloud, you can easily graft individual cloud features onto existing virtualization infrastructure.

In a fully developed cloud, our Monday morning disaster might have been avoided entirely. Instead of popping the server hardware question on you at the last moment, the accounting department would have known about the Web-based self-service portal, which it could have used to work with the software vendor to determine the VM specs and the likely quantity of first- and second-tier storage. And the accounting folks would know exactly what it was going to cost, enabling them to align their requirements with their budget without going back and forth with you.

Once the specs were set and the software prepared for installation, the system could automatically deploy everything to a secure network segment -- access to which the accounting deparment could control on a granular level. All of this could potentially happen without IT's involvement. Better still, when the new accounting application turns out to require more resources than first thought, the accounting folks can see exactly how much resources they're using, know what it will cost to upgrade, and initiate the upgrade on their own. Likewise, the engineering department could build, start, stop, and destroy VMs to its heart's content -- all while being charged only for the resources actually used.

Is there a catch?

Simply put, yes -- there is a catch. Several of them, in fact.

First off, implementing all of these features -- self-service portals and chargeback software -- is not free. You also will need to invest a significant amount of effort in building the policies that include the specifications and costs of the various "products" you're going to list for "sale" in the portal. Doing this incorrectly may result in undercharging business units for the services they consume or failing to deliver on service-level agreements.

You'll also need a lot of help from management -- right on up to the CFO -- to get buy-in for a sensible chargeback scheme. In some organizations, this may simply not be possible. If it isn't, a large portion of the benefit of implementing a private cloud dissolves. You really can't allow self-service cloud provisioning without some balancing cost component; otherwise, business will run rampant deploying new resources. However, you can still offer self-service management, which may be of benefit to the hypothetical engineering department.

Beyond that, the amount of effort you'll invest in monitoring available resources -- both compute and storage -- will be greater in a private cloud environment. Since business units can deploy new resources at a whim, keeping tabs on what resources are being consumed and ordering new blades and disks before they're required is crucial.

And it goes without saying: Deploying a full-featured private cloud in an environment where business units do not have the scale, interest, or ability to take advantage of its self-service capabilities would be a massive waste of time and money. Not everyone has business units with the IT expertise necessary to deploy and manage their servers, even with a portal.

Or to spin it in a positive way: Many, many larger businesses can benefit from the power of the private cloud. The really hard part is getting everyone to agree on the rules, rates, and policies for the automated system. Running that gauntlet may incur a string of miserable Mondays, but once you come out the other end, your entire workweek may look a whole lot better.

This article, "How I learned to stop worrying and love the private cloud," originally appeared at Read more of Matt Prigge's Information Overload blog and follow the latest developments in storage at For the latest business technology news, follow on Twitter.

Copyright © 2011 IDG Communications, Inc.