Lessons learned: Tribune Media rebuilds IT from the ground up

With everything software-controlled, workloads can be moved at will, and VMware’s NSX offsets the need for more than $1 million of network gear

1 2 Page 2
Page 2 of 2

He goes, "Here's what I want to do. I want to complete the virtualization stack with VSAN and set up a storage sandwich. Pure Storage (flash) on top to address high performance needs, VSAN on cheap servers in the middle to support the bulk of our needs, and Cohesity on the bottom for backups, replication, deduplication, and recovery, and then we'll automate the whole thing so the help desk can provision storage and give you millions back in savings over the next three years."    

You can't really argue with that. I asked him how long it will take, and it turns out he already had it running in the lab. So we are moving to a model where 25 to 30 percent of our capacity runs on flash, the lower 70 percent runs in VSAN, and Cohesity, which is coming online soon, does the deduplication and disaster recovery.

The team has set up auto-tiering to move applications to get the storage performance they need. The VSAN is running on commodity servers with JBOD with the software managing everything, and we're getting 2ms to 2.4ms latency with Cohesity doing backups at 35X data reduction.

The outcome is we are going to affect long-term operational costs while increasing capability.

One other example: We are using this software-defined data center architecture to move data centers. My team is doing it with four people, only one external. Think about that.

So it's all about wielding technology, changing the frame of reference of what can be done and how one uses the word can. "Why can't I?" vs. "I can't." 

How have you organized the team?

You have to have an organizational structure, but no one lives in a silo. The hardest part is taking most of the vertical structures and making them horizontal. I would argue I run a devops-type world. If we need to run a project, the project team collaborates with engineering, engineering talks to operations, operations talks to support. It's just a constant cycle. The engineers are third-level support. So if they build something crappy, they're up late at night dealing with it. That's only happened a couple of times.

Value isn't building something every day. My engineers aren't about building; they're about wielding technology. It's more about how are we going to leverage this capability and what can we do with this? Because it's just compute, storage, and networking.

You mentioned using NSX for microsegmentation. Can you expand on that?

Having to put physical firewalls in between everything is just not scalable. I've got 1,200 virtual servers. If those were physical, I'd have another seven racks just for the firewalls. Now I don't have to. And with NSX I can have a security policy and use that to wrap a server, an application, a piece of data, whatever I define, and that's really powerful. We're also using NSX APIs to integrate Cyphort, so we have them looking for advanced persistent threats between all my east-west traffic. I've never been able to think about doing that before. 

And going back to the indiscriminate computing idea, the ability to move from a private cloud to a public cloud, if you have an NSX capability you can ensure security is moving with it because everything is now a file and you wrap that file with a security posture.

Are your physical servers in one location and supporting all your locations?

Yeah, but obviously I have a DR [disaster recovery] site and I replicate everything. We use Riverbed's Granite, which lets us project a LUN. We take our applications and push them out to the edge, so the user feels like they're onsite. Only the changes come back, and if we lose connectivity it still runs. We do that as much as possible for data consistency. We had a user delete everything, totally bombed an app, and we reloaded the thing in five minutes. It works like a champ. 

Beside my idea around indiscriminate computing, I have another one: I don't want data ever leaving my data center. I want to project my data. I want to let you, depending on your rights, interact with it, but I don't want that data floating around on devices. That really reduces our security surface area.

You haven't mention container technology, which seems to be an increasingly popular tech. Do you have any container plans?

A VM is one operating system to one application, while a container is one operating system to many applications, so you can achieve greater density with your hardware. We've done some tests where you get up to 80:1 to 100:1, but of course it depends on the workloads. I think there's a place for it, but we're kind of waiting. VMware is working on it. And we're still in this tactical curve of rebuilding a $2 billion enterprise. 

We've launched everything. We've gone live with our app portfolio. We're operationalizing all that, and we're going in now to optimize it. In mid-2016, we'll step back and take another look at technology; containers are on our roadmap.

Any regrets starting from scratch vs. pushing to keep some legacy infrastructure?

My team got to do a greenfield, and it was a brutal pace. Last year was the hardest that I have ever worked. But we came out of it with a next-generation operating platform (infrastructure and application) so the business can face forward and think about increasing shareholder value.

From an IT perspective, teams tend to either spiral up or spiral down. My team is in spiral-up mode. For example, vSphere 6.2 just came out and my team is excited about what they can do with the technology. Some other shops will be like, "Oh my God, we have to do this upgrade," and spiral down.

It's having a team that's excited about technology and what it can do to help the business. We've tracked a couple of engineers to see how much they can do, and you can see just how much this technology empowers people. It's very different because historically the server guy is the server guy, the network guy is the network guy, etc. I don't have server or network or storage people. I have cloud engineers. Because all of that works as a system. People still have their natural affinities, what they're better at, but to all of them it's just bits. It's interesting to watch how they work together and how they communicate and how much they can get done because there aren't those artificial silos anymore. 

It sounds like you guys are having a blast.

I promise my team: One, you'll never be bored. Two, you'll never have as much fun, because I don't do technology for the sake of technology, it has to have an outcome. And three, we will develop a world-class team where we make people great and where they want to come to work every day.

We live by five operating principles founded on this one fundamental idea: Our job is to create a frictionless enterprise:

  1. We are here to help. Period.
  2. Enable the business to turn around and face forward.
  3. Make systems work for people, rather than people working for systems.
  4. Command technology to provide competitive advantage for the business.
  5. Provide management teams actionable information to make good decisions.

This story, "Lessons learned: Tribune Media rebuilds IT from the ground up" was originally published by Network World.

1 2 Page 2
Page 2 of 2