Morgan Keegan invests in resilience

2008 InfoWorld 100 finalist: To gird against downtime, financial services firm rearchitected resiliency into its systems, building a MAN from the ground up

For Morgan Keegan, the dire events of Sept. 11 drove home the importance of resiliency in advance of disaster. Inspired by those who rallied toward recovery in the immediate aftermath, and aware of the importance of operational continuity in the financial services sector, the regional investment firm began its project to build greater reliability into its systems the very next day.

"We could not afford to be down. Everything had to be up," explains CIO John Threadgill. "We needed to provide more reliability and uptime for our customers."

[ See the full list of this year's InfoWorld 100 winners, as well as the other Top 10 finalists ]

For Morgan Keegan, that meant a backup datacenter that could take over, instantly and seamlessly, in the event that its primary facility failed. Servers at Morgan Keegan already had some redundancy built in, but the real trick for the financial services firm was to ratchet up that redundancy without sacrificing performance. Third-party options left Threadgill and team unimpressed; the inherent latency of vendor solutions would ultimately slow down Morgan Keegan's most critical systems.

"We had to go through a lot of vendor testing. We found a lot of vendors promised the whole world but delivered a lot less," Threadgill explains. "While the technology was evolving, we had issues to get creative and smart about, real fast."

So the company set out to architect and build a MAN (metropolitan area network) replete with a physically separate datacenter to serve as backup should the primary datacenter go down.

"We took the clustered environment and split it," creating the second datacenter in raised-floor space 30 miles away, connected by redundant dark fiber running at 52Gbps, Threadgill says.

By keeping the distance less than 50 miles, Morgan Keegan also was able to build in active-active replication -- and complete the whole project without shutting down databases, trading applications, e-mail, analysis programs, IM, and other software critical to the company's trading operations.

Although the build-out touched on systems used by nearly all of Morgan Keegan's 3,000 employees across dozens of locations, trading continued uninterrupted throughout. Central to that effort was the company's reliance on virtualization, which also proved vital in containing the project's overall cost.

Through virtualization, Threadgill and his team offset a significant portion of the project's up-front cost. "We're migrating a lot to the virtual world, so we've not spent so much money," Threadgill says. "Two for the price of one is great to me."

The virtualization-centric datacenter provides automated redundancy, enabling its broker and dealer customers to continue making high-speed transactions and information flow -- essential to remaining competitive in the fast-paced financial services sector.

Perhaps more important, the new datacenter has given Threadgill and team peace of mind.

"Now we know that even if we lost a whole location, we'd still keep running, and we'd be up and running as soon as we lost that facility," Threadgill says.

The proof?

"We've had two telco-related problems where the data services coming into the office were blown out, but the network failed over and customers didn't notice," Threadgill explains.

Seamless performance -- exactly the kind of network reliability the financial services firm needed.

2008 InfoWorld 100 Award winners

Other Top 10 finalists
Accenture topples communication barriers
Centralized IT pays off for ADP
BlueStar sparks electricity services with SOA
IT delivers optimal route for Con-way
Direct Energy mines BI to conserve revenue streams
Global VoIP keeps Eaton connected
Old code proves key to modern IT at Midland
Qualcomm reaches for the clouds
Wells Fargo banks on next-generation ATMs