Will the financial meltdown slow IT innovation?

Capital markets have been key drivers of cutting-edge technology, calling into question the impact the financial crisis will have on the vanguard of IT

You might hate Wall Street -- and who doesn't this week, but if you work in IT you owe the Street a vote of thanks because the fat cats in financial services have been a major driver of technological innovation.

Given that role, with the markets now in meltdown mode, you might think that cutting-edge technology progress would slow.

[For a deeper look at IT innovation in the financial industry, see "How IT could have prevented the financial meltdown."]

I thought that, too. But after a series of conversations with some very smart folks at Sun Microsystems and IBM, I have a different take. Bear with me as I explain why.

Latency is the enemy
From an IT perspective, the segment of the financial services market that demands the most innovation is the capital markets: the investment banks, hedge funds, and electronic markets. Where the traders work, in other words. Banking and insurance companies are major IT buyers, to be sure, but they tend to be a beat or two behind the cutting edge, says Ambreesh Khanna, the head of Sun's Global Financial Services Group.

Traders are dealing in a slice of time so thin that they are measured in milliseconds. That's infinitely faster than any human can think, of course, but much of the trading that takes place around the globe is computer- and algorithm-driven. "One millisecond can equal $100 million in trading opportunity," says Khanna. And that means the real battle is a struggle to reduce latency.

That struggle plays out across hardware, especially the CPU, the network, and the software stack. Consumers are often baffled by the push to increase the speed and capacity of silicon, but Wall Street isn't. They're the ones buying massive servers running multiple multicore chips, made by Sun, IBM, Hewlett-Packard, Dell, and others.

Before I go further, let's admit that both Khanna and IBM's Suzanne Duncan, whom I also spoke with, toot their companies' respective horns a bit when describing the cutting-edge work driven by trading.

Nonetheless, even with such self-promotion factored in, here's a real-world example of Sun's work on latency that goes beyond the obvious hardware improvements the company has developed. Solaris has always used a container model -- apps running in a virtual slice of the OS on a single server. Generally, if an application needs to send data to a second app in a separate container, the data goes all the way back to the networking card and travels through multiple layers of the software stack. The result: latency.

Now Sun's TCP Fusion short-circuits that process and moves data directly from one container to container. The result: less latency as data moves at backplane speed.

"We used to talk about reducing latency to tens of milliseconds; now we talk about single-digit latency," says Khanna, a veteran of more than a decade at Sun.

Global markets demand innovation
Understanding risk in the global financial markets is extremely complex, and it's clear that the lack of insight into the complex derivatives developed to make money during the housing boom is an important contributing factor to the current crisis.

Foreign exchange, a huge market, is already made more rational by a mechanism called Continuous Linked Settlement, which ties together major banks and other players to give them a real-time view of risk, says Suzanne Duncan, financial markets consultant at IBM's Global Business Services.

Meetings are already happening with regulators, including the Securities and Exchange Commission and the Federal Reserve, to plan a similar system that would finally bring transparency to derivatives, Duncan says. (IBM, she notes, is a major technology provider to the CLS.)

That type of system won't be cheap. But Duncan, who speaks regularly with directors and executives of major financial services players, says those companies believe that targeted financial investment in advanced infrastructure is a key to success in today's very difficult environment.

Although those firms tend to have excellent transactional capacity, the crisis has produced huge spikes in volume that test the infrastructure. It's fairly obvious that building baseline capacity to handle those surges is wasteful. So expect those companies to invest in advanced cloud computing as a way to scale on an as-needed basis, she says. Major firms are already planning $1 billion and $2 billion cloud initiatives, an obvious boon to IBM, which has positioned itself as a provider of cloud technology.

That's not to say IT spending as a whole is likely to increase. It won't; at least not in the immediate future.

But as the wounded top-tier players pull back, the second tier "is smelling blood," Khanna says.

Indeed, capital markets companies that are doing reasonably well are planning to spend more on IT next year than this year, says Duncan. "They want to place a strategic bet that they can get ahead of their weakened competitors," she says. It is worth noting that the sentiment is noticeably strong in Asia, says Duncan, who just returned from meetings in that part of the world.

Khanna agrees: "The Asian operations of the tier-one banks are in turmoil. Their competitors will go after their clients, and the battle will start again as the top-tier stabilizes and tries to win back lost business."

Without a doubt, we're all in for a very rocky ride. There will be lots of disruption in IT markets as companies such as Goldman Sachs and Morgan Stanley morph into commercial banks, Bank of America digests Merrill Lynch, and the financial system copes with the costs of an enormous bailout.

But as Khanna and Duncan see it, there may be a pause while people take a deep breath -- but innovation will not stop.

(Disclosure: I hold a small number of shares in IBM.)

Send your comments to bill_snyder@infoworld.com.