The five missing pieces of SOA

Maturing Web services standards make an SOA seem practical. But should you start now or wait until they fill in the gaps?

The high concept of SOA (service-oriented architecture) continues to enthrall IT. Yet SOA’s promise of universal application integration is vague at best, confounding anyone who takes a closer look. Such scrutiny reveals major gaps -- in reliability, security, orchestration, legacy support, and semantics.

Peter Underwood, vice president of software development at brokerage firm Wall Street Access, says his team has had to do some serious thinking up front before planning SOA integration.

“You begin with the idea that [SOA] is bigger than a bread box. In other words, it’s just a framework,” Underwood says. Although SOA “has taken on a life of its own because of Web services” standards, Underwood believes a significant gap remains between Web services’ potential and its current capabilities.

Execs are happy to use Web services for simple needs, such as feeding information to Web-based portals. But complex, mission-critical jobs are another story -- and may demand Web services standards that are still under development. So when is a Web services SOA strategy advisable, and when is good old EAI better? It all depends on what you are trying to do and which gap in Web services’ capabilities you encounter.


The need for highly reliable, asynchronous messaging may be the most difficult to meet, at least in the short term. Aiaz Kazi, general manager of business integration at EAI stalwart Tibco, calls this kind of messaging “critical to enterprise-quality integration.”

Sam Boonin, vice president of marketing at Web services infrastructure vendor Blue Titan, says the need for reliability is similar to that which “we’re used to discussing in other computing paradigms. SOA is not quite ready for the utmost transactional reliability -- nonrepudiation, once-and-only-once delivery, and rollback -- but it’s only a matter of time until the standards and implementations mature to meet that requirement.”

In fact, several draft Web services specifications already address issues in mission-critical and lengthy processes. WS-ReliableMessaging, for example, is designed to guarantee that a SOAP message arrives at its destination. WS-AtomicTransaction, WS-Eventing, and several other proposed specifications would define ways of handling complex, stateful, and long-running business transactions. But unlike many security-related protocols (see below), widespread use of reliability standards such as these have yet to be realized.

Until then, says Chris Crowhurst, vice president of enterprise architecture at Thomson Prometric, a provider of computer-based testing and assessment services, “Reliable messaging [for Web services] is quite a burden. But at the end of the day, applications just need to build around it” because of the benefits of the interoperability Web services provides.

For now, “building around it” means coding applications to anticipate and to accommodate error conditions. It also means buttressing point-to-point SOAP interactions with an intermediary -- such as a Web services management broker -- that provides a standardized layer of abstraction. Available from independent players such as Actional, AmberPoint, and Blue Titan, these products enable managers to provide fail-overs and upgrades to software endpoints with minimal interruption to the production systems. (Useful Web services management must work across a range of platforms, which explains the absence of similar solutions from such major vendors as BEA, IBM, and Microsoft.)

On the other hand, as Dan Foody, CTO of Web services management vendor Actional, notes, “Not every problem requires the same kind of reliability.” Those that must be ironclad tend to be asynchronous, long-running transactions with many interdependencies such as complex financial transactions. For less demanding jobs, SOAP over HTTP works fairly reliably -- particularly with simple, synchronous interactions.

Rajan Jena, an enterprise architect at Bristol-Myers Squibb subsidiary Oncology Therapeutics Network, uses both conventional messaging-oriented middleware and Web services in his company’s integration infrastructure. He says that messaging solutions are appropriate when transaction volume is high and is transmitted in batches. On the other hand, Web services are best “when volume is low -- but it has to be totally real-time.”

Click for larger view.


Authorization, authentication, and encryption raise a serious red flag for IT managers contemplating Web services-based integration. Traditionally, access control has been a matter of requiring a log-in and authentication. In the distributed Web services world, where components of one application might easily go off and talk to components that live in different domains, keeping disparate but interconnected systems secure is a far more complicated problem.

As with reliable messaging, a bevy of standards for Web services-style interactions have been proposed. Two are particularly important and are being implemented widely: WS-Security and SAML. The former describes a highly extensible framework for itemizing various facets of a system’s security capabilities, whereas the latter defines a standard process of transmitting assertions to facilitate SSO (single sign-on) models of authentication.

For enterprise SOAs, analyst Phil Wainewright of the lively Loosely Coupled discussion site ( singles out a third, though not yet mature, proposal. WS-Policy would provide a means for different systems to declare what sorts of security mechanisms they require before accepting connections. “Without it, your ability to loosely couple across domains [would be] severely limited,” Wainewright says.

Although no proposed standard  meets the Web services security challenge by itself, as a group they provide a foundation on which to plan an ongoing strategy. Indeed, the Web Services Interoperability (WS-I) vendor and user group recently ratified portions of WS-Security for its Basic Security Profile, a collection of best practices designed to ensure interoperability. The organization is expected to add support for SAML and other standards in the near future.

In the short term, a relatively practical way to secure Web services is to employ the the techniques used to secure standard Web-based applications: transport-level mechanisms such as SSL.

One advantage of SSL is its support for two-way authentication in the form of client/server certificates, but the ongoing maintenance of a certificate infrastructure for a large number of clients can be a challenge. A complementary option that is available today is to employ generalized XML-savvy firewalls -- such as those from DataPower, Forum Systems, Reactivity, and Westbridge Technology -- and XML digital signatures to provide an additional measure of prevention for Web services network traffic.

After a distributed security architecture is in place, IT managers should find a silver lining in the security cloud. Ongoing maintenance of application security has often fallen on the shoulders of programmers -- a poor use of their skills and a potential security hazard. With Web services-based security, it becomes much more feasible to restrict that responsibility to authorized operations staff alone.

Oncology Therapeutics Network’s Jena describes this security approach as critical in his company’s transition to an SOA for its CRM and order management systems. “That means we can implement [security] from a policy perspective, as opposed to developers actually coding it” into the applications, he says. “We see that as a major advantage. It actually makes the applications more secure.”


The coordination of distributed software components for the purpose of creating meaningful business processes is at once the most complex and the best-suited to service-oriented styles of integration. The reason is clear: In the simplest of terms, applications built on SOAs are designed to be pulled apart and reassembled as needed. As the backbone of today’s BPM solutions, “orchestration,” as it is called, enables IT managers to string together new meta-applications from the functionality of packaged or homegrown applications already in place.

Blue Titan’s Boonin says this capability is important to many adopters of SOA integration strategies. “[They] are building composite applications on top of [SOA infrastructure] and orchestrating services into reusable business processes,” Boonin says. On the surface, assembling these pieces to guide the flow of a business transaction is a straightforward task. After all, the primary stumbling block for BPM solutions has been the monolithic nature of software. Web services, on the other hand, break software into components, each of which relating to a single business function.

The biggest challenge isn’t to create the modular applications but to change the way those systems represent the data they process. Thomson Prometric’s Crowhurst counsels “identifying a schema at the heart of the business,” rather than designing process flows around individual systems and their data records in isolation. In other words, it’s best to think about the output of business processes as documents that contain answers to each major question along the way. XML, the foundational element of Web services, enables this conception and makes it happen. “Thomson Corporation is XML-obsessed,” Crowhurst says.

Several standards for orchestration and BPM have been proposed. One of these, BPEL (Business Process Execution Language), has broad industry support and even appears in a number of BPM products -- although everyone concedes that the robust version, BPEL 2.0, is approximately 18 months from completion. Complementing BPEL, WS-AtomicTransaction and WS-CAF (Composite Application Framework) are designed to facilitate the complex transactions that make up long-running and stateful business processes.

BPM pure-plays, traditional EAI companies, and the major platform vendors all see promise in the ability to visualize, execute, and monitor processes using this new, standards-based form of BPM. Enterprise application vendors such as SAP and Oracle, which bought the BPM pure-play Collaxa this year, are also on board. SAP is taking a radical approach, actually redesigning its software to incorporate its own version of BPM-focused middleware, dubbed NetWeaver. Finally, XML-centric data stores specifically designed for SOAs, such as Blue Titan’s Data Director and Software AG’s Tamino, will represent an increasingly important aspect of addressing business process schema.

Legacy support

The software solutions used to connect applications with legacy systems are not unlike the kits that world travelers carry in their briefcases. Just as the right sort of dongle allows an American visitor in Greece to plug a laptop power cord into the wall, a specialized application adapter allows one system to pull data from another.

And just as each country seems to have its own standard for power outlets, each legacy system requires its own adapter. For years, EAI vendors emphasized their portfolios of legacy application adapters as key differentiators -- and as sources of revenue.

Fortunately, as standards such as Web services increasingly dominate interoperability, the cost of simple connectivity has fallen. And third-party application adapter specialists such as Attunity and iWay have increased competition and have vastly expanded the number of connectivity options available today.

A sometimes overlooked benefit of legacy application adapters is that they mask the complexity and obtuseness of many proprietary APIs. The role of a good adapter is not unlike that of a well-composed Web service: It provides a layer of abstraction that insulates the rest of the application infrastructure from all sorts of messiness. Some vendors, such as Software AG, have specialized in the “semantic integration” of legacy applications to XML-based integration backbones.

Packaged applications from companies such as Oracle and SAP already provide some degree of support for standards-based connectivity, typically by wrapping a proprietary API, such as SAP’s Business API, with a SOAP interface. As application vendors make use of Web services and add middlewarelike functions to their own products, the standard method of integrating packaged applications will come to reflect the best practices of SOA.

Even so, dealing with expensive, proprietary application adapters will continue to be the only option for connecting many truly archaic systems to a common integration framework.

There is one silver lining for IT shops with mainframes from companies -- such as IBM -- that have aggressively ported basic Web services stacks to their antique equipment. Ironically, their rigid APIs can be far simpler to expose through SOAP than untangling the mess of APIs found in many client/server systems.


Defining the business meaning of transactions and data is the most intractable issue that IT managers face. Although the semantics challenge predates Web services -- vertical industries have been developing their own XML schema for years, for example -- SOAs bring semantics to the fore. In fact, semantic interaction is a central part of well-designed SOAs.

1 2 Page 1
Page 1 of 2