Back when SOA first started getting traction, the goal was simply to make application functionality available as a shared service. Companies made up their architectures as they went along – and of course, they’re still doing that. The difference today is that, in the last couple of years, the business side has a better sense of the strategic value of IT, while IT has learned more about the competitive pressures business must endure. As a result, SOA now offers the possibility of greater alignment between IT and business than ever before.
Comcast builds its SOA on domain expertise
It’s tempting to start buying ESB, registries, and other tools when you decide to adopt the SOA approach. But that misses the key value of SOA, which is to have a way to align the applications you create and deploy to the business processes they execute, says Tom Adler, senior manager of application architecture and IT governance at Comcast. And starting with the architecture can help ensure you have the right framework to do that – both now and as needs change over time, he says.
“When we started this effort 18 months ago, we resisted the temptation of bringing in vendors. We brought in subject matter experts instead and figured out what we needed first,” Adler says. “All the vendors just wanted to sell us an ESB.” The architecture effort does more than set the framework, he notes: It also begins identifying where you already have redundancies, both in business processes and in applications. That’s key to getting business buy-in, Adler says, because it shows in very real terms where there are opportunities for savings that will help justify the eventual SOA infrastructure and tool investment, as well as show where the use of common services should help reduce maintenance and integration complexities, allowing more responsive development efforts in the future. “It’s the target that lets us eliminate redundant services,” he says.
After developing the architecture – what Adler calls the common domain model – Comcast’s next step was to develop the governance framework for the service development and deployment. “Services need to go through the governance gate,” he says, “otherwise no one will know that the services exist or follow the right policies and procedures.” Only services that pass the governance gate are added to the service registry and thus made available for others to reuse.
One governance challenge that came up quickly was deciding who owned the services. Comcast is fairly decentralized, so the culture naturally supported having the service originators own the services in their domains, Adler says. Common services, such as single sign on, reside with IT – their natural domain, he adds.
One step that Adler now realizes Comcast missed was developing a common data service model after defining the architecture. By not having standard data services to access corporate information and manage interactions across systems, developers have ended up designing their services to get the job done in different ways, leading to inconsistencies that break the SOA promise of allowing an easy mix and match of service components. “We underestimated the value up front,” Adler says, and the price has been reworking some services to impose that model after that fact.
The architectural focus of Comcast’s SOA effort has helped the concept be applied more widely than if viewed merely as a technology issue, Adler believes. For example, because Comcast didn’t start with the view that SOA means the use of Web services, the company has applied the SOA concept to all its efforts, not just those that are obviously Web-enabled. “A Web service is just one way to expose a service – it’s just an implementation detail,” he says. One result is that much of the initial internal SOA efforts were in fact directed at the legacy applications, reducing the integration points both within and outside the company (such as with billing vendors), a major pain point for the business.
Developers use a variety of tools and programming languages, for example, as best fit their knowledge and the application they are creating. By standardizing processes and policies instead of specific tools and technical methods, Adler says developers can better adhere to the architecture’s intent rather than trying to shoehorn each effort into the limitations or assumptions of a specific tool or technology. There’s also a practical reason to allow technological heterogeneity under the common architecture, he notes: In a 90,000-person company, it would be unrealistic to get everyone on the same methodology.”
A company of that scale must also adjust to changing business needs and technology opportunities, Adler says. It’s important to revisit the reference architecture periodically so that it doesn’t become a straitjacket or a document that everyone ignores; either way, you would lose the benefits of SOA. Adler revisits the architecture each month, though it is changed less often than that.
Leapfrog keeps its SOA options open with open source
It’s the classic problem faced by IT developers everywhere: Applications developed during the year by different groups just don’t work well together when brought into a common system such as a Web portal. The dilemma hit home for Leapfrog Enterprises in early 2007 when the toy company wanted to deliver its various applications in a consistent way to suppliers and customers alike, wanting to take better advantage of Internet-based commerce and transactions. In March, it decided it needed a new way to develop applications, so it began an SOA effort whose first efforts are now bearing fruit, says Eugene Ciurana, the director of systems infrastructure. “We wanted to lay the foundation for Web infrastructure and systems, so we stated with a clean slate,” he says.
Leapfrog had many of the same goals that typify a typical SOA initiative: greater reuse of code, faster development time, and easier integration. But the company did not want to approach SOA as simply a changing of the guard for development tools and integration platforms. Instead, it wanted to free its developers from conforming to a platform’s idea of best practices so that they could focus on the applications’ functionality and use a wide range of development technologies as best for each job. (Leapfrog’s developers use a mélange of Java 5 and 6, Microsoft’s C#, and Web services with various third-party libraries.)
For example, Ciurana did not want to force developers to all use the same transport. “The transport doesn’t matter,” he says. He chose to use the open source Mule ESB as a messaging backbone, relying on it to deal with transport interfaces. That way, “developers could focus as little as possible on the implementation of services,” he explains. Instead, their focus is on the functionality they are trying to achieve. The result is that developers tend to use HTTP as their transport mechanism, but some use REST (Representational State Transfer) and SOAP -- “whatever works best or they’re most comfortable in,” he says. With the Mule ESB’s approach, “they don’t need to worry about what’s in a specific SOAP stack or what IDE they’re using.” Ciurana had previously used Mule at Walmart.com, so he was convinced it was the right basis for Leapfrog’s “clean slate” efforts.
Leapfrog could take this approach because its focus is on integrating applications, Ciurana notes. “Most of the integration is happening at the app level, with apps talking to apps. So apps just do inputs and outputs,” he says. Developers write services as POJOs (plain old Java objects) and let the Mule ESB “wire” the POJOs to the messaging network, handling any transformation within the ESB. “Normally, when you’re working with SOAP and REST, you tend to think about how to connect to the outside world. With POJOs, you don’t need to do that,” he says.
Ciurana also likes the simplicity of the Mule ESB because it has no agenda other than to manage the messaging. “All the commercial vendors wanted to sell us a whole suite of products. Yet the whole point of SOA is to move from one locked-in system to another,” Ciurana adds. With the Mule ESB, Leapfrog has to assemble the various layers of the SOA stack, but Ciurana is happy to pay that price to have the flexibility to use whatever works best at the time.
Leapfrog uses two ESBs, one for managing data flow and application handoffs throughout internal systems such as ERP, ActiveDirectory, and data warehousing, and the other ESB for customer-facing Web-based applications, such as its customer account self-management application and the online games it offers consumers. Not only does this bring in a natural boundary for security and access management, it also provides a mutual backup capability, as each ESB can take over for the other if needed.
Leapfrog did have to create a common service-naming scheme so that services could run on either ESB, Ciurana notes. The exacting names “help us keep all the services straight,” he adds. It’s a small price to pay for ESB freedom.
United Airlines merges SOA with event-driven architecture
As sensible as the SOA concept is – decomposing business processes into their constituent elements and then developing stand-alone software services that let you mix and match the components as needed for a variety of business demands – it assumes that you’re dealing with discrete transactional functions. The basic SOA premise requires that function be building blocks that can be combined in nearly unlimited ways.
But many business tasks are not so decomposable, instead relying on specific sequences of events. Airlines are a classic example of an event-driven set of processes, so they typically have an EDA (event-driven architecture) in place to handle events. “EDA is very flow-oriented, while SOA is about discrete black boxes,” says Ramnath Cidambi, middleware engineering manager at United Airlines. Both have their place; after all, airlines also have transaction systems such as booking fares and assigning seats, not just event-driven ones such as dispatching fuel trucks when a plane lands or updating the status board for flight arrivals, he notes.
United has long invested in its EDA, using IBM’s WebSphere as its messaging bus for seven years. In the past year, it’s begun an SOA effort to handle the modern Web services used at its United.com Web site. But those two environments are fairly distinct, Cidambi notes, so they could exist in parallel. However, that’s beginning to change, as the company begins adding transaction services within its internal operations, such as notifying customer service reps by text message (using Web services) what their schedules are for the day, using the HR system to see who’s scheduled, who’s called in sick, and so on to assign individuals to various gates at each airport. That puts Web services into the same environment as event-driven processes, Cidambi says, causing the airline to begin an SOA effort beyond the United.com program.
United’s challenge is figuring out how to architect and deploy services in a business that has, and needs, two architectures. Although the airline’s internal operations have two architectures, it cannot treat them as completely separate. After all, a cancelled flight (an event) also has implications for transaction systems (such as rescheduling passenger flights, updating Web-based flight-status lookup tools, or issuing credit vouchers). Many processes have both an event and a transaction component: while customer service reps get their schedules for the day from the transaction system, changes in plane status due to cancellations, weather delays, and so on quickly render that schedule moot. The event-driven system tracks the plane status, and the scheduling transaction system then updates the staff’s assignments by polling that status periodically. (The flight display monitors use the same process.)
The biggest challenge has been figuring out the messaging system. “ESBs don’t use standards outside of Web services standards,” Cidambi notes, so how to deal with event-driven services is unclear and inconsistent from product to product and tool to tool. Yet Cidambi is drawn to using ESBs for both SOA and EDA purposes because they handle messaging, data transformations, and other critical data routing tasks well.
Today, United Airlines has two ESBs: one for EDA services and one for SOA services. It uses an IBM WebSphere integration broker as a publish-and-subscribe-oriented messaging platform for its event-driven services, propagating events as needed and handling any transformations among services – essentially acting as an EDA ESB. For the transport, its existing J2EE applications are very messaging-oriented, so all use JMS (Java Message Service) as the messaging standard rather than Web services.