The ABCs of BPM

With a BPM system, you can detach business logic from infrastructure to create new applications quicker than ever before

The benefits claimed by BPM (business process management) software sound almost too good to be true. Proponents crow about lower app dev costs, shorter time to market, improved compliance enforcement, and new points of leverage for optimizing business performance.

BPM software can't improve anything by itself, of course -- but it can be a powerful weapon when combined with business-oriented documentation and analysis. Within its own controlled, high-level app dev environment, BPM wraps IT solution development within business-driven modeling and performance measurement.

At the least, BPM provides an effective new medium through which the business side can communicate its requirements to IT. At best, it can distill functionality from existing applications and free business logic from the bonds of existing infrastructure to enable unprecedented agility.

One persistent problem for potential adopters, however, has been abject confusion. BPM solutions come in so many varieties that only a handful of consultants seem to know which solution is best for the task at hand.

Today, clarity is emerging in the form of the BPM suite, an integrated set of tools and runtime components designed to create software analogs of business processes. Together, these elements allow customers to model, deploy, and monitor BPM systems without having to staple together bits and pieces of technology from different vendors.

Properly utilized, BPM suites account for the fact that the internals of elemental process activities -- particularly those implemented by existing business systems -- may be difficult to modify. Instead, BPM suites enable IT to optimize business performance by changing the process logic that interconnects them. Process design in a BPM suite is akin to a flowchart, annotated with the necessary implementation detail. It requires little code, and the process logic is easily changed, qualifying BPM as a style of agile application development.

Click for larger view.

The basic BPM flow

BPM begins with process modeling, a business-driven exercise in which current and proposed process flows are documented in detail, linked to quantifiable performance metrics, and optimized through simulation analysis.

These optimized models automatically generate the skeleton of the IT implementation in a BPM suite's process designer, a graphical development tool that integrates human workflow, application integration, and business rules to create an executable process solution. Completed process designs are then deployed to the process engine and other components of the BPM suite runtime, where they route and track tasks, integrate with external business systems, and enforce business rules.

As process instances complete each activity, the process engine generates an event to mark the occasion. Those events are collected by the BPM suite's performance management component, which aggregates them into metrics that measure business performance.

Performance management dashboards graph metrics versus their target values, with drill-down analytics via OLAP queries. They also provide real-time alerts and automated escalation procedures when KPIs (key performance indicators) go off track, a capability of the BAM (business activity monitoring) component often bundled with a BPM suite. Actual performance data can be fed back to refine process models and begin a new cycle of incremental process improvement.

Process wars

Tally all that functionality, and you end up with quite a stack: software for business modeling, simulation analysis, human workflow, application integration, data mapping, business rules, performance analytics, BAM, and Web portals. All of these originated as independent tools from specialized vendors.

But today, within the BPM world, the trend is toward wrapping all these components inside the BPM suite, whether through mergers and acquisitions, OEM, or integration partnerships. This shift has created conflicts between BPM suite vendors and suppliers of modeling tools, BAM, and integration middleware, each of which tends to describe BPM in its own way.

Perhaps the greatest source of confusion, however, has derived from the two competing technical architectures for BPM. The one that has gotten the most media attention is based on the BPEL (Business Process Execution Language) standard, which implements processes by orchestrating Web services within an SOA environment. This is where the large infrastructure vendors play, including IBM, Microsoft, Oracle, and SAP.

On the other hand, most pure-play BPM suite vendors -- such as Fuego, FileNet, Pegasystems, and Savvion -- use an architecture that evolved from the workflow systems of the 1990s, one better suited to incorporating human tasks in the process model. In these offerings, SOA and BPEL play a more limited role and focus on application integration, rather than describing the end-to-end process.

The bottom line is fairly simple. The big-vendor solutions that emphasize BPEL work best for composing Web services into applications that involve little human workflow -- that is, without multistage handoffs to various users in various roles in an organization. Pure-plays have long emphasized implementation without programming, so their BPM solutions tend to provide the straightest line to a practical BPM deployment. The downside is that, as opposed to their big-vendor competitors, pure-plays' offerings can be more difficult to integrate into an existing application environment.

Modeling reality

Whether from a big vendor or a pure-play, modeling tools have one main purpose: to describe business processes in terms of their elemental activities and tasks, the resources required to perform each task, and the business rules interconnecting them -- all using a graphical notation understandable to business users.

Models play a critical role in aligning process design with quantifiable performance objectives and optimizing expected results through simulation analysis. By annotating each process activity with performance-related parameters such as expected time to perform, resource costs, availability, and branching ratios at forks in the flow path, models can be analyzed in a variety of scenarios using a simulation engine built into the modeling tool. In advanced modeling tools, the KPIs that will be used to measure the performance of the process implementation determine the parameters required in the model, ultimately closing the loop of performance improvement. This requires models that go beyond simple descriptions of activity flow to include modeling of organizational resources, process data, and process performance metrics.

Click for larger view.

For years, these capabilities have been the exclusive domain of business process modeling tools from vendors such as Casewise, IDS Scheer, Popkin (now Telelogic), and Proforma, often as part of a broader suite of enterprise architecture tools. Now, however, vendors such as Global 360, IBM, and Savvion are implementing the process modeling functions of these tools within the BPM suite itself. At the same time, modeling vendors are improving interoperability among BPM suites by leveraging BPMN (business process modeling notation), a standardized graphical notation from the Object Management Group.

Where once the output of process modeling was a business-oriented specification intended to guide IT in any implementation efforts that might be needed, BPM assumes an automated process implementation will be executed on the process engine.  Modeling standards such as BPMN and interchange formats such as CIF allow the output of a modeling tool to be imported into a BPM suite's design tool and a skeleton implementation design to be generated. This skeleton design lacks the implementation detail to be executable off the bat, but it creates a business-specified starting point.

The closed-loop problem

Even with standard BPM design languages such as BPEL, each vendor's process design tool is specific to its own runtime environment. Today, there is no such thing as a portable process design that can be executed on your choice of process engines -- unless, of course, you consider human tasks, business rules, and complex data mapping to be "external" to the business process design.

Most BPM suites today provide a unified design environment that hides the complexity of combining human workflow, application integration, business rules, and transaction management within a single executable design. The benefits this provides over treating these process components as independent entities in the enterprise architecture stack are a common data model and common state management over the entire end-to-end process. 

Like modeling, process design is mostly graphical. The tool provides a palette of activity types from which the designer selects, configures, and assembles the process steps. Unless custom activities need to be created, process design involves minimal programming. Behind the graphical design metaphor, the tool creates an executable process implementation in the BPM suite's particular process execution language.

Click for larger view.

In BPM suites based on workflow architecture, the language is typically proprietary but compliant with the XPDL (XML Process Definition Language) standard from the Workflow Management Coalition. Process activities may be one of several predefined implementation types (Web service, user task, integration activity)‚ each assigned to a resource, such as a human task role or an integration adapter. The configuration dialog for each activity depends on its type.

By contrast, BPM suites based on service orchestration rely on the BPEL language standard. BPEL provides a single activity type -- Invoke -- to call a Web service, human task, or integration adapter, all of which must be implemented as a service, with an interface described by WSDL. But Invoke must be addressed to a service end point -- a URL, not a task role. To accommodate human tasks, what gets invoked by BPEL is not the user task itself but a task manager service, which handles the workflow details.

Another difference is that workflow-based BPM suites support the notion of a subprocess, a reusable process fragment that shares context data and state management with the calling parent. BPEL provides no such concept. A subprocess is another BPEL process; data sharing and state synchronization must be explicitly defined in the process logic. To address these real-world limitations, IBM and SAP last summer outlined optional extensions to the BPEL standard, but the specs are not yet complete. In the end, regardless of architecture and coding differences, BPM suites tend to accomplish the same set of core functions.

Rolling out process-driven apps

The completed process design is then deployed to the process engine. As each instance of the process is triggered, the engine routes it through the defined sequence of activities, integrating external applications, routing workflow tasks to human participants, and managing deadlines and exceptions throughout the process. In offerings from app server vendors such as IBM, Microsoft, Oracle, or SAP, the process engine leverages unique capabilities of the app server and its associated middleware. Offerings from BPM pure-plays tend to run on the user's choice of app server platforms.

The process engine also reports snapshots of instance data and state, usually in the form of events, for the purposes of performance management. The performance management component of the BPM suite collects those events and uses them to update KPIs and other performance metrics defined in the modeling phase. Typically, metrics are aggregated in OLAP cubes, which can be charted and queried by users in management dashboards. OLAP-based performance management provides historical and "near-real-time" reporting and drill-down analytics, as updates are performed on demand by re-crunching the collected data set. Some BPM suites, including those offered by Adobe, FileNet, IBM, Intalio, and Savvion, support true BAM, providing real-time update of selected KPIs with rule-triggered alerts and escalation actions.

Metrics computed from running processes can be used to refine the model parameters used to generate expected values of those metrics, making the effect of process changes more predictable and stimulating additional rounds of process improvement.

The BPM choice

Picking the right BPM suite is a real challenge. Although virtually all the vendors promote the same list of capabilities in their brochures and Web sites, each offering is actually tuned to the requirements of a fairly narrow set of process types or use cases.

For example, a BPM suite designed for transactional, "straight-through" processes involving complex application integration but little human interaction might not be the best choice for collaborative, human-centric processes with minimal integration. Document-centric processes and production workflow processes where pools of users draw tasks from shared queues at high speed have their own unique requirements addressed by some, but not all, BPM suites.

Such complications aside, BPM is offering real return on investment to users today. When presented as an architectural stack, it can sound like a tangled mess. But the new generation of integrated BPM suites is untangling BPM and providing a new middle ground for business-IT collaboration.

Copyright © 2006 IDG Communications, Inc.