Transacting Business with Web Services Part I

Aug. 19, 2003 12:00 AM

Business transaction management (BTM) is a promising new development in general-purpose enterprise software. Most large companies are devoting significant resources to the problem of reliable, consistent integration of application services.

BTM offers previously inaccessible levels of application coordination and process synchronization, radically simplifying the design and implementation of transactional business processes. Business process management (BPM) needs to be enriched by BTM for users to see the potential value of BPM realized in practice.

XML is already widely deployed as a useful lingua franca enabling the creation of canonical data standards for particular industries, trading communities, and information exchanges. The extended family of Web services standards (clustered around the leading duo of SOAP and WSDL) is gaining growing acceptance as an important way of providing interoperable connectivity between heterogeneous systems. Many organizations are also examining the use of BPM technologies, exemplified by the current OASIS initiative, Web Services Business Process Execution Language (WS BPEL). Increasingly, attention is turning to the special problems associated with building transactional business processes and reliable, composable services. This is where BTM technology comes into its own.

In this article - which follows the excellent introductions by Mark Little and Jim Webber to WS-Coordination, WS-Transaction, OASIS Business Transaction Protocol, and BPEL (Web Services Journal, Vol. 3, issues 5-8) - I'm going to look at the rationale for and current status of BTM, and how vendors and users are thinking about the integration or fusion of BTM with BPM, particularly in the OASIS BPEL standardization effort. BPEL, as a special-purpose programming language designed to make processes portable across different vendors' execution engines, can become a very useful standard programming interface for business transactions in the Web services world.

Why Is BTM Needed?
Companies would like to minimize the costly, slow, and risky business of "reconcile and repair." Automating or improving the internal flow of transaction processing, spanning multiple disparate applications, is a major issue for most corporations: the finance industry's focus on straight-through processing is an emblematic case. The business model of the Internet (which can cynically be summarized as "make your customers your clerks") continues to drive efforts to offer automated transactional platforms or portals directly to consumers and to business clients. This push towards "self-care" makes reliable, consistent, and recoverable composition of back-end services more important, as inconsistencies and failures become quickly visible outside the call center or the operations room. Equally, customers and suppliers who transact business electronically need well-formed protocols which model the interactions of their business relationship: they want near-instant certainty and "assured common knowledge" about the outcomes (good and bad) of those interactions.

Automated business transactions are a new category, wider than historical data-centric local or distributed transactions. This "third generation" of transaction management builds out from core transactional technology, particularly the concept of a multi-phase distributed outcome ("two-phase commit" in conventional database/messaging transactions). Business transactions may be of split-second duration, but may also be long-running, i.e., may endure for periods that exceed the anticipated service cycles of participant systems. Business transactions retain the driving ambition of consistency. However, they are more flexible and sophisticated in their means - and often more modest in their goals, reflecting the imperfect determinism of real business processes (which are necessarily designed to cope with innumerable variations and with incremental and partial successes).

Business rules determine the set of viable outcomes, which may involve noncritical partial failures, or selection among contending service offerings, rather than the strict "all-or-nothing" assumption of conventional ACID transactions.

Business rules govern the duration and character of participation in a transaction. Provisional results are often revealed deliberately, to allow probabilistic inventory management. Tentative service or product offerings may be withdrawn spontaneously or after a declared interval, even if this disrupts the final outcome.

Both of these features are relevant to the work of the OASIS WS BPEL Technical Committee, whose charter foresees work over a minimum period of nine months (from May this year to January next) to achieve a recognized standard. In the remainder of this article I'll concentrate on the integration of a transactional coordination protocol with BPEL.

Exception and Recovery Handling in BPEL Today
The current WS BPEL 1.1 draft specification (jointly submitted by IBM, Microsoft, BEA, SAP, and Siebel) incorporates the notion of processes that can contain nested sub-processes or scopes, to any depth. Processes are expected to carry out "real work" (work which affects persistent records and external systems) by invoking WSDL-defined operations. While BPEL processes can define and manipulate variables, these variables are designed for use in tracking process state and controlling conditional flows. If processes wish to communicate with other processes then they can do so using "partner links," which essentially create sets of WSDL-defined services as access points for bilateral message sequences between defined roles in a collaborative exchange.

Processes are initiated by receiving inbound Web service invocations. BPEL, therefore, expects that external stimuli will arrive and substantive processing work will be invoked as Web service documents, typically delivered using SOAP messages.

Processes and scopes can declare fault handlers and compensation handlers, in addition to the normal sequence of "happy path" activities. A fault handler is designed to deal with exceptions during normal processing. A compensation handler is designed to reverse the effect of a completed scope, and can only be invoked when the normal work of the scope is finished (this gives the compensation handler a clean starting point for its reversal activity). Compensation handlers can be invoked by the fault handler of an enclosing scope - which can decide which compensations to process, and in which order - in order to roll back partial work in the event of failure.

Integrating BTM with BPEL BPM
Vendor and user companies in the WS BPEL committee, including Choreology Ltd, are discussing how BPEL will use BTM. (In this context BTM tends to be equated with WS-Transaction Business Activity [WS-T BA]. As we shall see, some features of OASIS BTP must also be taken into account, and the use of WS-Coordination must be made more precise.)

To set the scene, it's useful to look first at the relationship of BPEL processes to Web services. Figure 1 shows a BPEL process, a scope (sub-process) external Web services, and a WS client. Note that a BPEL process may receive messages from a Web service client. It can also call out to Web services, including other BPEL processes that offer Web service interfaces. BPEL processes can also execute nested scopes. (Scopes are effectively processes that happen to be initiated by a colocated process, and have direct access to control variables declared in enclosing scopes/processes.)

BTM features are superimposed on this basic structure. Figure 1 shows a BTM coordination service. (This might be "physically" located within a BPEL execution engine, or operate as a freestanding Web service.) It also shows the use of a business transaction context to connect or relate participant services and processes to a particular business transaction.

In looking at the integration of BTM into WS BPEL the technical committee is likely to consider four main areas:

How to propagate business transactions between Web services and business processes

How to propagate business transactions between business processes and nested scopes

How to initiate and terminate new business transactions within a business process

How to define the reaction of processes and sub-processes as participants in the progress of a business transaction

In another article, I'll discuss the specific changes proposed for BPEL in order to address these questions.

Sidebar

BTM: Protocols and Standards
Full-scale BTM software needs to implement interoperable protocols that define three phases of any transactional interaction (whether integrating internal systems, or automating external trades and reconciliations).

Phase One: Collaborative Assembly. The business-specific interplays of messages that assemble a deal or other synchronized state shift in the relationship of two or more services. A useful general term for such an assemblage of ordered messages is collaboration protocol. Examples include RosettaNet PIPs, UN/Cefact trade transactions, and the FIX trading protocol. In the future, BPEL abstract processes should help greatly in defining such protocols. Reliable messaging has an important role in this assembly phase, but as a subordinate part of a new, extended concept of GDP (guaranteed delivery and processing).

Phase Two: Coordinated Outcome. The coordination of an outcome that ensures that the intended state changes occur in all participant systems, consistent with the business rules or contracts which govern the overall transaction. Examples of relevant coordination protocols are WS-Transaction (Atomic Transaction and Business Activity, supplemented by WS-Coordination) and BTP (the OASIS Business Transaction Protocol) and the recently released WS-TXM (Transaction Management, part of the WS-Composite Application Framework [WS-CAF]).

A coordination protocol requires three related sub-protocols: a control protocol, which creates and terminates a coordination or transaction (present in BTP); a propagation protocol, which allows a unique transaction identity to be used to bind participating services to a coordination service (this sub-protocol is mostly defined by WS-Coordination); and an outcome protocol, which allows a coordination service to reliably transmit the instructions of a controlling application to the participants, even in the event of temporary process, processor or network failures. WS-T, BTP, and WS-TXM, contain very similar outcome protocols.

A coordination protocol is typically useful only if it is inherently reliable, utilizing persistent checkpoints and message retries to survive planned or unplanned interruptions in service.

Phase Three: Assured Notification. Notification of the result of the transaction to the parties involved, ensuring that they're all confident of their final relationship to their counterparties. Ideally, this requires a reliable notification protocol, which allows the different legal entities or organizational units to receive or check the final outcome, including partial or complete failures. There are no current examples of standardized reliable notification or termination protocols that I am aware of.

Current BTM products and standards focus on Phase Two, Coordinated Outcome. WS BPEL may prove a useful vehicle for creating a new, wider view of the scope of business tranaction management.

Related Links

▪ Figure 1

About Alastair Green
Alastair Green is CEO/CTO and co-founder of Choreology Ltd, the business transaction management (BTM) company. A co-author of the OASIS Business Transaction Protocol standard and an active member of the OASIS WSBPEL committee, he has spent over twenty years helping produce distributed transaction products for software vendors, and deploying them in end-user business processes, particularly in banks. He has been invited to overview web service transaction standards for the biennial High Performance Transactions Systems workshop this fall.