Product Review

Web services will revolutionize CRM (customer relationship management). This revolution will affect not only how sales, marketing, and service professionals interact with customers, but also how IT departments implement and support the technology. CRM is, in fact, suffering from precisely the constraints that Web services addresses architecturally. And several unique characteristics of CRM technology, data, and workflow exacerbate these constraints beyond other vertical applications.

The opportunities presented by this revolution will be balanced by equally powerful pitfalls. This revolution will demand a rethinking of the way many aspects of CRM technology are structured - in particular, data integration. As such, CRM is a unique case study in how to apply a new technology to an old problem.

What Is CRM, and Why Is It So Hard?
Broadly speaking, CRM is the set of business processes that address customer acquisition, retention, and cross-sell/up-sell opportunities across sales, marketing, and service functions. Thus, CRM is big, and big can mean costly, unwieldy, and complex.

In fact, today most large organizations are CRM victims whose senior executives cringe whenever someone (particularly a vendor) mentions a certain three letters in a certain order. Several characteristics unique to CRM are behind the current challenges:

1.   CRM is dynamic. Unlike back-office processes like logistics, customers are in control in CRM. Thus, organizations must constantly adapt to the way customers want to interact. Further, complex market and competitive forces are constantly affecting the way sales, service, and marketing happens. All this translates to a constant stream of system changes.
2.   CRM is multichannel. Customer interaction happens across multiple channels (phone, face-to-face, Web, e-mail, direct mail, third party, mass media). Activity in each of these channels affects how customers perceive activity in other channels. Thus, channel awareness is important.
3.   CRM cuts across organizational boundaries. Ask who owns the customer and practically every department in the front office will raise their hand. Product organizations, channel organizations, and geographic organizations all interact to create a set of complex dependencies for customer-facing processes.
4.   CRM processes are complex. Rarely do marketers ask the question, "How many customers do I have in California?" They do, however, ask, "How many customers do I have in California who have purchased at least twice, have children, and whose last purchase was over $200?" Then they start drilling down from there, resulting in perhaps dozens of variations on the same question.
5.   CRM data is typically everywhere, and metadata matters. Unlike most other applications, CRM can involve information from all other applications. In this respect, CRM is bigger than you think. Invoice data? Sure, it has purchase history. Service call? Of course, we need to gauge level of satisfaction. Third-party demographics? You mean I don't know if they have children by looking at my internal data? And the meaning of fields of information, even those with identical "tags" like "invoice_ amount", can mean very different things in different databases (think taxes and discounts, net versus gross, etc.). Tiny differences in the meaning of data that are not taken into account can cripple CRM analysis and implementation.

All of these factors combine to create very complex applications covering multiple touch points with proprietary data structures and self-contained integration.

The classic application suite approach that has evolved in response to these factors and all of the subsequent integration and maintenance headaches that have ensued provide the great opportunity for Web services. Simply put, organizations have over-invested in CRM and are not realizing the promise of their existing systems. In addition, these investments encompass multiple applications and databases. With today's strategic imperative to "grow revenue one customer at a time," CRM business processes are reaching maximum importance just when the enterprise has tired of big-footprint implementations. Rare is the company that will replace the functionality of one vendor with another just to standardize on a suite! The integration of such a coordinated, distributed architecture is ideal for Web services. Overcoming the integration challenge can provide the "killer app" all new technologies need.

The Integration Challenge
In order for CRM to work, organizations must be able to design, implement, and maintain business processes across applications and databases. One of the oldest problems in computing, application integration continues to impose constraints on the lines of business as they attempt to generate revenue through CRM business processes.

Very early solutions focused on writing custom code at the application layer, attempting to extend business logic across applications. The result, spaghetti code, became unworkable as scalability requirements increased. The current generation of solutions focuses lower on the integration "stack," at the data level. These solutions, from custom data movement code to extract-transform-load (ETL) synchronization and enterprise application integration (EAI), have focused on the movement of data from one application to another, either in batch mode (ETL) or in near real time as "transactions" (EAI) that automate workflow by transporting data from one application to another.

Today, however, the problem of implementing and maintaining application integration has not only remained unsolved, but has gotten worse. The reason for this is simple: CRM applications function best with centralized data repositories (warehouses or marts), resulting in the creation of myriad data silos, and creating a Gordian knot of data movement processes within large organizations, locking valuable data definitions, business logic, and transformation logic into custom code or proprietary ETL and EAI applications.

The result is a tangled, confusing array of interdependent programs both within the enterprise, and connecting the enterprise to the outside world. Maintaining this complex environment, and assuring that the business processes inherent in the programs contribute to high-level corporate goals and objectives is, increasingly, a nearly impossible task.

Figure 1 depicts a computing infrastructure that's not at all atypical. The blue lines indicate the various point-to-point ETL, EAI, and custom tools used to link systems and applications together to form CRM processes. Imagine the integration nightmare when a new department's Web site comes online, or when the data mart's schema needs to be updated!

The Web Services Revolution
So how does Web services architecture address this challenge? Web services promises to enable applications to communicate above the data integration level, and even above the application level. Web services enables a new generation of application integration applications that allow organizations to identify critical events and act on those events in real time, while maintaining all of the business logic already inherent in existing CRM investments within its applications.

Through lightweight adapters at each CRM application or database, events of interest to sales, service, and marketing that occur in one system, say a large withdrawal, or a complaint by a premium customer, can be readily and instantly identified, evaluated based on context and knowledge from around the enterprise via centrally stored business rules, and acted upon by sending a native transaction to another application that is also linked through a lightweight adapter. The adapter components of the architecture communicate with and extend the existing business logic in systems, dramatically reducing the application integration challenge (see Figure 2).

This approach, which I call "event coordination," provides the following advantages:

• Creates a "hub and spoke" messaging system, replacing the point-to-point nightmare described previously
• Reduces reliance on expensive consultants writing custom code
• Enables more efficient use of resources such as hardware and bandwidth through distribution of functionality
• Extends the integration framework to multiple service providers and third parties outside the firewall
• Provides a balance between decentralized application infrastructure and centralized coordination of activity across applications, overcoming data and functionality "silos"
• Utilizes all of the business logic already contained in an organization's infrastructure without modifying or transferring it

The approach plays to Web services' strengths and dovetails nicely with current trends in technology, specifically:

• Web services development is cheaper than proprietary approaches due to the amount of free basic infrastructure available.
• Web services standards are well developed, allowing the use of standards-based products for development and administration.
• The tremendous momentum of Web services has forced many EAI, ETL, and application suite vendors to reveal their business logic through SOAP APIs.
• An emphasis on Web services security enables distributed applications to extend beyond the firewall.

Data Provisioning: The Next Challenge
While facilitating application communication at the event level conforms nicely to the advantages of Web services architecture, provisioning data to decentralized functionality looms as a potential pitfall.

Application suites and other complex bundles of functionality evolved because of heavy reliance on standardized and reliable data in the form of centralized data warehouses and data marts. In fact, the decades-old trend toward data centralization is one of the main rationales for big-footprint applications. So, any Web services approach that seeks to decentralize functionality must address the data provisioning issue. Simply put, it is impossible to build a single database to provision all Web services, and it is equally impossible to implement custom code to provision data for each Web service.

Happily, one of the base enabling technologies in the Web services architecture promises to facilitate the data-provisioning challenge: XML. XML enables applications that can map decentralized data to straightforward business meanings. Thus, the term "customer" or "product" can be standardized for an audience and mapped to the various data elements in their native data stores. Further, business rules that describe how one definition of a product (say, an SKU code) can be standardized to conform to the business definition for that audience can be encapsulated in the XML. Thus, a lingua franca is born.

This common language can then serve as a reference point for disparate Web services to fetch data, transform it into the definitions expected by the audience, and then act accordingly. Otherwise known as a "canonical object" approach to metadata (where each term such as "customer," "product," or "campaign" is an object with specific attributes and meanings), this lingua franca overcomes one of the thorniest issues in CRM: data integration to support functionality. Most of the cost of the implementation of an application suite is the research, design, implementation, and maintenance of the processes that take data from around the organization, standardize it, load it, and update it over time. By leaving data in its original location, but making it accessible via mappings and business rules stored as XML, Web services architecture can dramatically reduce the cost of implementation and maintenance.

A side benefit to this approach is reusability. Definitions and business rules stored as XML can be provisioned to any application or process, not just the Event Coordination framework. And, all of the original research and design is no longer a one-off project: work that would have to be done anyway is accessible again and again.

Conclusion
Every new technology trend offers opportunities to rethink methodologies previously thought to be common knowledge. Web services offers this opportunity to CRM and, in doing so, promises to revolutionize an area of computing that is simultaneously critically important and desperately difficult.