Industry Commentary

Even if you work for a great company, if it isn't a technology company it will mean that no matter how cool, obvious, or forward thinking technological advances are, you will always have to be able to translate their benefits into real business dollars and sense.

Web services presents one of the simplest and most innovative technologies I have come across. Its potential looks amazing and it offers little down side that I can see. It makes sense to adopt it as a systems integration platform of choice because it represents two things that our business is interested in:

• Cost savings
• Agility

I'm going to talk about Web services as an integration technology, generally focusing on using it to pull legacy systems into a service-oriented architecture model. As I have gone through this journey, I can recommend dozens of things that you should have an understanding of if you are looking to adopt Web services for integration use; however, there are three things that I think are at the top of the list:
1.  Have a value proposition: Be able to show how the Web services integration architecture saves money in your company.
2.  Have a plan for all of your systems, not just the new ones: Including legacy systems in your Web service architecture has many benefits.
3.  Know what your strategy is going to be for data before you make any other decisions. This strategy will have substantial impacts on your design and scope.

I want to share what I believe to be a sound way of presenting these important business ideas to senior management. In this article I'll go over the models I have built to help translate the obvious technology benefit of Web services into language that works to cover the business costs of putting them in place. After that, I'll talk a little bit about how to get creative so you can use Web services for older legacy systems that needed to be included in a service-oriented architecture. Finally, I'll touch on some enterprise-wide data integration basics to get you thinking about a strategy.

The Problem Statement
Many companies today have a lot of different technologies, and a substantial part of there enterprise data in legacy systems. The industry is evolving, slowly, into more modern commercial-off-the-shelf (COTS) packages. Companies that are all COTS based I call an Application Oriented Architecture. This may have been the grand industry vision at one point, but somewhere along the way the vision changed (or was incomplete to start with), and has left many with new and legacy systems that do kind of the same thing. This creates a lot of integration needs to give the business a complete view of their lines of business.

Systems need data and we have to come up with cost-effective solutions for a blend of technologies. Additionally, we have to justify the cost of these solutions against a preestablished business benefit that has only moderate tangible (save money now) benefits, but an overwhelming set of elusive (save money later) benefits.

So as technologists our problem is twofold: find a set of integration technologies that offered a fit for a very wide array of systems, and describe the tangible benefits in a way that can be measured against existing financial cost expectations.

A service-oriented architecture (SOA) using Web services makes a lot of technical sense. It makes good business sense too. Since we are sitting somewhere between no architecture and an application-oriented architecture, the SOA model is the most efficient model to tie it all together. I think the SOA is the most complete vision of business application technology today.

Adopting a strategy to move to an SOA allows you to focus on the elements of business technology that matter most to the business.

A foundational model for maturing the enterprise integration architecture is:

Connect ‡ Integrate ‡ Optimize

One danger we have faced over the past couple of years is that the tactics used to connect systems, without a vision for an optimized environment, lead to point-to-point madness. Systems cannot be integrated, only connected, and reuse or other optimization is not possible.

A service-oriented architecture promises to speed development time and reduce integration costs. But for this to happen the services must be understood and implemented correctly. They must be understood from an enterprise perspective and organized so they can be reused. This is where the big payoff comes, as I'll talk about later. If planned properly, Web services for integration can be a very good answer indeed.

Now, how do we wrap this into language that can be used to communicate to a fast-moving, bottom line business?

Have a Value Proposition
Okay, I have the technology solution; how do I justify that this is a better business choice then a different architecture?

I've done a considerable amount of research analyzing how people spend their time on projects. Sometimes this can be a little bit of a challenge day to day, but it provided me with data I could analyze and determine how much on average current integration implementations were costing. For simplicity's sake let's assume you have the standard supply of FTP's, point to point, ETL's, users performing multiple entries, screen scrapping, and message queuing. If your inventory turns up similar to mine. You should now able to reduce your costs down to a quantifiable value that can be scaled by the number of touch points there are between systems across the enterprise.

I call this the Consumed Integration Point (CIP) Index. I use this term to identify points where an application will need to connect to or integrate with other applications to share data or processing. The CIP Index is calculated by counting the number of data consumption needs, per application, for data outside of the applications domain.

This index can then be tracked historically and trended forward over time. Efficiencies to the integration architecture can be measured and costing trends established against those efficiencies. And here is the great part: when you start to forecast your CIP Index on existing technologies against a Web service-based integration architecture you will see that the costs went down - by a lot!

The industry average for a new CIP pre-Web services costs just over $23,000. Since in our pre-Web services architecture an application needed to request and then receive the data, the complete work flow comprises a minimum of two CIPs, one for the request and one for the response. Since we adopted a request/response metaphor for our Web services generic scenario we get the benefit of two different reductions: first, the cost per CIP goes down just over 20%; second, we eliminate one of the CIPs. This is a substantial cost savings. Integration messaging that used to cost us $46,000 to implement now costs only $21,000.

This is a real savings and a strong business reason to look into Web services. The model below assumes that you have multiple ways of moving data that you will consolidate; that you can collapse five CIPs into three per; and is for the build and test phases of your SDLC only. The model becomes more accurate with more CIPs.

Cost Savings = Current Build Costs - Improved WS Integration Costs
Current Build Costs = Cost Per CIP * # of CIP
Improved WS Integration Costs = (Cost Per CIP * .8) * (# of CIP * .6)

For example, assume that your sales executives receive awards and recognition based upon their sales performance. This sounds obvious and straightforward. In reality, the system they use to sell your product, the valuation of the quality of the sale, and the criteria for the award are three different systems. For the awards and recognition system to produce output it takes six CIP's, which are:

1. Ask for Sales Executive's information
2. Receive Sales Executive's information
3. Ask for inventory information
4. Receive inventory information
5. Ask for sales to be evaluated
6. Receive sales to be evaluated

I have adopted the request/response style of using Web services for integration - there are technology reasons why this needed to be the model used. In this example, with the request/response method four interesting things happen. First, I have a tangible build cost reduction for each CIP. Second, I have a tangible elimination of three CIPs. Third, I have an elusive benefit of reusability, meaning that if I ever need this information for a different purpose or system I can completely reuse the Web service. Fourth, if I am adding to an existing Web service's infrastructure no additional support costs accrue.

Given all of the tangible benefits, my cost per CIP goes down, and the number of CIPs is cut in half. If you assume that you might have hundred's of scenarios like this one you can be talking about millions of dollars of savings, which should justify the costs of moving to Web services and an SOA.

Another obvious place to look for tangible cost efficiencies is in product maintenance fees. This article isn't about preaching one Web services platform over another, although I will tell you that the one I selected doesn't carry any additional maintenance costs than were being paid already. This means that my cost per CIP can go down just by changing over to a Web service and SOA platform. Eliminating a couple of products from the infrastructure is always a good goal when you have hundreds.

Have a Plan for All of Your Systems
By using the CIP index you should be able to cost-justify a project to implement an SOA using Web services and include your legacy systems. One of the first design challenges is the fact that the industry average shows that about 80% of the data and processing you need to provide to the enterprise as a service is still housed in the oldest and least SOA-friendly systems. The goal is still not to write too much new code in these systems because the trend is for organizations to focus on other technologies going forward. I wanted an example that would work for all types of systems.

What I landed on was a bit of a hybrid idea between a pure SOA and the older hub-and-spoke messaging model. Basically, the idea is to have a central hub where the majority of the Web services for legacy systems can be deployed. Other benefits come from this, such as a single point of audit and using SSL, a very graceful transport solution that can be used to meet all of the broader security compliance regulations and legislation. I call this hub the Network Enterprise XML Universal Service, or NEXUS.

I recommend adopting the W3C.Org as the standards body and to only write code for the NEXUS that is W3C compliant. You can use this hub to service intra- and extra-systems needs.

One of the down sides of a hub is that it can be a single point of failure. To address this I recommend a redundant, clustered-server environment that also serves as a load-balance platform for performance, so nothing goes to waste.

Have a Data Strategy
This last part, a data strategy, was a lesson learned with some pain. Decisions need to be made right up front about what your goals are. Do you want to have a system of record for every data element? Do you want the data to be accessible locally to each system that uses it? Does some data need to be more secure, or have audit trails? The questions go on and on. A nifty little way to start to talk about the decisions involves taking a high-level view and categorizing your goals.

The model I came up with has two elements that decisions need to be made on, for a total of four possible strategies (see Table 1):

Data Everywhere. This decision point established whether you want data to be locally available to each application. Data everywhere would push your SOA into a Transaction Of Record state, meaning that as data changes, or is needed, integration services need to keep the enterprise in sync. The opposite of data everywhere is systems of record, which would push your integration services to go get data as needed from the systems of record.

Pre-Sync. This decision point establishes whether you want to scrub and align your data before turning on integration services, or if you want the business rules created around the integration services to synchronize the data as it is accessed.

Looking Forward
Web services have lowered my integration development costs and positioned my architecture around agility. There is no question about it. With Web services, I can start to see the promise of technology that seemed so prevalent just five years ago. I am excited about a number of future plans and ideas I have for optimizing our systems on this platform and look forward to evolving uses of this unique technology.