Multiple entry-levels to SOA

From a technical point of view there are multiple entry points in a roadmap toward SOA. As I mentioned before, in large companies there is no one-size-fits-all approach to SOA. Depending on the maturity of the project environment, in which the enterprise's software development and operations take place, an adequate entry point may be chosen.

This is the maturity scale I tend to use as the distinct per project entry levels to SOA:

1. Batch-file transport over the ESB

2. Record (message) oriented data transport over the ESB

• at sending side: split batch file into SOAP-messages
• at receiving side: rebuild batch file from queued messages

3. Shift design-focus from applications and batch-files to asynchronous real-time point-to-point messages

4. Define intermediate (canonical) layer of message types to map semantics and decouple formats between senders and receivers

5. Break the applications down to well defined, documented and reusable components (services)

6. Make use of tools for Business Process Management (BPM) en Business Activity Monitoring (BAM)

By choosing an ESB-infrastructure for data exchange between applications, even at the lowest level of maturity, an optimal coherence between the old world (legacy) and the new world (SOA, EDA, BPM, BAM) will be achieved. By using web services technologies based on an ESB-infrastructure, a virtualization of location as well as technology will arise. The tooling around this kind of infrastructures will offer early visibility of the application landscape in terms of existence of and interrelationships between the applications.

So, avoid choosing one single approach to introduce SOA. But instead choose an entry point on the maturity scale for every single project, depending on the project's context. And realize it will take an odd 10 years - on average for big companies - before you may seriously be speaking of having an installed SOA base.

Creating an ESB-infrastructure at zero investment

The introduction of an ESB-infrastructure may be a big hurdle with regard to financial investments. Technology evolves rapidly and adoption by system development projects may evolve very slow. You might end in a situation of high costs and no view on any return on investment other then charging the projects excessively which lowers adoption even more.

What you really want when introducing an ESB-infrastructure - from a pragmatic point of view - is scalability on three aspects:

Deployment

You want a modular ESB-deployment on project or system bases. Based on the context an ESB-component is deployed for that specific situation or project. The context may differ on aspects as:

• Geography (different locations)
• Network topology (different network zones)
• Footprint (data center on one end and devices as PDAs/passing gates/vending machines on the other end)
• Technology (e.g. Linux versus Windows)

The ESB-product must allow for clicking the separate project based ESB-components together and behave as one - federated - ESB. Functionality and services on one ESB-component may be made available for other components.

Functionality

You want the ESB to offer just the functionality you need on a project basis. The product must allow for adding functionality at a later time.

License

You want the ESB-vendor to have you pay a reasonable fee for only the number of connections a project creates to the ESB with an upper limit if you scale above an agreed number of connections. Or you may want an equivalent model based on the number of messages travelling across the ESB.

This allows for having development projects pay their own ESB-deployment from the project budget while growing one homogeneous company wide ESB-infrastructure over time. The federated architecture allows for ultimate performance scalability on the fly as the interacting components may be deployed and redeployed in a distributed way; across multiple locations (data centers), servers and devices, and as fine grained as you prefer maintaining one single control view.

The evolvement of federated infrastructure services - from data centers to agents in devices - is a recognized trend in current evolving complexity to offer agility at a low cost. Examples are federated directory-, identity-, access-, and data management services. Also the flexibility in adding functionality on demand is a recognized trend; more and more even on a pay-as-you-go basis. And the same kind of on demand scalability applies to the licensing structures of modern vendors.

At this moment in time mature ESB-products are offered in the market that support the scalability requirements mentioned above. It is a wise decision to focus on these type of products when introducing an ESB-infrastructure in your organization.

Three ESB challenges for the enterprise

Enterprises that decide to introduce ESB concepts and products as a way of messaging and services platform are challenged in three ways:

• Functional domain boundaries
• Multiple ESB products
• Federated infrastructures

Functional domain boundaries

Policies have to be in place to (not) allow for cross boundary service calls and data reuse. The desired level of autonomy of functional domains determines the tolerance of these cross boundary dependencies.

As SOA at the organization level strives for loose coupling to achieve ultimate organizational flexibility, the ESB concepts have to support functional boundaries. This may result in the concept of domain buses that are connected via a corporate bus.

Policies have to guide ownership- and management responsibilities for the defined domains within the ESB concept, as well as for the data and services available on the platform.

Multiple ESB products

In the current era, enterprises are confronted with the deployment of multiple ESB products. An enterprise that uses e.g. SAP will most likely have a Netweaver PI implementation. And if the IT of one or more business units is based on Microsoft products, there will also be a Biz Talk Server implementation. Innovative parts of the organization that use Cordys BPM tools, will likely have a process server based on the Cordys ESB, or it may be Tibco or Oracle, or all of them. And perhaps the enterprise supports IBM Websphere ESB at the corporate level, not to mention that IBM's Advanced ESB (a.k.a. Websphere Message Broker) may be on the game as well.

Policies have to be in place to regulate what processes and services run on what products, taking the functional domains into account. Think of how to define your policies if one of the product implementations spans multiple functional domains where you defined a corporate bus of another vendor for inter domain communications. E.g. SAP Netweaver PI is used for the Financial domain as well as for the HRM domain. It will not always be very clear to the developers to use the corporate ESB to communicate between Finance and HRM in this example. Architectural policies will have to enforce this for the sake of autonomy and flexibility (design-to-change).

Federated infrastructures

Another challenge is the federated ESB infrastructure. If applications only run in a central data center, there will not always be a need for a federated ESB. But if applications also run on distributed locations, e.g. multiple data centers, offices, devices (mobile or not), shops, trains, stations, ASP's, etc. a central ESB deployment will not suffice. And even if the applications run in one data center, but in network zones separated by firewalls, some kind of federated ESB infrastructure will be needed.

Policies have to be in place to guide the implementation and use of federated ESB infrastructures, including federated control of the ESB, to allow for distributed services to smoothly communicate across geographic- and network boundaries.

Conclusion

Enterprises will have to spend serious efforts in architecting and governing an ESB infrastructure. All three challenges mentioned above have their own characteristics that are interdependent and must be balanced for optimal results in terms of efficiency, flexibility, manageability, stability, autonomy and governance.