Friday, August 24, 2007

What is EAI?


What is Enterprise Application Integration or EAI? Application Integration is – ironically – about application division. This article explains how applications are divided into separated components and how the components are glued together. Several “glue-areas” are recognized where mashups, portals, shared databases, SOA and EDA all play a role in gluing the pieces together.

In this article several steps are described to finally reach the “integrated application”. These steps however are not meant as a logical sequence. In practice these steps will evolve concurrently over time, led by architectural directions and roadmaps.

Let us start with “the chaos”.

Context


Many current application landscapes consist of an organically grown collection of monolithic applications that interact by many different application- and technology-specific interfaces. All these applications have their own implementations of access control, user interfacing, persistency and business process control. Interface control is merged with business rules within the different applications, representing the overall business processes.

This leads to:

  • Complex and long lasting maintenance efforts
  • Data inconsistency
  • Error-prone interface maintenance because of many (sometimes unknown) dependencies
  • Exponential growth of complexity when extending the system
In the current age where the pace of minor and major business-changes accelerates quickly to comply with global demands and where business more and more will be implemented and performed by IT, such an application landscape will not suffice anymore.

What is one thing we can do to bring order in the chaos? The answer is: strip the applications down to core business rules implementations by externalizing shared areas for exchange, access and persistency.

Externalize shared Exchange Area from applications



Bringing the interface control outside the application into a shared facility to be used by all applications results in uniformity of message exchange. Generic data transformation and routing services offer a layer of indirection between the communicating applications, which makes the applications less dependent on knowledge of each others data formats, technologies, addressing and location. In this layer generic facilities may be implemented to secure data transport in a standards based and sophisticated way, without any impact on the applications themselves.

Another huge advantage is that (changes of) process definitions can be accomplished much easier. Business processes that involve more than one application are established in the interface control mechanisms. Taking these mechanisms outside the applications allows for a more flexible configurable solution. This principle, combined with a design style to break down the business logic within an application into well defined components is the basis for BPM (Business Process Management).

The effects of externalizing an exchange area are:
  • Less maintenance efforts
  • More speed and flexibility in changing business processes
  • Uniformity of management
  • Linear growth of complexity when extending the system
Services at a shared external exchange area brings applications together at the business logic layer.

Externalize shared Access Area from applications




Bringing user interface and access control from the individual applications to a generic facility makes the application:
  • Independent from user devices
  • Independent from user location
  • Independent from user interface
  • Independent from application access control
Generic services can establish single sign-on, customized house-style decorations, remote access, personalization (language, time-zone, preferences).

This leads to:
  • Less maintenance efforts when changing overall presentation
  • Uniformity and consistency of presentation
Services at a shared external access area brings applications together at the user interface layer.

Externalize shared Persistency Area from applications


This externalization principle is very common for decades already: get all the fine grained data persistency logic out of the application and use shared database services. The persistency area offers services for:
  • Virtualization
  • Real-time data synchronization
  • Data warehouses
  • Recovery
  • Meta data management
Some of the benefits are:
  • Data more actual (real-time synchronization)
  • Consistency of data (synchronization, data warehouse)
  • Less database management (recovery, meta model)
Services at a shared external persistency area brings applications together at the data layer.

Virtual one single integrated “application”


Externalizing these three shared areas from the applications leaves a fourth area where the implementation of business logic resides. Business logic may be implemented by ERP, COTS, custom code and legacy. These concrete ways to deliver business logic don’t exclude each other, but overlap (e.g. the greater part of legacy will mostly be custom code).

Finally this architectural design leads to virtual one big integrated enterprise application.


Once in place, opportunities grow. In particular the exchange area is interesting. When business logic implementations are broken down to adequately sized and designed components, the exchange area will enable standards based BPM, BAM and CEP:
  • Business Process Management: Flexible business process definition/maintenance
  • Business Activity Monitoring: Real-time view on operational business state
  • Event processing: React proactively to potential bottlenecks; correlate detected events to generate new events
The exchange area will allow for BPM, BAM and CEP to make use of standards based access services and standards based persistency services in the context of SOA and EDA.

Mashups: integration at the client side


There is however one more aspect to application integration at the user interface layer. And that is the clients. Portals facilitate integration at the server side. At the client side (e.g. the web browser) new technologies are introduced based on Javascript and AJAX. Integration at the client side of the user interface are called a mashup. Mashups bring applications together in the browser while portals bring applications together with the help of portlets on the server.

The glue

The externalized areas can all be characterized as Enterprise Application Integration, each of which is very different from the other in nature. The areas each offer their own kind of services to glue the components together.




The multiple faces of Enterprise Application Integration recapitulated:
  • Integration at the data layer: the glue is shared databases, DBMS, data warehouse
  • Integration at the business logic layer : the glue is ESB, SOA, EDA
  • Integration at the user interface layer (server side): the glue is portlets in portals
  • Integration at the user interface layer (client side): the glue is mashups


3 comments:

Anonymous said...

Hi Jack,

Fascinating analysis. A bit oversimplified for my taste. Don't get me wrong: as a reference roadmap, it is fairly interesting, but there are elements that I believe make it impractical or unrealistic.

For example, in the first diagram, you show apps integrated only between middle layers. I disagree. Apps can be integrated with connections from db-2-db, middle-2-middle, ui-2-ui, and by using humans (app-2-human-2-app).

Failing to recognize that creates a simple model, but one that may miss a critical element: you can't move some of that integration into the EAI without forcing a complete repartition and rewrite of the app.

Second, pulling the U/I off of a set of apps is often untenable, because it assumes good design on the part of the original author (allowing the u/i to come off). You also lose sight of the composability aspects. Where does composition happen in this model?

Third, you do NOT want to move to shared databases. They are an astoundingly bad idea in SOME cases, and in MOST cases are actually infeasable. This is especially true with larger COTS packages.

I like your sensabilities, but there is a better way to create this unified view. It is more revolutionary than evolutionary, I'm afraid. But it can be done. Just not this way.

Jack van Hoof said...

Hi Nick,

Thanks for your input. Of course it is oversimplified as all of the patterns I present here. How can it be otherwise...

I started in the first diagram explaining how monolithic applications communicate: software that pass files to each other. Believe me, we got over 400 business applications that are linked together in this way.

I hope it is clear that I didn't want to define a roadmap; I tried to explain the different views on application integration.

You are right: Apps can be integrated with connections from db-2-db, middle-2-middle, ui-2-ui, and by using humans (app-2-human-2-app). That is exactly what I try to explain in this posting.

We have put in place a portal, an ESB and we standardize on two DBMS systems. That is our application infrastructure.

We certainly don't break up our applications. But we enforce all changes to applications and - more over - all new development efforts (including selecting packaged software) to adhere the use of our application infrastructure (our portal, our ESB and our data storgage facilities). And we prescribe the protocol standards like WSRP, JSR168, SOAP, SQL etcetera. Developers are free to build their applications the way they like, but the connectors to the application infrastructure must comply to our (open) standards, and they are enforced to use this infrastructure.

About shared databases: you may be right, but take it in a broader perspective. We offer data virtualisation, synchronization, back-up/recovery, and so on. These services are meant when I say (oversimplified) shared databases.

-Jack

Anonymous said...

Thanks for the info, you can also find Government Grants here,
or Free Grant Kit here,
Free Grant Kit Grant Kit