EDA is Lame and SOA is Blind

If you compare EDA and SOA with the human body, you might say that EDA is like your eyes and ears and SOA is like your hands and feet. When your eyes detect a lion jumping on your path, a message is send to your brain which figures out that your feet have to be notified to run.

Amlan Debnath, vice president for Oracle Server Technologies and a evangelist for event-driven architecture, talks about SOA and EDA. Read the clarifying interview...

Granularity of Services

I notice a misconception spreading around at a large scale between SOA-bloggers and article-writers, even by the great gurus. They keep a discussion alive about the optimal granularity of a service. But they are mixing up services which are functional components, and "Web Services" which is an interface implementation technology.

To me it makes more sense to discuss at which levels of granularity the use of web services technology is appropriate. And that depends on system implementation characteristics such as degree of distribution and heterogeneity and desired openness.

The granularity of functional components "goes to the bottom". It's just common modular and structured design and programming practice, originating from the 70's, every developer should now about. A nice pdf on this subject is available here. So nothing new to SOA. Read, for instance, "Reliable Software through Composite Design" by Glenford J. Myers, 1975.

Or study Yourdon's DFD (Dataflow Diagrams) method; very interesting is part III and chapter 22.

EDA is not just technical implementation

Some SOA-specialists tend to see events and event-driven architecture as a technical implementation; at a lower level of design. Well, it may be, but it needn't be. I'm thinking of EDA as modeling real-world events; at a higher level - even at a top level - of design. An event is a change of state: a certain condition coming true. Business events are the interesting real-world events that a company wants (has planned, is forced) to respond to. The response to the business event is the business process, which can generate new business events, and which can change over time. Business processes handle the life-cyle of the captured business events. Interesting is that a business event (real-world event we planned to respond to) mostly is stable, while the way we are responding (business process) mostly is not.

What I believe is that you can only start modeling business processes when you know the business events you want to respond to. You can only start modeling SOAs (which might be seen as a way to model business processes) when you defined the business events to focus on.

So in my vision events are not technical elements at all. Fortunately we have an ESB to capture the business events and publish them to all interested business processes which are implemented as SOAs. And fortunately we can use the same ESB to build an environment to monitor the state of our business in real-time. This state is represented by the collection of current business events. All based on open standards. Yes, that may be called technical, but with huge business benefits.

The starting point for SOA is...

I just read a blog of Steve Jones, called The starting point for SOA is... Where he comments a blog of CIO Magazine by Christopher Koch. Let me join the party.

I think events happening in the real world should be the starting point of an SOA-design. Real-world events are the triggers for the business processes by nature. When you support a business process with IT, then of course you must get the event into your IT-system. You may call this a service, but it can be all kinds of components (both software and hardware). What we do is starting with modeling the events. We describe events in canonical formats and we publish the events in a "global dataspace" (implemented by an ESB). We use web services interface technology to publish these events. Again, you may call these publishers services, but I'm not sure if that is always appropriate. And yes, the publisher uses a web service interface. But I wouldn't call that an SOA yet. In my opinion SOA starts with recognizing the real-world events (what to react on) that will be the basis for defining generic functional building blocks to compose SOAs (how to react; the process). It's a kind of contract-first design.

Modeling events is what we call an event-driven architecture (EDA), the layer between the real-world events and artificial IT-systems (SOA). So (business) events are our starting point. It's the way we obtain optimal alignment of our business and the supporting IT-systems.

Here is how the entire design-model looks:



Speak aloud (experiment)

In Search of Insight - the future of IT

Charles Nicholls wrote:

It is akin to asking you to drive a car where your dashboard has a speedometer showing the average speed yesterday, and the fuel gauge gives the fuel remaining in the tank at the end of last week. Oh, and you can only use the rear view mirror because there is no windscreen!
Clearly you wouldn’t drive your car like this – well at least not for long! But this is how we run businesses today, based on a rearward view of the business.

And this:

A key component of SOA is the underlying asynchronous message infrastructure; an Enterprise Service Bus (ESB) or Enterprise Application Integration (EAI) system. ESB’s are typically based on Web Services standards and provide foundational services for SOA via an event driven messaging engine.
At the heart of this architecture are events, specifically XML messages. Ultimately the processes themselves, perhaps managed in a Business Process Management tool, are actioned by events.

I read Nicholls' paper on the future of IT and I felt very delighted.

The Da Vinci Code and EDA

What has the Da Vinci Code to do with EDA?

I came across this link. Click on Play Video and enjoy...

Are you an old-style or a new-style developer?

Heard what Gartner said back in 2005 (or was it earlier)?

• The best designers of SOA applications are the architects that understand EDA

And another quote of them:

• Learn EDA to best understand SOA

I absolutely agree with Gartner; developers need to break out of synchronous request-reply thinking. Let me explain...

When I read all those SOA articles and hear all those SOA-gurus talk enthusiastically I see only part of the story, and unfortunately not the most promising part. I'm not sure if those gurus are driving us to deliver the right - or better: complete - designs yet. When they tell you about "calling services" they think RPC-style and that is old-style. The thing is that we have to think in events, even if we decide to implement a request/reply call-stack. To be able to judge when to "call a service" in stead of publishing an event, you need to know EDA. Events are the starting point of the design, the services are not. You don't agree? Well, than you are an old-style developer.

Did you ever notice that EDA is an inverse of RPC-style SOA? (RPC-style SOA, may I nowadays call that SOA 1.0 since the raise of SOA 2.0, huh?) Okay: in EDA the data-supplier takes the initiative in contrast to RPC-SOA where the consumer does. In EDA the supplier and consumer do not know anything of each others existence, they are decoupled. On the other hand in RPC-SOA the consumer calls the supplier by name and even waits for a reply. If the reply doesn't come, the consumer falls into an error procedure; that really is tightly coupled, isn't it? Not in your case? You use asynchronous queues to perform service requests? Well, you are tricking around, probably for good reasons, but it is not EDA; EDA is asynchronous by nature, you can't trick it to act synchronously. EDA has focus on data (message) while RPC-SOA has focus on function (service). Want more? EDA is declarative while RPC-SOA is procedural. That is in EDA you can tick to connect events to processes; in RPC-SOA you built algoritms with If-Then-Else and DoCase constructs in order to fire the appropriate calls. You don't agree? Okay, old-style developer.

What about granularity? How big must a service be? Our SOA-gurus say: not too big and not too small. Depending on what? Some gurus say: depending on performance. I think they are mixing up services which are functional components, and "Web Services" which is an interface implementation technology. To me it makes more sense to ask at which levels of granularity the use of web services technology is appropriate. And that depends on system implementation characteristics such as degree of distribution and heterogeneity and desired openness. So in fact there is no conceptual granularity-issue. You start - top down - designing focused on business events, which are events with business significance. When you reach a level too fine grained to recognize neat real-world events you leave the business oriented approach and enter the application oriented approach where RPC-SOA may become adequate. This is the level to compose adaptive business solutions - i.e. ways to handle business events - using independent generic components. So at this level you start creating SOA building blocks to construct modular applications. Though, that doesn't mean you must drop the even-driven paradigm here, preferably not. Don't agree? You old-style delevoper.

As you might have read between the lines, EDA has affinity with business processes and SOA has affinity with application construction. Taking into account that the EDA approach is an inverse of the well known SOA approach you might say the two paradigms are complements by nature.

Do you recognize the huge business potential of asynchronous, event oriented design? And do you believe that EDA by its nature will be the paradigm to realize the ultimate alignment of business processes and the supporting IT-systems? That the ultimate layer between our real world-events and artificial application constructs will be an EDA? And do you recognize the possibilities offered by the current IT-technology evolutions to support this paradigm? Then you truly are a new-style developer.

Context Sensitive Subscription - a pattern

We have a lot of trains - over 2000. On these trains we also have a lot of personnel like drivers and conductors. A main characteristic is that the trains and the personnel move across the country and so their locations change as time progresses. Another characteristic is that the movements of the physical entities do not have a stable pattern; one day a train runs from north to south and the next day the same physical train may run from east to west. And personnel change trains every few hours and they also have days off. This means that the context is changing all the time.

Trains have standard applications for instance to collect data about delays of other trains and to provide the passengers with relevant travel information. The same applies for the PDA's of the personnel on duty. So there are thousands of instances of the same applications deployed and running. Now it's getting interesting: how do I distinguish between the different instances of the same application when it comes to communicating with these applications? Sending messages from these moving application instances in changing contexts is not a big deal, but consuming just the right messages relevant in the specific and frequently changing context is another story.

All of our new application system designs take the event-driven architecture paradigm as a base. We are very keen on that. So what we need is a way to implement context sensitive subscriptions. We keep the application-layer independent from the network-layer by using an Enterprise Service Bus between the two layers. We therefore don't allow to identify or address our applications at the IP-level, so a unique logical identifier (name) per instance is required.

We already use the context aware services of Appear Networks (see video) to be able to deploy (or eliminate) business software depending on the - predefined - context of the concerning devices (owner, time, role, location). In trains, for instance, these context aware services may take advantage of on-board GPS-equipment. A pre-installed generic client initiates this process. Based on the use of these context aware services that we already got, we developed a very simple design-pattern for context sensitive subscriptions. Here is how it works...


The idea is as follows:

1. There are a lot of instances of the same application (one on every device or train)
2. Every instance of the application has the same name (identifier) on every device or train
3. To be able to have the infrastructure route the correct messages to the correct application instances, the instances must be distinguishable
4. Use a uniquely identifiable proxy (Subscriber) in front of the application instance (id based on predefined context characteristics)
5. On change of context (published by context services) generate a new proxy based on a template - including subscription parameters and an identifier both based on the context - and replace to old proxy
6. Have the proxy subscribe itself to the relevant publications (may be content based)

So the idea is to re-subscribe. In this pattern the "context sensitive" part is not a filter but a context-unique subscriber. We want the endpoints only to receive relevant messages. This makes the solution more scalable and - compared to a filter solution - it decreases the load on the mobile network and at the endpoints tremendously.

The pattern is not only of use in wireless mobile environments, but also in plug-and-play devices like gates and information displays on our railway stations. All gates on all stations function in the same way with the same software, but they differ in location. The same applies to the information displays. To be able to receive only the messages relevant for their own specific location, the devices must be uniquely identifiable at the application-level.

It as an enormous effort to pre-configure all those devices separately for the specific location where every device is planned to be installed. It's much cheaper and much more flexible when all devices are generically configured with general software such as location detection software (e.g. context aware service) and context sensitive subscription software. The location detection software may use detection appliances like a GPS-receiver, or it may make use of a simple user-interface to get a location-code. Once the location is determined, the generic software in the device generates a context sensitive subscription. In this way it doesn't matter which device is installed at which location. And devices can be moved to other locations without reconfiguring the software.

An example of the use of this pattern is an information display at a main railway station, which can be installed on the fly and can show information of other main stations in the country by a simply entering the desired location-code.

Another example is a traveler who buys a ticket via Internet. The Internet application issues an event which is published on the ESB. The ESB supports some enrichment by adding the location-codes of the railway stations on the route. Only the gates on this route receive and store the message of this traveler and allow passage (only today and only once) when the traveler puts his id-card in the reader of the gate; without any travel-rights loaded on the id-card and without querying a central database.

At this very moment of writing we are merely discussing the abstract - implementation agnostic - pattern of context sensitive subscriptions; the idea. But we think that an implementation of this pattern will bring us a lot of flexibility and ease.

Please feel free to react.