Monday, March 31, 2008

Event Processor tracks State of Objects

An object can be anything you decide to maintain data about; a human, a train, an order. When things relevant to these objects happen, the data about these objects may need to be changed to represent the new situation; a human gets ill, a train gets delayed, an order gets rejected. Things that happen are events.

Data about objects is maintained in databases. So events may trigger database updates. The database persists the state of objects. You may choose to persist the state-history (data warehouses), or not.

So far so good, we got our applications to handle these updates based on simple or complex algorithms. But things might get complicated in highly active operational environments with near real-time processing requirements. Consider the following cases.

The state change of an object is derived by correlating multiple events occurring within a time-frame.

A trains new estimated time of arrival at B depends on (1) trains departure time at A, AND (2) speed limit between A and B due to heavy weather, AND (3) congestion approaching B (because of other trains departed earlier from A and not yet arrived at B).

The state change of an object is derived from patterns in multiple events occurring within a time-frame.
Two gate passages are detected with the same access token within a time-frame that is too short to travel between the two gates. This pattern detects - in real-time - an illegal copy of the access token (new state: blacklisted) and may alert authorized personnel on duty to arrest the passenger instantly.

The state of objects changes quicker then a database can follow.

A huge number of stock quotes to be traded changes within milliseconds.

Actions are to be started instantly based on a specific state change of an object.
Inform passengers their train will get delayed (mind: the action is the result of a real-time correlation of multiple event types within certain time-frames, see above).

In case of high volume state changes, real-time event correlation or real-time event pattern recognition, wouldn’t it be a good idea to deploy a dedicated service in your SOA to process events, hold states and publish new derived events?

An event processor is a service that pulls multiple streams of event data through its memory for comparison. Boolean logic detects correlations and/or the occurrence of predefined patterns across multiple event instances within certain time-frames. The event processor may also instantiate the objects you want to track some predefined states of (e.g. delay states of all running trains), likely in memory. Boolean logic snaps relevant state changes of the object instances and will trigger instantiation of the new state (that can be queried). All in real-time and instantly within a few clock-cycles by applying boolean algebra and truth tables. Based on the results actions can be triggered including executing tasks, start processes (BPM), publishing derived events and new object states or driving business activity monitors (BAM).

If you are able to model the event data being idempotent this service will not only be potentially very powerful, but very robust as well.


Paul Vincent said...

Jack: fully agree - state is very important to model.
Note that some of us use the term "rule" to describe "actions.. on a specific state change".

PatternStorm said...

Hello Jack,

interesting post, please, I have some questions and some comments:


What do you mean "by applying boolean algebra"? Can you put an example?

Can you put an example also of what you mean by idempotency here?

What do you mean by "The event processor may also instantiate the objects you want to track some predefined states of (e.g. delay states of all running trains)"? An example would also help here...


Time is not the only possible dimension defining which subset of events are of interest, this can be genaralized and talk about context: any events falling inside the context are of interest.

You say an event processor is a service, however this can be misleading due to the usual meaning associated to the word service in a SOA: imho an event processor looks more like a runtime into which to deploy individual event processors arranged to form event processing networks. You can call this a service, anything can be called a service, but is more informative to say that it is an event-driven runtime to reactively process events through the deployment of event processing agents written with a specialized highly declarative, reactive, possibly visual, programming language approppriate for the task at hand.

Thak you ver much.


change rubber tracks said...

A blog worth reading on forward until the end! :) I really admire the wording in this blog, quite precise to the details but I just might change one or two things, never-the-less, bravo on well choiced words mate.. p.s.>> Thanks for sharing, I actually picked up some knowledge on this one :)
-Have an amazing day!