EDA in practice: Hi Jim, I need some data from your Global Data Space

At the phone

Jane: "Hi, Jim, we are working on a new BI-system for the marketing department and we need some ordering- and invoice data."

Jim: "Are you working on a new BI-system?"

Jane: "Yes, Business Intelligence, reports. We need more recent data, the old system can't supply the actuality we need."

Jim: "Ah, I see. What exactly do you need?"

Jane: "Well, what I said, I want all the incoming orders data and all the invoice data. Do you have it available?"

Jim: "You are lazy, Jane, we supplied everybody with our business events catalog. You could have looked it up by yourself."

Jane: "You are right, Jim, but you always are so helpful, and... I just wanted to hear your voice." [blush]

Jim: "You are flattering me, Jane, come over to my office and we will look it up together."

Jane: "You are so kind, I'll be there in a minute."

At Jim's office

Jim: "Look, here are all the business events that currently live in our Global Data Space. But wait, lets first get a cup of coffee. Milk and sucker?"

Jane: "Black please, Jim."

[...]

Jim: "Let's see, orders, yes, here it is, the order received event. You are lucky, the owner has declared the data publicly available to all internal departments. Notifications cost 10 billing points per message."

Jane: "Great, how looks the canonical format?"

Jim: "Here, below the description you can see the definitions of all available data."

Jane: "Could you please send me a copy by mail?"

Jim; "Of course, I'll enter your mail-address here and press this button... there you are, a copy in your mailbox."

Jane: "And the invoice data?"

Jim: "Hmmm, look, it's not publicly available, you need the ask the permission of the owner. Shall I do that for you?"

Jane: "I first want to study the canonical format, Jim."

Jim: "You know we always translate the canonical format to your local required format, do you?"

Jane: "Yes, I do know that, but I want to know if all data we require is present."

Jim: "If not, you must inform our Competence Center Integration, they will add enrichments to your format if possible, or even extent the canonical format itself if appropriate."

Jane: "How many billing points will be charged for the invoice messages?"

Jim: "Well, it's not very cheap, look, 30 billing points per message. Look here, the events are available for consumption almost instantly when they physically occur. That means that the invoice process is fully automated. I can't help it, but you have to pay for that quality-of-service."

Jane: "Well, you charge almost nothing for a billing point, so I suppose it's not a big deal. Who is the owner over the invoice receipt events?

Jim: "The owner is the manager of the Finance Department, you can see it here. His name is Robert, the new guy with the big mustache."

Jane: "O, don't mention him, he is always much too kind to me, it scares me."

Jim: "Haha, among us, he recently told me he's gay, perhaps this knowledge helps you, but it seems to scare me."

Jane: "Thank God!"

Jim: "Haha, okay, back to business, if you decide to consume these business events, contact the intake manager of our Competence Center Integration. As soon as he knows the data format you wish, he can have your system connected to our Enterprise Service Bus and configure your system to be subscribed to these events. It will take them about two days, including testing, I guess."

Jane: "Great! Can I offer you a lunch? I'm starving..."

At lunch

Jim: "Jane, do you have any plans after work...?"

SOA versus Service Calls - a short story

Suppose, you introduced an ESB as an infrastructural platform to build your SOA on. Let's say that you have a managed services environment available within your ESB environment that serves as your SOA-base and allows you to govern your SOA.

And now there is your first "reusable" service available. You deploy the service as a shareable component in your managed services environment, completely controlled by your SOA-governance. You decide that every piece of software within your enterprise is allowed to access this service. So you expose the service via the ESB infrastructure to the different environments within your organization. And you and your CIO feel happy, because you created your first SOA-molecule. Champagne...!


Several environments start calling the service. You can see that, because your SOA-governance monitors have shown some traffic to the service. The service seems very popular, because traffic is increasing. And increasing, and increasing, and increasing, wow...

Then complaints are starting to flow in at your service desk. The service response is very, very slow. This holds up the systems and so the business processes. Customers are waiting on the phone or placed in the waiting queue because the Customer-care department has to cope with a very slow system. Not always, but sometimes.

You could scale-up the infrastructure , but the costs are rather high. In fact you would have to up-level your infrastructure for only a few hours over the day. The rest of the day the performance is perfectly high.

Your SOA-governance tools can not exactly detect where the high volume of requests is coming from. So you schedule a meeting at the office of your network specialist.

You show him your reports and he hits some keys on his keyboard. Then he says: "Don't know, it's coming from a subnet in the Amsterdam area."

You know the department that is running some applications in Amsterdam. So you decide to pay them a visit.

Amsterdam: "Yeh, we collect some data and when it reaches a certain limit we run a program to process the data. It is a scheduled process with lots of time-based dependencies. By the way, we are very happy with the service you made available. It works great for us."

You: "Uhm, yes, thank you, but, uh, other departments can't access the service when you run your program..."

Back at your office your CIO drops in with the announcement that his plan to charge the consumers of the service per access call has been approved by the board. "You explained me that you can measure the calls with your SOA-governance tool, so when do you think we could start sending our reports to the Financial department to charge the consumers?"

"No, yes we can, no, the tools can measure the calls, but not where they come from. Well, not really true, we can see the IP-address. We should build something to convert the IP-addresses to the department code. Or something like that, because IP-addresses change over time, and some departments run software of other departments on their computers..."

And then you tell your CIO about the huge number of calls from Amsterdam, making the service unavailable for the Customer-care department. While you are talking to your CIO a phone-call comes in.

"Hi there, it's me, John. We are using your service, but it appears that you use numeric city-codes. That is how the Finance department use them. But we use another code-set of alphabetics, just like the Personnel department because we get our data from them, you see. Could you please change that?"

You: "But the Finance department also uses our service. That means we have to pass two different city-codes in the message."

"No problem, wait... Bill is just telling me that the office-codes are also different, same story. If you would be so kind to change that as well... We are very happy with the service, but the performance drops a few times a day. That is no problem for this application, but it would be too cumbersome for our on-line application, so we decided not to use the service for that one."

You sleep not very well that night. You have some awful nightmares; your CIO dropping you of the roof of the office building and everybody pointing at you and shouting: "Mister service, boooh!"

And then just before you wake up a beautiful woman enters your dream. She kisses you on your cheek and whispers in your ear: "I've got a solution for you."

She unfolds a sheet of paper and sticks it on the wall. Then she disappears again. You stare astonished at the picture on the sheet. It looks very familiar to you. It is yours, but with some slight differences:


Proxies? Proxies! With proxies you create environment representations within your managed services environment at the perimeter of your SOA-base. The environments are now visible for your SOA-governance tools and, in future, for your BPM-layer. And you can control the environments. You can throttle Amsterdam, you can measure the number of calls per environment and charge them accordingly, you can automatically redirect environments to a dedicated instance of the service as soon as the number of calls exceeds a limit, you can align formats and code-sets per environment by adding mediation services, you can add authorization per environment. You have overview and control of the whole.

All you have to do is to add basic authentication between the environments and the respective proxy services in order to guarantee their relationships. Initially all proxy services can look identical and just pass the call to the shared service. The trade-off is that only authenticated environments can call the service, but isn't that what you want from an enterprise level IT-architecture perspective?

The next morning, when you walk to work there is a happy feeling inside you, you feel relieved. Your eyes catch a beautiful woman across the street. She looks very familiar to you, as if you have recently met her, but you can't remember where. She looks at you with a smile on her face and she waves with her hand. Then she turns her back to you and disappears in the crowd...