SOA beyond the hype

The SOA-storm has tempered down. The extremely enthusiastic opinion-makers are cooling down. The hype is over, and SOA is there to stay. Tools are evolving toward maturity and enterprises are serious in adopting the architectural style of SOA. A new layer of governance, the hardest part of SOA, will slowly grow on top of today's IT-governance once the new CIO has convinced the board. Roadmaps are in place.

What next? Build and implement new applications with new technologies and new standards, conforming to new architectures. And that leads to new challenges: Selecting the right tools to design, build and implement SOA-based applications; defining services and compose them into business process implementations. And building the services in core code.

To get a picture of the current state, I ordered two books from Packt Publishing:

• Oracle SOA Suite Developer's Guide

• SOA Patterns with BizTalk Server 2009

I've chosen Packt Publishing as this publisher demands a practical perspective from its authors: concepts must be clearly linked to feasible implementations with available tools. You may click the links above to find out about the structure and content of these books.

Both books give a good overview of the respective products Oracle SOA Suite 10gR3 and BizTalk Server 2009. In both books the authors drill down from conceptual patterns to implementation solutions including code snippets and screen shots of the tools in practice. In this respect both books are worthwhile reading, depending on the tools your company is using, to allow yourself a head start. Or - as I did - just to get an idea of the capabilities of tools currently available.

I was surprised by the Oracle book. The authors did not only present a very clear picture implementing conceptual patterns with Oracle's SOA Suite, but they went one step further upwards in the design realm. This book explains higher level SOA design principles, such as the arrangement of services in different layers of concern and e.g. the great benefits of using proxies in front of the business services. This book has earned a special place on my bookshelf as it offers excellent insights in good SOA-design - from an IT-development point of view - which can be implemented with currently available tools.

The author of the BizTalk book walks quite another road. The author explains the Microsoft concept of Windows Communication Foundation (WCF) and how BizTalk relates to it. A bunch of patterns is described including orchestration, schemas and endpoints, versioning and asynchronous communications. Finally the book mentions the new SOA capabilities of BizTalk 2009: a set of supporting services, components and patterns called "ESB Guidance 2.0". Some low level SOA design principles come to the scene as well in a chapter called "Planning Service Oriented BizTalk Solutions". This chapter focuses on the concepts of a service and the different types of services including variations of request/response services and one-way services.

The difference between the two books is that the Oracle book might be seen as a SOA design guide, extensively illustrated by applying the Oracle SOA Suite; it has a main focus on the BPEL level (business processes). The BizTalk book is just a BizTalk guide; it has a main focus on the (web)service level and doesn't even mention BPEL.

And what about the current maturity of these tools? Build/Test/Acceptance/Production workflow-support and support for configuration-items (build) control is very poor (non-existent) as far as I could determine, but nevertheless I think both products are mature enough to be used in developing real life SOA solutions. The users of these tools need a good understanding of the SOA-design principles and modern XML-based technologies and standards in order to create added values. The learning curve will be rather steep, but once being accustomed to one tool it will be rather easy to switch to the other tool. And the pace of developing - and modifying - systems will really increase compared to what we are used to.

Diving into the Nerd's level of SOA

When we, architects, business process modelers, and system designers, finally got our concepts in place, and when the SOA governance is instituted, the runtime platforms are installed in development, test and production environments, and the steering committee says "go", then we call them in: The Nerds!

These "Nerds" deserve all the respect, because they are the talented ones who are able to change all our paperwork to smoothly running systems. Even the best and most ingenious architecture is worth nothing without someone who can build it and make it reality.

I came across a book with the subtitle "Build SOA applications on the Microsoft platform in this hands-on guide". That made sense to me. The main title sounds "WCF Multi-tier Services Development with LINQ". So I concluded WCF and LINQ must be the Microsoft platform to build SOA's. I started reading, because I was curious how the deep level developers are able to get our architectural SOA models to live.

WCF stands for Windows Communication Foundation and LINQ stands for Language Integrated Query. Both are heavily (no, totally!) bound to the .NET Framework. In fact this book are two books, one about WCF and one about LINQ. Don't blame me for this conclusion, I am an architect and architects look by nature for separation into demarcated components.

Reading the book I found out that WCF is Microsoft's unified model for building service-oriented applications on the Microsoft .NET Framework and covers an umbrella technology for web services, remoting, and messaging. With WCF programmers are able to surround the written business logic with web services technology like SOAP and WSDL, supporting WS-*, in combination with end-point definition (addressing), contract definitions (service-, operation-, message-, data- and fault-contracts) and asynchronous messaging and queuing.

I also found out that LINQ is used to access the persistent-data layer directly from natively embedded program statements in the source-code. LINQ is a set of features in Visual Studio that extends query capabilities to the language syntax of C# and Visual Basic. So you just code your SQL-queries as smart local statements in C# or Visual Basic and the compiler or interpreter converts these statements to real SQL-queries to access the SQL-aware data layer.

Well, I must say that - as a retired programming-geek - I really enjoined browsing this book. And I am definitely sure that contemporary C#-programmers can gain great insights in using WCF to build SOA-applications and to use LINQ to access the underlaying databases by reading this book. The book really offers a good and pragmatic hands-on guide with code and screen-layout examples. A valuable head start for every .NET developer in the current SOA-era!

[Click the picture for details]

SOA Governance

As we all know SOA is about business and SOA is about technology. A business oriented SOA (organization of well defined and autonomous business service units) needs a technology oriented SOA (modular standards based composition of functional autonomous software components) to support optimal continuity of the service delivery in an ever changing context. Whew…!

What is less commonly recognized is that a technology oriented SOA can also serve business continuity if the business itself is not service oriented. So starting with a technology oriented SOA in an organization that is not explicitly organized in a service oriented way (which I think is still common practice at the deeper levels than just the surface of most organizations today) is not a bad idea.

This being said most of us, clever architects, will agree that without a solid governance SOA will fail at the business level as well as at the technology level. From an IT-perspective SOA is not as much as another way of doing things we used to do, but it is much more an additional layer on the things we already used to do; doing things “SOA” adds an extra dimension to doing things as we are used to in the past decades. We still design, build and test software, we still implement user requirements, we still manage configurations, we still release versions, and we still organize and manage projects. With SOA we add additional requirements to the design, construction and deployment of software in the IT-realm. These requirements significantly broaden stakeholders’ involvement. That is why current development policies and processes need an additional governance layer on top of the current IT-governance to guide the design, construction and deployment of these additional requirements.

But how do you define and implement such an SOA governance layer? Who else but fellow blogger Todd Biske could give us a sound answer to this question. He has written a book on his ideas of SOA governance and states that SOA governance is the key to successful SOA adoption in your organization. Of course, reading a 200 page book will not put a SOA governance layer in place. But it will definitely help you evangelize the need for it in your organization. It will help you to find the answers on what, why, how, when, where and who with regard to the governance aspects of SOA.

Todd approaches SOA governance from the perspective of people, policies and processes to establish and maintain desired behavior in order to succeed in an IT-oriented SOA. Aptly illustrated around a fictive case of the enterprise architecture team of Advasco, a leading financial conglomerate, he teaches his readers the aspects of avoiding a BoS (Bunch of Services), controlling life cycles and versioning, governing design-time and run-time, establishing SOA governance at your organization (I think the hardest challenge of all), and celebrating success to help in changing behavior.

Reading this book felt like taking a hot shower. As professional architects, we all understand what Todd has written (or don’t we?). But owning one handy book of hardly 200 pages with all those thoughts structured and combined at an appropriate level of understanding feels like possessing a jewel. Thanks, Todd!

[Click the picture for details]

Business Process Driven SOA - using BPMN and BPEL

It is going to look like I am collecting the whole SOA-library of Packt Publishing. Not because of any commercial interest or benefit, but because I discovered that their collection is very appealing to me as an IT-architect who realizes that practice is always about the "dirty details" (which I often used to call the "golden details" to our developers).

How do you create an SOA that is driven by business processes? Use BPMN and BPEL is what Matjaz Juric and Kapil Pant evangelize in their book "Business Process Driven SOA - using BPMN and BPEL" where they explain how to get from business process modeling to orchestration and service oriented architecture.
This book starts from a business process perspective and explains how business processes and IT relate. The authors explain why SOA is needed and why we should believe this. They recognize business aspects, technical aspects, and organization aspects in an SOA approach. They continue in explaining how SOA and BPM relate and why it makes a perfect fit for the business process lifecycle.

After these academic exercises they dive into the world of BPMN and the BPEL. BPMN is a formal notation language to define process flows and BPEL is the resulting code to be processed by a process engine. Flow charts and screen shots are used to illustrate the ideas.

My heart opened when I saw back the principal ideas of structured programming. It proved that the authors know how algorithms should be designed. My advice to them is to introduce Nassi-Shneidermann diagrams in the next edition of the book.

The book is highly recommended for the next generation enterprise developers who are going to build enterprise-scale software systems in the context of service oriented architecture. However, one must be aware of the fact that BPMN and BPEL is not the holy grail. Without a technical mapping from the models explained in this book to hard-coded software modules, custom made or commercial off-the-shelf, based on service-contract interfaces, the whole idea of BPMN and BPEL will be not much more than a theoretical concept that can't be implemented or deployed in the real life IT-landscape.

[Click the picture for details]

SOA Cookbook

I am getting addicted to the SOA books of Packt Publishing. Now it is the SOA Cookbook that I've been reading with delight. Why with delight? Because, again, the focus is on practical use. The concepts and technologies in the book are challenged against existing products in the market, including tools of BEA, Oracle, TIBCO, and IBM. This is of very great value to practitioners.

The author roughly knocks down some simplifying myths around SOA. He teaches techniques in the modeling of orchestration processes. These processes belong to the process integration layer of the universal model stack provided by the leading vendors: BPM, Process Integration and ESB.

The book starts with the A (architecture) of SOA which has been approached from Kruchten's famous 4+1 model. The author combines aspects of this approach with the well known ARIS method based on the Event-driven Process Chain, EPC which is an evolvement toward EDA.

The book tells you how to separate SOA from BPM (including design tips) and how BPEL fits in this context. Orchestration versus choreography comes to the scene as well. The author explains how choreography is fundamentally decentralized and acts as a set of traffic rules to govern how participants interact, whereas orchestration builds a flow of control around these interactions. Real-life examples using BPMN support the understanding.

Long and short running processes come to the scene and the change problem of dynamic processes is addressed. You can read why the author calls versioning "Poor Man's Change" and why he thinks versioning is only a beneficial approach to vendors, but hurts adopters. "Design processes that are adaptable to begin with", he says.

The book ends with a chapter on measuring the complexity of SOA where Thomas McCabe's cyclomatic complexity measure (1976) is applied to BPEL and TIBCO's BusinessWorks.

Conclusion: the book is of great value to SOA practitioners in the semi-technical domain. That is, it doesn't deal with the organizational aspects of SOA neither does it deal with the hard-core deployment of services. The book implicitly assumes that your IT-organization is mature with regard the building and deploying software and with regard to procedures to control these development processes. This book adds value in teaching you the state of the art techniques of process integration on top of your current development processes.

[Click the picture for details]

SOA Approach to Integration

After I read Service Oriented Architecture with Java I decided to read another SOA-book of Packt Publishing. The book is titled:

SOA Approach to Integration

As an Enterprise Integration Architect I was very pleased to read this book and I would recommend every integration architect to read it. Why? Not only because it will teach you the ideas behind SOA from an integration perspective, but also because it teaches you the developers language.

The book is also of great value if you are a developer, because it teaches you the various technologies to implement SOA.

The book starts with positioning SOA as an integration style, which I do think is a valid perspective. The term Process-Oriented Architecture is introduced. Yet another acronym, POA, which sounds to me like BPM. BPM, however, is not mentioned in the index of the book. But let's stay away from this semantic discussion.

The authors explain best practices for using XML for integration. They explain the web services approach, including design guidelines. The WS-I Basic Profile comes to the scene and - in the context of Process Oriented Integration - BPEL is comprehensively explained, including some code snippets.

To me the most interesting part of the book was the last chapter. These 90 pages (a quarter of the book) deal with the SOA platform, being the Enterprise Service Bus (ESB). The ESB architecture is described as well as the concepts of Service Containers as the primary tier of the bus. Bus Services are mentioned (Mediation, Transformation and Process Flows); Security and Transactions; Reliability, Scalability and Management; Application Development Considerations; and finally Extending the ESB to Partners. These are the things developers are concerned with. It is a very good idea for an architect to have a notice of what puts the burden on the developers, and to be able to understand their language.

My conclusion is that this book supplies its readers with feet-on-earth knowledge of SOA between concepts and technical implementation. Very worthwhile reading.

[Click the picture for details]

Service Oriented Architecture with Java

Are you fed up with over 800 pages SOA-books full of conceptual blah blah? Letting you know it is completely nuts not to implement SOA? Telling you SOA is about business and not about technology, or just the other way around? Outlining huge expensive roadmaps involving every bit of the company? Stressing to you not to start if you didn't restructure your business- and IT-organization in advance?

Don't worry, there is some hope for you. I discovered a mind-sized book of less than 200 pages practical no-nonsense knowledge on SOA; 169 pages to be exact.

The authors clearly have a thorough understanding of SOA from a business point-of-view as well as of the application level implementation aspects. They succeeded in bringing SOA to earth, presenting no more and no less of SOA than it is. In their own words:

We convert the business processes to "services" and expose it to be "oriented" with its business goals. The software design "architecture" that conforms to this is SOA.

Not only does this book offer practical insights in the architectural and business aspects of SOA, why XML and web services is a good idea at the implementation level, the limitations of RESTful services, why using an ESB, aspects of data handling in SOA, and tight coupling (which has advantages and disadvantages) versus loose coupling (which has other advantages and disadvantages). The authors also demonstrate how all of this can be implemented with today's available tools and frameworks in a Java environment. You really can try out at home how these concepts work. For this purpose the Java code snippets used in the book are downloadable from the publishers website.

There is an easy case study included which compares a traditional EAI solution with an SOA-based solution of a simplified business problem. Really a brilliant and yet easy to understand illustration.

If you are a programmer (not necessarily a Java programmer) and you want your SOA-programming understanding to be state-of-the-art, this book is one that should be on your shortlist for reading shortly.
But also if you are a designer or an architect with little or no Java knowledge, the book is still very valuable in understanding when and how to apply an SOA-approach; you just skip the Java details.

As I said, it is not an overloaded book as many others are, but just a mind-sized set of interesting need-to-know knowledge from a holistic point-of-view, business as well as implementation, written in a style which offered me a few hours of delightful reading.

[Click picture for details]