Service Bus, Workflow, BizTalk Server, and Windows Azure BizTalk Services

I see questions from various customers and partners – Microsoft has so many technologies for doing “Integration”, when do I choose what? In this post, I will put forth my views on firstly, why there are so many technologies, and then based on that, it should become clearer when you should use what.

Let me first start with BizTalk Server. The diagram below tries to show BizTalk Server in terms of its components / capabilities. Not everything is shown here – just what is relevant to this post.

This is how BizTalk Server has been since 2004. The capabilities / components have been enhanced, but the overall architecture has remained the same.

 

 

Since then, there have been changes in other parts of the Microsoft ecosystem.

The first change was introduced with .NET 3.0. That was – the introduction of WCF. WCF was the new communication framework – if you need to implement a component to abstract your transport protocol, then WCF was the way to do it. BizTalk Server had its own Adapter Framework for developing Adapters, and the obvious limitation was that they could work only with BizTalk. WCF allowed you to write “Adapters” that could run in any .NET application. With BizTalk 2006 R2, we tried to bridge the difference between the frameworks, with the introduction of the WCF Adapter in BizTalk 2006 R2, and the WCF LOB Adapter SDK to make it simpler to write transports for LOBs using WCF.

The next change – Workflow – was also introduced with .NET 3.0, and was then revamped in .NET 4.0. Recently (post .NET 4.5) we shipped the Workflow Manager, which is a host for running Workflows (though the workflows that run here have some differences with respect to what they can do compared to if you use the execution options that ship with .NET (i.e., WorkflowInvoker, WorkflowApplication, and WorkflowServiceHost). BizTalk Server had its own framework for designing workflows for modeling the message flow across endpoints – orchestrations. But again, the biggest limitation is that this was tightly coupled with BizTalk (and can also be said that the other big limitation is that it hasn’t evolved much since its original incarnation). While, .NET Workflows can be used in any .NET application, and they’ve been evolving with every .NET release.

The next change was introduced somewhere around … I think the 2007 timeframe. You may remember it first being known as BizTalk .NET Services. This was essentially the Relay capability which is there in Service Bus today. Soon after was introduced the concept of a MessageBuffer, which was some sort of a “queue in the cloud”. All this ultimately evolved to what we know today as Service Bus, which comprises of Relay, Queues and Topics. A Message Bus. BizTalk Server had its own version of a pub-sub system – the MessageBox – but not only was this tightly coupled with BizTalk (just like Adapters and Orchestrations), but this wasn’t even something which developers/customers could directly use – you only worked with it indirectly when working with Send Ports (filters), Messaging (promoted properties), Orchestrations (filters). You knew it was there, but it wasn’t meant to be used directly from your code.

So, let’s see how the components listed above fit in.

 

Let’s now take a look at Windows Azure BizTalk Services. Here, we’re attempting to bring Integration to the cloud, a small set of components / features / capabilities at a time. We’ve release version 1.0 just recently (we announced the public preview just a few days ago), and hope to announce GA soon.

 

The above picture captures the components unique to BizTalk Services. One thing you’ll notice is that the position of the box that represented the “Message Bus” has changed. The reason for this is – Bridges in Windows Azure BizTalk Services aren’t built on top of a “Message Bus”. BizTalk Services can of course interact with the other components in Azure – so, we can interact with the “Message Bus” component in Azure, which is – Service Bus. Keeping with the current component pictures, we now arrive at this:

Over the course of the above pictures, it should now be clear how each of the technologies mentioned here is meant to be used. Depending on the capabilities that you need, the appropriate technology/product can be picked.

 

It’s also easy to hypothesize how BizTalk Services will evolve:

Comments

• Anonymous
  June 15, 2013
  Great post Mustansir, explaining the various individual parts of our integration stack.  

• Anonymous
  December 17, 2013
  The comment has been removed

• Anonymous
  December 13, 2014
  Mustansir, great post. Should we understand that BizTalk azure services will never integrate one day or another  BRE? we heavily use BRE on-premise