How to use Duplex MEP to communicate with BizTalk from a .NET application or a WF workflow running inside AppFabric Part 3
Introduction
In the first article of the series we discussed how to exchange messages with an orchestration via a two-way WCF Receive Location using the Duplex Message Exchange Pattern. This form of bi-directional communication is characterized by the ability of both the service and the client to send messages to each other independently either using one-way or request/reply messaging. In a service-oriented architecture or a service bus composed of multiple, heterogeneous systems, interactions between autonomous client and service applications are asynchronous and loosely-coupled. All communications require published and discoverable service contracts, well-known data formats and a shared communication infrastructure. In the second part of the article we saw how to implement an asynchronous communication between a client application and a WCF Workflow Service running within IIS\AppFabric Hosting Services using the Durable Duplex Correlation provided by WF 4.0. Besides, we discussed how to create a custom Activity for extending AppFabric Tracking with user-defined events and how to exploit the XML-based data transformation capabilities provided by the new BizTalk Server Mapper directly in a WF project thanks to the new Mapper Activity contained in the AppFabric Connect. In the final article of the series, we’ll examine how to implement an asynchronous communication between a WCF Workflow Service and an Orchestration using WS-Addressing and Content-Based Correlation.
Before explaining the architecture of the demo, let me briefly introduce and discuss some of the techniques that I used to implement my solution.
Correlation in WF 4.0
If you are a WF or a BizTalk developer, you are surely familiar with the concept of correlation. Typically, at runtime workflows or orchestrations have multiple instances executing simultaneously. Therefore, when a workflow service implements an asynchronous communication pattern to exchange messages with other services, correlation provides the mechanism to ensure that messages are sent to the appropriate workflow instance. Correlation enables relating workflow service messages to each other or to the application instance state, such as a reply to an initial request, or a particular order ID to the persisted state of an order-processing workflow. Workflow Foundation 4.0 provides 2 different categories of correlation called, respectively, Protocol-Based Correlation and Content-Based Correlation. Protocol-based correlations use data provided by the message delivery infrastructure to provide the mapping between messages. Messages that are correlated using protocol-based correlation are related to each other using an object in memory, such as a RequestContext, or by a token provided by the transport protocol. Content-based correlations relate messages to each other using application-specified data. Messages that are correlated using content-based correlation are related to each other by some application-defined data in the message, such as a customer number.
Protocol-Based Correlation
Protocol-based correlation uses the transport mechanism to relate messages to each other and the appropriate workflow instance. Some system-provided protocol correlation mechanisms include Request-Reply correlation and Context-Based correlation. A Request-Reply correlation is used to correlate a single pair of messaging activities to form a two-way synchronous inbound or outbound operation, such as a Send paired with a ReceiveReply, or a Receive paired with a SendReply. The Visual Studio 2010 Workflow Designer also provides a set of activity templates to quickly implement this pattern. A context-based correlation is based on the context exchange mechanism described in the .NET Context Exchange Protocol Specification. To use context-based correlation, a context-based binding such as BasicHttpContextBinding, WSHttpContextBinding or NetTcpContextBinding must be used on the endpoint.
For more information about protocol correlation, see the following topics on MSDN:
For more information about using the Visual Studio 2010 Workflow Designer activity templates, see Messaging Activities. For sample code, see the Durable Duplex and NetContextExchangeCorrelation samples.
Content-Based Correlation
Content-based correlation uses data in the message to associate it to a particular workflow instance. Unlike protocol-based correlation, content-based correlation requires the application developer to explicitly state where this data can be found in each related message. Activities that use content-based correlation specify this message data by using a MessageQuerySet. Content-based correlation is useful when communicating with services that do not use one of the context bindings such as BasicHttpContextBinding. For more information about content-based correlation, see Content Based Correlation. For sample code, see the Content-Based Correlation and Correlated Calculator samples.
A content-based correlation takes data from the incoming message and maps it to an existing instance. This kind of correlation can be used in the following 2 scenarios:
- when a WCF workflow service has multiple methods that are accessed by a single client and a piece of data in the exchanged messages identifies the desired instance;
- when a WCF workflow service submits a request to a downstream service and asynchronously waits for a response that could potentially arrive after some minutes, hours or days.
In my demo I used 2 different types of protocol-based correlation, respectively, the Durable Duplex Correlation to realize an asynchronous message exchange between the client application and the WCF workflow service and the Content-Based Correlation to implement an asynchronous communication between the WF workflow service and the underlying BizTalk orchestration.
Architecture of the Demo
The following picture depicts the architecture of the demo. The idea behind the application is quite simple: a Windows Forms application submits a question to a WCF workflow service hosted in IIS\AppFabric and asynchronously waits for the related answer. The AsyncMagic8Ball WCF workflow service uses the Mapper activity to transform the incoming request in a format suitable to be consumed by the underlying BizTalk application. For more information on how using the Mapper activity in a WF workflow to implement message transformation, please refer to the second part of this series. Next, the WCF workflow service sends the request message to BizTalk Server via via a one-way WCF-NetTcp Receive Location. In particular, the client endpoint used by the WCF workflow service to transmit the request to BizTalk is configured to use a custom message inspector called ReplyToMessageInspector. At runtime, this component assigns the URL where the WCF workflow service is asynchronously waiting for the response to the ReplyTo header of the outgoing request message. Once received, the request message is read and processed by a new instance of the AsyncMagic8Ball orchestration that returns a response message containing one of 20 standardized answers. In particular, the orchestration copies the request ID from the request to the response message. This piece of information will be used by the WF runtime to correlate the response message back to the appropriate instance of the AsyncMagic8Ball WCF workflow service. Then the orchestration reads the callback URL from the WCF.ReplyToAddress context property and assigns its value to the Address property of the dynamic send port used to return the response to the appropriate instance of the WCF workflow service. Upon receiving the response message from BizTalk, the WCF workflow service applies another map using the Mapper activity and returns the resulting message to the client application.
Message Flow
- The Windows Forms Client Application enables a user to specify a question and a delay in seconds. When the user presses the Ask button, a new request message containing the question and the delay is created and sent to a the WCF workflow service. Before sending the first message, the client application creates and opens a service host to expose a callback endpoint that the workflow can invoke to return the response message. In particular, the binding used to expose this callback contract is the NetTcpBinding, whereas the binding used to send the request message to the WCF workflow service is the NetTcpContextBinding. We will expand on this point in the next sections when we’ll analyze the client-side code.
- The WCF workflow service receives the request message of type WFRequest using the Receive activity of a ReceiveAndSendReply composed activity. In particular, the Receive activity is used to initialize 2 distinct correlation handles:
- The callbackHandle is configured to use the Durable Duplex Correlation and is used to correlate the incoming request with the response returned to the client application.
- The correlationHandle is instead configured to use the Content-Based Correlation and is used to correlate the outgoing request with the response returned by BizTalk Server.
- The WCF workflow service uses the CustomTrackingActivity to keep track of individual processing steps and uses an instance of the Mapper activity to transform the WFRequest object into an instance of the BizTalkRequest class. See the second part of this series for more information on the CustomTrackingActivity and Mapper activities.
- The WCF workflow service uses a composed SendAndReceiveReply activity to send the BizTalkRequest message to the WCF-NetTcp receive location exposed by the BizTalk application. The ReplyToMessageInspector assigns the URL where the WCF workflow service is asynchronously waiting for the response to the ReplyTo header of the outgoing request message.
- A one-way WCF-NetTcp receive location receives the request message and the XmlReceive pipeline promotes the MessageType context property.
- The Message Agent submits the request message to the MessageBox (BizTalkMsgBoxDb).
- A new instance of the AsyncMagic8Ball orchestration receives the request message via a one-way logical port and uses a custom helper component called XPathHelper to read the value of the Question and Delay elements from the inbound message.
- The AsyncMagic8Ball orchestration invokes the SetResponse static method exposed by the ResponseHelper class to build the response message containing the answer to this question contained in the request message. Then it copies the request ID from the request to the response message, reads the callback URL from the WCF.ReplyToAddress context property and assigns its value to the Address property of the dynamic send port used to return the response to the appropriate instance of the WCF workflow service. The response message is finally published to the MessageBox (BizTalkMsgBoxDb) by the Message Agent.
- The response message is retrieved by a one-way Dynamic Send Port.
- The PassThruTransmit send pipeline is executed by the Dynamic Send Port.
- The response message is returned to the WCF workflow service.
- The WCF workflow service uses a Mapper activity to transform the BizTalkResponse object into an instance of the WFRequest class.
- The WCF workflow service uses a Send activity to send back the response message to the client application. The Send activity is configured to use the callback correlation that contains the URI of the callback endpoint exposed by the client application.
Client Code
Please refer to the second article of this series for a full explanation of the code used by the client application to invoke the WCF workflow service using the Durable Duplex communication pattern.
The AsyncMagic8Ball WCF Workflow Service
In this section I will focus my attention on how the AsyncMagic8Ball WCF workflow service exchange messages with both the client application and the BizTalk orchestration. When I created the WCF Workflow Service, the initial workflow just contained a Sequence activity with a Receive activity followed by a SendReply activity as shown in the following illustration.I selected the Sequential activity and clicked the Variables button to display the corresponding editor. Next, I created a variable for each message to exchange with the client and BizTalk application and then I created two CorrelationHandle variables called respectively callbackHandle and correlationHandle. The first of these two variables is used to implement the Durable Duplex Correlation with the client application, whereas the second one is configured to use the Content-Based Correlation and holds the Id of the incoming request. This information is copied into the request sent to BizTalk and is contained in the corresponding response message. Upon receiving a response from BizTalk Server, the WF runtime uses this correlation handle to identify the appropriate instance of the AsyncMagic8Ball WCF workflow service to pass the message to. In order to expose a NetTcpContextBinding endpoint I configured the Receive activity as shown in the following picture:
In particular, I used the ServiceContractName property of the Receive activity to specify the target namespace and the contract interface of the service endpoint and I used the Action property to specify the action header of the request message. To initialize the callback correlation handle, I selected the Receive activity and then I clicked the ellipsis button next to the (Collection) text for the CorrelationInitializers property in the property grid for the Add Correlation Initializers dialog box to appear. As shown in the picture below, I specified callbackHandle as correlation handle and I selected Callback correlation initializer in as correlation initializer.
Before invoking the downstream BizTalk application, the WCF workflow service immediately returns an ACK message to the caller. Therefore, I configured the SendReply activity, bound to the initial Receive activity, to return a WFAck message, as shown in the picture below.
As you can notice, the workflow uses a CustomTrackingActivity to emit a user-defined event. This pattern is used throughout the workflow. Custom tracking records generated at runtime by the WCF workflow service can be analyzed using the AppFabric Dashboard . For more information on the CustomTrackingActivity and how to use the AppFabric Dashboard to monitor the runtime behavior of the WCF workflow service, please read the previous article of this series.
There are two options to invoke the WCF-NetTcp receive location exposed by the BizTalk application:
- The first possibility is to generate a custom WCF proxy activity using the Add Service Reference and use this activity to invoke the underlying WCF receive location in a synchronous way.For more information on this technique, please refer to the previous article of this series.
- The second alternative is using the the messaging activities provided out-of-the-box by WF and the Content-Based Correlation to implement an asynchronous communication between the WCF workflow service and the underlying orchestration. In this article, we examine this approach.
The following picture depicts the central part of the AsyncMagic8Ball WCF workflow service.
In a nutshell, this section of the workflow executes the following actions:
- Uses a SendAndReceiveReply activity to send the request message to the one-way WCF-NetTcp receive location exposed by the BizTalk application.
- Tracks a user-defined event using the CustomTrackingActivity.
- Uses a Receive activity to receive the response back from BizTalk.
The following figure shows how I configured the Send activity to transmit the request message to BizTalk.
As highlighted above, I used the ServiceContractName property to specify the target namespace and the contract interface utilized by the client endpoint and the EndpointConfigurationName property to define the name of the client endpoint used to transmit the request message to BizTalk. The bizTalkAsyncNetTcpBinding endpoint is defined in the configuration/system.serviceModel/client section of the web.config (the configuration file is shown later in the article). In particular, this endpoint is configured to use a custom message inspector that assigns the callback address of the workflow service to the ReplyTo header of request messages.
Next, I initialized the correlation handle used to implement the content-based correlation. To accomplish this task, I selected the Send activity and then I clicked the ellipsis button next to the (Collection) text for the CorrelationInitializers property in the property grid for the Add Correlation Initializers dialog box to appear. On the left panel of the dialog, I specified correlationHandle as correlation handle and then I selected Query correlation initializer as correlation initializer. Finally, I specified the correlation key by selecting the Id element from the data contract of the BizTalkRequest message.
To complete the configuration of the content-based correlation, I selected the Receive activity and I specified the correlationHandle as value for the CorrelatesWith property. The latter defines the correlation handle that is used to route the message to the appropriate workflow instance, whereas the CorrelatesOn property sets the MessageQuerySet used to query the message to extract correlation data.
To specify the correlation key on the response message, I clicked the ellipsis button next to the (Collection) text for the CorrelatesOn property to open the Add Correlation Initializers dialog box. Then, as shown in the picture below, I selected the Id property from the data contract of the BizTalkResponse message.
The last part of the WCF workflow invokes the callback endpoint exposed by the client application to return the response to the initial request. In particular, the latter contains the Id of the original request, and this allows the client application to correlate the response to the corresponding request, especially when the client has multiple in-flight requests. For more details on how the client handles the callback, please refer to the second part of this series.
This portion of the workflow performs just 2 steps:
- Uses a Send activity to send the response message back to the caller. This activity is configured to use the callback handle correlation.
- Tracks a user-defined event using the CustomTrackingActivity.
The following figure shows how I configured the Send activity used to transmit the response message back to the caller.
As highlighted above, I assigned to the the callbackHandle, previously initialized, to the CorrelatesWith property. Then I properly set the other properties like OperationName, Action, and ServiceContractName to match the characteristics of the callback service endpoint exposed by the client application.
The ReplyToMessageInspector component
In order to transparently add the ReplyTo header to outgoing request messages, I created a custom message inspector. For your convenience, I include below the code of this component along with the code of endpoint behavior used to register at runtime this extension.
ReplyToBehaviorExtensionElement Class
|
ReplyToEndpointBehavior class
public void Validate(ServiceEndpoint endpoint) { return; } #endregion }} |
ReplyToMessageInspector class
|
The following table shows an excerpt from the configuration file used by the AsyncMagic8Ball service and by the SyncMagi8cBall service introduced in the second part of this article. You can find the original configuration file in the companion code for this article. Note in particular the definition of the ReplyToBehaviorExtensionElement component in the configuration/system.serviceModel/extensions section.
|
The AsyncMagic8Ball Orchestration
The following picture shows the structure of the AsyncMagic8Ball orchestration.
The orchestration uses a one-way logical port to receive the inbound request message and dynamic send port to return the corresponding response message. The Trace Request Expression Shape contains the following code to extract the information from the request message. The namespace of the LogHelper and XPathHelper static classes have been eliminated for ease of reading.
|
You can use DebugView, as shown in the picture below, to monitor the trace produced by the orchestration and helper components.
Note My LogHelper class traces messages to the standard output using the capability supplied by the Trace class. This component is primarily intended to be used for debugging a BizTalk application in a test environment, rather than to be used in a production environment. If you are looking for a tracing framework which combines the high performance and flexibility provided by the Event Tracing for Windows (ETW) infrastructure, you can read the following whitepaper by Valery Mizonov: |
The value of the Delay Shape is defined as follows:
|
Therefore, the orchestration waits for the time interval in seconds specified in the request message before returning the response message to the caller. Finally, the table below shows the code used to set the response message. As you can see, the code specifies the value for the context properties exposed by the WCF Adapter. The penultimate line of code reads the callback URL from the WCF.ReplyToAddress context property and assigns its value to the Address property of the dynamic send port used to return the response to the appropriate instance of the AsyncMagic8Ball workflow service, whereas the last line of code specifies to use the WCF-NetTcp to send the response back to the initial caller.
|
Testing the Application
To test the application, you can proceed as follows:
- Makes sure to start the DuplexMEP BizTalk application.
- Open a new instance of the Client Application, as indicated in the picture below.
- Enter an existential question like "Why am I here?", "What's the meaning of like?" or "Will the world end in 2012?" in the Question textbox.
- Select one of NetTcpEndpointAsyncWF in the Endpoint drop down list.
- Specify a Delay in seconds in the corresponding textbox.
- Press the Ask button.
Now, if you press the Ask button multiple times in a row, you can easily notice that the client application is called back by the WCF workflow service in an asynchronous way, that in turn invokes the underlying AsyncMagic8Ball orchestration in an asynchronous manner. Therefore, the client application doesn't need to wait for the response to a previous question before posing a new request.
Make some calls and then open the AppFabric Dashboard. This page is composed of three detailed metrics sections: three detailed metrics sections: Persisted WF Instances, WCF Call History, and WF Instance History. These sections display monitoring and tracking metrics for instances of .NET Framework 4 WCF and WF services. Let’s focus our attention on the WF Instance History section, highlighted in red in the figure below. The latter displays historical statistics derived from tracked workflow instance events stored in one or more monitoring databases. It can draw data from several monitoring databases, if the server or farm uses more than one monitoring database for services deployed at the selected scope.
If you click the Completions link you can review the instances of the AsyncMagic8Ball service that completed in the selected period of time. You can use the Query control on the Tracked WF Instances Page to run a simple query and restrict the number of rows displayed in the grid below.
Finally, you can right-click one of the completed WF instances and select View Tracked Events to access the Tracked Events Page where you can examine events generated by WCF and WF services. On this page you can group events by Event Type, as shown in the figure below, and analyze the user-defined events emitted by the current WCF instance using the CustomTrackingActivity that we saw at the beginning of this article.
In particular, you can quickly investigate the details of a selected event in the Details pane, as highlighted in red in the figure above.
Conclusions
In the final article of this 3-posts series we have seen how to implement an asynchronous communication between a WCF Workflow Service and an Orchestration using WS-Addressing and Content-Based Correlation that represent probably the safest and most reliable way to correlate across truly disparate and disconnected applications in an asynchronous manner. As we have observed in the introduction of the present article, the use of asynchronous communication patterns can dramatically improve the scalability and performance of a distributed application platform where multiple systems exchange messages using WCF. In the first part of this series we have examined how to implement an asynchronous message exchange between a .NET application and an orchestration via a two-way WCF receive location using the Duplex Message Exchange Pattern, whereas in the second part we have seen how to realize an asynchronous communication between a client application and a WCF Workflow Service running within IIS\AppFabric Hosting Services using the Durable Duplex Correlation. In the second article we have also seen how to create a custom activity to emit user-defined events, how to use the AppFabric Dashboard to monitor custom tracking events generated by WF services, and finally how to exploit the Mapper activity provided by AppFabric Connect to implement message transformations in a WCF workflow service. I hope this article can provide you with useful ideas on how to improve the scalability and flexibility of your AppFabric/BizTalk applications. Here you can download the companion code for this article. As always, you feedbacks are more than welcome!