I have a process which starts a workflow and then an orchestration. I want to create an activity cross the workflow and the orchestration. So I create a ContinuationToken in the WF IC and create a ContinuationID in the Orchestration TPE. However, it does not seem work. The data from WF and the data from Orch are written into two records in the Completed table.

My questions are:

Is it possible for continuation to happen between WF/WCF and Orchestration? What specific steps I should be aware in order for that to happen? Any sample?

Here is my answer:  if you have a WCF data that is also available (flowed to) Biztalk Orchestration, you can use it to form a continuation token.   The important piece of information here is that you must prefix it with a constant and the constant must be the same name as the ContinuationID folder.

Here is a snippet of IC for a WCF application, in this continuation token node. Here the TokenID will be something like WCF2Orch1PO123 if POID contains PO123.

      <ic:ContinuationToken>

        <ic:Expression>

          <ic:Operation Name="Constant">

            <ic:Argument>WCF2Orch1</ic:Argument>

          </ic:Operation>

          <wcf:Operation Name="XPath">

            <wcf:Argument>//s:Body//po:POID</wcf:Argument>

          </wcf:Operation>

          <ic:Operation Name="Concatenate" />

        </ic:Expression>

      </ic:ContinuationToken>

In the TPE, you would then need to create a ContinuationID folder and rename it to “WCF2Orch1” and map the value under it to a piece of data that contains the flowed value of PO123.

*  When you deploy the tracking profile, you will get a warning that the tracking profile is missing the Continuation folder, but since we know what we are doing :), just ignore and proceed.

A common scenario is when the WCF application configuration file is updated and the IC is created and deployed, there is not data intercepted and persisted into the BAM PITDB (Primary Import Tables Database).   When the event log is checked out without error and that the bm.exe deploy-interceptor is successful, most likely the culprit is in the IC – somewhere in the IC, the XPath is incorrectly set.

Here is what I would do:

Enable WCF logging/tracing to produce the wcf log.  In the config file of the “interceptee” application (either or both client and service), splice the following highlighted code.

<configuration>

  <system.serviceModel>

    <diagnostics>

      <messageLogging maxMessagesToLog="30000"

              logEntireMessage="true"

              logMessagesAtServiceLevel="true"

              logMalformedMessages="true"

              logMessagesAtTransportLevel="true">

      </messageLogging>

    </diagnostics>

  </system.serviceModel>

  <system.diagnostics>

    <sources>

      <source name="System.ServiceModel" switchValue="Verbose, ActivityTracing" >

        <listeners>

          <add name="xml" />

        </listeners>

      </source>

      <source name="System.ServiceModel.MessageLogging" switchValue="Verbose">

        <listeners>

          <add name="xml" />

        </listeners>

      </source>

      <source name="Microsoft BizTalk Bam Interceptors" switchValue="All" >

        <listeners>

          <add name="xml"/>

          <add name="myListener" type="System.Diagnostics.TextWriterTraceListener"

               initializeData="TextWriterOutput.log" />

        </listeners>

      </source>

    </sources>

    <sharedListeners>

      <add name="xml"

           type="System.Diagnostics.XmlWriterTraceListener"

           initializeData="Traces.e2e" />

    </sharedListeners>

    <trace autoflush="true" />

  </system.diagnostics>

</configuration>

Rerun the application.  This should generate a WCF trace file: *.e2e.

Use the SvcTraceViewer.exe tool (comes with WCF SDK installation) to view the wcf log and copy out the message into a .xml file.  

 Here in this snapshot of the tool, you should copy out the s:envelop node. 

Use and xpath tool (such as VisualXPath) to drill down to the specific area of the message to produce a valid XPath.  Copy and past this into the interceptor configuration file.

Redeploy the IC and rerun the application.

Hopefully the above steps would help surface the problematic XPaths in the IC and speed up your work.

SvcTraceViewer1.JPG

In this series of blogs I will again as usual, delve on the BAM side of BizTalk offering and answer any questions from you.  In R2, there are two new interceptors added, namely the BAM Interceptor for WCF (Windows Communication Foundation) and the BAM Interceptor for WF (Windows Workflow Foundation).   Of the two new interceptors, I will, for the time being, concentrate on the WCF Interceptor.  To find out more on WF Interceptor, please checkout my colleague, Andy Shen’s blog site: https://blogs.msdn.com/andyshen/

Before I get into the BAM Interceptor for WCF, here is a summary of some howto’s from my past entries to those who are new to Biztalk BAM:

• Introduction and means to BAM-enabled a business process.
• Writing BAM events that are transactionally consistent with BizTalk pipelines using MessagingEventStream.
• Writing BAM events that are transactionally consistent with BizTalk orchestration using OrchestrationEventStream.
• Continuation scenarios with mix of TPE-enabled and BAM  EventStream API tracking.
• Programmatically retrieving data from BAMPrimaryImport database using supportable API.
• Programmatically retrieving server and database info of Biztalk MessageBox and BAM PrimaryImport database.

So what is this BAM Interceptor for WCF?  In a nutshell, it extends the capability of BAM interception to any WCF application of it message without having to recompile the WCF application.  (This is analogous to using BAM Tracking Profile Editor to map and enable interception on BizTalk application without having to recompile the BizTalk application.)

The integration of WCF application and BAM interception is achieved through WCF configuration file by adding a custom behavior to the WCF application endpoint node.  A separate file called IC (Interceptor Configuration) is required to filter and map WCF data of interest to corresponding BAM activity. 

Where in WCF application are the data intercepted?  It supports the following interception points for WCF client, service and callback applications:

·         Before the method call (Request)

·         After the method call (Reply)

·         When the fault occurs in method call (Fault)

What can be intercepted?  Basically, anything you can see in the soap envelope can be tracked, but more often than not, you are interested in the data in the soap body where information of the method parameters is located.

Here are the basic steps to enable BAM interception for a WCF application:

1.      Create BAM activity and deploy.

2.      On a WCF application, update its config file.

3.      Create a corresponding IC file and deploy.

4.      Execute the application.

5.      View data in Portal.

Only step 2 are 3 are new to this release and you can see details information on this in the BizTalk Server Documentation under the heading (I will update this to online link when it is available):

Business Process Management | Using Business Activity Monitoring | Implementing BAM Solutions | Using the BAM WCF and WF Interceptors

Additional resource:

MSDN forum specific for Microsoft BAM: https://forums.microsoft.com/MSDN/ShowForum.aspx?ForumID=1473&SiteID=1

To be continued…

In BizTalk Server Documentation, currently in its 2006 pre-released version, you will find information on the BAMManagementService API.  (Under Technical Reference | Developers Reference | BizTalk Server 2006 .Net Class Reference | Microsoft.BizTalk.Bam.Webservices.Management ). 

The BamManagementService class is what we are interested. This is a webservice API and you can also see its methods by pointing to the box where Biztalk is installed. (replace localhost with the machine name in the link below). 

https://localhost/BAM/BamManagementService/BamManagementService.asmx

• AddSubscription
• CreateAlert
• DeleteAlert
• GenerateEventForArtifact
• GetAlert
• GetAlertsSummary
• GetCapabilities
• GetPivotViewXml
• GetReferences
• GetRelatedActivityInstances
• GetViewDetailsAsXml
• GetViewSummaryForCurrentUser
• RemoveSubscription
• UpdateAlert
• UpdateSubscription

For example, the above GetViewSummaryForCurrentUser would allow you to retrieve all views that the user has access.  And it would allow you to retrieve the activity names from there.  See code example in help doc for this.

Once you have retrieve the activity name, you can then connect to the database and call the views that below to the specific activity. 

Sample code from help doc (some of these are still been updated):

using WS = BamManagementService;

class Program
{
    static void Main(string[] args)
    {
        GetViewSummary();

        Console.WriteLine("Hit Enter when ready.");
        Console.ReadLine();
    }

    static void GetViewSummary()
    {
        // Create a web service proxy object.
        using (WS.BamManagementService managementService = new WS.BamManagementService())
        {
            // BAM Management Web Service requires client to be authenticated.
            // The client must be a member of the BAM Portal Users group.
            managementService.Credentials = System.Net.CredentialCache.DefaultCredentials;

            // Get the list of all BAM views available to the current user.
            WS.ViewSummaryList viewSummaryList = managementService.GetViewSummaryForCurrentUser();

            // Print the info for all views on the screen.
            foreach (WS.ViewSummary viewSummary in viewSummaryList.View)
            {
                Console.WriteLine("View: {0}", viewSummary.Name);
                // Each view is defined on one or more BAM activities.
                foreach (WS.ActivitySummary activitySummary in viewSummary.Activity)
                {
                    // The activity data can be on a different server/database if distributed
                    // BAM infrastructure is deployed.
                    Console.WriteLine("    Activity: {0} {1}",
                        activitySummary.Name,
                        activitySummary.IsRemote ? "[ Remote ]" : "");
                    Console.WriteLine("        Server: {0}", activitySummary.Server);
                    Console.WriteLine("        Database: {0}", activitySummary.Database);
                    Console.WriteLine("        BAM Portal URL: {0}", activitySummary.PortalUrl);

                    // OLAP and Real-time aggregations aggregations can be defined.
                    if (activitySummary.Aggregation != null)
                    {
                        Console.WriteLine("        Aggregations:");
                        foreach (WS.Aggregation aggregation in activitySummary.Aggregation)
                        {
                            Console.WriteLine("            Name: {0} *** Type: {1} *** Cube Name: {2}",
                                aggregation.Name,
                                aggregation.Type,
                                aggregation.CubeName);
                            if (aggregation.PivotView != null)
                            {
                                foreach (WS.PivotView pivotView in aggregation.PivotView)
                                {
                                    Console.WriteLine("                PivotView: {0}", pivotView.Name);
                                }
                            }
                        }
                    }
                }

                Console.WriteLine();
            }

            if (viewSummaryList.PartialResults)
            {
                // Distributed BAM infrastructure is deployed and one of the referenced databases
                // is not accesible.
                Console.WriteLine("One or more of the referenced databases are not available. " +
                    "The given list of views may not be complete.");
            }
        }
    }
} 

Sample Output:

View: SalesManagerView
    Activity: PurchaseOrder 
        Server: purchasesrv
        Database: BAMPrimaryImport
        BAM Portal URL: https://purchasesrv/BAM
        Aggregations:
            Name: POCube *** Type: OLAP *** Cube Name: POCube
            Name: POByLocation *** Type: RTA *** Cube Name: POCube
    Activity: Shipment [ Remote ]
        Server: shipsrv
        Database: BAMPrimaryImport
        BAM Portal URL: https://shipsrv/BAM
        Aggregations:
            Name: SHCube *** Type: OLAP *** Cube Name: SHCube
            Name: SHByLocation *** Type: RTA *** Cube Name: SHCube

View: SupplyManagerView
    Activity: SupplyOrder 
        Server: purchasesrv
        Database: BAMPrimaryImport
        BAM Portal URL: https://purchasesrv/BAM
        Aggregations:
            Name: SupplyCube *** Type: OLAP *** Cube Name: SupplyCube
                PivotView: SupplyCost
            Name: SupplyTimes *** Type: RTA *** Cube Name: SupplyCube
                PivotView: SupplyTimes

Hit Enter when ready.

Do also tell us what feature you like and dislike.

Thanks,
Keith

This new feature is found in Microsoft.BizTalk.Bam.XLANGs assembly.  From you BizTalk project, you can write the following in your orchestration, e.g. Expression Editor (click on the appropriate shape in orchestration to launch this):

// Sample code for using OrchestrationEventStream from an orchestration, no need to new it:
Microsoft.BizTalk.Bam.EventObservation.OrchestrationEventStream.BeginActivity( "PurchaseOrder", "PO123");

Note: this only applies to Biztalk 2006.

* To use this in Biztalk 2004 SP1, you will need a hot fix, its kb article can be found here: https://www.kbalertz.com/Feedback_900083.aspx,

Here are the steps:

Create a console appplication project in Visual Studio,

Cut and paste the code below and run

Sample output:

InfoFound Management Server KW0081 Database BizTalkMgmtDb
InfoFound MsgBox Server KW0081 Database BizTalkMsgBoxDb
InfoFound PIT Server KW0081 Database BAMPrimaryImport
InfoFound DTA Server KW0081 Database BizTalkDTADb

Sample Code:

Let's pick two continuation scenarios: the first, a TPE segment continues to a non-TPE segment, and the reverse being the second.

1.      TPE-enabled segment continues to a non-TPE-enabled segment.

In this scenario, you would need only a Continuation folder. The naming of this folder is important since the concatenation of the folder name and its mapped value will form the continuation TokenID used in non-TPE-enabled segment.  For example, if the Continuation folder name is "TPE2OES" and the mapped value has "PO123" for instance, the continuation TokenID would be "TPE2OESPO123".  Once you have mapped other data items and milestones, you will deploy* this.

In the related orchestration, in the process somewhere downstream, you will need to insert the following:

// Continuation segment using OrchestrationEventstream**

// Where ContinuationTokenID based on this instance contains the value ""TPE2OESPO123".***

Microsoft.BizTalk.Bam.EventObservation.OrchestrationEventStream.UpdateActivity("PurchaseOrder",  ContinuationTokenID,

        "Column1", Value1,

        "Column2", Value2);

Microsoft.BizTalk.Bam.EventObservation.OrchestrationEventStream.EndActivity("PurchaseOrder",  ContinuationTokenID);

2.      Non-TPE-enabled segment continues to a TPE-enabled segment.

In the reverse scenario where an activity begins from an non-TPE segment,  the following code would be required:

// Sample main segment of continuation using OrchestrationEventStream.***

// Activity in this instance contains: "PO124"

Microsoft.BizTalk.Bam.EventObservation.OrchestrationEventStream.BeginActivity("PurchaseOrder",  ActivityID);

Microsoft.BizTalk.Bam.EventObservation.OrchestrationEventStream.UpdateActivity("PurchaseOrder",  ActivityID,

        "Column1", Value1,

        "Column2", Value2);

// CONTINUATION LINE HERE:

Microsoft.BizTalk.Bam.EventObservation.OrchestrationEventStream.EnableContinuation("PurchaseOrder",  ActivityID, "OES2TPE" + ActivityID);

Microsoft.BizTalk.Bam.EventObservation.OrchestrationEventStream.EndActivity("PurchaseOrder",  ActivityID);

To create the continue-to segment, in your tracking profile, you would need to add a ContinuationID folder, name this folder as "OES2TPE" and map the relevant item (one that will give you the above "PO124" for this particular instance of message.  Deploy* this and drop a message and see the beauty flows :)

*   You will get a warning that the tracking profile is missing the ContinuationID or Continuation folder, but since we know what we are doing :), just ignore and proceed.

** Note: there is no BeginActivity call since this is a continued-to segment. 

***The above can easilly be changed to used DirectEventStream or BufferedEventStream. If you do so, make sure you call the Flush method explicity or set the flush threshold to 1 or greater in the constructor.

DirectEventStream (DES) vs BufferedEventStream (BES)

Use DES when you don't want any latency of data appearing in BAMPrimaryImport database.   Since this is synchronous call, any error will be thrown back to the calling application.

Use BES if your concern is throughput.  This is particularly suited for scenarios with spurts of large chuck of events.  A small penalty for using this is that there will be small latency between when the event is sent and the event appearing in BAMPrimaryImport database.

For more info on EventStream: https://msdn.microsoft.com/library/default.asp?url=/library/en-us/sdk/htm/frlrfmicrosoftbiztalkbameventobservationeventstreamclasstopic.asp

Adapt the SDK CustomComponent (found in .../SDK/Samples/Pipelines/CustomComponent) sample, you will find two directories:

• FixMsg - the custom pipeline component (you will need to splice BAM code here) 
• PipelineComponentSample - custom pipeline and will use the above custom pipeline component.

Modify the FixMsg to include the below sample (Search for Execute method):

public IBaseMessage Execute(IPipelineContext pc, IBaseMessage inmsg)
{

IBaseMessagePart bodyPart = inmsg.BodyPart;
if (bodyPart!=null)
{

...
...
//Splice BAM Code here:
WriteBAMEvent(pc, "Execute", inmsg);

}
return inmsg;

}

// Add a new method to write to BAM

private void WriteBAMEvent(IPipelineContext context, string StageDescription, IBaseMessage inmsg)
{

EventStream _BAMEventStr = null;
// Create an EventStream instance
try
{

// Use can use BES/DES or MES
// DES - DirectEventStream - writes direct to BAMPrimaryImport (Synchronous call, no latency)
//_BAMEventStr = new DirectEventStream("Integrated Security=SSPI;Data Source=.;Initial Catalog=BAMPrimaryImport",1);

// BES - BufferedEventStream - writes to MessageBox (Asynchronous call, no wait in call)
// _BAMEventStr = new BufferedEventStream("Integrated Security=SSPI;Data Source=.;Initial Catalog=BizTalkMsgBoxDb",1);

// MES - MessagingEventStream - Async and participates in Pipeline transaction context).
// - Note: Available in BizTalk 2004 with SP1 patch and Biztalk 2006
_BAMEventStr = context.GetEventStream();

}

catch(Exception e)
{

throw (e);

}

// Sample BAM Event Code here:
string ActivityID;
//Create a new, unique activity identifier to use as the ActivityID in BAM
ActivityID="BAM_" + Guid.NewGuid().ToString() + "_" + DateTime.Now + "_" + StageDescription;

//Start the activity record identified by ActivityID
_BAMEventStr.BeginActivity("PurchaseOrder",ActivityID);

_BAMEventStr.UpdateActivity("PurchaseOrder",ActivityID, "ProductName","QFE1117");

_BAMEventStr.EndActivity("PurchaseOrder",ActivityID);

// Optional Flush
//_BAMEventStr.Flush();

}

Reference on MessagingEventStream:

https://msdn.microsoft.com/library/default.asp?url=/library/en-us/sdkmref[bts06]/html/M_Microsoft_BizTalk_Component_Interop_IPipelineContext_GetEventStream.asp

Reference on EventStream:

https://msdn.microsoft.com/library/default.asp?url=/library/en-us/sdkmref[bts06]/html/T_Microsoft_BizTalk_Bam_EventObservation_EventStream.asp

To understand the above and details on how to fetch message specific data, please refer to sectio on Developing Pipeline Components in the help or use this link:
https://msdn.microsoft.com/library/default.asp?url=/library/en-us/sdk/htm/ebiz_prog_pipe_qdri.asp
You would at least need to implement these interfaces.
IBaseComponent, IComponent, IComponentUI.

The above is tedious and is the only means for writing to BAM in Biztalk 2004. In Biztalk 2006, you can do some of this easily via TPE drag and drop.