Integrating BizTalk applications with Mainframe programs can be done using many methods. One of the most convenient methods is to use the TCP CICS integration model using the standard listener TRM or enhanced listener ELM program link. Both these methods requires installation of a concurrent server (listener) on the Mainframe to handle TCP and sockets communication. Host Integration Server comes packed with an example concurrent server. One problem that you might face when using this model is the case of a processing of a transaction on the Mainframe and while this processing is happening on the Mainframe the socket communication fails. In this case you will face a condition where the transaction is already done on the Mainframe but the application on BizTalk is not aware that the transaction is performed so if there is no mechanism against this then the same transaction can be executed twice.

Discussion & Resolution

A resolution of this problem is to edit the concurrent server program on the Mainframe to issue a CICS ABEND in the case of a socket failure. This insures that if the response is not delivered to the application then the transaction would be rolled back on the Mainframe, this though requires that all Mainframe programs to be used should not implement any SYNCPOINT.

To change the HIS provided sample for the concurrent listener to implement this approach the following should be added to the program MSCOMTI.cbl in the “STANDARD-LISTENER” procedure and after processing the “WRITE-BUF-TO-SOCKET” procedure.

Then within the same procedure after the loop ends add the following code to issue the ABEND.

This will make sure that the transaction on the Mainframe will rollback if no response can be delivered to the application due to any connectivity problems.

Full source of the customized listener is here.

HostApps adapter is BizTalk way to communicate with Mainframe programs using CICS integration. To use this adapter as part of the BizTalk ESB toolkit then you will need to allow dynamic ports to be able to send requests to this adapter. The ESB way to do this is by implementing a custom adapter provider.

The steps to implement a new custom adapter provider are simple. The following are the steps required to implement a HostApps adapter provider:

1. 1-     Create a new C# class library DLL and strongly sign the DLL.

2. 2-     Add references to the following DLLs

a.      Microsoft.BizTalk.Messaging.dll

b.      Microsoft.BizTalk.Pipeline.dll

c.      Microsoft.Practices.ESB.Adapter.dll

d.      Microsoft.Practices.ESB.Itinerary.Pipelines.dll

e.      Microsoft.XLANGs.BaseTypes.dll

1. 3-     Add a new class with any name and inherit from BaseAdapterProvider

2. 4-     Override the public properties AdapterName and AdapterContextPropertyNamespace and return any name for your adapter provider. The following is an example for the HostApps adapter provider.

1. 5-     Override the method SetEndpoint and do all your processing to add context properties required by the adapter. In the case of the HostApps adapter you need simply to add an undocumented context property called “AdapterConfig” with all properties required including the assembly mapping. The following is an example for full configuration to be set in this property

.

1. 6-     You can implement the way to take all these properties any way you want but the below is a full example of how to implement this.

1. 7-     Where the Config class is a just a helper class to allow serialization of the properties to XML and it is as follows.

1. 8-     Then create the manifest file as follows to allow development inside the itinerary designer.

1. 9-     Copy the manifest file to the folder “C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\Extensions\Microsoft.Practices.Services.Itinerary.DslPackage” on any development machine.

2. 10-  Build the DLL and then GAC it.

3. 11-  Add the following line to the Esb.Config file

<adapterProvider name="HostApps" type="MyDll.AdapterProviders.HostAppsAdapterProvider, MyDll.AdapterProviders, Version=1.0.0.0, Culture=neutral, PublicKeyToken=32323232" moniker="HostApps" />

1. 12-  Now open the itinerary designer and you will be able to see the new adapter provider in the list of adapters for the static resolver to configure the required properties. Also you can use the same configuration using any dynamic resolver such as BRE resolver.

Happy BizTalking J

Host Integration Server allows application integration with the Mainframe system using the Transaction Integrator (TI) component. The method to implement this integration involves building what is called a TI assembly. TI Assembly is actually built using the definition of the interface between the Mainframe program and the definition of the input and output COM areas. This involves in many cases reading large COBOL files and manually adding them to the TI library. The HIS development tools have added the import utility to simplify importing from COBOL copy book files but as we will see in this post this still have many challenges and I will provide in this post a systematic approach to facilitate importing a COBOL copy book file.

Discussion

COBOL copy books are the way Mainframe programs define the inputs and outputs. You can think of these as the signature of C# methods. These are mainly text files with specific structure. You can find more details about the definition of the structure of the COBOL copy book files here. A COBOL copy book file would look something like the below:

Now usually you would receive two COM area definitions like the above, one for the input COM area and another for the output (if they are different) from the Mainframe team. So the challenge here is how to use the HIS import utility to import these into one method in the TI assembly. Usually when you try to import the import wizard would crash and would not give you enough description of where was the problem so you would end up adding these things manually and that would be ok for smaller programs (like the above) but for more complex programs where you have more than 20-50 fields in input and outputs that would be a very tedious task. The section below would provide a simple approach and walkthrough on how to import these files and use the import wizard successfully every time.

Solution & Walkthrough

The HIS import wizard is very sensitive to the spaces and structure of the COBOL copy book file. So for example a following misalignment would fail the import.

So the first thing you need to do when dealing with COBOL copy books is to align them perfectly. To do this I recommend using a tool such as Notepad ++ as it understands this type of file and it actually shows you very helpful vertical guidelines.

So this simplify spotting misalignments.

The second thing you need to do is to merge the inputs and outputs COM area definitions in one COBOL copy book file while aligning the outputs as if it is part of the input and I will show you later how these would be separated. Just give it a name with the same program name while adding any suffix such as “-OUT” to it. Please also note that every line has to be terminated with a dot.

Now that you have created and saved the COBOL copy book file open Visual Studio and create a new client TI library in a TI project. Now import it as follows:

1-    

2-    

3-    

4-    

5-    

6-    

7-    

8-     Now the method is created as follows:

Host Integration Server (HIS) is used to integrate with Legacy Host technology. One of the Host integration scenarios is to call Mainframe programs or transactions using a component in HIS called Transaction Integrator (TI). The way to use TI is to create what is called a TI assembly and then use this to call the Host systems from the Windows System. The TI assembly can be either used in a folder or deployed to the Global Assembly Cache (GAC). I wanted to deploy the assembly to the GAC to make sure it is available from anywhere in the integration solution and because I had strong named assemblies in the solution that required to reference this assembly. This blog post will provide the challenge and problems faced while trying to do this and how these were resolved to finally deploy the TI assembly to the GAC and use it from there in a BizTalk solution.

Discussion

Host Integration Server (HIS) is used to integrate with Legacy Host technology. One of the Host integration scenarios is to call Mainframe programs or transactions using a component in HIS called Transaction Integrator (TI). TI technology has been around for a while (it was used to be called COMTI) but it went through many changes and updates. In HIS 2013 the TI component undergone major changes. There are two scenarios to use the TI, either Host Initiated Processing (HIP) where the call is initiated from the Host server to the Windows server, in this scenario the Windows server acts as the backend and the Host is acting as the channel or initiating system. The other scenario is Windows Initiated Processing (WIP) where the call is initiated from the Windows platform and it is routed to the Host system. The diagram below illustrates this scenario.

Figure 1: Windows Initiated Processing Scenario

To be able to implement a WIP scenario you need to implement what is called a TI assembly. This assembly actually governs the interface and mapping of parameters between the Windows system and the Host system. The way you create this assembly (as this is a special assembly type) is using the TI project template that ships with the HIS 2013. I am not going to describe how to create the TI assembly as you can find many references on how to do this in the HIS 2013 SDK here and on MSDN here.

Once you have created the TI assembly you will have the following view in VS 2012.

Figure 2: TI Assembly in VS 2012

Now to be able to deploy an assembly in the GAC you need to have that assembly signed with a key and then would have a strong name. If you click on the Library node (root node in the above figure) and check the properties you will see that it has the following properties:

Figure 3: TI Library Properties

The property I am interested in is the KeyFile property which is according to the documentation if you provide a Key file in this property then the generated assembly would be signed with this key and would have a strong name (sweet). Unfortunately for some reason this does not work as expected and I could not get it to work and the assembly generated is always not signed.

I have investigated many other options to sign the assembly after being compiled and this is doable but I wanted something more streamed and easier to be done if I needed to re-generate the assembly.

Once I signed the assembly (as I will show you in the next section) and GACed it. When I create a BizTalk port using the BAHA adapter and try to configure it with the GACed assembly I found that the dialog to select the assembly from the GAC is always empty!  As per the below figure.

Figure 4: Select Assembly from GAC form

The way to resolve also this issue and finally to configure the GACed assembly will be also presented in the following section.

Solution & Walkthrough

So let’s get to the way to get this working. The steps are as follows:

Part 1: GAC the TI Assembly

1-     Create the TI Assembly as you need and configure the calling model of the assembly to be “DirectCall, WCF, WS, BAHA”

2-     Now in the same properties make sure you have set the Debug property to “True”

3-     Now save the TI assembly and you will find the following are the generated artifacts in the output folder:

4-     Now take notice of the generated CS file marked above. Now create a new class library project and name it as you see fit. In that Class library project make sure that the .NET version is set to 4.0 at least (as HIS 2013 is upgraded to .Net 4.0) and then add the generated CS file to that project ( you might even add it as a link to make sure if the CS file is re-generated it is automatically updated in the class library project)

5-     Now make sure you add the following line to the AssemblyInfo.cs
[assembly: System.CLSCompliant(true)]

6-     Now go to the properties of the class library and make this library a signed library as you do with any C# library

7-     Add references to the following assemblies:

C:\Program Files\Microsoft Host Integration Server 2013\system\Microsoft.HostIntegration.TI.ClientContext.dll
C:\Program Files\Microsoft Host Integration Server 2013\system\Microsoft.HostIntegration.TI.TBGen.dll
C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0\System.ServiceModel.dll
C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0\System.Web.Services.dll

8-     Now build the assembly it should build successfully and then add it to the GAC using Gacutil.exe as you would do with any other assembly. The TI assembly is now GACed and ready to be used.

Part 2: Use the GACed TI assembly from BizTalk

1-     Since the TI assembly is a .Net 4.0 assembly it will be GACed in the .Net 4.0 GAC and hence will require the .Net 4.0 Gacutil to be handled. Which comes installed with VS 2012.

2-     Now for some reason the HIS HostApps adapter is not linked correctly to the VS2012 installation folder and hence I had to change things a little to make sure it is able to see the TI assembly in the GAC.

3-     Open the Regedit application and browse to the path “HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\.NETFramework”

4-     Add a new string value with the name “sdkInstallRootv2.0” and set the value of this string to “C:\Gacutil4”

5-     Now open the .net 4.0 tools folder “C:\Program Files (x86)\Microsoft SDKs\Windows\v8.0A\bin\NETFX 4.0 Tools” and copy all the contents of this folder to the folder “C:\Gacutil4\Bin” (just create this folder).

6-     Now open the BizTalk administration console.

7-     Create a new send port in BizTalk

8-     Select the HostApps as the adapter and then click configure and configure the connection string

9-     Now click add and then add assemblies (GAC) option

10-  Now you will see all assemblies installed in the .Net 4.0 GAC and you can select your TI assembly that you installed in Part 1 above

11-  Now complete the configuration as usual and click ok to configure the send port

12-  Now if you check the connection string created and compaire the TI GACed assembly with a normal one you will find this as below:

13-  Now you can use this send port normally.

Conclusion

The HIS 2013 TI assembly can be installed in the GAC to simplify your deployment configuration. The walkthrough provided above can be used on the development machine to be able to configure the solution and build it but on the production environment no changes are required at all.

In the previous post here I showed how to write a new provider hosted App using a step by step process. In this post I will build up on that knowledge and show you the following:

• How to customize your provider hosted app so that you get information from the SharePoint site. (Such as user email).

• How to customize the look and feel of your provider hosted App so that it reflects the SharePoint theme so that the user does not feel a completely disconnected experience. Also how that can be selective so that if the same web site was viewed outside of SharePoint it is displayed as a normal web site.

• How to link back to SharePoint and publish information back to the site including posting to the personal feeds.

• How to host the provider hosted App on Azure web site and how to register this on your private tenant.

So let’s get started, shall we.

Getting Information from SharePoint

To allow the application developers to connect to the source SharePoint site, the request coming from the SharePoint site has a query parameter called “SPHostURl” this contains the source URL of the SharePoint site. So to retrieve information you need to use the client object model to connect to the SharePoint site. Now another problem is how to establish authorization and get the current user identity to be able to perform actions using the currently logged on user identity. To establish this you will already find in any new SharePoint provider hosted App a class called “TokenHelper” this will specifically help you to create the client context using the token. Now once you got the token you do not want to re-connect with the SharePoint site to retrieve the token with every request and hence the need of a token cache. I have included with this article a “TokenDictionary” class that would allow you specifically to cache your tokens.

Now let’s see the code to get the user email address:

Of course for this to work you will need to add a reference to the SharePoint client object model DLL “Microsoft.SharePoint.Client” and “Microsoft.SharePoint.Client.Runtime”, one good thing is that this is already done for you as part of a new SharePoint App project. J

Customizing the look and feel

Once you click on a provider hosted App you are routed to that app and hence you lose the look and feel of the original SharePoint site and the user really feels disconnected. So how to resolve this issue. You simply need to customize your app to look the same like the SharePoint site instead of looking something like this.

The code required to do this is placed in your site theme. If you are using HTML5 then it would be the “_SiteLayout.cshtml” file. You need to add the following line to the header section:

This line will allow the site to get the SP script file from your original farm.

Now on the page DIV you need to add the SP chrome as below:

This will allow the page to look as below:

Also we do not need to show this on site if you open the site from an external user. So simply we add the following code to our default.chtml file:

Also on the public default file we add the following to change the layout file to use the public file that does not contain the SharePoint chrome.

Publishing to the SharePoint User Feeds

Now we need to write information back to the SharEPoint site and usually this is simply done using the Client Object Model in the same way we have read. As an example here I will show you how to write to the user feed.

The first thing you need to do is to add a reference to the DLL “Microsoft.SharePoint.Client.UserProfiles”. Now you need to do the following:

Also please note that you need to add proper access for your app to be able to do this as follows.

Hosting the App on Azure Websites

Now for the last piece of the puzzle how to host your site on Azure Websites. So here are the steps.

1-     First you need to create a new Azure WebSite. So logon to https://manage.windowsazure.com and go to the websites and click create.

2-     Once the web site is created now download the publishing profile.

3-     Now go to your app and import this publishing profile and publish your app to azure.

4-     Click save to save the created profiles but do not publish yet as it will fail. Now open the file “backeryapp - Web Deploy.pubxml” using VS and edit it to insert the following lines

5-     Now publish your site to azure and it should succeed

6-     Now you should be able to open your site and see the normal public page (not linked to SharePoint).

7-     Now open your “AppManifest.xml” file and make sure you change the URL or your app to the Azure URL (Note that the URL must be HTTPS based)

8-     Now do not deploy it to SharePoint yet as it will fail, you need to register your app to SharePoint (same as what VS is doing automatically for you) so go to your tenant and open the page https://<tenant>.sharepoint.com/_layouts/15/appregnew.aspx and click on the two generate buttons and fill the information as required:

9-     Now copy both the Client Id and the Client Secret and put them in your site web.config file replacing the values there. Republish your site to azure.

10-  Now click on Create on the application registration page

11-  Now publish the SharePoint App.

12-  Now go to your SharePoint tenant and upload the App package.

13-  Now test your app from the link below

You should now be routed to the Azure web site and your app would be able to work as expected and link back to your SharePoint tenant.

You can find the solution helper files here.

Then you copy all contents of this tab into a newly created XSD schema in a BizTalk project.

Now the problem I faced was that this tab is missing in HIS 2013 :( So how can we do this. At first I thought that maybe this is an installation issue as I had in that environment HIS 2010 and VS 2010. But it was not. It turned out that now you simply save the DLL and VS 2012 automatically generates for you the XSD (along with many other things such as WCF and ASMX services) inside a bin folder. So Vola use that XSD.

Happy HISing ;)

Receiving messages and requests using WCF endpoints and receive locations is a very well-known practice that is used in many BizTalk integration projects. These WCF services can be either a published orchestration or using specific input and output schemas. In any of these there are two known messaging patterns that can be used either in-process or isolated. In-process means that the BizTalk server host instance will be responsible for hosting the WCF host and any requests will be directly received by the BizTalk host. Isolated model means that the WCF service host will be hosted inside IIS and a request received by this service will then be processed inside IIS W3WP worker process and once it requires persistence it will be submitted to the BizTalk Message Box database for normal publish and subscribe processing to occur. In this case this request can be picked up by any of the BizTalk processing host instances and hence will traverse the processes boundaries. In this post I will not go any further in describing these two models but I want to concentrate on a specific problem I faced.  

So imagine the following scenario, you want to implement an in-process WCF receive service but you want the MEX endpoint to be published on IIS so you want the consumer of this service to be able to submit specific requests and expect specific responses rather than submitting and receiving Message objects. This can be done easily if you want to host the processing or receiving endpoint inside IIS, so working in isolated mode. But unfortunately this is not supported for in-process. Usually we do this by using the WCF publishing wizard and publish a metadata exchange endpoint only to IIS and point to the receive location required to process the request which is usually hosted inside IIS (isolated). Now if you select an in-process receive location it will publish the endpoint as usual and you will be able to create the MEX WCF service on IIS but when you try to use it you will receive the following error.  

This is normal and expected as the in-process receive location have to be specified using an absolute URL rather than a relative one as you usually would configure an isolated one. Since the URL is absolute the MEX host will try to add it as a new base address. Now the problem is that this address is not supported by IIS. Now if you try to configure the service using the suggested configuration in this error this will not work in any way.  

Solution  

Usually this is not a supported configuration and hence there is no out of the box method to get this to work. Now I wanted this configuration to work so I started reflecting the BizTalk assemblies to figure out the problem.

Now if you open the MEX endpoint service SVC file you will find the following in it.  

So as you can see here it is using a custom host factory to create the WCF service host. This custom host factory is inside an assembly called “Microsoft.BizTalk.Adapter.Wcf.Runtime” that is hosted in the GAC. So I reflected this assembly to see what is happening there and here is what I found inside the CreateServiceHost. 

So as expected, it adds the URL to the list of base addresses if the address of the receive location is in absolute URL format. Now this is correct and needed if the processing endpoint is hosted inside the IIS but since this is hosted inside BizTalk worker process this is not needed and hence we would not need such a line.  

So what I ended up doing is to create new service host factory. I started by thinking of inheriting from the already existing host factory, well you guessed it, this class is sealedL. That leaved me with the only other option which is to create a new class with exactly the same code (reflected) but with these lines removed. So the CreateServiceHost function is now as follows:  

removed. So the CreateServiceHost function is now as follows:

And as you can see I have commented out the lines to add the receive location URL as a base address. Of course I needed to reflect more classes as they were also sealed and private (or internal) so I could not get my new host factory to compile. I could have used reflection to create objects of these classes but I thought it would be simpler to simply reflect them and add them to the project. These classes were:

·        ConfigurationInfo

·        ReceiveLocationProxy

Now the last thing I need to do is to change the SVC file of the MEX endpoint service to use the new host factory instead of the out of the box one as follows:

Now GAC the DLL and reset IIS (or recycle the app pool) and vola now it is working and you can submit messages to the in-process receive location using a MEX endpoint hosted inside IIS.

Happy BizTalking.

In this blog post I will present a very simple solution to integrate with Apache ActiveMQ platform. Actually I got this question from a friend I could not believe how easy it is to integrate with this platform using the new BizTalk 2013 WCF-WebHTTP adapter. The adapter provides many features out of the box that really makes the integration engineer life much easier and better.

In this scenario I have installed Apache ActiveMQ on a private VM, created a test queue and made sure that I am able actually to send messages to that test queue. The target then was to create a way to receive messages from this queue to BizTalk server.

Discussion

The following procedure is what I did to perform this POC.

1-     First I started by installing ActiveMQ and to my surprise that was super easy. So I downloaded the binaries from here and I also downloaded the JDK from here. I installed the JDK then extracted the ActiveMQ to any folder of my choice.

2-     I then had to set the JAVA_HOME environment variable (I do not know why the JDK setup did not do this) but I did that manually to point to the JDK folder as C:\Program Files\Java\jdk1.7.0_45

3-     I then started the ActiveMQ platform by browsing to the folder where I extracted it and then typing the command:

4-     Then I opened the new administration portal using the URL (http://localhost:8161/hawtio) then I logged in using the username admin and password admin.

5-     Clicked on “ActiveMQ”

6-     Then I created a new Queue called “Testing” by clicking “Create” then “Create Queue”

7-     Click on the queue on the left then click “Send” then make sure you change the format to XML type in a valid XML message and finally send the message.

8-     If you now go to the browse of the queue you will find that the message is ready for pickup as per below:

9-     Now the ActiveMQ supports the REST API to be able to pick up messages (or post them) the address is as follows (http://localhost:8161/api/message/Testing?type=queue&clientId=BTSconsumer). So note this address as we will use it later in BizTalk.

10-  Now let’s go to the BizTalk administration console. I created a new send port to be able to pick up messages from the Queue using the WCF-WebHTTP adapter. So I created a new send port and configured it as follows:

11-  Then click configure and change the address and operation as follows:

12-  One key point here (and a new feature that I really like about this new adapter) is that if you are using the HTTP GET operation then you cannot sent anything in the BODY of the HTTP request. But BizTalk out of the box have to send a message so the adapter solved this by adding a key feature called suppress outbound messages body for verbs GET, so you need to configure this as below:

13-  Now I need a way to issue a GET verb using this send port. So usually what you would configure is a scheduled task adapter to schedule checking of messages from the queue. In my case I wanted something simple and without installing anything else on the VM. So I configure a messaging approach to issue a GET verb after I send an empty file through a file receive location and then created another file send port to pick up the final message received form the Queue and write it to a folder.

14-  Started everything and BINGO worked like a charm J I have sent the message to the queue and then I drop the empty file to the input folder to issue a GET verb and finally the sent message is written to the output folder as follows:

Conclusion

The conclusion here is that REST is very powerful and makes you do amazing things. I really think everything should support REST and the other very important conclusion is that I need to thank the BizTalk product group to support this out of the box in BizTalk Server 2013 J

Introduction

In this series of posts I will walk you through the processes of creating an Active/Active SQL server cluster using Hyper-V and Microsoft iSCSI target software for virtualized SAN. The target is to create first a storage server hosted on a normal Windows 2008 R2 server. Then connect to this server using two other machines as iSCSI initiators. Then I will create the windows cluster along with the DTC clustered service. A clustered SQL server instance will then be created. Finally another clustered SQL server instance will be created and Active/Active configuration of both instances will be applied.

Solution Architecture

The solution is fairly simple as per the below configuration.

Revisiting the Cluster configuration and Dependencies

The old dependency was not set correctly since the SQL server service and the associated DTC service are not the same clustered service but are managed differently and I established the link between them using an external service dependency. This rendered the SQL service not movable since the cluster really does not know that it is required to move the DTC service to be able to move the SQL service but instead it treats it as an external service that must exist for the SQL service to start.

The old dependency diagram was like this:

The problem is actually in the dependency on an external clustered service rather than a cluster service resource.

Then I remembered that the Failover Cluster can add the DTC as a generic service resource to the same clustered service, or even better it actually knows of DTC and has a special type of resource as per the below image:

So to configure everything correctly I performed the following:

         1-     Brought all services offline including all two SQL server services and even disks.

          2-     I removed all DTC related configuration by deleting the two clustered services for DTC including all dependencies from the SQL server services.

          3-     Then I started by configuring the first SQL instance (let’s say BCINST) I clicked on “Add Distributed Transaction Coordinator” as per the image above.

          4-     I then added the DTC disk to the SQL server clustered service. So I now have two disks in this clustered service as below:

          5-     Then after it is added I simply configured the services dependencies so that the new DTC is configured as below

So as you see it is dependent on the name and the disk.

          6-     Also I configured the IP to depend on the SQL data disk and the DTC disk also to make sure that the disks are the first thing to be brought online and before anything else.

          7-     Then I configured the SQL service to depend on three things (The name, the DTC service, and the Disk) as follows:

     8-     Then I crossed my fingers and brought all services online. It worked like a charm and I was finally able to move the SQL service from one node to another with no problems. I then configured the other service exactly the same way. So as you can see below now there are no more external DTC services but only internal clustered service resources as per the below:

The final dependency reports for both services is displayed as below.

This makes the two SQL server clustered services operate as expected and makes moving one service from a node to the other successful.

Happy clustering

BizTalk mapping tool is a very powerful tool once you get the hang of it. Sometimes you would need to produce some structure in the output schema that did not exist in the source schema, the Table Looping functoid is built for this specific task.

In some other cases you would need to generate a parent child kind of structure. In this blog post I will present the solution to use out of the box functoids to be able to generate this structure.

Problem

Suppose you have the following as the input schema.

And the following output schema.

The requirement is to generate two Message elements in the ProcessRoot output schema. It is also required to generate some fields in the fields array inside each message. These fields are specific for each message and depend on the message ID field. So to be more specific let’s say that we have the following XML input:

And we want to generate the following XML output from it:

As you can see we have two challenges:

1-     You need to create not just an array of two messages for one node in the input message, but also some sub-fields within each message.

2-     The fields generated are dynamic and dependent on the value of one of the fields of parent array.

Solution

Because I want to generate some structure that does not exist in the original message then we have to use the Table Looping and Table Extractor functoids. Now the challenges listed above is the key here on how to develop this. So I had many ways I tried to implement this but mainly they all involved creating two Table looping functoids one for the parent structure and one for the child array. The challenge I had was how to link them and make the second table looping functoid dependent on the first one. Here is the map I ended up implementing:

There are some key points here to consider.

First the first table looping functoid generates the parent array of messages and hence it is linked to the output node of the messages as per this figure:

Also the table of this functoid has the following structure to be generated:

The second notice here is that we have a link from the first column of the parent table looping functoid and the first column also of the child table looping functoid linked to the output fields array to make the generation of the fields conditional if it is equal to the message ID of its parent. This means that second table looping functoid has to include the message ID as part of the table also it will not be generated in the output.

Now ideally this would mean that the table of the child table looping functoid should be as follows:

But when I tried this it never worked (and until now I do not know why) when I debugged the generated XSLT I noticed that the first two rows generated from this functoid are always wrong. When performed further testing I noticed that the child functoid always has bad rows generated out of it that are exactly equal to the number of rows generated from the first table looping functoid. So what I ended up doing is that I added two extra dummy rows that I made sure will never generate anything in the output schema by making sure that the value that would be compared is always not equal. So what I ended up doing is configuring the second table looping functoid as follows.

This achieved exactly what I wanted and produced the required output XML.

The sample input schema, output schema, and the above described map can be found here.

Happy BizTalking J