The objective of this document is to provide a step-by-step guidance for creating a new virtual machine using the BizTalk Server 2013 beta image available in Windows Azure platform images gallery and then using the same VM to configure single server BizTalk Server environment. Finally, the page provides step by step guide to create and test a simple message routing scenario using the same BizTalk Server environment.

2.       Pre-requisites

You need to have Windows Azure subscription for Windows Azure Virtual Machine (preview) features. You may sign-up for Virtual Machines & Virtual Networks preview feature through the signup options available on Windows Azure Website.

3.       Create BizTalk Server 2013 Virtual Machine

After signing up for Windows Azure subscription, login to the Windows Azure Management Portal.

  I.            In the command bar (at the bottom-left of the portal), select New -> Compute -> Virtual Machine -> From Gallery.

      II.            This brings up the Create Virtual Machine wizard. In the first step of the wizard, select Microsoft BizTalkServer 2013 Beta image from all images. Microsoft BizTalk Server 2013 Beta (64-bit) on Windows Server 2012 image contains the Beta version of BizTalk Server 2013.

Click Next (arrow icon on the bottom right of the wizard).

    III.            On the next page, provide virtual machine configuration details:

• Virtual Machine Name: This is the name of the BizTalk Server virtual machine. You should consider using a name for the virtual machine that is descriptive enough to tell what it is used for, but short enough to make it easy to use e.g. BTS2013Demo.

The length of the name must be between 3 and 15 characters. The name can contain only letters, numbers, and hyphens. The name must start with a letter and must end with a letter or a number.

• New User Name: This is the username with administrator privileges that will be used to connect to the virtual machine. You may not change the initial user name – Administrator. However, you could create additional login or service user accounts after the virtual machine is created.
• Password: This is the password for the Administrator user account. It should be at least 8 characters.
• Size: Virtual machine may be created with different sizes. Each size has an associated CPU cores, memory, disk space etc. For demonstration purpose, you may select the Small size which comes with 1 core and 1.75 GB memory.

Click Next.

    IV.            On the next page, provide virtual machine mode details:

In this lab, we will create a standalone virtual machine. However, it is possible to connect multiple virtual machines to balance the load of traffic to your application. For standalone virtual machine, provide following details:

• DNS Name: Specify the name of the cloud service that will be created for the virtual machine. The DNS name can contain only letters, numbers and hyphens. The DNS name must start and end with a letter or a number.
• Storage Account: Name of the Windows Azure storage account in the current subscription that will be used for storing the virtual machine image that gets created. You may specify to create a new storage account or select an existing one, if available.
• Region/ Affinity Group/ Virtual Network: Select the name of the region, affinity group, virtual network where the virtual machine will be created.

Click Next.

      V.            On the next page, provide virtual machine options:

An availability set is a group of virtual machines that are deployed across fault domains and update domains. An availability set makes sure that your application is not affected by single points of failure, like the network switch or the power unit of a rack of servers.

For this scenario, leave the default selection – None for availability set.

Click Submit (Tick icon).

At this point, Windows Azure will start provisioning the virtual machine, as per the configuration details provided through the wizard.

Windows Azure Management Portal will show the status as Starting (Provisioning), as shown below:

Similarly, you would see the storage account status (based on whether you selected to create a new storage account or use an existing storage account

In roughly 4-5 minutes the virtual machine should be created. You would also see the virtual hard disk file created in the storage account (navigate to storage account -> Containers -> select container), as shown below:

    VI.            To start virtual machine select the virtual machine in Windows Azure Management Portal and click Start in the command bar on the bottom.

  VII.            To remote desktop into virtual machine, click Connect in the command bar on the bottom.

VIII.            Provide user name and password to connect to the virtual machine. You need to use same username (Administrator) and password that was provided in the wizard, during creation of virtual machine.

 Upon completion of this, you should now be connected to the newly created BizTalk Server virtual machine.

 4.       Configure BizTalk Server 2013

In this section, we will configure BizTalk Server environment on the single machine.

        I.            Open BizTalk Server Configuration wizard.

Tip: You may locate the BizTalk Server Configuration by searching for BizTalk in the sidebar on remote BizTalk Server.

      II.            Select Basic Configuration.

    III.            Provide Database server name - same as BizTalk Server machine as we are setting up single box environment, username and password. You may choose to use the same administrator account or create a local account with less-privileges to configure BizTalk Server, to avoid the following warning message. If selecting the administrator account, then select Yes to proceed ahead.

    IV.            Click Next. BizTalk Server will start configuring, as shown below.

Upon successful completion, you should see following features configured in your BizTalk Server machine.

Also following databases should be created on the SQL Server:

      V.            Open BizTalk Server Administrator Console.

    VI.            By default, the administrator console will not be connected to any BizTalk Server Group. Since we already have BizTalk Server configured, choose Connect to existing group.

  VII.            This will bring up the following dialog box. Specify the SQL Server name (local server name) and Database name as BizTalkMgmtDb. Click OK.

This would show the BizTalk Server artifacts, as shown in the figure below:

5.       Configure Receive and Send Ports

In this step, we will create and configure send and receive ports for a simple pass-thru message routing scenario.

I. Expand existing application – BizTalk Application 1. Right click Receive Ports and select New -> One-way Receive Port…

This brings up the Receive Port Properties dialog box.

ii.                   Provide receive port name as FileReceivePort.

iii.                  Select Receive Locations and click New.

This brings up Receive Location Properties dialog box.

iv.                 Specify receive location name as File Receive Location.

Select Transport Type as FILE.

Select Receive pipeline as PassThruReceive. This means that BizTalk Server will not do any processing on the incoming message and save the same into BizTalk MessageBox as-is.

v.                   Click Configure. This brings up File Transport Properties dialog box.

vi. Specify Receive Folder Path.

Make sure that receive folder exists on the server.

Also change the file mask to * so that BizTalk can pick up any file irrespective of the file extension type.

Click OK to save the changes and close the dialog boxes.

 vii.                To create a new send port, right click Send Ports. Select New -> Static One-way Send Port.

 viii.              Specify send port name as FileSendPort

Select Transport Type name as FILE.

Select Send pipeline as PassThruTransmit.

ix. Click Configure. This brings up File Transport Properties dialog box.

x. Specify Destination Folder path.

Make sure that the destination folder exists on the server. Also change the file name to %MessageID%.

Click OK.

xi.                 We will now specify the routing filter conditions on the send port.

Select Filters.

Select Property as BTS.ReceivePortName and Value as FileReceivePort.

This creates a BizTalk subscription so that every message that is picked up by the receive port named ‘FileReceivePort’ is sent to this send port.

 xii.                Click OK to save the changes and close the dialog boxes.

 6.       Start BizTalk Application

Although we have created and configured BizTalk receive and send ports, the subscriptions are not activated until the ports are enabled. You could enable the ports one-by-one or start the BizTalk application at once.

I. To start the BizTalk application, right click BizTalk Application 1 and select Start.

 This brings up Start ‘BizTalk Application 1’ Application dialog box.

ii.                   Click Options >> for more details. Click Start.

iii.                  Make sure that receive location, send port and host instance are enabled and started, as shown below:

 7.       Test Scenario

To test the scenario, drop a file in the receive location folder that you specified while creating the receive port/ location. The file can be of any type e.g. xml, txt, exe, etc. because it is a simple pass thru messaging scenario, where no message validation is performed by BizTalk server.

BizTalk Server should pick up the file you dropped and create a new file named as a unique identifier in the destination folder.

Summary: When considering IaaS, lot of focus has to be on architecting and designing for non-functional requirements such as security, integration mechanism with other systems that continue to remain on-premise, performance, reliability etc. Once we get this right, the development for cloud is not much different from the on-premises environment.

• BizTalk comes with Tracking Profile Editor (TPE), which provides developers a graphical way to instrument orchestrations and ports by mapping the data with the items in the BAM activities. Although for many solutions this works great, it has certain limitations. These can be met by using BAM API.
• Consider using OrchestrationEventStream and MessagingEventStream instead of DirectEventStream, when using BAM APIs. These temporarily writes the data to MessageBox database unlike DirectEventStream which writes the data directly into BAM database. The tracking host (TDDS service) moves the data from message box database to BAMPrimaryImport database, periodically. This performs better as calls are asynchronous i.e. caller does not wait for data to be committed in BAM database, and it is transactionally consistent with orchestrations and pipelines i.e. if orchestration rolls back for some reason then BAM entries are also rolled back.
• If you use DirectEventStream, you will need to specify the database connection string, as it directly writes to database i.e you will need to store the connection parameters in your application. The best way to do this is by storing this as configuration data (in BizTalk configuration file or in SSO database). When using OrchestrationEventStream, you don’t have to worry about the server and database names as BizTalk automatically takes care of database connections based on BizTalk configuration.
• It is better to use BAM to track message body data than the built in message body tracking for performance reasons. Built in message body tracking is good for debugging scenarios, temporarily. But if you always need to track message body data, consider using BAM.
• In general, avoid BAM continuations if you can as it incurs performance overhead. If you can not do without it, but try your best to limit to using 2 continuations with a BAM activity.
• The data traced from BizTalk Server is received in UTC format. If you use the event stream APIs to provide date and time tracking data in local time format, the data in the BAM portal will be incorrect as the expectation is that all times in BAM data are in UTC format.
  If you have existing applications that use local time, and you are now upgrading and planning to use the BAM portal, you must modify your data to make it conform to UTC format. You also need to modify your custom application to convert to UTC format. 
• It is recommended to review and monitor the BAMPrimaryImport database size over a period of time. By default, the configuration is to archive the data older than 6 months into BAMArchive database. For more information, see this article: Archiving Primary Import Database Data.
• To identify the bottleneck with BAM ensure that the active Instances count is not climbing, the SQL agent service is running, and BAM_DM_ (Data Maintenance) job is scheduled to run at periodic intervals. Also if OLAP Analysis is configured ensure that the BAM_AN_ job is running at periodic intervals.

There is a great whitepaper available on MSDN: Business Activity Monitoring in Depth for Developers. This is recommended for all developers starting to work on BAM.

Please do leave comments, if any questions or issues or suggestions. Thanks!

The KB Article https://support.microsoft.com/default.aspx?scid=kb;EN-US;925731 provides the download links for Message Packs to support the multiple versions of the BizTalk Accelerator for SWIFT. 

BizTalk Accelerator for SWIFT 2009 Message Pack includes the following features:

• The repackaging of all SWIFT FIN FF message types and of all business rules
• Updates to schemas and to business rules to make the schemas and the business rules fully compliant with the SWIFT 2009 certification requirements
• All the hotfixes that are related to schemas and to business rules but that are not superseded by the SWIFT 2009 certification requirements
• BizTalk Accelerator for SWIFT Message Pack 2009 has no components other than schemas, business rules, and Microsoft Office InfoPath forms. 

Note: Microsoft intends to release a new message pack for BizTalk Accelerator for SWIFT each year to coincide with the annual updates to the SWIFT Standards Release Guide.

Please do leave comments, if any questions or issues or suggestions. Thanks!

BizTalk Server provides following out-of-box monitoring tools that are normally used by developers and administrators.

• Health and Activity Tracking (HAT): Health and Activity Tracking (HAT) is a web-based user interface that you can access over HTTP to see tracking data through different views and queries, depending on your role. You can monitor the progress of your business process, you can examine state as your business process runs, and you can create custom views to get just the information that is important to you.
• Business Activity Monitoring (BAM) Portal: The BAM portal in BizTalk Server provides real-time, end-to-end visibility into a business process. It is a Web-based feature that consists of a collection of ASP.NET 2.0 pages. You can customize BAM to enhance the performance and experience for your users.
• BizTalk Server Administration Console: The BizTalk Server Administration Console is a Microsoft Management Console (MMC) that you can use to manage and monitor BizTalk Server, and that you can use to deploy and manage your BizTalk Server applications.

Although these tools address most of the needs for monitoring and managing BizTalk solutions, below are some requirements that are not met in effective manner, especially for complex solutions:

• Ability to search for BizTalk orchestration and messaging instances based on process identifiers. In some situations, it is required to locate the corresponding BizTalk instances (one or more) for a particular process based on process identifiers e.g. order number, order dates etc. It can get very difficult to locate the instances based on such properties when a large number of messages flow through the system.
• Ability to resume business processes from latest possible checkpoint in the process flow, in case the orchestration is suspended in non-resumable state for any reason. This eliminates the need to restart the process from the initial state, which may have required rework if the process includes manual steps.

To fulfill the above requirements, BAM and HAT integration is one of optimal solutions. Although some integration between BAM and HAT already exists, it is very primitive and is good only for very basic needs. The integration feature is limited to availability of ‘Assistance’ button on activity status page in BAM portal. The assistance request is placed in the Application Event log. You can then use Health and Activity Tracking (HAT) to retrieve the assistance request.

To achieve better integration between BAM and HAT, another possible solution is to develop custom BAM portal, which integrates very well with health and activity tracking. You may utilize class members in Microsoft.BizTalk.Operations namespace and BAM API to query the data from BizTalk databases and present the same on a unified interface, as per your needs. Here are related links for the same:

BAM and HAT Integration

https://msdn.microsoft.com/en-us/library/aa578154.aspx

Microsoft.BizTalk.Operations Namespace

https://msdn.microsoft.com/en-us/library/microsoft.biztalk.operations.aspx

BAM Portal

https://msdn.microsoft.com/en-us/library/aa560294.aspx

Business Activity Monitoring in Depth for Developers

https://msdn.microsoft.com/en-us/library/dd320628.aspx

Using Health and Activity Tracking

https://msdn.microsoft.com/en-us/library/ms943797.aspx

Please do leave comments, if any questions or issues or suggestions. Thanks!

• BizTalk Server 2009 performance optimization guide: The BizTalk Server 2009 Performance Optimization Guide contains prescriptive guidance for optimizing BizTalk Server performance, based upon hands-on experience of IT professionals who have worked extensively with BizTalk Server. This guide contains four main sections: Finding and Eliminating Bottlenecks, Optimizing Performance, Scaling a Production BizTalk Server Environment, and BizTalk Server Performance Testing Methodology. The BizTalk Server 2009 performance optimization guide is available here.
• Video: BizTalk 2009 – End to end performance testing: These 2 video recordings are from the BizTalk 2009 launch and Massive scalability night at Microsoft in Stockholm on April 16, 2009. This session introduces the BizTalk Performance Testing Guidance. This session will provide a full interactive run through of the performance lab process. It will illustrate the tools and techniques and documentation that we use and are documented in the guide. Also learn about performance lab characteristics on BizTalk Server 2006 R2, BizTalk 2009 (physical) and BizTalk 2009 on Hyper-V platform. Watch it here: Part 1 & Part 2.
• Presentations from the BizTalk Customer Advisory Team: There are 4 separate presentations available:

1. BizTalk Server 2009: Performance Characteristics on Hyper-V and Physical Platforms; Ewan Fairweather and Paolo Salvatori
   The BizTalk Customer Advisory Team quantifies performance of BizTalk Server 2009 vs. BizTalk Server 2006 R2, as well as performance of BizTalk Server 2009 on Hyper-V. The team also provides guidance on how to effectively do performance testing of BizTalk Server 2009.
2. BizTalk Server 2009 Performance; Ewan Fairweather, Tim Wieman, and Paolo Salvatori
   The BizTalk Customer Advisory Team provides examples of successful performance labs. The team also provides information about the tools, techniques, and processes that were used to run effective BizTalk Server performance labs.
3. BizTalk Server End-to-End Performance Testing Guidance; Ewan Fairweather and Paolo Salvatori
   The BizTalk Customer Advisory Team discusses how they run BizTalk performance assessments, demonstrate tools and techniques that used during the assessment, and introduce the BizTalk Server Performance Optimization Guide.
4. BizTalk Hyper-V Virtualization Guidance; Ewan Fairweather and Clint Huffman
   The BizTalk Customer Advisory Team shares best practices for using BizTalk Server with Hyper-V. They also provide guidance on the performance characteristics of BizTalk Server on Hyper-V, and introduce the BizTalk Server Hyper-V Guide.

Download them all from here.

• Technical Webcast: Ewan Fairweather, who is a member of BizTalk customer advisory team and is primarily focused on BizTalk Server performance, is presenting a webcast on BizTalk performance this Tuesday September 01, 2009 1:00 PM Pacific Time along with Trace Young, who wrote the majority of the BizTalk Documentation.  They will cover a summary of all the performance tests they have performed in the lab this year and answer any questions.  Please tune in if you have any performance related questions or if you just want to see what we have been up to.  Registration link is here. I will update the link once it is available for on demand.

Please do leave comments, if any questions or issues or suggestions. Thanks!

Version info & changes for the pack:
8/31/2009 - Version 6.0.6679.0 - Updated release of MP, for supporting BizTalk Server 2009 and addressing high CPU utilization issue.

Overview:
The BizTalk Server Management Pack for Operations Manager provides comprehensive discovery and monitoring of BizTalk Server components and applications.

Feature Bullet Summary:
This management pack focuses on the following key scenarios which are discussed further in the management pack guide provided within the download:

• Suspended Message Alerts
• Alert Suppression Policy
• BAM Technical Assistance Alerts
• BizTalk Message Boxes and Hosts
• Orchestration Failures
• Service Monitoring
• Application Monitoring
• Monitoring of various aspects of RFID

The Management Pack has already been downloaded and tested by 11 customers through our Technology Adoption Program (TAP).

Please do leave comments, if any questions or issues or suggestions. Thanks!

·     Follow standard naming conventions for BizTalk artifacts: This is one area which may seem a minor point in the beginning but as the solution grows larger it plays an important role in maintaining the solution. It is a good idea to maintain a guidelines document that everybody on the development team must walkthrough so that there is a consistency across the project in terms of maintaining a standard.

·     Encourage reusability: Before you start writing code, it is advisable to check if the code already exists in BizTalk SDK or somewhere on the internet as an open source e.g. on CodePlex etc. Keep in mind that for each additional line of code, you are also adding an additional maintenance effort. Also in large solution, it is always to check with the other developers whether the utility functions already exist to meet a desired functionality.

·     Before adding any new artifact in the BizTalk solution, check if this is really required? I have seen solutions having multiple message schemas when the functionality could be met with very few of them. E.g. consider using multi part messages for situations where a message needs to exist which is only a combination of two other messages.

Or avoid creating maps where you could construct a simple XML or XLangMessage message in the user code. Just because you need to create another message, you don’t need to use a map. There exist different ways to do this, see Constructing Messages in User Code for more information.

·     Do lot of tracing in the code: Do not hesitate to instrument your code enough. This really helps debugging the problems without getting into the code in many situations. Use features in System.Diagnostics class library to enable tracing your code. With this it becomes easier to turn on/off at run time using configuration. It will also be possible to switch a particular level of tracing on e.g. verbose, error, info etc. Ideally each function should have some tracing enabled with the function duration, passed parameters and how far you did get in your function.

·       Leverage pub-sub architecture of BizTalk: It is very important to understand the way BizTalk server works. Build your systems loosely coupled as much as possible. BizTalk orchestration designer built into VS.Net provides an easy way to drag-drop shapes and define your business processes. But before doing that take some time to really think about how the business process may look like in future? Are there any possibilities of adding any new systems in the future to participate in the current process? Is it easy to plug-in new components to existing process without changing the design too much? Really think hard about the extensibility, and maintainability of the solution. Think about the people who are going to maintain the solution after you build it.

Also, here is a nice code review checklist compiled by Richard Seroter.

Please do leave comments, if any questions or issues or suggestions. Thanks!

In this blog, we are going to talk about new utility "SSO Configuration Application MMC Snap-In". This snap-in is available for download here.

Here is the overview of the snapin (taken from the documentation):

 The downloaded zip file extracts to 3 another zip files:

- SSOMMCSnapInSetup.zip: Extracts to setup and msi file for installing mmc snapin.

- SSOConfigurationApplicationClientHelper.zip: Extracts to SSOClientHelper.cs that can be used in your application to read the values stored in SSO. 

- SSOConfigurationApplicationMSBuildImportTask.zip: Contains MSBuild custom task that can be used to automate the deployment of your applications’ SSO configuration data.  All you need to do is export the application and then use the MSBuild task to automatically import it into your environment

Attached screenshot (SSO Application Configuration Snap In Screenshot.png) shows how the snap in looks like after installation. I have created a dummy application called POApplication and a key-value pair in the application called SQLConnectionString.

Please do leave comments, if any questions or issues or suggestions. Thanks!

SSO Application Configuration Snap In Screenshot.png

Snapsot replication is easiest to understand as it is similar to database backup and restore in a way. Everytime a snapshot replication happens, it copies all the specified articles/ data from publisher to subscribers. Instead of writing only the changed data, snapshot replication rewrites the whole data. This means that you should use this type of replication only when the amount of data is small and rather static.

Transactional replication suites more volatile scenarios and is one of the mostly used replication type. It initially copies the whole set of data and from then onwards copies only the modified data. It uses transaction logs to copy the modified data and keeps the publisher and subscribers in sync.Peer-to-peer transactional replication is a new kind of transactional replication that lets multiple servers subscribe to the same schema and data, permitting simultaneous changes in multiple servers. Limitations of transactinal replication is that it does not allow to change the data in subscribers.

When the solution requires both publisher and subscriber to allow changes and keep them in sync, Merge replication is required. Merge replication does not rely on transaction logs to sync the data, rather in this each server modifies the replicated data.

To replicate the data, SQL Server uses programs called replication agents which are sql server agent jobs. When replication is configured these jobs are automatically created for you. The replications agents include snapshot agent, Log reader agent, distribution agent, merge agent and queue reader agent.  All replication types use snapshot agent initially to create the starting copy of the database.

Replication involves 2 types of subscriptions: Push and Pull subscription. In push subscription, distributer is responsible for distributing the data to subscribers. Pull subscriptions run on subscriber servers that retrieve the publications from the distributors.

LINQ is divided into following 3 areas:

• LINQ to SQL:

LINQ to SQL is a component of .NET Framework version 3.5 that provides a run-time infrastructure for managing relational data as objects. 

• LINQ to XML:

LINQ to XML provides an in-memory XML programming interface that leverages the .NET Language-Integrated Query (LINQ) Framework. LINQ to XML uses the latest .NET Framework language capabilities and is comparable to an updated, redesigned Document Object Model (DOM) XML programming interface.

• LINQ to Objects:

The term "LINQ to Objects" refers to the use of LINQ queries with any IEnumerable or IEnumerable<(Of <(T>)>) collection directly, without the use of an intermediate LINQ provider or API such as LINQ to SQL or LINQ to XML. You can use LINQ to query any enumerable collections such as List<(Of <(T>)>), Array, or Dictionary<(Of <(TKey, TValue>)>). The collection may be user-defined or may be returned by a .NET Framework API.

Here is the list of reqources available for more information:

Additional samples and/or documentation may be available from the following sources:

Cheers!

Teekam.