BizTalk Server 2006 - Enterprise Production Considerations - Part 4 - BAM

  • Article

 

Obligatory Introduction

BAM is such a good name for this technology for a few reasons. The first is that it reminds me of the kid Bambam on the Flinstones (You know, the baby with the club that had a thing for Pebbles). Very powerful and very cool.  Also because "BAM" is the sound my forehead makes while it collides with the palm of my hand as I discover I could have simply used BAM instead of writing a custom reporting app.

Sample to Whet the Appetite

Here is a fictitious BAM portal Aggregation for one of my lab projects

The Basics

This article will be a fairly unstructured summary of my experiences with BAM under BTS 2006.  If there is interest, I will create some more in-depth posts on the topic. I'll be brief here because most of you know this already. The basic way BAM works is the following:

  1. Create a Business Activity showing the LOGICAL progression and points of interest in a process (using Excel, or Visio ODBA)
  2. Deploy that Activity (as .xls or .xml to the biztalk server) using BM.exe
  3. Associate low level technical events (i.e. hitting a shape in an orch or a method in a .net class) with items you defined in the activity (using the BAM API  or the Tracking Profile Editor (TPE)

The end result of this is that you can take the mess of code, orchestrations etc and transform it back into a logical/business view that a CEO could understand.  All in all, very cool.

Some Pragmatic Advice

"DO" List

  1. Try both ODBA and Excel for creating activities (I like excel better, but hey, who cares?)
  2. Become familiar with the BM.exe utility. You really only need the "deploy-all" and "remove-all" commands to start with
  3. Keep your activity files versioned! If you deploy an activity, then change the activity file, then try to undeploy you may have problems. Then you have to undeploy by manually deleting tables and rows in the BAMPrimaryImport table.
  4. I recommend that you manipulate your activity definition in Excel. When you are ready to deploy it, export it to xml and then deploy the xml. That way if you fiddle with the Excel file, you can still undeploy the activity using the same xml you used to deploy it.
  5. Before you re-export your activity to XML, undeploy the activity. Or even better export your Activity to a new version of the xml (i.e. with a version number in the file name
  6. Start with the tracking profile editor, master it, then move onto the BAM API calls.

"DON'T" List

  1. Don't try to map milestones from Message Constructs or Loops (or other special cases). That is advanced and you may need to use the BAM API call to make that work as you would expect
  2. Don't put spaces or special characters in your Activity Names. It can lead to problems later where the Event Log will say that "stored procedure does not exist" and your BAM Aggregation will not work.
  3. Don't have a group shape as the last shape in the orchestration you are tracking in TPE. For some reason it won't work. Just add any shape (i.e. Expression) after the group to fix the issue.

Continuations & Relationships

I can't do these justice in this entry so this will be a bit shallow. I will probably blog on this separately. The more comments i see below requesting this, the sooner I will get around to it.

Continuations allow a single activity defintion to span muliple orchestrations or even other classes/systems seamlessly. It can also allow you to use different unique identifiers accross those systems to  When using TPE to create a continuation. Create a continuation and name it, then create a continuationID and name it the exact same thing.  Map some kind of ID value (PO ID, RequestID, etc.) from a messaging shape (receive, construct, send as described in the above DO List) in the first orchestration to the continuation. Then in the second orchestration map that same ID value to the continuationId. The idea is that the engine will recognize that those to values match and will correlate them so that both instances can contribute data to the same Activity instance

Use Continuations when the Relationship is 1:1
Use Relationships when the Relationship is 1:Many

Real-Time vs. Scheduled Aggregations

There is a little button in the Excel view (if you have installed the BAM addon) that will mark the aggregation as RealTime or Scheduled.  Again, this is complex so I will summarize below.

RealTime

Scheduled

Maintenance

Low maintencance - The data is kept in the relational tables and periodically cleaned. All you have to do is set the TimeSlice and TimeWindow values to determine the trimming behavior

High Maintenance - This data is kept in cubes so you need to have Analisys server setup, schedule the packages, and make sure they execute as expected

Amount of Data

Small Amount of Data - Because this is kept in the relational tables the volume of data has to be kept rather trim

Large Amount of Data - Because this is kept in cubes you can have almost unlimited amount of data

Accuracy of Data

Very Accurate - This provides up to the second view of data and can show in progress activities

The data you have is correct, but you will be missing any data collected since the last execution of the packages

Comprehensive Measures

Limited - You can not use MIN,MAX, and apparently AVG functions in RTA

Unlimited - You can use any type of measure in scheduled packages

Performance

This does contend with SQL and BizTalk for resources on the production box.

This is presumably a separate machine serving as a DataWarehouse so it should not contend with the production server for resources. (except to execute the packages)

BAM Alerts

For the most part, real time data is better for driving alerts, as they are usually time sensitive.

Common Problems & Solutions

Problem: The BAM Activity and the Tracking Profile have been deployed, the process has been run, but there is no data displayed for the activity in the BAM portal or LiveData Excel sheet.

Solution: When you deploy a non-Real Time Aggregation (this is default) while using SQL 2005. It will create the <package>_DM and <package>_AN packages. These packages need to be run to populate the BAM tables and get you the BAM Data.  To run these packages you should select "Integration Services" in the SQL Management Console (not database engine). Once you have found them, run them. Hopefully, you're data will appear.

Problem: The BAM portal or LiveData Excel sheet shows "BLANK" for one column and one row in the Aggregation display. It seems like the measures, and the milestones are not being combined into the same activity. Also, the data shows up in two rows (or more) in the BAMPrimaryImport.dbo.bam_<ActivityName_Completed table.

Solution: This could be a number of things, but if it was the same problem that I had the solution is the following. When you are in the TPE and you are associating data from the message, be sure to use the orchestration schedule.  Right click on your receive, Message Construct, or Send shape and select Message Payload and map the items from there. Many people go to the "Select Event Source" and use the "Messaging Payload" from there. YOU ONLY WANT TO DO THAT IF YOU ARE JUST USING SEND RECEIVE PORTS TO MOVE MESSAGES. If you are using an orchestration, map everything from the orchestration schedule. 

Problem: Your BAM aggregation is not working and you see "Could not find stored procedure" in your event log.

Solution: This is probably caused becaues you have a "space" or some other bad character in your Activity Name. Undeploy your activity, rename it so it does not have the space. Redeploy it. Adjust your TPE (if required). This should fix the problem.

Wish List

  1. Not being able to have spaces in the Activity Name AND not being warned about it is a serious limitation
  2. The length of Activity Names and Item names is too short. I wish for longer names.
  3. Measures using Average also seem to not show for RealTime Aggregations (at least they did not in mine).
  4. I would like to see this integrated with the Visual Studio Environment to a much greater degree. My vision is that we use Office and VS and nothing else.
  5. I also want to be able to deploy BAM stuff with MSI easier