Today's my last day here at Microsoft, and as of now, this blog is locked down.  But, my BizTalk party will continue over at http://seroter.wordpress.com.  Thanks to all who have contributed, and see you on the other side.

I've posted a few things over to the new blog lately and thought I'd keep this blog in sync until it shuts down next week.

First, I did a walkthrough of how do use the out-of-the-box BizTalk XML pipeline to debatch messages coming inbound from the SQL Adapter.  Fun stuff.

Second, check out the latest post by Lee on the BizTalk Core Engine blog which covers some of the underlying SQL magic contained in the MessageBox.   He also threatens physical violence against anyone who changes the query plans or stored procs.  Ok, not really.  Still a good read, even sans violence.

Technorati Tags: BizTalk

Next Thursday, 1/11/07 will be my last day at Microsoft.  I've decided to take an "Architect" position with one of my customers in the Los Angeles area.  It was a tough choice to leave my great colleagues and customers, but I'm fairly jazzed about the new opportunities I now have.

So, much like Han Solo in The Empire Strikes Back, this particular blog will be frozen.  Next week it will get locked from any future usage.  My new employer has indicated support for my blogging habit, so, I've set up a new blog at http://seroter.wordpress.com.  I plan on continuing to do the same things on that blog that I do on the current one.  So, look out for more code snippets, walkthroughs, demos, reviews, pointers to cool resources and more.  I just posted an article on the new BizTalk blog about "considerations for resuming suspended orchestrations".  Thrilling stuff.

I've really appreciated all the great feedback and questions resulting from the 160+ posts on this blog, and hope you'll continue participating in the new one.

Technorati Tags: BizTalk

Just recently I had a customer ask me if they could use their existing BizTalk infrastructure to manage events that were spawned from their primary application.   So, I built a small demonstration that I've outlined here.

Basically, this company's primary application triggers events that should result in email notifications.  They wanted something that could listen for these events, send emails, and be flexible and easy to modify.  The key is that last part.  I wanted to build something that would not require a code change or redeployment if new events were added or email formats had to change.

I started with a basic schema.  All that I get from the primary application is the "event source" and "event ID".  So I need an orchestration that will fetch the additional data needed to send and format the email alert.  From the base schema, I created another schema that BizTalk will actually use to send the alert.  It looks like this:

I've now got a "Template" node which will hold the physical location of the XSLT template, and an "OptionalText" node which will hold any additional text we want to include in the email message (HTML markup supported!).

Next I needed a pipeline to do the XSLT conversion.  If you recall, the BizTalk SDK ships with a sample for using XSLT to build an HTML formatted email message.  I needed to modify that sample because instead of getting the "template" property at design time, I need to use a run-time value.  So, I copied the code from the SDK sample, and modified the core Execute method:

public IBaseMessage Execute(IPipelineContext pc, IBaseMessage inmsg)
{
//variable to grab out template location (promoted value)
string templatePath = inmsg.Context.Read("TemplateLocation", "http://CompanyA.PrimaryApp.EmailNotification.ApplicationAlertPropSchema").ToString();
//call helper method, passing in stream and template path
inmsg.BodyPart.Data = TransformMessage(inmsg.BodyPart.Data, templatePath);
inmsg.BodyPart.ContentType = "text/html";
return inmsg;
}

 Ok, now I had to consume this custom pipeline component.  My new "send" pipeline in Visual Studio looks like this:

I threw the XML Assembler in there just to be sure that my "TemplateLocation" value would get promoted into context.  Probably unnecessary, but what the heck.

Next we needed the Business Rules that will store the key information needed to construct the email message.  Based on the source of the event, we need to designate a recipient, subject, extra "body" information, and template to be used.  One of the rules I built looks like this:

Note that the (optional) body text field contains HTML markup.  You may also see that I perform the Halt command at the bottom.  If that rule fires, then I want to halt further rule execution. I wanted to be able to handle unknown events, so each defined alert had a Priority number set in the Business Rules Composer. Finally, I created one final rule with a Priority of 0, which by design, will only fire if all preceding rules have been ignored (that is, it's an event that wasn't recognized by any earlier rules and processing halted).  That "catch all" rule looks like this:

Almost there.  Now, an orchestration is needed to tie all this together.  My process flow consists of receiving a message, instantiating a .NET class (which holds all the email details), calling my Business Rule(s), building the output message, and sending the message out.  The flow looks like this:

The main guts of this process exist in the final Construct shape.  You'll see that the Construct shape exists in an Atomic Scope.  That's because I needed to use a variable of type XmlElement.  Why?  Here's the code for the Message Assignment:

Create outbound message from inbound message
AlertOutput = AlertInput;
Get the XML element holding the optional text
xmlElem = xpath(AlertOutput, "/*[local-name()='ApplicationAlert' and namespace-uri()='http://CompanyA.PrimaryApp.EmailNotification']/*[local-name()='OptionalText' and namespace-uri()='']");
Set the "innerXML" equal to the HTML markup
xmlElem.InnerXml = EmailData.EmailBody;
Set the template to use
AlertOutput.Template = EmailData.EmailTemplate;
Set all the needed email properties
SendFASTAlertPort(Microsoft.XLANGs.BaseTypes.Address) = "mailto:" + EmailData.EmailTo;
AlertOutput(SMTP.From) = EmailData.EmailFrom;
AlertOutput(SMTP.Subject) = EmailData.EmailSubject;
AlertOutput(SMTP.SMTPHost) = "WS03VSSQL05";

 I had to do the whole "XmlElement" thing because just setting the HTML string equal to the OptionalText field caused the HTML markup to get jumbled.  By using InnerXML on the XMLElement, the HTML (really just valid XML) gets thrown under the OptionalText field as XML data, tags intact.  Pretty handy.

Finally, I built and deployed this whole thing.  I then wrote a few XSLT templates to format my HTML email.  After feeding a few different alerts into BizTalk, my Outlook mailbox contained the following messages:

You'll see the last sentence in each email "body" represents the HTML markup set in the Business Rule.  So, now I have an architecture where someone could change just the XSLT template and modify the entire alert message.  If you wanted to add new events to be specifically handled, all that's required is a new business rule, and optionally, a new XSLT template.  Pretty simple!

Technorati Tags: BizTalk

I spent a bit of time today reviewing some best practices for assembly version in BizTalk applications, and wanted to also elicit community feedback.

One of the first things that a new BizTalk developer is told when building up a BizTalk solution is to not cram all the various artifacts into a single .NET project.  Often that developer will be told to put schemas in one project, maps in another, pipelines in another, and finally orchestrations in an isolated project.  I've become less of a fan of this rigid structure.  But first, let's see how I'd do an upgrade of a solution configured this way.

I'm starting with a simple project, containing a single schema file.  The message looks like this:

I built and deployed this project and constructed a simple "receive location" and "send port" where the send port had a subscription/filter on BTS.MessageType.  I tested it, and everything travels through just fine.

Then I built a simple orchestration that makes a decision and writes to the event log.  It looks like this:

I referenced the schema project, built, deployed and tested the orchestration successfully.  The next step was to create a new version of the schema.  So, I added a "Status" node, and then changed the .NET Assembly Version.

I then rebuilt and deployed the schema project.  Now given that you've constantly been told that namespace#root has to be unique, the screenshot below may worry you. 

If you want to read more about how pipelines resolve schemas, read here.  So you may expect that I would get a "ambiguous reference" sort of error when I try and submit the inbound message now.  However, when I drop the file, the orchestration still picks it up, processes it and continues on its merry way.  But, what happens if I "stop" the orchestration so that I can see the message in flight, and investigate the SchemaStrongName property assigned by the pipeline?  What you'd see is this:

Surprise.  The schema resolved to the most recent version, and THIS is the one yanked in by the orchestration.  This surprised me a bit.  I would have expected the send port containing the BTS.MessageType filter to yank the most recent version of the schema since it only cared about "type", but I had thought the orchestration would only grab the version it explicitly referenced.  Now if I try and remove the "1.0" schema assembly via the Admin Console, I get an error stating that the orchestration is referencing it.

So to get rid of the "1.0" schema project, I'd have to reversion the Orchestrations project with an updated build.  Then after gracefully transitioning to the new "1.1" orchestration, I could safely remove legacy assemblies. 

Now, if I used a custom receive pipeline, THEN it seems I can dictate the schema version the orchestration consumes (old schema or new schema).  I tested this by putting a receive pipeline (with XML Disassembler and Validator components) in the project with the schema.  I deployed that (version 1.0 again).  Then, I upped the version to "1.1" and built and deployed.  So, now I have schema/pipeline projects for "1.0" and "1.1" and an orchestration project still built with "1.0".

By switching the pipeline, the orchestration would grab specific versions of the schema.  For instance, if I dropped a "1.0" message (no "Status" field) when the "1.0" pipeline was selected, everything ran fine.  If I dropped a "1.1" message with the "1.0" pipeline still in place, I got a suspended message.  Same situation when I flipped to the "1.1" pipeline.  The orchestration consumed either the "1.0" OR "1.1" message.

So, any conclusions here?  Changing .NET Assembly Version has broad impact, regardless of how you've factored your projects.  One recommendation I've seen is to only change the Assembly File Version so that you still have an incremented build number, but the core .NET version stays the same (allowing for lower impact changes).  However, that could get dicey if you are truly introducing breaking changes.  You'll also see recommendations to version schemas by changing the namespace#root combination (e.g. changing the namespace to http://Microsoft.Project.BizTalkStuff.v1).  Of course then you HAVE to change all the dependent artifacts (maps, pipelines, orchestrations) because you've made a core modification.

This leads me to a solution structure that is less segmented by artifact type, but more by artifact relevance.  That is, group related artifacts in the same project.  You still may have a "base schemas" project, but it's also ok to have a subsequent project that contains a few schemas, maps and orchestrations.  Try and group the items that will change and version together.  Now if I have to version the schema, I can fairly naturally version the sibling artifacts and deploy fewer assemblies.

So I ask you: any other versioning strategies that you prefer when you want to lessen the impact of production modifications?  How do you like organizing your solution structure?  If you only have to make a slight change, how do you prevent the rebuild of every dependent project?

Technorati Tags: BizTalk

After reading Charles Young's fantastic post today on BizTalk transaction compensation, I was finally inspired to write a post that's been on my blog "to do" list for 7 months.  Specifically, I wanted to demonstrate simple compensation handling, and dealing with loops and transactions.

 I'd like to walk through a simple solution I just built and demonstrate the transactional considerations.  The first thing to show here is that data (variables, messages) manipulated within an atomic scope do NOT have their changes committed unless the atomic scope successfully completes.  In the picture below, you'll see a very simple orchestration I built that receives a message, sets a value for a variable ("Status"), jumps into an atomic transaction, does a calculation, updates the "Status", and finally completes by writing the "Status" value back out.  Basically what I'm trying to show is that the variable "Status" may have gotten changed by the action in the atomic scope, however, if the scope fails for any reason, that change isn't committed.

I'm using the (now-Microsoft) tool DebugView to view the Debug.WriteLine statements embedded in my orchestration.  When I pass in a simple message, you see here that indeed the value is initialized ("Starting") and changed by the atomic transaction ("Updated").

If I pass in a bad message (in this case, dividing by 0), then the orchestration is suspended and the compensation code does NOT fire.  Why?  Compensation only fires for committed transactions, and since an operation within the atomic scope failed, there's technically nothing committed, so nothing needs to be compensated.  Now what if I wrap the atomic transaction in an long running (LR) transaction so that I can catch any exceptions and gracefully handle it?

Now, when I run this process, the exception block catches the error, and I continue on in my orchestration.  As you'll see here, because the atomic transaction did not commit, the "Status" field change was not persisted.

What happens if we instead cause an exception AFTER the atomic transaction has completed?

You'll see here that the step following the atomic transaction raises an exception.  What do you expect to happen when I drop a standard (valid) message into this orchestration?

The compensation code did NOT fire since I didn't do anything to trigger it.  All that happened was that the shapes in the Exception block were processed.  Let's force a compensation to happen.  We could, in the Exception block, use a Compensation shape and explicitly compensate the atomic transaction.  But, what if I had 4 atomic transactions in there and didn't want to explicitly roll back each?  What I can do is compensate the Long Running transaction, which in turn will execute the compensation code for each contained transaction in reverse order.  We'll see more of that in a second.  My compensation now looks like this:

If I spin up the orchestration now, you can see here that the Exception block AND compensation code both fire.  Sweet.

What happens if I add a second atomic scope to our LR transaction and raise the exception after the new transaction?

Without changing anything in the LR transaction, it went ahead and executed compensation on both transactions in reverse order.

The last thing I want to show here (and probably the neatest), is something Charles mentioned in passing in his article.  I like to show this when I teach BizTalk workshops and it always amuses folks. What happens if I'm executing transaction in the confines of a loop?  Is BizTalk actually smart enough to keep track of this and handle it appropriately?  I've modified my orchestration to now contain a Loop shape and cycle through 5 times.  On each pass, it executes the atomic transaction.  Following the loop, I raise an exception.  I still haven't changed anything with regards to compensation in the LR transaction.

Any idea what you'll see when I run this?  As you'll notice below, it goes ahead and executes each transaction, and exception is thrown and caught, and each of the looped transactions have their respective compensation code called.  Neato.  I'm always impressed to see that.

There you go.  Read Charles' article for significant more depth on the topic, but hopefully this provides a more simple take on a fairly complex process.

Technorati Tags: BizTalk

Also, if you've never undergone a code review by Irena, you haven't truly lived.  It's a truly terrifying and rewarding experience from a brilliant MCS asset.

Yesterday I did a quick evaluation of the Beta 1 release of BizTalk Server 2006 R2, and thought I'd walk through the installation process and highlight some new things I noticed.

The Beta 1 release looks to be for early adopters only, so there is no "public" release at the moment. Hopefully there will be one in the Spring. Also please note that anything shown here is subject to change by Beta 2 or final release.

First, I walked through the standard installation.

Next I chose the components to install. You can see from my image below that there are additional EDI/AS2 components, Windows Communication Foundation (WCF) bits, and additional BAM Eventing (new interceptors for both Windows Workflow and WCF).

After installing, I jumped right to configuration. As you can see here, not a whole lot of changes.

You will see additional configuration, however, for the new EDI/AS2 functionality.

Next I went through and installed the new BizTalk .NET adapters. These adapters allow you to ping various Line of Business (LOB) systems directly through .NET code without going through BizTalk.

Once everything was installed, I went to see if anything new showed up in the BizTalk Administration Console. First thing I saw was a new "Application" called BizTalk EDI Application. Inside, there were a couple of orchestrations, lots of schemas, ad a few pipelines.

Next I checked out the "Adapters" section and saw all the new WCF adapters available. Nice.

I read in one of the documents that the BAM Management tool had been upgraded, so I looked at the bm.exe command line tool to see what new options were available. You'll see below that there are now options to deal directly with interceptors. This will allow you to instrument your WF and WCF process to feed data directly into the common BAM infrastructure. I like it.

I wanted to see all the options available for the WCF adapters, so I went ahead and build a couple new receive locations. The first receive location uses the WCF - WSHttp adapter.

Then I built a WCF - Custom location. Check out all the various options below:

Finally, I jumped into Visual Studio.NET 2005 and kicked off the WCF Service Publishing Wizard. If you've ever used the standard Web Services Publishing Wizard, then you'd be fairly familiar with this one.

In future posts I'll dig more into how to actually use all these great new functions. Some folks are already digging into the functionality, like my buddy Jesus Rodriguez who has a post on Invoking TCP-Hosted WCF Services using BizTalk Server R2.

Technorati Tags: BizTalk

Good article in this month's MSDN Magazine called Integrate E-Mail Processing Into Your Business Solutions. It covers a variety of topics, including the POP3 adapter, orchestration and some BizTalk plumbing. I also found the part on writing a custom adapter to deal with email message attachment parts fairly interesting. The custom adapter takes each message part and adds it to a custom database. Check it out.

Technorati Tags: BizTalk

I caught a glimpse of something in the BizTalk Server 2006 Help the other day that surprised me a bit. The note said:
If a property schema is associated with a message schema, then these two must be in the same BizTalk project. Separating property schema from its associated message schema in different BizTalk projects is not supported.
I had seen that mentioned elsewhere and had mentally discarded it, but I wanted to see whether it's not possible, or it's not supported.

First step was to build two separate projects; one to hold my property schema, another to hold my working schemas.


The property schema has a single field, OrderID. After building and compiling the project, I referenced that project from my primary schemas project. I then tried to promote the OrderNumber value in the schema, and sure enough, I was allowed to pick a schema from a referenced assembly. So far so good.

After also promoting the value in my second schema, I deployed all the projects. Next step was to build a send port that had a subscription filter in the OrderID property field. Finally, I dropped two different messages into BizTalk (via FILE receive location), and both messages were picked up by the send port.

So, from a technical standpoint, this seems to work fine. Clearly there's some reason for us making this an unsupported configuration, and I'm all ears as for why that's the case. It would seem to me that separating my property schemas into a separate assembly creates a much more flexible architecture than forcing it to live with the schemas that use it. Thoughts?

BTW, I'm jazzed that this is my first post using Microsoft Windows Vista RTM, Office 2007 RTM, and Expression Web Designer (to write the post itself). After installing Windows Vista, I finally had to discard my trusty Allaire Homesite which wouldn't install correctly. I've used Homesite (now owned by Macromedia) for years to write all my HTML, but finally have to upgrade my toolset. Of course, I'll still do the HTML by hand, but now in a snazzier shell.

Technorati Tags: BizTalk