Side-by-Side Execution of Workflow Services: Step-by-Step Example

  • Article

Introduction

In this blog post, we go through a complete example of a workflow service that handles multiple versions.

The MSDN Documentation covers this topic, and a sample is available in the Samples section of MSDN. In this post we are going to cover the topic step-by-step, building a complete sample from scratch. We will also try to emphasize the relevant pieces and to provide a motivation for them. We will focus on IIS/WAS-hosted workflow services, even though we will also briefly touch on the underlying API that is available to self-hosted workflow services. Last, we will cover the more advanced scenario of changes that extend outside the workflow definition itself, to activities used by the workflow.

Note: the step-by-step procedure is relative to Visual Studio 2015, the current version as of this writing. It is possible, though not guaranteed, that the same steps, possibly with minor adjustments, also work with other Visual Studio versions. Since SxS versioning is a new feature of WF4.5, the .Net Framework must be 4.5 or higher.

The steps are highlighted in yellow throughout the post.

Defining Side-by-Side Execution

Workflows (including workflow services) can be long-running. With long-running workflows, you may need to update the structure of a workflow while some instances are running. This poses the problem of what happens to the running instances.

In WF4.0, you do not have much choice: it is up to you to maintain the new and old definition at different endpoints, so the client has to explicitly decide which version to use. This obviously creates tight coupling between clients and the service.

WF4.5 addressed this lack of flexibility by providing two options:

  • Side-by-side execution
  • Dynamic update

Side-by-side execution is defined as follows:

  • all the existing instances will continue to run, until completion, using the original workflow definition (that is, the definition that was current when the workflow instance started execution)
  • new instances start execution using the most current workflow definition.

Dynamic update, on the other hand, implies the dynamic update of the running instances, so to migrate them to the new definition. If the dynamic update completes successfully for all the running instances, it is not necessary to keep multiple versions, because all instances (existing and new) will use the most recent version at this point.

In this post we focus on side-by-side execution only. This is the easiest and most common scenario.

Creating the Service

We start by creating the workflow service from the default template. As we’ll see, several additions are necessary before we can create a working side-by-side example. Nonetheless, starting with the default template is much faster than building everything from scratch.

To create the service:

  • Open Visual Studio 2015
  • Select the File | New | Project menu item
  • In the dialog box, in the left pane, select Visual C# | Workflow
  • Select WCF Workflow Service Application
  • Give a name to the project (for instance, SxSWfSvc)

Note: you may want to open Visual Studio as Administrator. This is not required for the steps above, but it will make it possible to run the service under the debugger. Without administrative privileges, the service won’t be able to open the endpoint to listen for incoming requests.

Note: the default project configuration is such that, when running the project, it will open the URL to the currently selected document in the project (see Project Properties, Web tab, Start Action: it is set to Current Page). If anything other than Service1.xamlx is selected (for instance, web.,config), we won’t get the expected behavior. To fix this, change this setting: in Start Action, select Specific Page and set it to Service1.xamlx.

This service is already operational. You can double-check this by running the project in the debugger (F5). This fires up WcfTestClient; use it to execute the GetData operation and check that it completes successfully:

TemplateTesting

Adding a Second Operation

At the moment, our service has a single operation. Once a client calls the operation, the service completes execution. This service cannot be a long-running service. We need a service with at least two operations, in sequence. The first operation starts the service. Once the operation completes, the workflow service continues to run and is idle. When the client calls the second operation on the same workflow instance, the service completes execution. This way, the lifetime of the service is entirely in the hands of the clients, and workflow services can be as long-running as the clients wish.

In our case, the Message Exchange Pattern (MEP) between the client and the service, once we have added the second operation, will be like this:

MEP

InstanceId is the identifier (a GUID) of the running workflow instance. Note that this is just for the sake of this sample, a real-life workflow can return any business-related data.

To add a second operation:

  • Drag and Drop a ReceiveAndSendReply “activity” to the workflow, at the end of the workflow

This is enough to add a second operation, however we also take the chance to do some cleanup, fix-up and cosmetic changes:

  • In the Variables, rename __handle1 to SecondRequestResponse and chance its scope to “Sequential Service”
  • Also in the Variables, rename handle to FirstRequestResponse. Ignore the validation error for now.
  • Also in Variables, remove data, as we won’t use it anymore.
  • Also in Variables, define a new variable at Sequential Service scope, of type System.Guid, and name it InstanceId
  • Also in Variables, define a new variable at Sequential Service scope, of type string, and name it sInstanceId
  • Change the display name of the Receive activities to ReceiveFirstOp and ReceiveSecondOp, respectively
  • Change the display name of the Send activities to SendFirstOp and SendSecondOp, respectively
  • Drag and Drop ReceiveSecondOp and SendSecondOp outside of their Sequence activity, directly below SendFirstOp
  • Remove Sequence, which is empty at this point
  • Change SendFirstOp: set Content to sInstanceId
  • Change ReceiveFirstOp: set OperationName to StartWf and remove data from the Content. In CorrelationIntializers, set FirstRequestResponse instead of handle. Validation errors should disappear at this point.
  • Change ReceiveSecondOp: set OperationName to ContinueWf and Content to sInstanceId
  • Change SendSecondOp: set Content to “Done with " + sInstanceId.

FirstRequestResponse needs to return the InstanceId to the client, so we need to retrieve it. This information is available in the activity context, so the easiest way to retrieve it is by means of a code activity:

  • Add a class to the project, name it WfInstanceId and implement it this way:
 public class WfInstanceId: CodeActivity
 {
     public OutArgument<Guid> InstanceId
     {
         get; set;
     }
     protected override void Execute(CodeActivityContext context)
     {
         InstanceId.Set(context, context.WorkflowInstanceId);
     }
 }

Note: mind to make the class public, otherwise the activity won’t show up in the Toolbox in the next steps.

  • Build the project (necessary for the WfInstanceId activity to show up in the Toolbox)
  • Drag and Drop the WfInstanceId activity from the Toolbox to after ReceiveFirstOp and before SendFirstOp
  • Set the InstanceId property of WfInstanceId to the InstanceId variable
  • Drag and Drop an Assign activity from the Toolbox to after WfInstanceId. Assign to sInstanceId the value “InstanceId.ToString()”

This is how the workflow should look like at this point:

Wf

Adding Persistence

Though this is not a strict requirement, long-running workflows usually don’t stay in memory for their entire lifetime: they are persisted and unloaded when appropriate (typically, when they are idle, that is when they are waiting for some external event like an incoming message or an expired timer to occur, before resuming execution).

Persistence adds the following benefits:

  • reliability: if the host crashes without having ever persisted a workflow instance, the instance state is lost. If, on the other hand, the workflow instance has been persisted, it can be resumed from its last persistence point.
  • scalability: if thousands or workflows are executing at the same time (which is well possible, if workflows are long-running), persisting and unloading idle workflows decreases the memory usage and increases scalability. Scalability is further increased by the possibility to persist and unload from one machine and reload from a different machine.

Adding support for persistence is straightforward and can be done declaratively in the configuration file. In addition, we also configure the workflow to be persisted whenever it reaches an idle point:

  • add the SqlWorkflowInstanceStore and WorklfowIdle behaviors to web.config:
 <system.serviceModel>
   <behaviors>
     <serviceBehaviors>
       <behavior>
         <sqlWorkflowInstanceStore connectionString="Initial Catalog=WorkflowInstanceStore;Server=(local);Integrated Security=true"/>
         <workflowIdle timeToUnload="0:00:0" timeToPersist="0:00:0"/>
       </behavior>
     </serviceBehaviors>
   </behaviors>
  </system.serviceModel>

Configuring Correlation

Let’s try the sample. Run the project, use WcfTestClient to execute, in the correct sequence, the operations StartWf and ContinueWf. Make sure to copy the instance id, output of StartWf, to the input of ContinueWf (without “”).

When you execute the ContinueWf project, you should get this error message in WcfTestClient:

System.ServiceModel.FaultException was unhandled
Message: An exception of type 'System.ServiceModel.FaultException' occurred in mscorlib.dll and wasn't handled before a managed/native boundary
Additional information: There is no context attached to the incoming message for the service and the current operation is not marked with "CanCreateInstance = true". In order to communicate with this service check whether the incoming binding supports the context protocol and has a valid context initialized.

Why is that? The reason has to do with how correlation is configured for this workflow.

You may recall from the previous steps that we have two variables of type CorrelationHandle in our workflow: FirstRequestResponse and SecondRequestResponse. These correlation handles are initialized with a Request-reply correlation initializer (see the CorrelationInitializers property of ReceiveFirstOp and ReceiveSecondOp). They are used to correlate each Receive activity to its corresponding SendReply activity (or activities).

However, we don’t have anything that relates the first operation to the second operation. In other words, we need something that, when the client calls the second operation, is able to identify to which running workflow instance this operation is directed.

There are two types of correlation that can be used for this purpose: context-exchange correlation and content-based correlation. This MSDN topic and its subtopics have all the details. We will be using content-based correlation in this case, because the workflow can be long running. The lifetime of a long-running workflow should not be constrained by the lifetime of the client that initiated it (in other words, it should be possible for one client to start the workflow, and for other clients to continue its execution). Since a context, established by context-exchange correlation, is limited to the lifetime of the client, it is not suitable for our needs.

Content-based correlation uses values (called instance keys) contained in the messages exchanged by client and service, that uniquely identify a workflow instance. In our MEP described above, this role can be played by the InstanceId. Therefore, we will configure content-based correlation using the InstanceId as the key. To do this, follow these steps:

  • Add a variable named InstanceCorrelation of type CorrelationHandle, at Sequential Service scope
  • Set the CorrelationInitializers property of SendFirstOp: type InstanceCorrelation on the left, then select “Query correlation initializer” and select Content: String in the XPath query
  • Set the CorrelatesOn property of ReceiveSecondOp: set CorrelatesWith to InstanceCorrelation, and select Content: String in the XPath query

The steps above look somewhat esoteric, but in fact they are pretty simple: we are basically specifying that the key that uniquely identifies the message is the InstanceId contained, in string form, in the response of the first operation and in the request of the second operation.

Let’s now test that this fixed the previous FaultException: run with F5, execute the first operation, copy the output GUID and paste it into the input of the ContinueWf operation (make sure not to include the “”). This time everything should work and you should get “Done with <GUID>” in the Response of the ContinueWf operation.

Let’s further test the effect of content-based correlation: as before, execute StartWf. Copy the returned GUID. However, this time after pasting the GUID in ContinueWf, modify it so that it does not match the one returned by StartWf and execute.

You should get this exception:

System.ServiceModel.FaultException was unhandled
Message: An exception of type 'System.ServiceModel.FaultException' occurred in mscorlib.dll and wasn't handled before a managed/native boundary
Additional information: The execution of an InstancePersistenceCommand was interrupted because the instance key 'b4c6f11b-a465-87ce-fe6e-7cf3d2558b6c' was not associated to an instance. This can occur because the instance or key has been cleaned up, or because the key is invalid. The key may be invalid if the message it was generated from was sent at the wrong time or contained incorrect correlation data.

The key that we provided is invalid, because there isn’t any running workflow instance with that instance id, hence the exception.

Note: if you query the InstancesTable table of the persistence database, you should find an instance with the GUID used in the last test: since we failed to execute the second operation, that workflow instance is still running, persisted in the database.

Adding Support for Multiple Versions

After sorting out all the previous steps, we are now ready to implement side-by-side execution. The steps are pretty easy, but let’s first look at the api provided at WF Runtime level since WF4.5, as this helps understand what happens in IIS/WAS-hosted workflows.

WorkflowService in WF4.5 has a new property, DefinitionIdentity of type WorkflowIdentity, which identifies a particular version of a workflow service definition. In addition, WorkflowServiceHost has a SupportedVersions property, which is a collection of WorkflowService instances.

When explicitly using WorkflowServiceHost (self-hosting), a host that supports multiple versions side-by-side would use the following code:

 WorkflowService initialService = new WorkflowService
 {    
     Name = "MyWorkflowService",
     Body = new MyWorkflowService1(),
     DefinitionIdentity = new WorkflowIdentity("Initial Version", new Version(1, 0, 0, 0), null)
 };
  
 WorkflowService updatedService = new WorkflowService
 {
    Name = "MyWorkflowService",
    Body = new MyWorkflowService2(),
    DefinitionIdentity = new WorkflowIdentity("Updated Version", new Version(2, 0, 0, 0), null)
 };
 WorkflowServiceHost sh = new WorkflowServiceHost(updatedService);
 sh.SupportedVersions.Add(initialService);

This host will implement SxS versioning, in that

  • when a new workflow is activated, version 2.0.0.0 is used
  • when a request directed at an existing instance arrives, the correct workflow definition will be used, if it is 1.0.0.0 or 2.0.0.0.

For IIS/WAS-hosted workflow services, this same effect can be achieved by using a well-defined naming convention for workflow services and the folders that contain them. Specifically:

  • the .xamlx file at the URL used by the client is the “main version”, equivalent to the one used as an argument of the WorkflowServiceHost constructor in the code above
  • all the .xamlx files placed under subfolders of App_Code, named after the workflow service itself, are “supported versions”, equivalent to those added to the SupportedVersions collection in the code above.

The DefnitionIdentity property of WorkflowService can be set directly in the workflow designer.

In our sample, we opt for a change to the InstanceId that is used to uniquely identify the workflow. Instead of the GUID, we want to use the syntax InstanceId: GUID. Before making the change, we run StartWf and get the Instance ID back. This is to ensure that, after making the change, we’ll have an instance running with the previous workflow definition.

In order to support SxS execution in our sample:

  • Set the DefinitionIdentity property of Service1.xamlx: set Name to “Service1” and Version to 1.0.0.0. Leave Package empty.
  • Create a subfolder called App_Code
  • Create a subfolder of App_Code, named Service1
  • Copy Service1.xamlx to the Service1 subfolder, possibly changing the file name (for instance, Service1_v1.xamlx)
  • Make changes to the top-level Service1.xamlx file: set Version in DefinitionIdentity to 2.0.0.0; set SendFirstOp’s Content to ‘“Instance Id: “ + sInstanceId; set SendSecondOp’s Content to "V2: Done with " + sInstanceId
  • Build and run
  • Execute StartWf. This will start a new workflow that uses version 2.0.0.0, so you should get a response like “Instance Id: <GUID>”
  • Copy the string and paste (without quotes) in the Request of ContinueWf. You should get the Response: “V2: Done with Instance Id: <GUID>”
  • Execute ContinueWf using this time the GUID returned by StartWf before we introduced versioning. You should get the Response “Done with Instance Id: <GUID>”. This shows that this workflow instance continued execution using the original workflow definition, thus achieving the desired SxS versioning behavior.

Constraints of Multiple Versions

You may have noticed that, when upgrading our workflow service from version 1.0.0.0 to version 2.0.0.0, we haven’t changed the MEP between client and service. Note that the client code does not change and the choice of which version to use is taken at runtime on the service side. Any change to the MEP between client and service, for instance

  • Changing the names of the operations
  • Changing the number or type of the arguments of the operations
  • Adding / removing operations to the MEP
  • Changing the type of the operation from Request/Response to OneWay or vice-versa

For the sake of completeness, let’s see what happens if we change the MEP:

  • Under App_Code\Service1, copy Service1_v1.xamlx to Service1_v15.xamlx
  • Change DefinitionIdentity for Service1_V15.xamlx: set Version to 1.5.0.0
  • Modify SendFirstOp of Service1_V15.xamlx: set Content to InstanceId instead of sInstanceId
  • Build and run.

When WcfTestClient starts, it should display an error message. If you look at it in Html View, you should see an error message like this:

Exception loading supported version 'App_Code\Service1\Service1_v15.xamlx': Two SendReply objects with same ServiceContractName and OperationName 'StartWf' have different ValueType.

Before going forward, remove Service1_v15.xamlx, otherwise the service won’t start.

Updating Code Activities

So far in our example, the changes were relative to the .xamlx file only. Workflows, however, are made of activities, and some of them can be custom activities, which in turn may also use custom assemblies in their implementation. When it comes to modifying a workflow definition, therefore, the changes are not necessarily restricted to the .xamlx workflow service; they may also affect custom assemblies (eventually the same assembly where .xamlx files are located).

Note: the following considerations on assembly version handling apply to code activities and other custom assemblies as well.

In order to exemplify the concept, let’s now make a variation to our previous change: instead of changing the returned string by editing the Content of SendFirstOp, we change the implementation of the WfInstanceId activity to return a string with the new syntax. This also requires changing the type of OutArgument of WfInstanceId, from System.Guid to String.

This change requires modification to both the .xamlx file and the compiled assembly, where WfInstanceId is defined. Mind that, if we want to support SxS for the workflow service, we need to keep both versions of the assembly: running instances that use the previous version will still need access to the previous version of WfInstanceId.

In .Net it is possible to keep multiple versions of an assembly in this way:

  • give different assembly version to the assemblies
  • put the assemblies in the GAC

Note: assembly version is different from the file version. In .Net, assembly version is set with the AssemblyVersion attribute, the file version is set with the AssemblyFileVersion attribute.

Note: in order to put the assemblies in the GAC it is also necessary to give them a strong name.

All this is simplified if we factor out the custom activity in a separate assembly, so let’s do it first:

  • Add a new project to the solution, of type class library. We can call it ActivityLibrary (but feel free to choose a different name)
  • Remove Class1.cs from the project, as we won’t use it
  • Move WfInstanceId.cs from the previous project to the new project. You may want to change the namespace for the WfInstanceId activity. Also fix-up assembly references (add System.Activities).

Now we need to give a strong name to the assembly built out of this new project, and to put the assembly in the GAC at the end of each successful build:

  • In the Project Properties of ActivityLibrary, in the Signing tab, check “Sign the assembly” and choose a .snk file (or create new, if you don’t have one already)
  • In the Project Properties of ActivityLibrary, in the Build Events tab, add a Post-build event “gacutil.exe –i $(TargetPath)” (you’ll need to use the full path to gacutil.exe, if it is not in the PATH environment variable)

If you build the ActivityLibrary project, you should see that the assembly is now in the GAC (look under %WINDIR%\Microsoft.NET\assembly).

The assembly we have just built has version 1.0.0.0, this is determined by the AssemblyVersion attribute in AssemblyInfo.cs. Now let’s build version 2.0.0.0 of this assembly, with the updated WfInstanceId activity:

  • Set the AssemblyVersion attribute to 2.0.0.0 in AssemblyInfo.cs
  • Modify the WfInstanceId activity in this way:
 public class WfInstanceId: CodeActivity
 {
     public OutArgument<string> InstanceId
     {
         get; set;
     }
  
     protected override void Execute(CodeActivityContext context)
     {
         InstanceId.Set(context, "Instance Id: " + context.WorkflowInstanceId.ToString());
     }
 }
  • Build the solution (including the SxSWfSvc project). If you look at the GAC now, you should see that both version 1.0.0.0 and version 2.0.0.0 of the assembly are present.

Let’s recall that different versions of our workflow service will need to use different versions of the WfInstanceId activity. Let’s open the two .xamlx files in the designer. You should see something like this:

WfWithErrorInActivity

The reason of the error is that the referenced activity is still the one contained in SxSSvcWf, which is no longer present. At the same time, in the toolbox you will see the new WfInstanceId activity under the ActivityLibrary category. For both .xamlx files, let’s fix the error:

  • Remove the old WfInstanceId activity (the red box)
  • Drag&Drop the new WfInstanceId activity in the same position

We also need to bind the InstanceId argument of the WfInstanceId activity. You’ll notice, if you try to bind it, that for both XAMLX files the InstanceId argument appears to be of type String:

NewInstanceIdArgument

In other words, both versions of the workflow service are now using the WfInstanceId activity contained in version 2.0.0.0 of the ActivityLibrary assembly, This is not what we want, and it is happening because the activity in the toolbox is version 2.0.0.0. Let’s make the following changes:

  • Remove the WfInstanceId activity from the Toolbox
  • Right-click on the ActivityLibrary category in the Toolbox and select “Choose Items…”
  • In the dialog box, click the Browse button and select ActivityLibrary.dll version 1.0.0.0 and ActivityLibrary version 2.0.0.0 from the GAC, as shown below:

WfInstanceIdBothVersions

 

When you click Ok, you should get a cryptic message box with this message:

The following controls were successfully added to the toolbox but are not enabled in the active designer: WfInstanceId

This message means that we do have both activities in the toolbox, but only one of them is shown when the current workflow service is open. If we hover on the WfInstanceId activity shown in the Toolbox, we find out that it is version 2.0.0.0. If we open the other version of the workflow service, the same thing happens. So our workflow services are still referencing version 2.0.0.0 of the ActivityLibrary assembly.

Let’s have a look at how ActivityLibrary is referenced by our workflow services. In order to do this, we need to switch to the XAML view of the workflow services. Right-click on Service1.xamlx and select “View Code”. Near the beginning of the xamlx file you should see something like this:

 xmlns:a="clr-namespace:ActivityLibrary;assembly=ActivityLibrary"

In other words. the assembly reference is qualified only by name, not by the fully qualified assembly name, including version and public key token. This is how the workflow designer adds assembly references automatically, and it has its own advantages in some scenarios. If there are multiple versions in the GAC, the latest is loaded. This explains why we could only see version 2.0.0.0 in the Toolbox.

The good thing is that you can manually modify the default, and fully qualify an assembly reference. Since we need to reference different versions of the assembly from our workflow services, do the following:

  • open the top-level Service1.xamlx and replace the assembly reference with:
 xmlns:a="clr-namespace:ActivityLibrary;assembly=ActivityLibrary, Version=2.0.0.0, Culture=neutral, PublicKeyToken=4a41ef8fd7501054"

 

  • open the xaml under App_Code and replace the assembly reference with:
 xmlns:a="clr-namespace:ActivityLibrary;assembly=ActivityLibrary, Version=1.0.0.0, Culture=neutral, PublicKeyToken=4a41ef8fd7501054"

Note: The PublicKeyToken depends on the signing key and will be different in your case, You can find the full assembly reference using ILDASM, ILSpy or another disassembly tool.

After saving the .xamlx files, open them under the designer. If you try to bind the InstanceId argument of WfInstanceId, you should find that each workflow service is referencing its own version. So in the workflow service under App_Code you should see that the argument is of type Guid:

OldInstanceIdArgument

Now let’s complete the binding. For the workflow service 1.0.0.0:

  • Bind InstanceId to the InstanceId variable, as before

For the workflow service version 2.0.0.0:

  • Bind InstanceId to the sInstanceId variable
  • Remove the Assign activity that follows the WfInstanceId activity in the workflow

If you try this time, you should see that both new workflows and existing workflows can run successfully.

SxSWfSvc.zip