My reaction to this was to wonder why we needed a new scripting language when there were already so many in existence! This is especially true considering each component probably would require its own XML processing engine so there are probably multiple out there.

One additional benefit to using scripting for your tests is that since scripts are often interpreted, your build times should be reduced over implementing tests in C#. But there are also some down sides:

• Detecting API changes and incompatibilities won’t happen until runtime when using scripting languages.
• You may have to install additional binaries to run your scripts
• Many people are probably not proficient enough in a scripting language such that they would consider using it for testing. It takes time to learn a new language.
• Hosting a scripting language requires additional learning and research if you require a mixture of compiled and scripted code.

In this post, I will show some examples of scripts to run a simple scenario using the Unified Communications Managed SDK (UCMA), which I’ve used once or twice before.  I will attempt this in Iron Python 2.7 and Iron Ruby 1.1.3.

Note: PowerShell is another possible choice, but there are some additional things to overcome so I hope to handle it in a future post.

Challenges

Each language has some strengths and some challenges when it comes to dealing with existing .Net code.  In particular, the APIs we are testing have to have equal or better usability for the following C# code:

endpoint.RegisterForIncomingCall<InstantMessagingCall>(this.CallReceived);

call.StateChanged += this.CallStateChanged;

Basically, the scripting languages need to deal with passing generic delegates, and registering event handlers that use an EventHandler<TArgs> signature.

Using Iron Python

I am more familiar with Python than Ruby, so out of the gate Python has an advantage and it may be visible in my writing in this post.  If you are fluent in Iron Ruby and would like to point out some corrections to put Ruby on a more even playing field, feel free to leave a comment at the bottom of the post.

Iron Python has built in support for generics, which I happened to know when writing the sample scripts. This helped a lot in making the RegisterForIncomingCall usage easy. This section of code is shown here:

print 'Registering for incoming call'

self.__endpoint.RegisterForIncomingCall[InstantMessagingCall](self.__IMCallReceived)        

Also in Iron Python, registering for event handlers is similar to C# and doesn’t require much of a learning curve:

self.__imCall.StateChanged += self.__CallStateChanged

But to load a dll so you can use it is a bit unintuitive at first:

import clr

import sys

sys.path.append('<path to dll>')

clr.AddReference("Microsoft.Rtc.Collaboration.dll")

and then python requires importing the symbols you need. I did it the hard way:

from Microsoft.Rtc.Collaboration import ServerPlatformSettings,CollaborationPlatform,ApplicationEndpointSettings,ApplicationEndpoint

from Microsoft.Rtc.Collaboration import Conversation,InstantMessagingCall,CallEstablishOptions

from Microsoft.Rtc.Signaling import FailureResponseException

it could probably be simplified a bit like this:

from Microsoft.Rtc.Collaboration import *

from Microsoft.Rtc.Signaling import *

but it is probably the case you would pull in too many type names as .Net probably is not as conservative on what is exposed as Python libraries are.

The rest of the issues with python are the usual preference items that come up when comparing languages. For example, you will see “self” repeated a lot in python classes, and the space significance of the blocks may not be to your liking.

Using Iron Ruby

Not being too familiar with Ruby, I had to do a little research to get started. After picking up the Ruby necessities my first challenge was getting the reference set up to the API I was interested in testing. After trying a few things, I was able to get the API to test loaded, but it probably not the best way:

require 'C:\windows\microsoft.net\assembly\GAC_MSIL\Microsoft.Rtc.Collaboration\<version>\Microsoft.Rtc.Collaboration.dll'

include Microsoft::Rtc::Collaboration

Note that it is being included directly from the GAC. I should be able to specify the assembly information and it would hopefully find it in the GAC, but that didn’t seem to work unfortunately.

The next thing was to figure out the RegisterForIncomingCall invocation. This is what I came up with:

# The C# equivalent of these two lines is:

#

# endpoint.RegisterForIncomingCall<InstantMessagingCall>(this.IMCallReceived)       

#

p = Proc.new { |sender,e| IMCallReceived(sender,e) }

@endpoint.method(:RegisterForIncomingCall).of(InstantMessagingCall).call(p)

As you can see, this is probably not the most optimal way to do it in Ruby.  A small interactive experiment shows that Ruby does support generic syntax:

>>> require 'System'

=> true

>>> include System::Collections::Generic

=> Object

>>> x = List[System::Int32].new

=> []

>>> x.add(4)

=> nil

>>> x

=> [4]

>>> x.add(7)

=> nil

>>> x

=> [4, 7]

>>>

but that is how I initially got it working.  I probably could go back and change the code now, but finding the better way to register the event handlers is really bugging me.  It seems that this should have worked:

# The following should also be valid, but doesn't seem to work. Because of generics?

#

#  @imCall.InstantMessagingFlowConfigurationRequested.add do |sender, e| 

#    FlowConfigurationRequested(sender,e)

#  end

But it didn’t work for me as you can see from the comment.  I eventually got it to work as follows:

# In C#: call.InstantMessagingFlowConfigurationRequested += this.FlowConfigurationRequested

p = Proc.new { |sender,e| FlowConfigurationRequested(sender,e) }

@imCall.InstantMessagingFlowConfigurationRequested.add p

Although it works, it isn’t really intuitive to someone coming from C#. Maybe I can do something with the += operator to make this easier, but out of the box it didn’t seem to allow that. If you have experience with this in Iron Ruby feel free to leave a pointer in the comments on what the best way to register the event handlers.

Other than that, everything is pretty straight forward and again, the only thing left are tidbits related to language syntax and preferences.  Ruby is not sensitive to whitespace to determine its block boundaries like Python is, but you need to be sure to match up your “ends” correctly.

Ruby has more support for creating DSLs (sub languages) based on Ruby syntax which I am interested in trying out at some point. With a little more time spent with Ruby I’ll hopefully be able to pick up the best practices and improve my example.

Choosing your language

Given that some language aspects are likely to bug people and affect their choice of languages, it is very likely that both languages will need to be available in the test environment. Unfortunately, Iron Python and Iron Ruby currently use different versions of the hosting API which makes having them both present somewhat tricky.

The recommended solution is to download the source and recompile so they use the same hosting APIs.  This may be a barrier to entry that is too high for some people and drive them back to inventing their own scripting languages again. I quickly attempted this, but it will take more time and investigation to get it working.

Summary

In this post I presented two existing scripting languages that people could try instead of rolling their own.  Below I’ll include the full text of the implementations so you can see more clearly. Hopefully you will be inspired to spend one or two days learning a scripting language and consider using an existing one rather than rolling your own. By looking through the code below you may find one language appealing.  Maybe it will reduce the anxiety about learning one of the languages.

test.py

import clr

import sys

sys.path.append('<path to dll>')

clr.AddReference("Microsoft.Rtc.Collaboration.dll")

print "Loaded ... "

from Microsoft.Rtc.Collaboration import *

from Microsoft.Rtc.Signaling import *

class Client:

    def __init__(self,ownerUri,port):

        self.__platform = None

        self.__endpoint = None

        self.__ownerUri = ownerUri

        self.__port = port;

        self.__conversation = None

        self.__imCall = None

        self.__imFlow = None

    def get_Port(self):

        return self.__port

    def get_OwnerUri(self):

        return self.__ownerUri

    def Initialize(self,proxyPort):

        settings = ServerPlatformSettings("UserAgent", "localhost", self.__port, self.__ownerUri + ";gruu")

        self.__platform = CollaborationPlatform(settings)

        self.__platform.EndStartup(self.__platform.BeginStartup(None, None))

        settings = ApplicationEndpointSettings(self.__ownerUri,"localhost", proxyPort)

        print "creating app endpoint"

        self.__endpoint = ApplicationEndpoint(self.__platform, settings)

        print 'Registering for incoming call'

        self.__endpoint.RegisterForIncomingCall[InstantMessagingCall](self.__IMCallReceived)        

        print 'Establishing app endpoint'

        self.__endpoint.EndEstablish(self.__endpoint.BeginEstablish(None,None))

    def SendMessage(self,message):

        self.__imFlow.EndSendInstantMessage(self.__imFlow.BeginSendInstantMessage(message,None,None))

    def MakeCall(self,ownerUri):

        self.__conversation = Conversation(self.__endpoint)        

        self.__imCall = InstantMessagingCall(self.__conversation)

        self.__imCall.InstantMessagingFlowConfigurationRequested += self.__FlowConfigurationRequested

        self.__imCall.StateChanged += self.__CallStateChanged

        try:

            self.__imCall.EndEstablish(self.__imCall.BeginEstablish(ownerUri, None, None, None, None))

        except FailureResponseException,e:

            print "Exception:", e.Message

    def Terminate(self):

        if self.__imCall != None:

            self.__imCall.EndTerminate(self.__imCall.BeginTerminate(None,None))

    def __CallStateChanged(self,sender,e):

        s ="[" + self.__ownerUri + "] Call state changed: " + e.PreviousState.ToString() + " => " + e.State.ToString()

        print s

    def __IMCallReceived(self,sender,e):

        print "Call received."

        self.__imCall = e.Call

        self.__imCall.InstantMessagingFlowConfigurationRequested += self.__FlowConfigurationRequested

        self.__imCall.StateChanged += self.__CallStateChanged

        e.Call.BeginAccept(self.__CallAcceptCompleted,None)

    def __FlowConfigurationRequested(self,sender,e):

        print "[",self.__ownerUri,"] Flow configuration requested."

        self.__imFlow = e.Flow

        e.Flow.StateChanged += self.__FlowStateChanged

        e.Flow.MessageReceived += self.__MessageReceived

    def __FlowStateChanged(self,sender,e):

        s = "[" + self.__ownerUri + "] Flow state changed: " + e.PreviousState.ToString() + " => " + e.State.ToString()

        print s

    def __MessageReceived(self,sender,e):

        s = '[' + self.__ownerUri + '] Message received: '+ e.TextBody

        print s

    def __CallAcceptCompleted(self,result):

        try:        

            self.__imCall.EndAccept(result)

        except RealTimeException,e:

            print "Call accept failed:", e.Message

c1 = Client("sip:one@myhost", 5061)

c2 = Client("sip:two@myhost", 5062)

print 'Initializing client 1'

c1.Initialize(c2.get_Port())

print 'Initializing client 2'

c2.Initialize(c1.get_Port())

print 'Making call'

c1.MakeCall(c2.get_OwnerUri())

c1.SendMessage("Hello World!")

c1.Terminate()

c2.Terminate()

test.rb

require 'C:\windows\microsoft.net\assembly\GAC_MSIL\Microsoft.Rtc.Collaboration\<version>\Microsoft.Rtc.Collaboration.dll'

include Microsoft::Rtc::Collaboration

class Client

    def initialize(ownerUri,port)

       @platform = nil

       @endpoint = nil

       @ownerUri = ownerUri

       @port = port

       @conversation = nil

       @imCall = nil

       @imFlow = nil

    end

    def Port

        @port

    end

    def OwnerUri

        @ownerUri

    end

    def CallAcceptCompleted(result)

        puts "Call accept completed."

        @imCall.EndAccept(result)

        #rescue Microsoft::Rtc::Signaling::RealTimeException => e

        #    puts "Accept failed: " + e.Message

        #end

    end

    def IMCallReceived(sender,e)

        puts 'Call received'

        @imCall = e.Call

        # The following should also be valid, but doesn't seem to work. Because of generics?

        #

        #  @imCall.InstantMessagingFlowConfigurationRequested.add do |sender, e| 

        #    FlowConfigurationRequested(sender,e)

        #  end

        # In C#: call.InstantMessagingFlowConfigurationRequested += this.FlowConfigurationRequested

        p = Proc.new { |sender,e| FlowConfigurationRequested(sender,e) }

        @imCall.InstantMessagingFlowConfigurationRequested.add p

        # In C#: call.StateChanged += this.CallStateChanged

        p = Proc.new { |sender,e| CallStateChanged(sender,e) }

        @imCall.StateChanged.add p 

        p = Proc.new { |result| CallAcceptCompleted(result) }

        @imCall.BeginAccept(p,nil)

    end

    def MessageReceived(sender,e)

        s = '[' + @ownerUri + '] Message received: ' + e.TextBody

        puts s

    end

    def CallStateChanged(sender,e)

        s = '[' + @ownerUri + '] Call state changed: ' + e.PreviousState.ToString() + '=> ' + e.State.ToString()

        puts s

    end

    def FlowStateChanged(sender,e)

        s = '[' + @ownerUri + '] Flow state changed: ' + e.PreviousState.ToString() + ' => ' + e.State.ToString()

        puts s

    end

    def FlowConfigurationRequested(sender,e)

        s = '[' + @ownerUri + '] Flow configuration requested'

        puts s

        @imFlow = e.Flow

        p = Proc.new { |sender,e| FlowStateChanged(sender,e) }

        @imFlow.StateChanged.add p

        p = Proc.new { |sender,e| MessageReceived(sender,e) }

        @imFlow.MessageReceived.add p

    end

    def Start(proxyPort)

        settings = ServerPlatformSettings.new("UserAgent", "localhost", @port, @ownerUri + ";gruu")

        @platform = CollaborationPlatform.new(settings)

        puts 'Creating app endpoint'

        settings = ApplicationEndpointSettings.new(@ownerUri, "localhost", proxyPort)

        @endpoint = ApplicationEndpoint.new(@platform,settings)

        # The C# equivalent of these two lines is:

        #

        # endpoint.RegisterForIncomingCall<InstantMessagingCall>(this.IMCallReceived)       

        #

        p = Proc.new { |sender,e| IMCallReceived(sender,e) }

        @endpoint.method(:RegisterForIncomingCall).of(InstantMessagingCall).call(p)

        puts 'Creating platform'

        @platform.EndStartup(@platform.BeginStartup(nil, nil))

        @endpoint.EndEstablish(@endpoint.BeginEstablish(nil,nil))

    end

    def SendMessage(message)

        @imFlow.EndSendInstantMessage(@imFlow.BeginSendInstantMessage(message,nil,nil))

    end

    def Terminate()

        if @imCall then 

            puts '[' + @ownerUri + '] Terminating call.'

            @imCall.EndTerminate(@imCall.BeginTerminate(nil,nil))

        end

        if @endpoint then

           puts '[' + @ownerUri + '] Terminating endpoint.'

           @endpoint.EndTerminate(@endpoint.BeginTerminate(nil,nil))

        end

        if @platform then

            puts '[' + @ownerUri + '] Terminating platform.'

            @platform.EndShutdown(@platform.BeginShutdown(nil,nil))

        end       

    end

    def MakeCall(ownerUri)

        @conversation = Conversation.new(@endpoint)

        @imCall = InstantMessagingCall.new(@conversation)

        # In C#: call.InstantMessagingFlowConfigurationRequested += this.FlowConfigurationRequested

        p = Proc.new { |sender,e| FlowConfigurationRequested(sender,e) }

        @imCall.InstantMessagingFlowConfigurationRequested.add p

        # In C#: call.StateChanged += this.CallStateChanged

        p = Proc.new { |sender,e| CallStateChanged(sender,e) }

        @imCall.StateChanged.add p

        @imCall.EndEstablish(@imCall.BeginEstablish(ownerUri, nil, nil, nil, nil))

        #rescue Microsoft::Rtc::Signaling::FailureResponseException => e

        #    print 'Exception: ' + e.Message

        #end

    end 

end

c1 = Client.new("sip:one@myhost", 5061)

c2 = Client.new("sip:two@myhost", 5062)

puts 'Starting client 1'

c1.Start c2.Port

puts 'Starting client 2'

c2.Start c1.Port

puts 'Making call to ' + c2.OwnerUri

c1.MakeCall c2.OwnerUri

puts 'Sending message.'

c1.SendMessage("Hello world!")

puts 'Terminating client 1'

c1.Terminate

puts 'Terminating client 2'

c2.Terminate

201101127_Scripting.zip

Avoiding a race condition using ContinueWith

First lets assume we have the following code calling an asynchronous method which computes a value and stores it in m_field:

 1: private void RunContinueWith()

 2: {

 3:     Task task = null;

 4:  

 5:     Console.WriteLine("In calling method.");

 6:     // task = this.DoContinueWithAsync();

 7:     task = this.DoContinueWithRaceConditionAsync();

 8:     Console.WriteLine("Returned to calling method.");

 9:  

 10:     task.Wait();

 11:            

 12:     Console.WriteLine("Hashcode = {0}", m_field.GetHashCode());

 13: }

It may seem natural to implement this method as follows:

 1: private Task DoContinueWithRaceConditionAsync()

 2: {

 3:     Task task = Task.Factory.StartNew(

 4:         () =>

 5:         {

 6:             Console.WriteLine("Task part 1.");

 7:         });

 8:  

 9:     task.ContinueWith(

 10:             t =>

 11:             {

 12:                 Console.WriteLine("Task part 2.");

 13:                 Thread.Sleep(3000);

 14:                 m_field = new object();

 15:             });

 16:  

 17:     return task;

 18: }

There is a subtle race condition in this method implementation, however. Do you see it? It was probably introduced because the person writing the code wanted it to be a bit more readable. 

Because the ContinueWith method creates a second task that runs after the first task, the variable task now returns the first task and not the second task because of the code formatting. The caller is expecting the entire operation to complete when the returned task completes, but since it doesn’t return the correct task, there is a NullReferenceException when accessing m_field.GetHashCode().

There are two ways to fix the problem. Change task.ContinueWith to update the variable task as follows:

 1: task = task.ConintueWith(

or make sure that task is assigned the value of the second task from the beginning by ensuring that the methods are chained together:

 1: private Task DoContinueWithAsync()

 2: {

 3:     Task task = Task.Factory.StartNew(

 4:         () => 

 5:             { 

 6:                 Console.WriteLine("Task part 1."); 

 7:             })

 8:         .ContinueWith(

 9:             t => 

 10:             { 

 11:                 Console.WriteLine("Task part 2.");

 12:                 Thread.Sleep(3000);

 13:                 m_field = new object();

 14:             });

 15:  

 16:     return task;

 17: }

Like I mentioned earlier, this is very subtle but important for the stability of your code.

Unobserved Exceptions

Another problem that had to be addressed before signing off was exceptions reported from the finalizers of the tasks during garbage collection.  If an exception is not observed before the task is garbage collected, the finalizer will throw an exception.  It is very difficult to trace the problem back to the code where the exception should have been handled.  You can see an example unhandled exception and some code to cause the exception to be reported in the finalizer here:

 1: private void RunUnobservedExceptions()

 2: {

 3:     this.Unobserved();

 4:  

 5:     Console.WriteLine("Press <Enter> to force collection.");

 6:     Console.ReadLine();

 7:  

 8:     GC.AddMemoryPressure(1 * 1000 * 1000);

 9:     GC.Collect();

 10:     GC.WaitForPendingFinalizers();

 11: }

 12:  

 13: private void Unobserved()

 14: {

 15:     Task t = DoUnobservedAsync();

 16:  

 17:     while (!t.IsCompleted)

 18:     {

 19:         Thread.Sleep(1000);

 20:     }

 21: }

 22:  

 23: private Task DoUnobservedAsync()

 24: {

 25:     return Task.Factory.StartNew(

 26:         () =>

 27:         {

 28:             throw new InvalidOperationException("Failed.");

 29:         });

 30: }

Because the IsCompleted property of the task doesn’t result in the exception being observed, you get a dialog from Visual Studio when running in the debugger that looks like this:

Well, that is no fun. To avoid this we need to make sure we check and handle the exceptions before the Task goes out of scope.

How to observe exceptions

There are a few ways to observe exceptions. The first it to observe using the Wait method:

 1: private void ObservedUsingWait()

 2: {

 3:     Task t = this.DoUnobservedAsync();

 4:  

 5:     try

 6:     {

 7:  

 8:         t.Wait();

 9:     }

 10:     catch (AggregateException exception)

 11:     {

 12:             // TODO

 13:     }

 14: }

A second way to observe the exception is to check the Exception property explicitly:

 1: private void ObservedUsingException()

 2: {

 3:     Task t = this.DoUnobservedAsync();

 4:  

 5:     while (!t.IsCompleted)

 6:     {

 7:         Thread.Sleep(1000);

 8:     }

 9:  

 10:     if (t.Exception != null)

 11:     {

 12:         Console.WriteLine("Failed with exception: {0}", t.Exception.Message);

 13:     }

 14: }

This is a good option if you don’t want the exception re-thrown and caught.

If the task is computing some result, you can observe any exceptions by accessing the result:

 1: private void ObservedUsingResult()

 2: {

 3:     Task<int> task = Task.Factory.StartNew<int>( 

 4:         () => {

 5:             throw new InvalidOperationException("Failed.");

 6:         });

 7:  

 8:     try

 9:     {

 10:         // Will wait and throw exception on failure.

 11:         Console.WriteLine("Result is {0}", task.Result);

 12:     }

 13:     catch (AggregateException exception)

 14:     {

 15:         // TODO

 16:     }

 17: }

The last way to observe the exceptions, is to handle the exceptions individually using the Handle method:

 1: private void ObservedUsingHandle()

 2: {

 3:     Task t = this.DoUnobservedAsync();

 4:  

 5:     while (!t.IsCompleted)

 6:     {

 7:         Thread.Sleep(1000);

 8:     }

 9:  

 10:     // Mark all exceptions as handled.

 11:     t.Exception.Handle(

 12:         (ex ) => 

 13:             {

 14:                 Console.WriteLine("Ignoring exception with message: {0}", ex.Message);

 15:                 return true; 

 16:             });

 17:  

 18: }

The down side of this approach, is that it is way too easy to be sloppy and just ignore all exceptions. If an exception is not ignored (i.e. false is returned), however it is re-thrown and it needs to be handled elsewhere.

Observing Exceptions and ContinueWith

Things get a little more complicated when you need to observe an exception and are using ContinueWith. The first thing to remember is that ContinueWith creates a task for itself but it needs to observe the task that preceded it before it goes out of scope. Here is an example:

 1: private void ObserveExceptionInContinueWith()

 2: {

 3:     Task t = Task.Factory.StartNew(

 4:         () =>

 5:         {

 6:             throw new InvalidOperationException("Failed.");

 7:         })

 8:         .ContinueWith(

 9:             previousTask =>

 10:             {

 11:                 if (previousTask.Exception != null)

 12:                 {

 13:                     Console.WriteLine("Exception: {0}", previousTask.Exception.Message);

 14:                 }

 15:             });

 16:  

 17:     while (!t.IsCompleted)

 18:     {

 19:         Thread.Sleep(1000);

 20:     }

 21: }

In this case, the first tasks exception is observed, and the the second task cannot fail, so everything is fine. You can also have the ContinueWith run only if there is a fault by specifying a special option:

 1: private void ObserveExceptionInContinueWithOnlyOnFaulted()

 2: {

 3:     Task t = Task.Factory.StartNew(

 4:         () =>

 5:         {

 6:             throw new InvalidOperationException("Failed.");

 7:         })

 8:         .ContinueWith(

 9:             previousTask =>

 10:             {                       

 11:                 // We know previousTask.Exception is not null. No need to check.

 12:  

 13:                 Console.WriteLine("Exception: {0}", previousTask.Exception.Message);

 14:             }, TaskContinuationOptions.OnlyOnFaulted);

 15:  

 16:     while (!t.IsCompleted)

 17:     {

 18:         Thread.Sleep(1000);

 19:     }

 20: }

This works fine as well. The only problem is when you use tracing to observe the exception and it is only available in debug builds:

 1: Debug.Assert(false, previousTask.Exception.Message);

In this case you may test extensively on your debug builds, but get exceptions in your retail product. Be careful. The OnlyOnFaulted option does not observe the exception. You still need to add the code to do it.

The last case that can be tricky is returning false from the delegate passed to the Handle method:

 1: private void ObserveHandleInContinueWith()

 2: {

 3:     Task t = Task.Factory.StartNew(

 4:         () =>

 5:         {

 6:             throw new InvalidOperationException("Failed.");

 7:         })

 8:         .ContinueWith(

 9:             previousTask =>

 10:             {

 11:                 // We know previousTask.Exception is not null. No need to check.

 12:                 previousTask.Exception.Handle(

 13:                     (ex) =>

 14:                     {

 15:                         Console.WriteLine("Exception: {0}", ex.Message);

 16:                         return false;

 17:                     });

 18:             }, TaskContinuationOptions.OnlyOnFaulted);

 19:  

 20:     while (!t.IsCompleted)

 21:     {

 22:         Thread.Sleep(1000);

 23:     }

 24: }

Since you are touching the Exception property of the previousTask you would think this should be fine. Since the Handle method will re-throw any unhandled exceptions, however, the ContinueWith task will have an exception and if unobserved, will be discovered in the finalizer. So beware.

OnlyOnFaulted and NotOnFaulted

You may be tempted to use OnlyOnFaulted and NotOnFaulted to separate your success and failure code into separate delegates like this:

 1: private void ContinueWithFaultedNotFaulted()

 2: {

 3:     Task t = Task.Factory.StartNew(

 4:         () =>

 5:         {

 6:             throw new InvalidOperationException("Failed.");

 7:         })

 8:         .ContinueWith(

 9:             previousTask =>

 10:             {

 11:                 // We know previousTask.Exception is not null. No need to check.

 12:  

 13:                 Console.WriteLine("Exception: {0}", previousTask.Exception.Message);

 14:             }, TaskContinuationOptions.OnlyOnFaulted)

 15:             .ContinueWith(

 16:             previousTask =>

 17:             {

 18:                 Console.WriteLine("Continue on success.");

 19:             }, TaskContinuationOptions.NotOnFaulted);

 20:  

 21:     while (!t.IsCompleted)

 22:     {

 23:         Thread.Sleep(1000);

 24:     }

 25: }

Unfortunately, this doesn’t work. It runs both the success and failed cases:

 1: Exception: One or more errors occurred.

 2: Continue on success.

 3: Press <Enter> to force collection.

Why? Because the success case runs if the previous task doesn’t fault. Since the previous task is the ContinueWith that runs only when the original task faults, it will run. This may be confusing at first.

The reverse also does not work:

 1: private void ContinueWithNotFaultedFaulted()

 2: {

 3:     Task t = Task.Factory.StartNew(

 4:         () =>

 5:         {

 6:             throw new InvalidOperationException("Failed.");

 7:         })

 8:         .ContinueWith(

 9:             previousTask =>

 10:             {

 11:                 Console.WriteLine("Continue on success.");

 12:             }, TaskContinuationOptions.NotOnFaulted)

 13:         .ContinueWith(

 14:             previousTask =>

 15:             {

 16:                 // We know previousTask.Exception is not null. No need to check.

 17:  

 18:                 Console.WriteLine("Exception: {0}", previousTask.Exception.Message);

 19:             }, TaskContinuationOptions.OnlyOnFaulted)

 20:             ;

 21:  

 22:     while (!t.IsCompleted)

 23:     {

 24:         Thread.Sleep(1000);

 25:     }

 26: }

In this case, the first task fails, so the first and thus subsequent ContinueWith statements do not run, and the exception is not observed. To handle both the success and failure cases, you really need to handle both in the first ContinueWith statement:

 1: private void ContinueWithCombined()

 2: {

 3:     Task t = Task.Factory.StartNew(

 4:         () =>

 5:         {

 6:             throw new InvalidOperationException("Failed.");

 7:         })

 8:         .ContinueWith(

 9:             previousTask =>

 10:             {

 11:                 if (previousTask.IsFaulted)

 12:                 {

 13:                     // We know previousTask.Exception is not null. No need to check.

 14:                     Console.WriteLine("Exception: {0}", previousTask.Exception.Message);

 15:                     return;

 16:                 }

 17:  

 18:                 Console.WriteLine("Continue on success.");

 19:             });

 20:  

 21:     while (!t.IsCompleted)

 22:     {

 23:         Thread.Sleep(1000);

 24:     }

 25: }

Summary

In this post I covered some issues that may catch you off guard using the Task Parallel Library.  Hopefully you will find these tips helpful, and it will help keep bugs from creeping into your code. There is enough scary stuff going around this time of year. You don’t want your code to be scary.

This gives a lot more possibilities to experiment with software and blog about those experiences. It became clear to me though that with multiple virtual machines to setup and maintain I would have to organize myself a little better. You can get away with a lot if you only have one machine.

First, it would be nice to set up development environments automatically.  I already spend a lot of my time watching progress bars tick along. It would be nice to be automated as much as possible so I can start something running and walk away.

Second, my source files, examples, and install packages are split between my laptop and desktop machine in no particular pattern.

Lastly, I’ve spent some time learning PowerShell and like putting together useful scripts. Now is a perfect opportunity to create some useful utilities, and can organize them so I can possibly find them again.

After I spent a little time trying to organize the materials I had, and collecting some script snippets that could be useful soon, I decided to look into automated installation.

Windows Automated Installation Kit (AIK)

I found my way to the Windows AIK as a method for installing Windows automatically.  There are a lot of features and tools there and my strategy of reading the documentation wasn’t particularly effective.  I think this strategy might have gone better for me:

• Scan the documentation reading about what tools are available, the scenarios etc.
• Satisfy my curiosity how things worked.
• Go back and understand which scenarios best fit what you want to do.  Make a plan.
• Hit the sections in detail that are needed.

I read the part that seemed interesting and skipped the planning part.  It took me a while to realize I probably needed two installation plans. One for the reference computer and one for target computers. At first I figured I could use the same plan for both, but that isn’t necessarily the best option.

The other part that threw me a little, is that a setting up a development environment is not really a target scenario for the documentation. This comes into play when deciding which roles can be running on a VM and which need to be on the physical computer. In addition, I needed to learn a few things about Hyper-V and setting up virtual machines.

In any case, reading the documentation for Windows AIK carefully seems to be critical.  Then experimenting to see what works and what does not to get familiar with it. This takes a bit of time waiting for OS installs to complete to see the results and get some experience.

Settings things up

After downloading all the ISOs I needed, I set up the Windows AIK on a virtual machine, and created another virtual machine to be my reference computer. The documentation recommends using a USB (removable drive) for storing the answer file for setup.

This seemed to be a bit of a problem at first. There was no way to get a USB attached to a VM that I could see.  But floppy drives also were fine, so after a bit of searching the internet I found some script that would create a VFD that I could plug into the VM.  I couldn’t find an automated way to format the floppy, so I did it manually.  (I now think this is possible with diskpart.exe, but have not tried).

The PowerShell I used to create the VFD was:

 1: $s = gwmi -namespace root\virtualization Msvm_ImageManagementService

 2: $s.createvirtualfloppydisk("C:\floppy.vfd")

You can also translate from the C# example on MSDN.

I created an answer file with the minimum recommended settings, put it on the floppy, loaded both the floppy and the answer file into a VM and successfully deployed.  The install was mostly automated, but I was disappointed to see that there was one menu (the OS selection) that I could not find a solution for.  Since the dialog comes up at the start of the install, I lived with it for a while.

Recently I found the solution, however. To select an OS image from inside install.wim, you need to set the following:

ImageInstall\OSImage\InstallFrom

Path = install.wim

ImageInstall\OSImage\InstallFrom\MetaData

Key = /image/index

Value = 3

3 seems to be the 2008 R2 Enterprise Full install. How did I find this out?  By looking at the Unattend.xml file generated by Microsoft Deployment Toolkit. 

Adding additional software

The base image needs a lot of updates to bring it current. It needs to pull in IE9 and 2008R2 SP1 among others.  The total number of updates is currently around 80.  It seemed it would be more efficient if I could automatically install the big ones and then automate windows update.

I found some examples using PowerShell with the Windows Update COM Api. If you are interested search “Windows update PowerShell” or something equivalent. MSDN has an example in VBScript that can be easily translated to PowerShell.

After downloading the IE9 and 2008 R2 SP1 installer I ran into a bunch of problems automating the install. I still don’t have a really good solution so if you have suggestions, please leave some bread crumbs in the comment section.

There are few problems that have to be solved to automate this:

1) When you download installer packages, they are marked as “from the internet” and pop up all kinds of security dialogs.

2) Both IE9 and 2008 R2 SP1 installers need to reboot the machine. This means your automation need to figure out where it left off and continue.

3) 2008 R2 SP1 installer is 900M+. It didn’t seem like a good idea to copy it to the VM’s expandable HD and run it from there. I’d rather run it from a share on the host machine.

4) Running installer from a network share has its own set of problems. The most notable, is the open file security warning dialog.

Here are some of the solutions I have found.

Unblocking installers from the internet

It turns out that files downloaded from the internet onto an NTFS volume are marked by using alternate NTFS file streams with the zone it came from.

Normally, you can unblock content by right clicking the file and choosing properties. Then clicking the “unblock” button. There is an automated way to do this in PowerShell by deleting the alternate file stream.

You can search more about alternate file streams and PowerShell or you can read this blog post.

Running from a network share

I have successfully gotten rid of the dialog that happens running from the network share by modifying the registry with the following script:

 1: $securityPath = HKCU:Software\Microsoft\Windows\CurrentVersion\Policies\Associations

 2:  

 3: if (-not (test-path $securityPath))

 4: {

 5:    new-item -itemtype directory -force -path $securityPath

 6:    set-itemproperty $securityPath -name ModRiskFileTypes ".exe;.msi"

 7: }

There may be better ways of doing this, especially if you can integrate the installers into the setup installer. This is the best solution I’ve found thus far. There also may be security issues with relaxing the restriction. In a closed VM environment I think it is fine for my purpose.

I was able to net use the share, run this script, and run one of the IE9 or SP1 installers as separate steps, but not together as one smooth install yet. One installer also isn’t helpful since I have to have a solution that runs two installers with a reboot in between.

Running a second installer

I’ve tried various things to run a second installer.

1) Add the unattended command lines to the specialize step of unattended setup. Setup can handle reboots in the middle so it seemed like a good approach.

Result: Although I did see the IE9 installer dialog, the net result was IE8 was still the browser after setup completed. It may be that it is too early in setup to run installers. It seems that way if you read the AIK documentation carefully.

2) I did a bunch of net use and installer launches in the oobe setup step. This phase doesn’t seem to have the same reboot management support that the specialize step did so I needed to find a solution to this.  The RunOnce registry key seems to be one possibility. Also, at the time I tried this I hadn’t found how to get rid of the open file security dialog yet so it would run the first installer and pop up a dialog.

3) Create a series of PowerShell scripts that manage the installation sequence.

Result: PowerShell execution policy has not been updated yet so you need to ensure you pass “-executionpolicy bypass” for scripts entered as synchronous commands in the unattend.xml file.  This was the first solution I was able to get to work, in combination with the RunOnce registry key.

Working with RunOnce registry key

Since both installers reboot, I considered using the RunOnce registry key to initiate the second installer after reboot and auto logon. Here is a script I used to install an example registry entry:

 1: pushd hklm:\Software\Microsoft\Windows\CurrentVersion

 2: if (-not (test-path RunOnce))

 3: {

 4:     new-item -type directory RunOnce

 5: }

 6:  

 7: set-itemproperty RunOnce -Name First -Value "C:\install\IE9-Windows7-x64-enu.exe /passive"

Note that when there are multiple values, it uses them in alphabetical order, so be aware.

The solution

So the solution I came up with to build the reference computer is made up of the following steps:

specialize

Copy the scripts and IE9 installer to the local VHD

Install RunOnce to install RunOnce for the SP1 installer

Install RunOnce to start the IE9 installer

oobe

Install IE9 and reboot

RunOnce

Install R2 SP1

Install windows update script into RunOnce

Install windows updates.

Specialize step in Autounattend.xml

net use z: \\host\install /u:host\install <password>

powershell.exe –executionpolicy bypass z:\reference.ps1

Reference.ps1

This script copies files to the c:\install directory and installs the RunOnce entry installing a RunOnce entry after reboot.

 1: new-item -itemtype directory -force c:\install

 2: copy-item z:\ie9\*.exe c:\install

 3: copy-item z:\*.ps1 c:\install

 4:  

 5: pushd hklm:\Software\Microsoft\Windows\CurrentVersion

 6: if (-not (test-path RunOnce))

 7: {

 8:     new-item -type directory RunOnce

 9: }

 10:  

 11: set-itemproperty RunOnce -Name 001 -Value "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -executionpolicy bypass c:\install\install-sp1-runonce.ps1"

 12: set-itemproperty RunOnce -Name 002 -Value "C:\install\IE9-Windows7-x64-enu.exe /passive"

 13: popd

install-sp1-runonce.ps1

 1: pushd hklm:\Software\Microsoft\Windows\CurrentVersion

 2: if (-not (test-path RunOnce))

 3: {

 4:     new-item -type directory RunOnce

 5: }

 6:  

 7: set-itemproperty RunOnce -Name sp1 -Value "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -executionpolicy bypass c:\install\install-r2sp1.ps1"

 8: popd

install-rs2sp1.ps1

This script uses the install share, installs the RunOnce entry for the windows update script, disables the dialog for the SP1 installer, and starts the install. It uses a registry setting available on VM clients to find the host name of the VM host machine.

 1: rite-host "Setting up network share."

 2:  

 3: # Find the VM host machine name by querying the registry

 4: $hostMachine = (get-itemproperty "HKLM:\Software\Microsoft\Virtual Machine\Guest\Parameters").HostName

 5:  

 6: # Get the network share.

 7: net use z: /d

 8: net use z: \\$hostMachine\install /u:$hostMachine\install <password>

 9:  

 10: write-host "Network share setup complete."

 11:  

 12: . c:\install\install-update-runonce.ps1

 13:  

 14: # Set security update

 15:  

 16: $securityPath = "HKCU:Software\Microsoft\Windows\CurrentVersion\Policies\Associations"

 17:  

 18: if (-not (test-path $securityPath))

 19: {

 20:    write-host "Making security changes to allow running from share."

 21:    new-item -itemtype directory -force -path $securityPath | out-null

 22:    set-itemproperty $securityPath -name ModRiskFileTypes ".exe;.msi"

 23: }

 24:  

 25: write-host "Starting install."

 26: start-process -wait Z:\r2sp1\windows6.1-KB976932-X64.exe -arg /unattend,/nodialog

install-update-runonce.ps1

 1: # Install windowsupdate into runonce

 2:  

 3: pushd hklm:\Software\Microsoft\Windows\CurrentVersion

 4: if (-not (test-path RunOnce))

 5: {

 6:     new-item -type directory RunOnce

 7: }

 8:  

 9: set-itemproperty RunOnce -Name share -Value "net use z: /d"

 10: set-itemproperty RunOnce -Name update -Value "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -executionpolicy bypass c:\install\windowsupdate.ps1"

 11:  

 12: popd

windowsupdate.ps1

This script searches for updates that need to be installed, downloads them and installs them. If a reboot is needed, it does one final reboot to finish the install. It doesn’t install the optional updates however.

 1: function Convert-ToWIAStatusValue($value)

 2: {

 3:    switch -exact ($value)

 4:    {

 5:       0   {"NotStarted"}

 6:       1   {"InProgress"}

 7:       2   {"Succeeded"}

 8:       3   {"SucceededWithErrors"}

 9:       4   {"Failed"}

 10:       5   {"Aborted"}

 11:    } 

 12:  

 13: }

 14:  

 15: $needsReboot = $false

 16: $UpdateSession = new-object -ComObject Microsoft.Update.Session

 17: $updateSearcher = $updateSession.CreateUpdateSearcher()

 18:  

 19: write-host "- Searching for updates."

 20:  

 21: $searchResult = $updateSearcher.Search("IsAssigned=1 and IsHidden=0 and IsInstalled=0")

 22:  

 23: write-host " -Found [$($searchResult.Updates.Count)] Updates"

 24: write-host

 25:  

 26: foreach ($update in $searchResult.Updates)

 27: {

 28:     # Add Update to collection

 29:     $updateCollection = new-object -comobject Microsoft.Update.UpdateColl

 30:     if ($update.EualAccepted -eq 0)

 31:     {

 32:         $update.AcceptEula()

 33:     }

 34:     $updateCollection.Add($update) | out-null

 35:  

 36:     # Download

 37:     write-host " + Downloadin update $($update.Title)"

 38:     $updatesDownloader = $updateSession.CreateUpdateDownloader()

 39:     $updatesDownloader.Updates = $updateCollection

 40:     $downloadResult = $updatesDownloader.Download()

 41:     $message = " - Download {0} " -f (Convert-ToWIAStatusValue $downloadResult.ResultCode)

 42:     write-host $message

 43:  

 44:     # Install

 45:     write-host " - Installing update"

 46:  

 47:     $updateInstaller = $updateSession.CreateUpdateInstaller()

 48:     $updateInstaller.Updates = $updateCOllection

 49:     $installResult = $updateInstaller.Install()

 50:     $message = " - Install {0}" -f (Convert-ToWIAStatusValue $InstallResult.ResultCOde)

 51:     write-host $message

 52:     write-host

 53:  

 54:     if (-not $needsReboot)

 55:     {

 56:         $needsReboot = $installResult.rebootRequired

 57:     }

 58: }

 59:  

 60: if ($needsReboot)

 61: {

 62:     restart-computer

 63: }

Note: you can find example PowerShell scripts by searching for “Windows update PowerShell” . This variation came from a MSDN forums thread.

Summary

Although my solution works, there is probably a more elegant solution. Specifically, there is probably a way to add the installers to the autounatted.xml by creating a new ISO image.  If you have suggestions for improvement, please leave a comment.

My next task is to try to deploy a domain controller and example corp deployment automatically. I would like to set it up on an internal Hyper-V network, but have not successfully gotten such a host to connect back to the VM host machine to access the install share. If you have a hint on that, I’d appreciate a comment on that as well.

Inspired by the book, I decided to try some of the techniques I had not used before to separate responsibilities in the example code. Specifically:

• Refactoring API into a Facade that I can replace
• Calling out contracts explicitly using interfaces
• Using dependency injection. I also chose to experiment with abstracting the container away using the Service Location API.
• Using a mocking framework

Implementing the Facade pattern

Implementing the Facade pattern on the API was relatively easy to do.  Treating the API in the same way as a “view” for a UI, I was able to separate out the SctpSocket and Association classes into thin wrappers over the implementation classes. I used two interfaces per class for communication between the Facade and the inner implementation.  You can see below how simple the Facade layer classes became:

public class Association : IAssociationNotification

{

    private IAssociation m_association;

    public Association(SctpSocket socket)

    {

        var socketToLayer = socket.Internal as SCTPApi.Interfaces.ISctpSocketToLayer;

        m_association = new SCTPApi.Internal.Association(

            socketToLayer,

            this);

    }

    internal Association(SctpSocket socket, IAssociation user) : this(socket)

    {

        user.SetNotification(this);

    }

    public IAsyncResult BeginConnect(

        int peerPort,

        AsyncCallback asyncCallback,

        object state)

    {

        return m_association.ConnectAsync(

            peerPort,

            state).ContinueWith(

               task =>

               {

                   if (asyncCallback != null)

                   {

                       asyncCallback(task);

                   }

                   else if (task.Exception != null)

                   {

                       // Exception is observed

                   }

               });

    }

    public void EndConnect(IAsyncResult result)

    {

        var task = (Task) result;

        task.Wait();

    }

    // ...

    public event EventHandler<EventArgs> Active;

    public event EventHandler<EventArgs> Terminated;

    public event EventHandler<MessageReceivedEventArgs> MessageReceived;

    #region IAssociationNotification

    void IAssociationNotification.Active()

    {

        var handler = this.Active;

        if (handler != null)

        {

            handler(this, EventArgs.Empty);

        }

    }

    void IAssociationNotification.Terminated()

    {

        var handler = this.Terminated;

        if (handler != null)

        {

            handler(this, EventArgs.Empty);

        }

    }

    void IAssociationNotification.MessageReceived(byte[] message)

    {

        var handler = this.MessageReceived;

        if (handler != null)

        {

            var args = new MessageReceivedEventArgs(message);

            handler(this, args);

        }

    }

    #endregion

}

After refactoring, I was quickly able to to implement another API variant on top of the implementation using the Async pattern. Not having implemented the Facade pattern before, I was glad to find it was as straight forward as I thought it would be. This refactoring should make it easy to experiment with creating a Fluent API in a later post.

Extracting interfaces and contracts

I usually don’t use interfaces in my code because it is usually a bit more difficult to version an interface than a class if you want to add a new feature.  In addition, the “Go to definition” feature does exactly what it supposed to, not “Got to implementation” which is what you want most of the time. This can be frustrating.

In any case, I decided to try using interfaces anyway. I’ve been seeing advice in various books that says in essence, don’t design your code based on the features available in your IDE. Seems like reasonable advice.

I didn’t think this would be too difficult a task, but when I got started I quickly realized that in the existing code, the contracts between the components were not as clear as I thought. It is easy to cheat when you have classes and can put something where it does not belong. Also, taking testability into account changes your thinking on how to divide up responsibilities between objects.

After trying a few different interface strategies, I settled on a coarse separation initially. Have one interface on each object that goes to the rest of the objects in the layer.  Here is an example of the interface for the SCTPApi.Internal.SctpSocket interaction with its own layer:

public interface ISctpSocketToLayer

{

    void RaiseAssociationReceived(Association association);

    IKeyManager KeyManager { get; }

    IChunkDispatcherManager PacketReceiverManager { get; }

    int ListenPort { get; }

    Task<int> SendAsync(byte[] buffer, IPEndPoint destination, object state);

    void StartReceive();

}

From the testability perspective, I also realized that using the framework networking APIs through an interface would make testing the packet processing more possible. I added an interface INetworkSocket to accomplish this:

public interface INetworkSocket

{

    void Bind(IPEndPoint endpoint);

    void Close();

    Task<int> SendToAsync(

        byte[] buffer,

        int offset,

        int count,

        SocketFlags socketFlags,

        IPEndPoint destination,

        object state);

    Task<Tuple<int,IPEndPoint>> ReceiveFromAsync(

                byte[] buffer,

                int offset,

                int length, 

                SocketFlags flags,

                IPEndPoint remoteEndpoint,

                object state);        

}

For testability, and to strictly adhere to the Single Responsibility Principle (SRP), it seems that I should do additional work separating some dictionaries with a method to do special processing logic (like tracking out of sequence packets) into separate components as well. But at this point, it feels like I would be testing a dictionary implementation several times.

Making things testable at a granular level, also seems to require exposing a lot of types and methods public that you would usually not wish to expose, especially if your product is an API.  Specifically, in my case the classes used to parse, build, and pass packet information around internally.

Some of the issues may be resolved by coming up with a better way of picking the interfaces and component responsibilities. Some hints may be gotten by looking at the Enterprise Library which uses Unity underneath. For right now, I’m not sure I’ve gotten the results I was hoping for.

Adding dependency Injection

Once you have interfaces clearly defined, it is not too difficult to add in dependency injection.  This usually involves shifting dependencies to constructor parameters. There was some unexpected work in bootstrapping in such a way API users could use the API without using dependency injection. For this I relied on some things I remembered from a description of how these issues were solved in the Enterprise Library.

The solution was to have a Library object which creates a static Current property which contains a properly configured container. I also used the Service Location container abstraction. The solution looks like this:

public static partial class Library

{

    private static IServiceLocator s_current;

    public static IServiceLocator Current

    {

        get

        {

            return s_current;

        }

        set

        {

            s_current = value;

        }

    }

}

Then to use a specific container, I added a partial class initializing that container as the default.

public static partial class Library

{

    public static IUnityContainer ConfigureUnityContainer()

    {

        UnityContainer container = new UnityContainer();

        container.RegisterInstance<Func<ISctpSocketNotification, ISctpSocket>>(

            outer => { return new SctpSocket(outer); });

        container.RegisterInstance<Func<ISctpSocket, IAssociationNotification, IAssociation>>(

            (socket, notification) =>

            {

                var socketToLayer = socket as ISctpSocketToLayer;

                return new SCTPApi.Internal.Association(socketToLayer, notification);

            });

        container.RegisterInstance<Func<ISctpSocketToLayer, IAssociationToLayer, IPacketBuilder>>(

            (socketToLayer, assocToLayer) => { return new PacketBuilder(socketToLayer, assocToLayer); });

        container.RegisterInstance<Func<IChunkDispatcherManager>>(

            () => { return new ChunkDispatcherManager(); });

        container.RegisterInstance<Func<IKeyManager>>(

            () => { return new KeyManager(); });

        container.RegisterInstance<Func<INetworkSocket>>(

            () => { return new NetworkSocket(); });

        return container;

    }

    static Library()

    {

        s_current = ConfigureUnityContainer().ToServiceLocator();

    }

}

This is not an ideal solution in that the containers available have to be compiled into the source creating a depencency on the specific container. Ideally, there would be some way of loading the container from an external DLL at runtime. This seems like a solvable problem, I chose not to go there at this time though.

I also tried to incorporate feedback from a previous post that resolving Func<xxx> instead of the interface itself would help abstract away the container. Although this solution works, I think the intent of the suggestion was that it be used in place of the Service Location abstraction not in addition to.

One interesting thing about the Service Location abstraction, is that it is a small subset of what the container can do.  This was tricky when it came time to testing. In order to customize the container for testing, I split the initialization into 1) Configuring the container and 2) Creating the service locator.  The ToServiceLocator implementation is show below:

public static class UnityContainerExtensions

{

    public static IServiceLocator ToServiceLocator(

        this IUnityContainer container)

    {

        if (container == null)

        {

            throw new ArgumentNullException("container");

        }

        UnityServiceLocator locator = new UnityServiceLocator(container);

        return locator;

    }

}

This may not be the best available solution for intercepting container initialization for testing, but I did not see an obvious way to get my container back from ServiceLocator. My test code then looked like this:

// Substitute a custom network socket.

var container = Library.ConfigureUnityContainer();

container.RegisterInstance<Func<INetworkSocket>>(

    () =>

    {

        var mns = new MemoryNetworkSocket();

        mnsList.Add(mns);

        return mns;

    });

Library.Current = container.ToServiceLocator();

Avoiding “Everything looks like a nail” issues.

While refactoring to interfaces, I realized that not all object creations should necessarily go through the container. Just because I can do it (i.e. I have a new hammer), doesn’t mean it is necessarily right. An object and its state machine, for example, work closely together so it may not make sense to put an interface between them. Likewise, objects within the same layer may not need separation yet if I am not putting an interface between them currently.

Using a Mocking Framework

The last thing I did while refactoring was trying out a mocking framework to create a test. After trying a simple example, I jumped into the deep end of the pool. I tried mocking an  interface with the following method:

public Task<Tuple<int, IPEndPoint>> ReceiveFromAsync(

            byte[] buffer,

            int offset,

            int length,

            SocketFlags flags,

            IPEndPoint remoteEndpoint,

            object state);

I worked on it a while and came up with the following, before giving up:

mockNetworkSocket.Setup(s => s.ReceiveFromAsync(

           It.IsAny<byte[]>(), 

           It.IsAny<int>(), 

           It.IsAny<int>(), 

           It.IsAny<SocketFlags>(), 

           It.IsAny<IPEndPoint>(), 

           It.IsAny<object>()))

    .Callback(() => { Trace.WriteLine("NetworkSocket.ReadFromAsync called."); })

    .Returns<byte[], int, int, SocketFlags, IPEndPoint, object>((buffer, offset, count, flags, endpoint, state) =>

         {

             return Task.Factory.StartNew<Tuple<int, IPEndPoint>>(() =>

             {

                 return new Tuple<int, IPEndPoint>(count, endpoint);

             });

         });

This is pretty close to what I needed, but it seemed like a lot of work to programmatically define a class when I could just type it in.  It seems the class is more readable when I just write the class normally:

public class MemoryNetworkSocket : INetworkSocket

{

    // ...

    public MemoryNetworkSocket()

    {

        m_outbound = new List<ArraySegment<byte>>();

        m_inbound = new List<ArraySegment<byte>>();

    }

    public Task<Tuple<int, IPEndPoint>> ReceiveFromAsync(

                byte[] buffer,

                int offset,

                int length,

                SocketFlags flags,

                IPEndPoint remoteEndpoint,

                object state)

    {

        m_tcsReceive = new TaskCompletionSource<Tuple<int, IPEndPoint>>(state);

        m_remoteEndpoint = remoteEndpoint;

        m_receiveSegment = new ArraySegment<byte>(buffer, offset, length);

        this.CheckReceiveCompleted();

        return m_tcsReceive.Task;

    }

    private void CheckReceiveCompleted()

    {

        lock (m_inbound)

        {

            if (m_tcsReceive == null)

            {

                return;

            }

            if (m_inbound.Count > 0)

            {

                int c = Math.Min(m_receiveSegment.Count, m_inbound[0].Count);

                Array.Copy(m_inbound[0].Array, 0, m_receiveSegment.Array, m_receiveSegment.Offset, c);

                m_inbound.RemoveAt(0);

                var tcs = m_tcsReceive;

                var remoteEndpoint = m_remoteEndpoint;

                m_tcsReceive = null;

                m_receiveSegment = new ArraySegment<byte>();

                m_remoteEndpoint = null;

                tcs.TrySetResult(new Tuple<int, IPEndPoint>(c, remoteEndpoint));

            }

        }

    }

    public void AddInbound(byte[] buffer, int count)

    {

        lock (m_inbound)

        {

            m_inbound.Add(new ArraySegment<byte>(buffer, 0, count));

            this.CheckReceiveCompleted();

        }

    }

    // ...

}

That’s a lot of code to generate using the API. It seems I will need more time with Mocking frameworks to understand how to apply them best.

Summary

In this post I went through a quick tour of my experiences applying a bunch of techniques I’ve not used before to my example code base.  I’m sure I’m just scratching the surface but did learn a lot in a short amount of time about where the challenges lie in these techniques.  If you feel the need to leave a comment to set me straight, please do. 

I hope that if you haven’t tried some of these techniques you consider giving them a try as well.  I think it's quite valuable to actually use the technique to validate your understanding rather than just reading about them.  In any case, this post gives a good indication how the example code has been transformed and hopefully will be helpful if someone wants to refer back in a later post.  I hope to get on to trying some more techniques now that my example code is prepped and ready.

20110926_RefactoringToInterfaces.zip

Why use an asynchronous state machine?  I’ve seen a lot of asynchronous code suddenly go synchronous and blocking once it gets to the state machine of the code.  It is difficult to implement without introducing threading safety issues and odd program states because the application is effectively in two states on different executing threads.  It would be nice to have a easy pattern for this so we can take advantage of the increasing number of cores we have available.

I recently had the opportunity to refactor the code to use the Task Parallel Library, which greatly simplified the implementation. I am using the State Pattern here, where the methods on the class indicate the events the state machine has to handle.  The signature of each state is now cleaned up significantly:

internal class AssociationState

{

     public virtual void Enter(Association association, IAssociationToLayer notification);

     public virtual void Exit(Association association, IAssociationToLayer notification);

     #region Network events

     public virtual Task<AssociationState> Receive_InitAckAsync(

         Association association,

         IAssociationToLayer notification,

         InitAckChunk chunk,

         object state);

     public virtual Task<AssociationState> Receive_AbortAsync(

         Association association,

         AbortChunk chunk,

         object state);

     public virtual Task<AssociationState> Receive_InitAsync(

         Association association,

         InitChunk chunk,

         object state);

     public virtual Task<AssociationState> Receive_CookieEchoAsync(

         Association association,

         CommonHeader common,

         CookieEchoChunk inChunk,

         object state);

     public virtual Task<AssociationState> Receive_SackAsync(

         IPacketBuilder packetBuilder,

         SackChunk chunk,

         object state);

     public virtual Task<AssociationState> Receive_CookieAckAsync(

         IAssociationToLayer notification,

         object state);

     public virtual Task<AssociationState> Receive_ShutdownAsync(

         IPacketBuilder packetBuilder,

         IAssociationToLayer notification,

         ShutdownChunk inChunk,

         object state);

     public virtual Task<AssociationState> Receive_ShutdownAckAsync(

         IPacketBuilder packetBuilder,

         IAssociationToLayer notification,

         object state);

     public virtual Task<AssociationState> Receive_ShutdownCompleteAsync(

         IAssociationToLayer notification,

         object state);

     public virtual Task<AssociationState> Receive_DataAsync(

         Association association,

         DataChunk chunk,

         object state);

     #endregion

     #region API Events

     public virtual Task<AssociationState> AssociateAsync(

         Association association,

         IAssociationToLayer notification,

         IPAddress address,

         int peerPort,

         AssociateOptions options,

         object state);

     public virtual Task<AssociationState> AbortAsync(

         IPacketBuilder packetBuilder,

         object state);

     public virtual Task<AssociationState> ShutdownAsync(

          object state);

     public virtual Task<AssociationState> SendMessageAsync(

         Association association,

         byte[] message,

         object state);

     #endregion

     #region Internal Events

     public virtual Task<AssociationState> DataQueueClearAsync(

         Association association,

         object state);

     #endregion

 }

There is much less noise in the interface of the state.  As it is a networking API, events can come from the stack, from the application, and from internal activity (such as expiration timers).

Sequencing the events

To operate reliably, the state machine needs to sequence (or at least coordinate) events to ensure that they get processed in the correct state. This is not difficult to accomplish with TPL. Tasks can be scheduled on a TaskScheduler that schedules tasks sequentially.  There happens to be one example of a custom TaskScheduler in existence already, the LimitedConcurrencyTaskScheduler available from MSDN. When you set the concurrency level to one, it sequences the tasks.

Implementations of the methods can then specify the TaskScheduler to use as shown below:

internal static class AssociationStates

{

    public static LimitedConcurrencyLevelTaskScheduler Scheduler = new LimitedConcurrencyLevelTaskScheduler(1);

    // ...

}

public override Task<AssociationState> Receive_CookieEchoAsync(

    Association association,

    CommonHeader common,

    CookieEchoChunk inChunk,

    object state)

{

    Task<AssociationState> outerTask = Task.Factory.StartNew<AssociationState>(

           (state2) =>

           {

               CookieData cookie = inChunk.CookieData;

               var peerEndpoint = new IPEndPoint(IPAddress.Loopback, common.SourcePort);

               association.PacketBuilder.SetPeerEndpoint(peerEndpoint);

               association.PacketBuilder.ProcessCookieEcho(inChunk, cookie);

               association.InitializePacketReceiver(association.PacketBuilder.Mtu, cookie.RemoteTSN);

               association.PacketReceiver.ProcessCookieEcho(inChunk, cookie);

               // Send cookie Ack

               var chunk = new CookieAckChunk();

               association.PacketBuilder.SendAsync(

                   new Chunk[] { chunk }, null/*state*/).Wait();

               return AssociationStates.Established;

           },

        state,

        default(CancellationToken),

        TaskCreationOptions.None,

        AssociationStates.Scheduler);

    return outerTask;

}

This works great for sequencing things, but you have to be careful. When you create tasks within a already executing Task it uses the same scheduler as the parent if not explicitly specified. Since the custom task scheduler schedules sequentially, this results in a deadlock unfortunately.

My way to get around the problem was to ensure the new tasks are explicitly routed to the default TaskScheduler. Here is an example from within the PacketBuilder.SendAsync method:

Task.Factory.StartNew(

    this.ProcessSendChunks, 

    default(CancellationToken), 

    TaskCreationOptions.None, 

    TaskScheduler.Default);

I might also have been able to avoid the deadlock by modifying the TaskScheduler to allow child tasks to run so they don’t block.

Summary

Refactoring to the Task Parallel Library greatly simplifies asynchronous code and allows you to see the logic of your application more clearly.  The TPL will likely be an important part of the solution to make an understandable pattern for asynchronous state machines.

Unity posting a time 363X slower than the leader was a little surprising. That usually is an indication that something isn’t right. Since the source code for Unity is available, I decided it might be worth a look.

Creating a baseline

I didn’t have a lot of fancy equipment available to dig into the problem. Just my somewhat dated laptop, and and the ability to create debug builds of Unity from source.  I figured that should be sufficient to narrow down the issue though. I ran the tests using the code and binaries I assumed were used to develop the numbers in the post:

Doing 5 runs of 100000 DI loops each. Program: singleton_loaded.

Autofac        :     253 ms.

Castle Windsor :     420 ms.

Ninject        :   1,277 ms.

Spring         :     306 ms.

StructureMap   :     489 ms.

Unity          :  76,423 ms.

The numbers were a bit larger than those from the post. I expected that because my machine is probably not as hefty as the machine it was run on. But also, the numbers may not be collected the same way as was for the post. My numbers seem to be averaged over the 5 runs.

The next step was to drop in the binary I built from the source to see what the delta from the baseline was. My source was for building version 2.1 against .Net 4.0. My build was a debug build so some slow down was inevitable:

Doing 5 runs of 100000 DI loops each. Program: singleton_loaded.

Autofac        :      225 ms.

Castle Windsor :      423 ms.

Ninject        :    1,294 ms.

Spring         :      312 ms.

StructureMap   :      488 ms.

Unity          :  115,946 ms.

Pretty scary, but it’s a place to start.

Profiling the test

I next ran the test in the profiler and found that the IsRegistered method called from the LoadedUnityRunner.Run method was using 95% of the time:

if (k.IsRegistered<IDummy>())

    k.Resolve<IDummy>().Do();

I’ve seen this pattern before – usually with a dictionary however:

if (m_dictionary.Contains(key))

{

    m_dictionary.Remove(key);

}

The problem is this pattern is duplicating work. Specifically, searching for the specified key twice.  I was curious though why the IsRegistered method was so much more expensive.  By looking at the implementation I found the problem:

public static bool IsRegistered(this IUnityContainer container, Type typeToCheck, string nameToCheck)

{

   Guard.ArgumentNotNull(container, "container");

   Guard.ArgumentNotNull(typeToCheck, "typeToCheck");

   var registration = from r in container.Registrations

                      where r.RegisteredType == typeToCheck && r.Name == nameToCheck

                      select r;

   return registration.FirstOrDefault() != null;

}

This algorithm is very likely different than the Resolve implementation. Also, since container.Registrations returns IEnumerable<> and not IQueryable<>, there probably isn’t much opportunity for the Linq expression to optimize the performance.

We are lucky in this case that the IsRegistered method really isn’t relevant to the test and can be removed. Resolve should indicate if it can’t find the item. In any case, the test was written in so that way it should always find the item.  I rewrote the the code as follows:

var r = k.Resolve<IDummy>();            

if (r != null)

{

    r.Do();     

}

This is in fact similar to how other runners were written. After inspecting all the other runners and applying similar fixes to the Spring and Windsor runners I was ready to try again.

Rerunning the test

Here are the results from the next run:

Doing 5 runs of 100000 DI loops each. Program: singleton_loaded.

Autofac        :    238 ms.

Castle Windsor :    381 ms.

Ninject        :  1,328 ms.

Spring         :    227 ms.

StructureMap   :    509 ms.

Unity          :    645 ms.

Not bad huh?  Spring performance improved by 27%, Windsor by 9.9%, and Unity by well, a lot. Although Unity is not the fastest, it is at least in the ballpark now.

Contacting the Author

At this point I was ready to contact Philip, share my results, and ask him to rerun the tests with the changes above. It turns out he had already posted an update with the changes and additional information on Unity performance. In short, IsRegistered is not to be used for other than debugging purposes.

Summary

Although my numbers aren’t directly comparable to those from the original post, I think it is safe to say that the reason the Unity numbers were off, was because the task asked of each container was not the same. Philip did a good follow up to his original post explaining the difference in the original numbers. If you are using Unity, you are probably not going to suffer the scary performance hits unless you use container.IsRegistered in the inner loop of your code.

Philip's post was a good reminder that one should always be aware of the performance behavior of the code you are building on. It provided me with a good opportunity to get hands-on with the Unity container.

As you are probably aware, in .Net 4.0 there is a new solution for solving the problem of doing work asynchronously, in the form of the Task Parallel Library.

The Task Parallel Library makes implementing the IAsyncResult pattern trivial:

private void Run(string[] args)

{

    this.EndDoNothing(this.BeginDoNothing(null, null));

}

public IAsyncResult BeginDoNothing(

    AsyncCallback userCallback,

    object state)

{

    Task t = Task.Factory.StartNew(() =>

    {

        // Do Nothing

        Thread.Sleep(1000);

    });

    return t;

}

public void EndDoNothing(IAsyncResult result)

{

    Task t = (Task)result;

    t.Wait();

}

So if coming up with new improvements to your implementation was a hobby of yours, you may be disappointed. Other than that, the Task Parallel Library creates a lot of opportunity to solve new problems.

Using the task parallel library is a big topic, and there are a lot of resources already on how to use it. What you may not have known is that there is an Async CTP available for download. In this post, I’d like to cover a few things I found interesting in the Async CTP.

Getting started with the Async CTP

To install the latest Async CTP, you will need to make sure you’ve installed the Visual Studio 2010 SP1 update on your machine first. After that, you can download the Async CTP from here or type "async ctp download” into Bing and it will get you right there.

The Async CTP has a lot of content to explore:

• White papers explaining the proposed “async” and “await” keywords for C# 5.0
• Lots of examples showing how to integrate asynchronous code with user interfaces, windows phone, and Silverlight.
• Examples demonstrating asynchronous processing of computationally intensive work, and I/O intensive work.
• Examples how to report progress and cancel asynchronous processing
• A new set of APIs and white paper to support data flow

There is definitely a lot of information and lots of code to explore.  Since I don’t do much with Silverlight, UI, and windows phone currently, the parts that caught my eye were the Whitepapers, 101 Asyncs samples, and the TPL Data flow.

101 Asyncs

The 101 Async application is very clever way to present samples.  It lets you see the code while you are running it.  Here is a screenshot of the application so you can see what I mean:

You can select the sample you want to try, get a description of what it is about, and run it without digging around in folders and opening project files.  In addition, it has syntax highlighting for the C# code.  You can crack open the code for this sample, and you can see how in just a few lines of code they were able to achieve the syntax highlighting.

Doing samples in a clear and consistent way in addition to creating the product is very difficult to do well. This is one of the first things I noticed when looking through the CTP.

TPL Data Flow

The data flow APIs and library let you design your application considering how the data should get processed through your application.  This is a somewhat different way of thinking about how to create applications.

Usually there is a lot of effort in creating an object model, dividing up responsibilities and data storage between the objects, and deciding on what the control flow should be.  How the data gets into and out of the objects, and into and out of the application doesn’t usually get quite the same emphasis, and there really isn’t a whole lot of tool support to help you unless you are working with SQL or a database perhaps.

The TPL Data flow looks very interesting and seems to allow you to express exactly how your data is to be processed in your application. Hopefully you will see a blog post or two from me in the future exploring this.

Summary

In this post, I wanted to show some interesting things to look at in the Async CTP.  I think you can probably spend quite a bit of time understanding everything that’s available, and understanding the code samples.  With the new support for asynchronous code, I think there is opportunity to find better solutions to existing problems, and free us up to solve more difficult programming problems.

The documentation for Unity is pretty detailed, so it is probably a good place to start for background information.  I’ve also included a link to the download area in the references section at the bottom of the post.

What is Unity and why would you use it?

When you start a project, the code base is usually small so it is not difficult to maintain or test. As the project gets larger, it is often the case that it gets broken down into components, and each component is created separately, and then integrated into the rest of the project. 

In the beginning, the components are self contained and testable on their own if the person doing the work was paying attention to object oriented design principles.  Even if the component was carefully constructed, at some point during integration it is usually the case that component boundaries begin to break down and components start having strong dependencies on each other.

Part of the break down of component boundaries happens because components start to inherit responsibility for configuring the dependencies with other components.  Once a component has built in knowledge on how to create and configure another component, it becomes difficult to separate the components and test them independently.

This is where Unity comes in. Not only does Unity take the responsibility for creating and configuring components, it does so in a consistent way. Components don’t have to know where their dependent components come from. In addition, Unity can automatically create the dependent components if, necessary, or use existing instances.

Inversion of Control (IoC)

Inversion of control is a term that shows up often with dependency injection. It refers to a plug in architecture where the application supplies customizations to the platform it is written on, and the platform holds the logic to use the application supplied plug-ins.

I won’t be covering much about creating plug in architectures in this post. You can probably see how this can be accomplished  using Unity by the end of the post though.

The Unity Container

In Unity, everything begins with a container. It can be created simply like this:

IUnityContainer container = new UnityContainer();

If you want to request that the container create you an instance of a class, all you need to do is call the Resolve method.  It can create instances of classes it doesn’t even know about if they have default constructors:

var c = container.Resolve<MyClass>();

c.DoSomething();

Although this is interesting, it isn’t particularly useful. You can’t do much with an object in isolation. You have to connect to other objects. Lets try the following class that requires two classes as inputs to the constructor:

public class MyClass2

{

    private DependantClass1 m_class1;

    private DependantClass2 m_class2;

    public MyClass2(DependantClass1 class1, DependantClass2 class2)

    {

        m_class1 = class1;

        m_class2 = class2;

    }

    internal void DoSomething()

    {

        Console.WriteLine("MyClass2.DoSomething");

    }

Is the following code going to work?

var c2 = container.Resolve<MyClass2>();

c2.DoSomething();

Certainly, so long as the container can find existing instances or constructors it can use to satisfy the dependencies.  Although this is a bit more interesting, a program using default arguments still isn’t that useful. You need to be able to vary the behavior of the program by providing some input.

Registering instances

At some point, you will need to pass in several objects of the same type but have different uses. You can distinguish instances of the same type by naming them explicitly.

To show this example, I have created a few classes to configure an API that I have some familiarity with.  The initialization code traditionally looks like the following:

var platformSettings = new PlatformSettings();

platformSettings.ListeningPort = 5060;

platformSettings.ProxyHost = "sip.contoso.com";

platformSettings.ProxyPort = 5060;

var platform = new Platform(platformSettings);

var endpointSettings = new EndpointSettings();

endpointSettings.AddressOfRecord = "sip:helpdesk@contoso.com";

var endpoint = new Endpoint(platform, endpointSettings);

var conversationSettings = new ConversationSettings();

conversationSettings.Subject = "This weeks news article.";

var conversation = new Conversation(endpoint, conversationSettings);

To use the container to build the object graph, I can distinguish between the string inputs by naming them with special names. The following shows one way of doing this with the Unity container:

IUnityContainer container = new UnityContainer();

container.RegisterInstance<int>("proxyPort", 5060);

container.RegisterInstance<string>("proxyHost", "sip.contoso.com");

container.RegisterInstance<int>("listeningPort", 5060);

container.RegisterInstance<string>("aor", "sip:helpdesk@contoso.com");

container.RegisterInstance<string>("subject", "This weeks news article.");

var conversation = container.Resolve<Conversation>();

Now this is starting to get useful?  I saved a few lines creating object instances and configuring objects.  You can see how I gave special names to various strings and integers so they can be plugged in the proper places.  Doing this requires adding a few attributes to your classes as shown below:

public class PlatformSettings

 {

     public PlatformSettings()

     {

     }

     [Dependency("listeningPort")]

     public int ListeningPort

     {

         get;

         set;

     }

     [Dependency("proxyHost")]

     public string ProxyHost

     {

         get;

         set;

     }

     [Dependency("proxyPort")]

     public int ProxyPort

     {

         get;

         set;

     }

 }

 public class EndpointSettings

 {

     public EndpointSettings()

     {

     }

     [Dependency("aor")]

     public string AddressOfRecord

     {

         get;

         set;

     }

 }

 public class ConversationSettings

 {

     public ConversationSettings()

     {

     }

     [OptionalDependency("subject")]

     public string Subject

     {

         get;

         set;

     }

 }

 public class Platform

 {

     private PlatformSettings m_settings;

     public Platform(PlatformSettings settings)

     {

         m_settings = settings;

     } 

     public int ListeningPort

     {

         get

         {

             return m_settings.ListeningPort;

         }

     }

     public int ProxyPort

     {

         get

         {

             return m_settings.ProxyPort;

         }

     }

     public string ProxyHost

     {

         get

         {

             return m_settings.ProxyHost;

         }

     }

 }

 public class Endpoint

 {

     private EndpointSettings m_settings;

     private Platform m_platform;

     public Endpoint(Platform platform, EndpointSettings settings)

     {

         m_platform = platform;

         m_settings = settings;

     }

     public Platform Platform

     {

         get

         {

             return m_platform;

         }

     }

     public string Uri

     {

         get

         {

             return m_settings.AddressOfRecord;

         }

     }

 }

 public class Conversation

 {

     private ConversationSettings m_settings;

     private Endpoint m_endpoint;

     public Conversation(

         Endpoint endpoint, 

         ConversationSettings settings)

     {

         m_endpoint = endpoint;

         m_settings = settings;

     }

     public Endpoint Endpoint

     {

         get

         {

             return m_endpoint;

         }

     }

     public string Subject

     {

         get

         {

             return m_settings.Subject;

         }

     }

 }

Creating objects later

Just because we have this cool container object, doesn’t mean that our need to create additional objects at a later time goes away. That’s no problem, we can just call container.Resolve<xxx>() any time we need anything.  But to do that we need to have the container available in each object.

Why not have the container inject itself for any object that needs it?  We can start by having the container register an instance of itself:

IUnityContainer container = new UnityContainer();

container.RegisterInstance<IUnityContainer>(container);

Then in each class that uses it, take the container in the constructor:

public Conversation(

     Endpoint endpoint,

     ConversationSettings settings,

     IUnityContainer container)

{

     // ...

     m_container = container;

     // ...

}

The class Conversation is now automatically configured to create other objects when it needs to.

Let’s assume that an application could choose to make an audio call or an instant messaging call. We don’t know the application’s intent when the conversation is created, so it is probably not a good idea to pre-create both and waste memory. We can use the new Lazy<> class in .Net 4.0 to lazy create the calls only when needed. The new conversation class looks like this:

public class Conversation

 {

     private ConversationSettings m_settings;

     private Endpoint m_endpoint;

     private Lazy<InstantMessagingCall> m_call;

     private IUnityContainer m_container;

     public Conversation(

         Endpoint endpoint,

         ConversationSettings settings,

         IUnityContainer container)

     {

         m_endpoint = endpoint;

         m_settings = settings;

         m_container = container;

         m_call = new Lazy<InstantMessagingCall>(

             () =>

             {

                 return m_container.Resolve<InstantMessagingCall>();

             }, true);

     }

     public Endpoint Endpoint

     {

         get

         {

             return m_endpoint;

         }

     }

     public string Subject

     {

         get

         {

             return m_settings.Subject;

         }

     }

     public InstantMessagingCall InstantMessagingCall

     {

         get

         {

             return m_call.Value;

         }

     }

 }

There is one problem with the implementation though. If I print out the instance of conversation attached to the call, and the original one I find they are different:

Console.WriteLine(

    "IMCall is hooked up to the same instance of conversation: {0}", 

    object.ReferenceEquals(conversation, call.Conversation));

The code above yields the following output:

IMCall is hooked up to the same instance of conversation: False

Why is it that the conversation has an IM Call that has a pointer to another conversation instance? It is because the container didn’t know about the existing conversation and automatically created a new one. We don’t really want that behavior. To reuse the same instance, we can add the following line to the conversation constructor:

m_container.RegisterInstance<Conversation>(this);

And then the instant messaging call will contain a pointer back to its parent as expected.

IMCall is hooked up to the same instance of conversation: True

There is another way to do this in Unity for some cases. You can specify that the lifetime of the conversation is tied to the container (i.e. singleton within the container).  Once one object is created, all references to the conversation will be resolved back to this singleton instance.

Summary

In this post I briefly did some exploration of the Unity Application Block.  Using the Unity container and dependency injection requires you to re-think how you design your code.

One great article by Chris Tavares I found was a description of how the use of dependency injection impacts library design. In library design, there are many challenges, including maintaining backward compatibility when you change the underlying structure of the code. Of particular interest, they were able to remove quite a few (200) classes from their code once Unity was integrated.

References

Download Unity 2.1

MSDN Documentation

Using Dependency Injection in Libraries

In this post, I would like to wrap up a quick tour of compiler extensibility with a few thoughts on how it compares to Text Templates.

Text Templates

It is easy to get started with text templates and the tools.  In most cases, you probably have a file that looks similar to what you want to generate that you can edit slightly to make a template file.

It is also very flexible as to what type of files it can generate. It is also very easy to visualize what you will get since the template should be pretty similar to the generated output.

Because the template engine allows you to mix C# in your template, it is very easy to handle conditional generation and generating output within a loop.

Once you have your template ready, you do need to know how to integrate the tool into your build system.  You can edit your templates in a simple text editor, but it would be nice if there was more tool support for it.  Visual Studio doesn’t seem to currently have syntax highlighting to help match the open/close tags or a way to configure the tool to pass arguments to the template engine from the UI.

Once you get going, it takes a lot of time fiddling with the templates to get the white space the way you want it.

The Text template engine does provide helpful diagnostics when your C# code in the template doesn’t compile or run as expected.

Compiler Extensibility

There is a bigger initial cost to using compiler extensibility for code generation.  You need to understand the compiler, the extensibility possibilities, and have a rough idea of how the syntax of the language will affect what you can do.  There is also time needed to understand the AST structures.

Extending the compiler involves a mix of code and simple templates used to create and manipulate the AST. The quasi quotation mechanism is apparently not quite as flexible as the template system. The result of this is that it is not always clear what code structure you will get from your compiler extension.

Once you get familiar with compiler extensibility, though, there are things you can do more easily than with text templates. One is to combine output from multiple extensions in the same class, including merging code in constructors. The second is that you have access to the entire AST, not just the part you are generating.

Summary

For most people, it is much easier to get started with text templates since you probably already have a file similar to what you want your output to be.  It is just a matter of integrating the code generation into your build system.  You can accomplish quite a bit with text templates and clever factoring of your code using partial classes.  If you are really doing advanced things, then compiler extensibility may be the way to go.

If you have worked with code generation and have some different way of doing it, or have some helpful tips to share. Please leave a comment.

Series

Start of series previous

Initially, I thought it would be straighs't forward to modify the model created earlier with text templates for use in compiler extensions.  I did run into some interesting problems.

First, I realized that the model I had created was really two integrated models. One for capturing metadata about API exception behavior, and one for modeling the IAsyncResult design pattern.  Since my goal was to model the IAsyncResult design pattern, I dropped the portions of the model related to exception handling.

Second, I quickly found that the syntax I wanted to use was not legal Boo syntax and had to change strategy.   My first syntax attempt was something like this:

operation Send(host as string, port as int, buffer as (byte), offset as int, size as int): MyClass

MyClass is the class on which the asynchronous operation would be implemented. The problem is that this is a C# notation, and the parser was not happy.

The solution

The solution that seemed more compatible with Boo turned out to be as follows:

partial class MyClass:
    operation Send(host as string, port as int, buffer as (byte), offset as int, size as int):
        m_socket as Socket
        
        def Helper(something as bool):
            print something
            
        step Connect:
            m_socket = Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp)
            m_socket.BeginConnect(m_host, m_port)
            
        step Send:
            m_socket.BeginSend(m_buffer, m_offset, m_size, SocketFlags.None)
    
        step Disconnect:
            m_socket.BeginDisconnect(true)

By nesting the operation in the class it was to be applied, the macros are now able to find the class where to add the BeginXXX and EndXXX methods by navigating to its parent in the AST.

In addition any additional member variables or methods (e.g. m_socket and Helper) can be defined within the operation and copied to the class derived from AsyncResultNoResult.

You can see some other aspects of the syntax if you look closely:

• The name of the operation appears as the method name after the operation declaration. Send in this example.
• Arguments passed into the Begin method are specified as arguments to the operation
• The userCallback and state parameters are not specified in the BeginXXX calls within the steps. These are auto generated method names and are inserted automatically for you.
•  The step macro specifies the name of the Step, which is used to construct generated callback methods.

Step macro

Because macros are evaluated innermost to outermost, the step macro will be evaluated before the operation macro.  Since the step macro can’t render all that it needs to when evaluated, it saves its information in the operation macro for when it is evaluated. Its implementation is as follows:

    macro step:
        nameExpression = step.Arguments[0] as ReferenceExpression
        
        stepName = nameExpression.Name
        
        d = operation["steps"] as List[of Step]
        if d == null:
            d = List[of Step]()
            operation["steps"] = d
             
        stepBody = step.Body
            
        d.Add(Step(stepName, stepBody))

        return null

As you can see it is very simple. It saves the name and body of the step in a collection of Steps on the operation macro for evaluation later.  It then returns null to remove itself from the AST.

Operation macro

The operation macro first creates the BeginXXX and EndXXX methods on the class it is extending. Since the operation macro is nested within the class it is extending, navigation of the AST will get the class declaration it needs to extend:

macro operation:    
                    
    # Get the class definition and class name
    if operation.ParentNode != null:
        definition = operation.ParentNode.ParentNode as ClassDefinition
    else:
        definition = null

    if definition == null:
        # There must be a better way to fail the compilation than this.
        assert false, "operation must be used within class definition."
        return null
        
    className = definition.Name
    
    # Keep the module to add classes to
    module = definition.ParentNode as Module
    
    # Extract the name of the method and arguments.
    methodExpression = operation.Arguments[0] as MethodInvocationExpression
    assert methodExpression != null
    
    nameExpression = methodExpression.Target as ReferenceExpression
    assert nameExpression != null
    
    shortName = nameExpression.Name
    resultClassName = nameExpression.Name + "AsyncResult"

    # This prevents the method name from being 'quoted'. Probably not the right node type
    # but I believe it may get turned into a string and reparsed anyway.    
    methodReference = ReferenceExpression(resultClassName)
          
    # Build a partial class to hold Begin/End methods
    
    extendedClass = [|
        partial internal class $className:
            
            def    $("Begin" + shortName)(asyncCallback as AsyncCallback, state as object):                
                result = $methodReference(asyncCallback, state, self, $(shortName))
                result.Process()
                return result
                
            def $("End" + shortName)(result as IAsyncResult):
                AsyncResultNoResult.End(result, self, $(shortName))
    |]

At this point, we have the class name and a prototype class with BeginXXX and EndXXX members. From the AST assigned to extendedClass, we can extract the members and add the arguments to the BeginXXX method. Note that the arguments are also passed to a prototype constructor of the AsyncResult derived class so it is done in the same loop. Here is the prototype for the asyncresult class:

    # Build a template for the derived AsyncResult class
    resultClass = [|
        partial internal class $resultClassName(AsyncResultNoResult):
            internal def constructor(
                asyncCallback as AsyncCallback, 
                state as object, 
                owner as object, 
                operationId as string):
                super(asyncCallback,state,owner,operationId)
                                
            internal override def Process():
                pass
    |]

Here’s the code that finds the constructor and the begin method, and appends the arguments to the prototypes of the methods:

    # Find Begin method which needs additional arguments
    beginMethod = extendedClass.Members[0] as Method
    assert beginMethod != null
    
    # Find the constructor call on right side of assignment statement
    es = beginMethod.Body.Statements[0] as ExpressionStatement
    assert es != null
    
    be = es.Expression as BinaryExpression
    assert be != null
    mie = be.Right as MethodInvocationExpression
    assert mie != null
    
    # Find the constructor which needs additional arguments
    ctor = resultClass.Members[0] as Constructor
    assert ctor != null
    
    for a in methodExpression.Arguments:
        # Add parameter to begin method
        beginMethod.Parameters.Insert(-2, ArgumentToParameterDeclaration(a))
        
        # Pass the parameter through to the constructor
        mie.Arguments.Insert(-4, ArgumentToPassedArgument(a))
        
        # Add parameter to ctor
        ctor.Parameters.Insert(-4, ArgumentToParameterDeclaration(a))
        
        # Add field to result class
        resultClass.Members.Insert(-2, ArgumentToField(a))
        
        # Add field assignment to constructor
        ctor.Body.Statements.Add(ArgumentToAssignFieldStatement(a))

In addition to adding the arguments, it also adds the fields and assignment statements for the parameters of the constructor to the fields of the result class in the constructor body.

Building the steps of the AsyncResult

Building the sequence of steps necessary to do the asynchronous operation is similar to how it was done using text templates in my earlier post. The first step is started in method called Process on the AsyncResultNoResult derived class, and no exception handling is added.

Each subsequent step starts the following step when its completed handler is called. Each completed handler needs exception handling to make sure that Complete() is called on the result class on failure. The very last step also has to call Complete() on success.

The following code achieves this:

    first = true
    processMethod = resultClass.Members["Process"] as Method
    previous as Step = null
    method as Method = null
    
    steps = operation["steps"] as List[of Step]
    for step in steps:        
        if  first:
            first = false
            for s in step.Body.Statements:
                processMethod.Body.Statements.Add(s.CloneNode())                
        else:
            ts = method.Body.Statements[1] as TryStatement
            assert ts != null
            
            for s in step.Body.Statements:
                ts.ProtectedBlock.Statements.Add(s)
                
            resultClass.Members.Add(method)
            
        previous = step
        method = step.GetCompletedMethod(false)

    method = previous.GetCompletedMethod(true)
    resultClass.Members.Add(method)

Note that steps beyond the first have a try/ensure/finally inside the handler where the statements are added. The first step does not have exception handling so the body statements are added directly to the Process() method.

Copying member and field declarations

In our DSL we allow developers to declare fields and methods that should be included on the result class. In this case it was m_socket and a method called Helper().  After the step macros were expanded, they returned null to remove themselves from the operation body. That means everything that remains should be copied to the result class as is. The following code does just that:

    # Add member statements to resultClass
    for s in operation.Body.Statements:
        decl = s as DeclarationStatement
        
        assert decl != null, "Only declarations are supported in operation body."
        
        c = [|
                class temp:
                    $(decl)
            |]
            
        for m in c.Members:
            resultClass.Members.Add(m)

Since both fields and methods both are DeclarationStatements, the code copies only DeclarationStatements to the result class for simplicity.

Finishing up

Once the result class has all its fields, methods and the arguments are added to the constructor, the class needs to be added to the model. The Begin and End methods need to be added to the hosting class, and we need to remove the operation macro from the AST by returning null. This is done by the few lines shown below:

    # Copy the Begin/End methods to the class the operation state was nested in.
    for m in extendedClass.Members:
        definition.Members.Add(m.CloneNode())
        
    # Add the result class to the module
    module.Members.Add(resultClass)
    
    # Everything has been added to the module or class
    # so remove the node.
    return null

Summary

In this post, I finally got to show you how you can use some macros in Boo to extend the compiler to support asynchronous results.  The solution takes a little while to understand but once you have extended the compiler, the compiler supports a pretty simplified syntax for implementing asynchronous operations. Making things simple should improve productivity.

Previously we evaluated using text templates with the code DOM for generating code. In the next post, I will review how text templates compare with compiler extensibility as a mechanism for generating repetitive code.

Series

Start of series previous next

20110711_CompilerExtensibilityIAsyncResult.zip