The CVAC Pod has given me a measure of health and freedom from pain that I never thought I would experience.

Unfortunately, it is not enough to allow me to return to work.  Given my condition with Dercum's Disease, I simply cannot work full time any longer.  Microsoft has been nothing short of wonderful to me over these 14 years and especially in the last 2 years as I became more and more sick.  It is a great company filled with great people... caring people who do their very best every day to make the  world a better place and I am better for having been a part of it.

My new role is President and CEO of the Dercums Foundation.  In this role I am going to do everything I can to make sure that every last person on the face of the earth who suffers from this terrible disease can get access to the treatment that has helped me so much.  Today the CVAC pod is not an approved medical device so insurance will not pay for treatments and it is not available outside of the US.  I'm going to work as hard as I can to change this situation because for those suffering from Dercum's disease the pain is devastating.  You probably won't see me at Tech-Ed or Build any more but I'm around on Facebook and you can always find me at the Dercums Foundation.  

Thanks for your kindness to me across the years.

Your friend,

Ron Jacobs

My old position is no longer available so the first thing I have to do is to find a position.  I'm making contacts with friends in Microsoft to try and find something that will work and have some promising leads. I hope to announce something soon.  I have to say that Microsoft has been very good to me.  I am still a Microsoft employee (on leave) and I am still covered by their medical insurance which is a tremendous help to me.  For those of you outside the US you might find that strange but in the US most people get health insurance through their employer.

When I got really sick I moved to California to live with my mother who took care of me (thanks mom!).  This weekend I'm heading back to Seattle to find a place to live, meet up with some friends on the Microsoft campus for lunch on Monday, talk with some guys about a possible position and shoot an update for the This Week on Channel 9 show. 

Because of my condition, anyone who offers me a position is taking a risk that I might get sick again and be unable to work but I promise you and them (whoever they are) that as long as I am able to sit at a computer and type I will commit myself to producing great samples, great videos and blog posts and to helping you to succeed with whatever you are doing.

Thanks for your support.  I'll post an update next week.

It's Worse

Since I wrote my previous post in January my condition has become much worse. I have now reached the point that I can no longer walk my dog and I rarely leave home. I spend most of my days in severe pain and try to sleep as much as possible because it is the only time I don't feel the pain but even sleep is impossible without medication.

But there's new hope

I recently learned about a new experimental treatment for Dercums Disease. Unfortunately, this treatment is not covered by insurance so I've started a fundraiser to get the money I need to pay for this treatment. I've spoken with 2 people who had this disease and have recovered wonderfully after treatments with a CVAC machine. If you want to donate or learn more just click on the right and thanks for your support

As for the future, no one really knows what this disease will do to me in the long run.  The doctors have been very surprised to see this devastating impact so quickly.  Not only is this disease causing a great deal of pain, it is also attacking my brain making it very difficult to do even simple tasks.

Over the years I have seen farewell blog posts from many of my colleagues at Microsoft.  Most are moving on to bigger and better things.  I wish this was the case for me.  Unfortunately it is not.

I’ll never forget the first time I heard someone mention a “Race Condition”.  In my younger years when I was more worried about what people thought of me I was to embarrassed to admit that I didn’t really know what they were talking about.  I knew it had something to do with multi-threaded programs and the unpredictability of order and locking and blah, blah, blah…  To be honest it seemed like so much noise to me at the time.  After all, most of the programs I wrote never had to contend with this.

But Then…

Then when multi-core hyper-threaded systems became common place and I started working on Windows Workflow Foundation and dealing with multiple threads and race conditions became unavoidable.  My current computer is an Intel I7 6 Core hyper-threaded system which means that in Windows Task manager it shows 12 CPUs.  When I started using NCrunch to run my unit tests in parallel I found many threading bugs and race conditions in my code.

Why is this so difficult?

“With great power, comes great responsibility” – Spiderman

It just is… What can you say? Let’s just agree that your life would be simpler if you always use a request per thread model and avoid access to shared state.  You can do a lot with this type of programming but then one day you will have to cross that line and when you do… don’t hurt yourself.

Why is this a race?

Imagine each of the runners in the picture above is a CPU Core.  Each of them is working all at once.  If they don’t interfere with each other or depend on each other then no problem but what if…

What if one runner has to wait for another runner to hand off some data before it can continue?  Now you have to coordinate this race.  There are two ways to do this, locking and waiting.  Jeff Richter taught me that waiting is better than locking.  In fact, the best designs avoid locks entirely. 

Imagine a relay race where the runners approach the stadium to find a guard there who will allow one and only one runner on the track at a time.  Each runner, enters the stadium, runs their leg of the race, exits the stadium and then the other runner is allowed to enter.  Sure, it works but there are problems… performance, deadlocks, live locks, recursion and others that I won’t get into here.

Now imagine that all runners enter the stadium, each takes their place and then the race begins.  Our relay team has only one runner active and the others… wait.  They wait until a signal (in this case the baton) tells them that the other thread is done and they can now begin.

Fine but… what if there were three runners and three batons and our runner must begin when one of two of the three complete and if the other one completes the race is over with a disqualification?

Outcomes

Where you will get into trouble is when you assume that you know the outcome of a race.  You may notice that when you run your program in the debugger that the race always ends up in the order [a, b, c].  So you ship your program only to find that once in a while a customer reports an error that can only occur when the order is [c, a, b].  How can this be?  You setup a test environment and run the program a million times and it never does [c, a, b] so you close the bug as “not repro”.  But is the customer just crazy?  Or unlucky?  Or did you create a program that has a threading bug because of a race condition?

In a race, there are only three possible outcomes

1. None
2. All
3. Any

Given a point in time t

• None – none of the racers have completed the race
• All – all have completed the race
• Any – some have completed the race

If you make any other assumption about the order of the race participants you will be wrong sooner or later. 

So What?

As I’m currently thinking about API design, I’m asking this question.

“Can I make Windows Workflow Foundation easier to use?”

One reason that it is difficult is that we have created a race and asked you to solve it.  Any time you run a workflow, you are engaging in multi-threaded programming.  You found this out the first time you called WorkflowApplication.Run() only to find that Run just starts the workflow. 

How do you know when the workflow is done?  Well that depends…

Imagine that there is a race

• In lane 1 – Idle
• In lane 2 – Faulted
• In lane 3 – Completed
• In lane 4 – Canceled

Run is like the starting pistol.  Now you have to wait for the outcome of the race.  You have to write some complicated code to take into account the possible outcomes, code which most developers have never written previously.

I’d like to propose that the solution to making this easier is to use a Task to represent some of the runners in our race and exceptions to represent others.  For example, we can assume that typical outcomes would be Idle or Completed and that exceptional outcomes would be Faulted and Canceled.  Once we have taken that step Task has some helpful methods.

• If I want a task that will complete when the workflow becomes idle or completes I can call Task.WhenAny(idleTask, completedTask)
• If I want to wait with a timeout I can call Task.WaitAll() or Task.WaitAny()

Now instead of pushing the problem of waiting onto you, Workflow can give you tools that make it easy to do the right thing.

Once again, let me say I’m thinking out loud here.  This is not a promise, just a dream for now…

Happy Coding
Ron Jacobs
http://blogs.msdn.com/rjacobs
Twitter: @ronljacobs

I am a musician. Over the years I have played keyboards in many bands, traveling and performing mostly as a hobby. I’ll never forget what one of my mentors taught me. He said

“The rest is just as important as the note”

The rest is the part of the song where you don’t play. Young musicians often forget this and take great delight in trying to dazzle you with their dexterity. Put a group of young musicians together and you get a cacophony of noise. The purpose, the beauty, the soul of the music is lost in the means of its production.

When I enter a planning process in building software, I want to keep this principle in mind. The things I leave out are just as important as the things I put in.

What Problem Are We Solving?

Unlike music or art which can exist simply for their beauty, software is utilitarian. It is a tool and no tool ever existed without a problem that it seeks to solve. The first step in building a great tool is to be very clear about the problem.

• What is the problem?
• Who has this problem?
• When does this problem occur?
• What would make it better?

Why This Matters

“Well those drifter's days are past me now
I've got so much more to think about
Deadlines and commitments
What to leave in, what to leave out”

- Bob Seger – “Against the Wind”

The art of building great software begins with the art of making great choices. We can’t do everything… we need to do the best thing that we can with the time we’ve and resources we’ve got.

Taking a Risk

Right now I’m still in the stage of defining it but I’m going to be as transparent as possible with you in this process with the hope that you can give me feedback. There is some risk in this. You might disagree with my conclusions. You might wonder about the priorities that we have as a team. You might get the wrong impression… blah, blah, blah…

We live in a time of increasing secrecy in the tech industry. Companies take delight in hiding the details and then in one dramatic moment pulling back the curtain to a round of applause. I suppose that for some things that is the best approach, but you and I are in a different place. We need each other and we need to be transparent which means we run the risk of misunderstanding and that’s ok.

What Problem is Ron Solving?

I’ve been given the job of making Workflow and System.Threading.Task work well together. I began by researching the situation as it exists today. I might have come to conclusion that there is no problem or that even if there is a problem that it is so small it’s not worth addressing. Or I could have concluded that the problem is so massive that nothing can be done about it. I came to my initial conclusions by talking with some of you, my own experience and many hours prototyping and trying to work with tasks and workflows together.

What is the problem?

There are two key problems and one opportunity.

1. Problem: Windows Workflow Foundation (WF) has a large API surface composed of three key classes (WorkflowInvoker, WorkflowApplication and WorkflowServiceHost) none of which is Task enabled as defined by the Task Asynchronous Pattern.
   
2. Problem: Developers who want to implement the Task Asynchronous Pattern in their hosting code and activity code will find it difficult if not impossible to do so.
   
3. Opportunity: Because we have to introduce new API surface to solve these problems we may be able to at the same time make learning and using Workflow both easier and more powerful if we are very careful.

Problem: WF does not implement the Task Asynchronous Pattern (TAP)

Here are a few examples

As more and more developers become familiar with System.Threading.Task and the way it works with other classes in the .NET Framework they will expect that Workflow works the same way. If it did work the same way then someone who is familiar with Task will find Workflow more familiar.

Problem: Implementing TAP With Activities

Suppose you want to create an activity that access a database and you want to use SqlConnection.OpenAsync(cancellationToken). This new API is great because if the database server is down, you can cancel the Open operation on demand. Your first challenge is how to pass a CancellationToken from your workflow hosting code to the database activity. You could pass it as an in argument and require workflow authors to pass it to your activity but that doesn’t seem like the right approach.

So instead, you decide to pass it as an extension only to find that when you call workflowApp.Extensions.Add(cancellationToken) that you get an error stating that extensions must be reference types. No problem, you create a class that allows you to pass the token as an extension to the activity.

Now you’ve got it right? So you create a unit test to verify that if you try to connect to a server that does not exist and then cancel the token using CancellationTokenSource.Cancel that the activity immediately cancels the OpenAsync call and returns.

At first you think it works but then you notice that your workflow terminates with a TerminationException of AggregateException –> OperationCanceledException.

Is that ok? Should a cancel cause a workflow to terminate?

No, it should not. Workflow already has a model for cancellation of activities. Someone may use your database activity inside of a CancellationScope. If your activity faults when it is really just canceled then the cancellation handler will not be invoked.

This is just one of several problems I’ve identified with using tasks inside of activities.

Opportunity: Make Workflow Easier and More Powerful

Any time you change an existing API there are both risks and opportunities. We could try to make it better, simpler and more powerful and actually end up making it worse, more complicated or break existing things. We must be very careful about how we approach this opportunity.

When it comes to workflows and tasks here are my design principles.

1. Embrace, don’t try to hide the asynchronous nature of Workflow
• When a caller wants to wait for something to happen with a workflow (complete, idle, load, unload, cancel etc. etc.) they should receive a task that allows them to wait for that thing.
2. A caller may create 0..n tasks when using Workflow
• Users may “Fire and Forget” a workflow (0 tasks)
• Users may want to create a task to wait for a workflow to become idle in addition to a task to wait for the workflow to complete (2 tasks)
  
3. If a caller has requested a task to wait for a specific event (idle, complete, unload, etc.) and the event that they received it for
• Did happen the task will complete.
• Did not happen yet but still could happen within the lifetime of the AppDomain, the task will remain running
• Did not happen yet and will never happen within the lifetime of the AppDomain the task will be canceled.
  
4. If the workflow faults then
• The main task will be faulted, all others will cancel

Ok, ready for the risk disclaimer?

None of this might happen. Or what eventually happens might be radically different than what I am thinking right now. Blogs live forever so if you come back in 5 years and read this you might laugh… but that is a risk I’m willing to take.

Happy Coding!

Ron
http://blogs.msdn.com/rjacobs
Twitter: @ronljacobs

Can I use a LINQ Query such as the following in a Workflow?

private static void ShowQueryWithCode(IEnumerable<string> names)
{
    Console.WriteLine("Linq Query in Code - show names that start with 'R'");

    // Assuming there are no null entries in the names collection
    var query = from name in names where name.StartsWith("R") select name;

    // This is the same thing as 
    // var query = names.Where(name => name.StartsWith("R"));

    foreach (var name in query)
    {
        Console.WriteLine(name);
    }

    Console.WriteLine();
}

Answer

Yes, you can use LINQ with a Workflow in exactly the same way.

Download Sample Code

Step 1: Create a Workflow with an In Argument of type IEnumerable<string>

Here you can see that I've added the in argument

Step 2: Add a Variable for the query of type IEnumerable<string>

Before you can add a variable you need to include an activity which has variables.  In this workflow I've added a sequence.

Step 3: Assign the query the LINQ expression you want to use

You can use a method chain or query syntax.

Step 4: Iterate over the collection

In the completed workflow I used a ForEach<string> activity to iterate the list of names and write them to the console.

Summary

This example uses C# in .NET 4.5 but the same technique can be used with Visual Basic.

When you run it you can see that both methods work the same.

Happy Coding!

Ron Jacobs
http://blogs.msdn.com/rjacobs
Twitter: @ronljacobs

1. Create a portable library that could be used by almost anything including Console Apps, Windows Phone, ASP.NET, Xbox 360
2. Make it data binding friendly for targets like WPF, Silverlight and Windows 8 XAML Metro Apps by supporting property and collection changed notifications.
3. Make it efficient so that users who don’t need data binding are not penalized with a performance hit and having to reference libraries they don’t need.
4. Write Unit Tests to demonstrate that the solution works for both data binding and non-data binding users.

I was a little stumped so I decided to ask about this on StackOverflow see C# Class library collections ObservableCollection T vs Collection T and based on what the community came up with, here is my solution – can you do better?

Download Sample Code

The Base Class: Foo

If you don’t want data binding, Foo is for you.  It is a simple class.  It has a property “Num” and a collection “Strings” with a method called Populate().  You’ll notice that I created some virtual members to allow the derived class to do what it needs to do.

public class Foo
{
    private readonly Collection<string> strings = new Collection<string>();

    /// <summary>
    /// Gets the Num value
    /// </summary>
    /// <remarks>
    /// This property is virtual so that the derived Observable class can provide property changed notifications
    /// </remarks>
    public virtual int Num { get; set; }

    /// <summary>
    /// Gets the collection of strings
    /// </summary>
    public ReadOnlyCollection<string> Strings
    {
        get
        {
            return new ReadOnlyCollection<string>(this.strings);
        }
    }

    /// <summary>
    /// The collection of strings
    /// </summary>
    /// <remarks>
    /// This property is protected virtual so that the Observable class can substitue an
    /// ObservableCollection 
    /// </remarks>
    protected virtual ICollection<string> StringsCollection
    {
        get
        {
            return this.strings;
        }
    }

    /// <summary>
    /// Populates the string collection
    /// </summary>
    /// <param name="max">The maximum size for the collection</param>
    public void Populate(int max = 1000)
    {
        // If no num was specified, create a random value
        if (Num == 0)
        {
            var r = new Random();
            this.Num = r.Next(1, max);
        }

        if (Num > max)
        {
            throw new InvalidOperationException("Num is greater than max");
        }

        for (var i = 0; i < this.Num; i++)
        {
            this.StringsCollection.Add(string.Format("string {0}", i));
        }
    }
}

The Derived Class: ObservableFoo

ObservableFoo is just like Foo except that it is data binding ready because it supports INotifyPropertyChanged and uses an ObservableCollection<string> to do it’s work.  The first thing ObservableFoo must do is to get the base class Foo to use an ObservableCollection<string> so it overrides the StringCollection property.

Second, Foo.Strings is a ReadOnlyCollection<string> which won’t do for data binding.  Instead we need to return a  data binding friendly ReadOnlyObservableCollection<string>. To do this I hide the Foo.Strings property with the new modifier.

Third, I override the Num property so I can fire a PropertyChanged notification when it is set.

Finally, I place ObservableFoo into a different assembly named FooLibrary.Observable since this project must reference System.Windows and forces all those who use it to do the same.

public sealed class ObservableFoo : Foo, INotifyPropertyChanged
{
    private readonly ObservableCollection<string> strings = new ObservableCollection<string>();

    public event PropertyChangedEventHandler PropertyChanged;

    public override int Num
    {
        get
        {
            return base.Num;
        }
        set
        {
            base.Num = value;
            this.OnPropertyChanged("Num");
        }
    }

    public new ReadOnlyObservableCollection<string> Strings
    {
        get
        {
            return new ReadOnlyObservableCollection<string>(this.strings);
        }
    }

    protected override ICollection<string> StringsCollection
    {
        get
        {
            return this.strings;
        }
    }

    private void OnPropertyChanged(string propertyName)
    {
        var handler = this.PropertyChanged;
        if (handler != null)
        {
            handler(this, new PropertyChangedEventArgs(propertyName));
        }
    }
}

The Unit Tests

Testing Foo is pretty straight forward so I won't go into detail about that here, but testing ObservableFoo was quite interesting.  I have found that in most projects with Model / View / View Model (MVVM) that people neglect to test property change notifications.  Then you get subtle bugs in the UI because the property change notifications are not firing as expected.  I wanted to create a test that would verify that ObservableFoo was indeed observable so I created the FooObserver.  First a test that uses it.

/// <summary>
/// Given
/// * An ObservableFoo
/// When
/// * Populated
/// Then
/// * The number of CollectionChangedNotifications is equal to Num
/// * The number of PropertyChangedNotifications is equal to Num * 2
/// </summary>
[TestMethod]
public void ObservableFooRaisesChangeNotificationsWhenPopulated()
{
    // Arrange
    const int Expected = 100;
    const int ExpectedPropNotifications = Expected * 2;

    var foo = new ObservableFoo() { Num = Expected };

    // Create an observer to monitor the notifications
    var observer = new FooObserver(foo);

    // Act
    foo.Populate();

    // Assert
    // Each item that was added to the collection should trigger a CollectionChangedNotification
    Assert.AreEqual(Expected, observer.CollectionChangedNotifications.Count);

    // Each time an item is added 2 property changed notifications are raised
    // Count and Item[]
    Assert.AreEqual(ExpectedPropNotifications, observer.PropertyChangedNotifications.Count);
}

Here you can see that my observer simply attaches itself to the ObservableFoo and captures the notifications received so I can assert them.  The implementation is not too difficult but interesting.

public class FooObserver
{
    private readonly Collection<NotifyCollectionChangedEventArgs> collectionChangedNotifications =
        new Collection<NotifyCollectionChangedEventArgs>();

    private readonly Collection<PropertyChangedEventArgs> propertyChangedNotifications =
        new Collection<PropertyChangedEventArgs>();

    public FooObserver(ObservableFoo foo)
    {
        ((INotifyCollectionChanged)foo.Strings).CollectionChanged += this.ObservableStringsOnCollectionChanged;
        ((INotifyPropertyChanged)foo.Strings).PropertyChanged += this.OnPropertyChanged;
        foo.PropertyChanged += this.OnPropertyChanged;
    }

    public ReadOnlyCollection<NotifyCollectionChangedEventArgs> CollectionChangedNotifications
    {
        get
        {
            return new ReadOnlyCollection<NotifyCollectionChangedEventArgs>(this.collectionChangedNotifications);
        }
    }

    public ReadOnlyCollection<PropertyChangedEventArgs> PropertyChangedNotifications
    {
        get
        {
            return new ReadOnlyCollection<PropertyChangedEventArgs>(this.propertyChangedNotifications);
        }
    }

    private void ObservableStringsOnCollectionChanged(object sender, NotifyCollectionChangedEventArgs args)
    {
        this.collectionChangedNotifications.Add(args);
    }

    private void OnPropertyChanged(object sender, PropertyChangedEventArgs args)
    {
        this.propertyChangedNotifications.Add(args);
    }
}

What Now?

Frankly, I hate to ship another assembly, thankfully NuGet will make this less painful. If this works as well as it appears to, I suppose I will start shipping data binding friendly versions of some of the types found in Microsoft.Activities.Extensions. Am I on the right track? After all, I could be missing something obvious. I could just make the base classes data binding friendly and make everyone reference System.Windows. The performance penalty is not that great, but then when there is a good solution why not use it?

Happy Coding!

Ron Jacobs
blog: http://blogs.msdn.com/rjacobs
Twitter: @ronljacobs

ReSharper

A few years ago I was sitting in Glenn Block’s office when I noticed some strange additions to Visual Studio.  When I asked him about it he said “That’s ReSharper” and I was hooked.  I had to have it.

ReSharper helps me to write better code faster period.  This is my #1 must have tool.

NCrunch

This is my most recent addition to the list.  I’m a big believer in unit testing and understanding your code coverage is crucial.  Yes Visual Studio has unit testing and code coverage but… if you have a large project the cycle of build, test, check code coverage just takes way too long.

With NCrunch I just work away and it builds, tests and analyzes code coverage in the background.   Before you know it you have red, green and black dots telling you about the coverage.  If a test fails, I see it right away and can fix it even before I built the project or even saved the file!.

StyleCop

I remember the good old days when teams used to fight about things like tabs vs. spaces in code or where to put your curly braces.  StyleCop stops all the fighting – it has a set of rules for code style that for the most part I agree with.  It analyzes my code and tells me when I’m not complying with the style as I type.

If you put StyleCop together with ReSharper – now you can quickly format your code.  Got to have this tool.

CleanProject

Have you ever wanted to email a project to a colleague or upload it to some website only to find that with all the bin / obj / ReSharper / etc. folder full of stuff that the zip file was huge?  Or maybe you open a project from a website or a friend and Visual Studio informs you that the project was under source control and the server is not available.

I created CleanProject as a tool to remove crap that I don’t care about from the zip file and to remove source control bindings.  Seems that several thousand of you agree that CleanProject is pretty useful.

.NET Reflector

Yes, I work at Microsoft and yes I have access to all the .NET Framework source (you do too by the way) but often I find that it is faster and easier for me to analyze what some piece of the .NET framework is doing by simply going to .NET Reflector.  They recently release a great extension for Visual Studio 2012 which makes it even better.  I can’t tell you how many times where understanding the underlying class has helped me to solve a difficult bug.