OS Family 1 is associated with Windows Server 2008 SP2, Family 2 is Windows Server 2008 R2 SP1, OS Family 3 is Windows Server 2012, and OS Family 4 is Windows Server 2012 R2 – additional details are available on our Guest OS Matrix page.  Our support lifecycle policy states that we support, at a minimum, the last 2 OS Families.

Many customers I’ve spoken to are not aware that they have some responsibility with Azure Platform as a Service (PaaS) applications to ensure they maintain an environment that contains the latest operating system security patches and that is commercially supported.   When applications are built in Visual Studio, they are built against a specific version of the Azure SDK (latest versions available here) – which has an association with the OS Family the application will ultimately support being deployed into.  Details of the OS Families and the supported SDKs are below:

Once a selection has been made for the application, some decisions need to be made on how to maintain an update-to-date operating system hosting the deployment.  A couple of options available are outlined in this blog, officially on MSDN and summarized below.

1.  Specifying a specific OS Version and Family in the application’s service configuration file (.csdef) prior to deployment

• Pros:
• Provides direct control over the version of the underlying OS provisioned to host your application
• Useful if your application has a specific binary compatibility requirement with a component of the Azure SDK or is perhaps using a 3rd party component that is supported by a specific version of Windows
• Cons:
• Requires thoughtful monitoring of Guest OS versions and family updates – The RSS Feed would be a good place to start
• Monitoring Guest OS versions may defeat some of the benefits of using PaaS in the first place – creating an administrative burden to maintain the solution
• If the underlying OS version (patch level) is not manually updated within the 60 day grace period, the application will be forcibly moved to the latest patch level to ensure the application is running on the most secure version (Family) of OS as discussed in this FAQ.

2. Specifying a specific OS Family (pinning to a version of the Windows OS) and allowing updates to be deployed automatically in the .csdef by specifying an OS version of “*”  This would be a preferred configuration to minimize

<?xml version="1.0" encoding="utf-8"?>
<ServiceConfiguration serviceName="IOTCloud" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration" 

osFamily="4" osVersion="*" schemaVersion="2013-10.2.2">

• Pros:
• Provides direct control over the version of the underlying OS provisioned to host your application
• Ensures your application is always running with the latest guest OS version (patch level) without any user/administrative intervention

• Cons:
• If your application is running with components having dependencies on the underlying OS; some security patches may impact the behavior of the operating system and require testing/validation. 
• You are still required to ensure your application is running on a supported Guest OS Family and monitoring the End of Life (EOL) dates shown on the OS Support matrix.

So, with this explanation, the next question is how to determine whether or not your deployed solution is pined to a specific version and to determine what SDK is being leveraged by deployments in your Azure subscriptions.  One option is to leverage the Azure REST APIs for service management to extract details for each deployment. 

My preferred method is to use PowerShell!  The script below (you can download here) will walk through all of the subscriptions and deployments in associated with a given organizational or user account:

foreach($subscription in Get-AzureSubscription) {

    Select-AzureSubscription -SubscriptionName $subscription.SubscriptionName

    $deployments=get-azureService | get-azureDeployment -ErrorAction Ignore | where {$_.SdkVersion -NE ""}

    $deployments | ft @{Name="SubscriptionName";Expression={$subscription.SubscriptionName}},

        ServiceName, SdkVersion, Slot,

        @{Name="osFamily";Expression={(select-xml -content $_.configuration -xpath "/ns:ServiceConfiguration/@osFamily" -namespace $namespace).node.value }},

        osVersion, Status, URL

}

Understanding the output

The script above will emit tabular details on each of your subscriptions as shown in the following image:

Details on the important fields are provided below:

• SDKVersion:
• This is the detected version of the Azure SDK your application is using – important as older SDKs may not be supported with later Guest OS Families
• Slot:
• PaaS Applications can have either a production or staging slot – each could have different versions of the OS or SDK referenced.
• OSFamily:
• If specified, the version will appear hear.  If unspecified, the OS Family will be select based on the SDK version referenced in code.  (In my case, SDK 2.2 can only run on OS Family 4; and is the version of the OS provisioned under the covers.)  The default OSFamily was previously 1 if unspecified.
• OSVersion:
• This is where you can see if your deployments are ‘pinned’ to a specific version of the OS Family.  If anything other than “*” appears hear, you would have manually maintain the version and ensure that the deployment is running on one of the latest 2 versions.

Action to take

Should you have any instances running anything less than SDK version 1.3; your deployment is likely running on an OS Family 1 instance.  You should be receiving e-mails if you have a deployment in this state sent to the service administrator of the Azure subscription in addition to seeing a banner in the Azure portal instructing you to take action prior to June 1st. 

While on paper one would hope for a direct linear increase in performance if you break a complex task across N physical nodes, there are theoretical limits involving the overhead of breaking work apart, physical properties of electrons and the size of the dataset being evaluated by a given application.  There are models one can use to identify the potential speed-up of a given task described by Amdahl’s Law involving a calculation based on the relationship of serial vs. parallel components of the computational algorithm used.  Gustafson’s law picks up on Amdahl’s law to prove that, with larger sized data sets and larger compute clusters, more work can be performed in the same amount of time as a smaller dataset on smaller clusters.

Theory aside, my colleague came up with a brilliant and simplistic solution for partitioning work across physical nodes using the Task Parallel Library (TPL) in .NET 4.5.  Check out the details of the solution and download the code sample from the October 2012 issue of MSDN magazine.

I’m not usually one to endorse products; but I have to admit that I’ve fallen into the category of users who tend to drop files ‘to-look-at-later’ onto my desktop.  [Side-note: This is an extremely in-efficient means of organization and things tend to get lost in the weeds if you don’t regularly circle back and address what you’ve placed onto your desktop.]   In looking for some tools to help bring organization to this black-hole I stumbled across a tool from StarDock called Fences that works with both Windows 7 and Vista.  This tool allows you to create ‘ring fences’ around groups of files/icons on your desktop bringing order to a world of chaos and confusion. 

All ‘fences’ active – double clicking the desktop makes me look organized and neat to the untrained eye!  (And a subsequent double-click brings my fences back.)

Elevated users could add documents, specifying a value for the Geography field without issue.  ‘Normal’ (Non-site admin) users could also specify a value and save a document to the library without error; however, the field would not be populated with a value upon saving the document.  (The managed metadata column would simply be blank.)

Issue

The hidden taxonomy list (accessible via the following url: http://<sitecollectionURL>/lists/taxonomyhiddenlist) had been modified by some automated tools ran in the customer’s environment and had removed authenticated users from this list; hence the reason why site-admins were able to properly add records but domain user’s could not.  Adding nt authority\authenticated users back into this hidden list resolved the problem.  Hopefully this saves you some pain in tracking down this problem if you’re impacted by it!

I was recently looking for a way to batch update items in a list that had “Require Content approval” (see Figure 1 below) enabled.  My requirement was that, after updating the item, it would be in an approved and not be placed into a pending status.  After doing some digging I was able to find the required command (<SetVar Name=”Cmd”>Moderate</SetVar>) that allowed this to work.  I’ve included a very simple test program below that updates the title field in a list and sets it to approved status (_ModerationStatus==0).  You can also assign other status values based on the enum SPModerationStatusType (values listed below).

public enum SPModerationStatusType
{
    // Summary:
    //     The item is displayed in public views of the list or document library.
    Approved = 0,
    //
    // Summary:
    //     The item is not displayed in public views of the list or document library.
    Denied = 1,
    //
    // Summary:
    //     The decision about displaying the item in public views of the list or document
    //     library is pending.
    Pending = 2,
    Draft = 3,
    Scheduled = 4,
}

Figure 1 – Content Approval Option

Code Sample (sans adequate error checking) that ‘batch-updates’ a single item

static void Main(string[] args)
{
            StringBuilder methodBuilder = new StringBuilder();
            string batch = string.Empty;
            string batchFormat = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>" +
              "<ows:Batch OnError=\"Return\">{0}</ows:Batch>";

            string methodFormat = "<Method ID=\"{0},Update\">" +
             "<SetList>{1}</SetList>" +
             "<SetVar Name=\"Cmd\">Moderate</SetVar>" +
             "<SetVar Name=\"ID\">{2}</SetVar>" +
             "<SetVar Name=\"urn:schemas-microsoft-com:office:office#Title\">{3}</SetVar>" +
             "<SetVar Name=\"urn:schemas-microsoft-com:office:office#_ModerationStatus\">0</SetVar>" +            
             "</Method>";

            using (SPSite site = new SPSite("http://moss.litwareinc.com/sites/aspnet"))
            {
                using (SPWeb sourceWeb = site.OpenWeb())
                {
                    try
                    {
                        int itemID = 1;
                        SPList list = sourceWeb.Lists["TestList"];
                        sourceWeb.AllowUnsafeUpdates = true;
                        string listGuid = list.ID.ToString();
                        methodFormat = string.Format(methodFormat, itemID, listGuid, itemID,"SomeValue1a1aa");
                        batch = string.Format(batchFormat, methodFormat.ToString());
                        string batchReturn = sourceWeb.ProcessBatchData(batch);
                    }
                    catch (Exception oEx)
                    {

                    }  // End exception block
                } // End using spweb
            }   // End using SPSite

} // End Main

This series assumes a basic understanding of WCF fundamentals and will provide specific usage scenarios for different WCF addresses, bindings and contracts. (Many articles already exist on this topic including one here.  Additional information may be found here and in Justin Smith’s Inside Windows® Communication Foundation.)

Streaming data with WCF
There may be instances where you need to transfer multi-megabyte files or blocks of data from your service to a consumer.  While you could have a method that returns a stream or byte array defined in your service contract, there are some potential memory implications this could introduce to your host process.  When a block of memory is allocated within a managed process that exceeds 85,000 bytes in size, the CLR performs this allocation in a special region called the Large Object Heap. (LOH – Details can be found here.)  Large, short-lived allocations in the LOH can lead to unexpected memory consumption of your host process since this heap is only collected during a Gen-2 garbage collection (that occurs less frequently).  This consumption of memory may also lead to a premature Out-of-Memory (OOM) exception within your host process since the LOH is not currently compacted (as of the 2.0 CLR) if there is not a large enough contiguous free-region within the process’s memory space to fulfill the allocation request.  These sorts of problems typically don’t arise until your place your WCF host under extreme load and sometimes, unfortunately, surface once the application is in production.  (This further underscores the importance of executing appropriate test plans!)  Luckily, there are a couple of options available to mitigate this problem.  One option is to take advantage of the streaming option available with the BasicHttpBinding, NetTcpBinding, and NetNamedPipeBinding; while another is to allow consumers to request chunks of data that is smaller than the 85,000 LOH threshold.  In this posting, we’ll look at both.

Configuration

Project setup:
The solution contains 4 projects (See Figure 1) to demonstrate large-data transfers using WCF.  The service contract is defined in the SharedInterface project that is referenced by both the client and the service implementation.  The actual service implementation is in the ServiceImpl project and the client, capable of illustrating all 3 methods of requesting data,  is in the FormsClient project.   The server project is a simple console application you can use to run the solution outside of the IDE.  (The implementation is separated from the sample host process so we can leverage the built-in WCF host within Visual Studio 2008.)

Figure 1 – Project Layout

To start with, we’ll define our service contract to have to two call-options.  One returning a stream while the other returns chunks of data.  The ‘chunking’ method is an alternative means of returning data to a consumer through a series of multiple calls where the consumer tracks its progress through the bytes of the file.  This method returns a struct with the total file size as well as the requested byte region from the file and is declared in the SharedInterface.

    [DataContract]

    public struct ChunkedDataResult

    {

        [DataMember]

        public byte[] ChunkedBlock;

        [DataMember]

        public long StreamSize;

    }

Then, we configure our bindings.  In this example, I have defined 1 binding to stream the response defined as follows and another to use the (default) buffered transfermode.  Both endpoints have relative addresses (Streaming and Buffered) that are rooted on the port and service name described by the baseaddress setting.

<system.serviceModel>

    <bindings>

      <basicHttpBinding>

        <binding name="StreamedBinding" messageEncoding="Mtom" transferMode="Streamed"  />

        <binding name="BufferedBinding" messageEncoding="Mtom" transferMode="Buffered" />

      </basicHttpBinding>

    </bindings>

    <services>

      <service behaviorConfiguration="WcfServiceLibrary1.Service1Behavior"

                      name="WCFSamples.ServiceImpl">

        <endpoint binding="basicHttpBinding" bindingConfiguration="StreamedBinding"

                            name="WCFSamples_Streamed" contract="WCFSamples.IStreamingService"

         address="Streaming">

        </endpoint>

        <endpoint address ="Buffered"

                            binding="basicHttpBinding" bindingConfiguration="BufferedBinding"

                            name="WCFSamples_Buffered" contract="WCFSamples.IStreamingService">

        </endpoint>

        <endpoint address="mex" binding="mexHttpBinding" contract="IMetadataExchange"

                            listenUriMode="Explicit">

        </endpoint>

        <host>

          <baseAddresses>

            <add baseAddress="http://localhost:8500/WCFSamples" />

          </baseAddresses>

        </host>

The key components of this binding is the transferMode option.  For our streamed tests, we set it to StreadmedResponse.  When using this transfermode (that, again, is available with the basicHttpBinding  in addition to the TCP and NamedPipe bindings) within the context of a web application hosted in either IIS, there are a couple of considerations to keep in mind:

1)     Visual Studio’s  webdev server (Cassini) cannot handle streaming over HTTP.   For testing, you need to host in a console application or through the WCF host utility.  (As I have done in the sample).
2)     IIS uses the ASP.NET maxRequestLength setting for the max length not the setting defined WCF setting.   You will need to add this to your web.config to ensure IIS is able to return the data requested.
3)    IIS cannot transfer more than 2GB of data. You'll need to self host to get around this if you need to send more data than that.
4)    To avoid timeouts, you will need to increase the receive timeouts on the client (as I have done in the FormsClient’s configuration shown below.  Note the extended recieveTimeouts in addition to both the increased maxBuffersize and maxRecievedMessageSize).

<binding name="WCFSamples_Buffered" closeTimeout="00:01:00" openTimeout="00:01:00"

     receiveTimeout="00:10:00" sendTimeout="00:01:00" allowCookies="false"

     bypassProxyOnLocal="false" hostNameComparisonMode="StrongWildcard"

     maxBufferSize="2097152" maxBufferPoolSize="524288" maxReceivedMessageSize="2097152"

     messageEncoding="Mtom" textEncoding="utf-8" transferMode="Buffered"

     useDefaultWebProxy="true">

     <readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="65536"

      maxBytesPerRead="4096" maxNameTableCharCount="16384" />

Evidence
Using the Debugging tools for Windows®; I collected some memory dumps of the host process to illustrate the allocations that occur when one of our images is sent to a consumer using the buffered transfer mode.  (While collecting a memory dump is outside the scope of this article, the command I used to collect a dump is: adplus –quiet –hang –pn server.exe.  Adplus is a VB Script  that is located in the installation directory of the Debugging tools. )  Upon inspection (See Figure 2), we can clearly see the large byte-array allocations within the LOH.

Figure 2 – Allocations that occur when transferMode = Buffered

When we examine the LOH (See figure 3) when transferMode=Streaming (or StreamedResponse); we can see that there are no large byte array allocations that occur to ‘buffer’ the data for the client.  (The object arrays are pinned regions of memory not related to the buffering of data from what I can tell.)

Figure 3 – Allocations that occur when transferMode = Streaming/StreamedResponse

With our ‘chunked’ transfer mode, we also see that these LOH allocations disappear (See figure 4) and we instead see many smaller allocations that are slightly larger than our 65,535 byte block-size we defined in our server implementation.  (The additional bytes are overhead with the WCF message.)  These blocks are controlled by the following line in our service contract implementation.  (StreamingImplementation.cs).

private const int BLOCKSIZE = 65535;

Figure 4 – Allocations that occur with our ‘chunking’ solution

While this article presented a configuration-based solution and an application-implemented solution; one could also leverage WCF’s many extensibility points to develop a channel that behaved similarly to  ‘chunked’ receive method shown above.  There’s a great starting point of such a solution here.

The code examples used for this solution can be downloaded from here.