When calling one of the Cognitive Services APIs you may receive an HTTP 401 Access Denied error with a message “Access denied due to invalid subscription key. Make sure to provide a valid key for an active subscription.”

Here are the most common causes and solutions for this issue.

Using the wrong API account key in the Ocp-Apim-Subscription-Key header

When first starting out with Azure or with Cognitive Services it can be confusing to understand which key to use in your application.  It is common to see developers use the Azure subscription key, which is a GUID with a format like 4b2714d2-3d2e-4b96-ad95-677f764e7b39, instead of the API account key.

Make sure you are using one of the 2 API account keys which look like 18de248475134eb49ae4a4e94b93461c.  This key can be found in the Azure Management portal by going to a Cognitive Service and selecting Keys under Resource Management.  There is a walkthrough for creating an account and finding the keys at https://docs.microsoft.com/en-us/azure/cognitive-services/cognitive-services-apis-create-account.

You can also call Get-AzureRmCognitiveServicesAccountKey from Powershell to get the correct account key.  I use this script to print out all Cognitive Services accounts and keys in my subscription:

PS C:\Users\kwill> $formatstring = "{0,-29} {1,-19} {2,34} {3,34} {4, 5}"; $formatstring -f "Account", "Type", "Key1", "Key2", "SKU"; Get-AzureRmCognitiveServicesAccount |
ForEach-Object {$account = $_; $keys = Get-AzureRmCognitiveServicesAccountKey -ResourceGroupName $_.ResourceGroupName -Name $account.AccountName; $formatstring -f $account.Ac
countName, $account.AccountType, $keys.Key1, $keys.Key2, $account.sku.Name}
Account                       Type                                              Key1                               Key2   SKU
kwillAcademic                 Academic              21b2b86e7b78abcde282bb5d602f8867   6f77ba002185abcdeeed04127da5e70d    S0
kwillBingSearchS5             Bing.Search           c7d85f71945abcdebdff2303c3c4d285   8f0de6ed747446abcdee3db939b401c0    S5
kwillBingSpeech               Bing.Speech           98de24847513abcdeae4a4e94b93461d   ad2269940791abcdea36b7389167c838    S0
kwillBingSpellCheck           Bing.SpellCheck       a8007a55a725abcde80ffd96d26664d3   6c118bba971d4abcde7d8d71e12fee6a    S1

...

Using an account key for the wrong API type

Each Cognitive Services API has its own account key, and you can’t use an account key for one API when making a call to another API.  Generally this is a pretty easy concept and not common for developers to run into, but for the Translator API this can be confusing because there is a Translator Text API and a Translator Speech API.  They both use the same API endpoint, but the two keys cannot be used interchangeably. 

You can confirm that you are using the correct API type that matches the API that you are calling by checking the API type and Endpoint in the Azure management portal.  Make sure that the URL you are using for the API call matches the key for the API account.

Using the incorrect regional endpoint

Most of the Cognitive Services APIs are region specific, which means that during API account creation you select which region you want to create the account in.  These APIs have region specific endpoints such as westus.api.cognitive.microsoft.com or eastus2.api.cognitive.microsoft.com, and an API key for an account created in one region will only work using the endpoint for that specific region.  This means that if you create an API account in West US you will not be able to call the eastus2.api.cognitive.microsoft.com API endpoint.

You can verify the region and endpoint in the Azure management portal.

This also applies to the web based testing console.  The API documentation will show the different regions available and links to the different regional testing consoles.  Here is the Face API documentation as an example:

There are a few additional scenarios where issues with the regional API keys are commonly seen:

• Applications that use Cognitive Service APIs.  If you are using a packaged application that asks you for an API key (for example, a Translator application using the Translator API) the application *should* also ask for the endpoint or region.  However, many applications were built before the Cognitive Services APIs had multiple region specific endpoints so they may not provide an option to configure an endpoint.  In this case the app is usually built using the West US region (westus.api.cognitive.microsoft.com).
• If you use the client library SDKs you will often instantiate the client using code like:

      IFaceServiceClient faceServiceClient = new FaceServiceClient("key");

  This will use the default West US endpoint, which will fail for accounts in other regions.  To resolve, pass in the optional endpoint parameter like this:

      IFaceServiceClient faceServiceClient = new FaceServiceClient(key,"https://westeurope.api.cognitive.microsoft.com/face/v1.0");   

• The LUIS.AI portal also has regional endpoints, and creating an app in the LUIS.AI portal must be done in the same region as the LUIS API account that was created in the Azure management portal.  For example, if you create a LUIS API account in West Europe you must use the https://eu.luis.ai portal to create the LUIS application.  In this scenario you may also get the error "Bad Argument, Invalid Subscription Key" from the LUIS.AI portal.

Using the incorrect version endpoint

Some APIs, such as Bing Search, have multiple versions.  As we release new versions of the APIs we release them as free APIs that can be obtained from https://azure.microsoft.com/en-us/try/cognitive-services/, but these new versions may not yet be available in Azure, and calling them with your Azure API account key will fail with a 401.

As of this blog post, the Bing Search API accounts created via Azure Cognitive Services use the v5.0 version of the API with the endpoint https://api.cognitive.microsoft.com/bing/v5.0.  We have also released a preview version of the v7.0 API with the endpoint https://api.cognitive.microsoft.com/bing/v7.0.  If you call the v7.0 endpoint with an Azure account key it will fail.  You can verify the correct endpoint in the Azure management portal:

Using the LUIS programmatic key instead of the API account key

There are three different API keys that can be used with LUIS applications:

1. Programmatic Key, which is used to automate the management of LUIS applications created via the LUIS.AI portal.  This is created and accessed via the LUIS.AI portal.
2. API Endpoint Key, which is used by a LUIS application to call into the LUIS API for processing.  This is created and accessed via the Azure Management portal in the Azure Cognitive Services LUIS account blade.
3. External Keys, which are used by a LUIS application to provide additional functionality such as Spell Check.

Further documentation about these 3 keys can be found at https://docs.microsoft.com/en-us/azure/cognitive-services/LUIS/manage-keys.

Each of these API keys are specific to their use cases and you will get a 401 if you try to use an API key against the wrong endpoint. 

• You cannot use the Programmatic Key which you get from the LUIS.AI portal when making a LUIS API call to https://westus.api.cognitive.microsoft.com/luis/v1/application such as Analyze Intents. 
• You cannot use your API account key that you get from Azure when making a LUIS Programmatic call to https://westus.api.cognitive.microsoft.com/luis/api/v2.0/apps/ such as Add Application.

Calling an API immediately after creating an account or regenerating keys

The API keys can take a few minutes to be provisioned and propagated within the various subsystems that make up the Cognitive Services service.  If you try to call an API before the keys have propagated you will get a 401 error.  Wait up to 10 minutes and then try making the API call again.  This is also noted in the Account Keys blade in the Azure Management portal:

Trial API Keys

The free trial API keys have 30 day expiration dates, and the same restrictions for region and version.  If you are using the trial keys you can go to https://azure.microsoft.com/en-us/try/cognitive-services/my-apis/ to manage your API keys (if you are not already logged in then just click one of the ‘Create’ buttons and you can go through the wizard to login and see your existing API keys), and you will also see the expiration date and endpoint.

Troubleshooting

If you are troubleshooting a 401 error, the best place to start is the API testing console.  Usually the testing console requires minimal input (often just the API key) so it is ideal to help isolate problems that may be caused by something like custom code or machine specific issues.

You can get to the testing console by going to one of your Cognitive Services API accounts in the Azure management portal and clicking Quick start and then API reference:

You can also get to the testing console by going to the main Cognitive Services documentation page at https://docs.microsoft.com/en-us/azure/cognitive-services/, selecting a service under the “Services/APIs” list on the left, and then going to the Reference –> API Reference section.

Templates

Initial Setup

Existing Virtual Network

Static Public IP Address

PS C:\Users\kwill> New-AzureRmPublicIpAddress -Name staticIP1 -ResourceGroupName ExistingRG -Location westus -AllocationMethod Static -DomainNameLabel sfnetworking

Name                     : staticIP1
ResourceGroupName        : ExistingRG
Location                 : westus
Id                       : /subscriptions/1237f4d2-3dce-1236-ad95-123f764e7123/resourceGroups/ExistingRG/providers/Micr
                           osoft.Network/publicIPAddresses/staticIP1
Etag                     : W/"fc8b0c77-1f84-455d-9930-0404ebba1b64"
ResourceGuid             : 77c26c06-c0ae-496c-9231-b1a114e08824
ProvisioningState        : Succeeded
Tags                     :
PublicIpAllocationMethod : Static
IpAddress                : 40.83.182.110
PublicIpAddressVersion   : IPv4
IdleTimeoutInMinutes     : 4
IpConfiguration          : null
DnsSettings              : {
                             "DomainNameLabel": "sfnetworking",
                             "Fqdn": "sfnetworking.westus.cloudapp.azure.com"
                           }

Service Fabric Template

Existing Virtual Network / Subnet

        "subnet0Name": {
            "type": "string",
            "defaultValue": "default"
        },
        "existingVNetRGName": {
            "type": "string",
            "defaultValue": "ExistingRG"
        },
        "existingVNetName": {
            "type": "string",
            "defaultValue": "ExistingRG-vnet"
        },
        /*
        "subnet0Name": {
            "type": "string",
            "defaultValue": "Subnet-0"
        },
        "subnet0Prefix": {
            "type": "string",
            "defaultValue": "10.0.0.0/24"
        },*/

        /*old "vnetID": "[resourceId('Microsoft.Network/virtualNetworks',parameters('virtualNetworkName'))]",*/
        "vnetID": "[concat('/subscriptions/', subscription().subscriptionId, '/resourceGroups/', parameters('existingVNetRGName'), '/providers/Microsoft.Network/virtualNetworks/', parameters('existingVNetName'))]",

New-AzureRmResourceGroup -Name sfnetworkingexistingvnet -Location westus

New-AzureRmResourceGroupDeployment -Name deployment -ResourceGroupName sfnetworkingexistingvnet -TemplateFile C:\SFSamples\Final\template_existingvnet.json

Static Public IP Address

        "existingStaticIPResourceGroup": {
            "type": "string"
        },
        "existingStaticIPName": {
            "type": "string"
        },
        "existingStaticIPDnsFQDN": {
            "type": "string"
        }

        "existingStaticIP": "[concat('/subscriptions/', subscription().subscriptionId, '/resourceGroups/', parameters('existingStaticIPResourceGroup'), '/providers/Microsoft.Network/publicIPAddresses/', parameters('existingStaticIPName'))]",

                "frontendIPConfigurations": [
                    {
                        "name": "LoadBalancerIPConfig",
                        "properties": {
                            "publicIPAddress": {
                                /*"id": "[resourceId('Microsoft.Network/publicIPAddresses',concat(parameters('lbIPName'),'-','0'))]"*/
                                "id": "[variables('existingStaticIP')]"
                            }
                        }
                    }
                ],

                "fabricSettings": [],
                /*"managementEndpoint": "[concat('https://',reference(concat(parameters('lbIPName'),'-','0')).dnsSettings.fqdn,':',parameters('nt0fabricHttpGatewayPort'))]",*/
                "managementEndpoint": "[concat('https://',parameters('existingStaticIPDnsFQDN'),':',parameters('nt0fabricHttpGatewayPort'))]",

Internal Only Load Balancer

        "internalLBAddress": {
            "type": "string",
            "defaultValue": "10.0.0.250"
        }

            "apiVersion": "[variables('lbApiVersion')]",
            "type": "Microsoft.Network/loadBalancers",
            "name": "[concat('LB','-', parameters('clusterName'),'-',parameters('vmNodeType0Name'))]",
            "location": "[parameters('computeLocation')]",
            "dependsOn": [
                /*"[concat('Microsoft.Network/publicIPAddresses/',concat(parameters('lbIPName'),'-','0'))]"*/
                "[concat('Microsoft.Network/virtualNetworks/',parameters('virtualNetworkName'))]"
            ],

                "frontendIPConfigurations": [
                    {
                        "name": "LoadBalancerIPConfig",
                        "properties": {
                            /*
                            "publicIPAddress": {
                                "id": "[resourceId('Microsoft.Network/publicIPAddresses',concat(parameters('lbIPName'),'-','0'))]"
                            } */
                            "subnet" :{
                                "id": "[variables('subnet0Ref')]"
                            },                
                            "privateIPAddress": "[parameters('internalLBAddress')]",
                            "privateIPAllocationMethod": "Static"
                        }
                    }
                ],

                "fabricSettings": [],
                /*"managementEndpoint": "[concat('https://',reference(concat(parameters('lbIPName'),'-','0')).dnsSettings.fqdn,':',parameters('nt0fabricHttpGatewayPort'))]",*/
                "managementEndpoint": "[concat('https://',reference(variables('lbID0')).frontEndIPConfigurations[0].properties.privateIPAddress,':',parameters('nt0fabricHttpGatewayPort'))]",

New-AzureRmResourceGroup -Name sfnetworkinginternallb -Location westus

New-AzureRmResourceGroupDeployment -Name deployment -ResourceGroupName sfnetworkinginternallb -TemplateFile C:\SFSamples\Final\template_internalonlyLB.json

Internal and External Load Balancer

        "internalLBAddress": {
            "type": "string",
            "defaultValue": "10.0.0.250"
        }

        "loadBalancedAppPort1": {
            "type": "int",
            "defaultValue": 80,
            "metadata": {
                "description": "Input endpoint1 for the application to use. Replace it with what your application uses"
            }
        },

        /* Add internal LB networking variables */
        "lbID0-Int": "[resourceId('Microsoft.Network/loadBalancers', concat('LB','-', parameters('clusterName'),'-',parameters('vmNodeType0Name'), '-Internal'))]",
        "lbIPConfig0-Int": "[concat(variables('lbID0-Int'),'/frontendIPConfigurations/LoadBalancerIPConfig')]",
        "lbPoolID0-Int": "[concat(variables('lbID0-Int'),'/backendAddressPools/LoadBalancerBEAddressPool')]",
        "lbProbeID0-Int": "[concat(variables('lbID0-Int'),'/probes/FabricGatewayProbe')]",
        "lbHttpProbeID0-Int": "[concat(variables('lbID0-Int'),'/probes/FabricHttpGatewayProbe')]",
        "lbNatPoolID0-Int": "[concat(variables('lbID0-Int'),'/inboundNatPools/LoadBalancerBEAddressNatPool')]",
        /* internal LB networking variables end */

        /* Add a second load balancer, configured with a static privateIPAddress and the "-Int" LB variables */
        {
            "apiVersion": "[variables('lbApiVersion')]",
            "type": "Microsoft.Network/loadBalancers",
            /* Add '-Internal' to name */
            "name": "[concat('LB','-', parameters('clusterName'),'-',parameters('vmNodeType0Name'), '-Internal')]",
            "location": "[parameters('computeLocation')]",
            "dependsOn": [
                /* Remove public IP dependsOn, add vnet dependsOn
                "[concat('Microsoft.Network/publicIPAddresses/',concat(parameters('lbIPName'),'-','0'))]"
                */
                "[concat('Microsoft.Network/virtualNetworks/',parameters('virtualNetworkName'))]"
            ],
            "properties": {
                "frontendIPConfigurations": [
                    {
                        "name": "LoadBalancerIPConfig",
                        "properties": {
                            /* Switch from Public to Private IP address
                            /*
                            "publicIPAddress": {
                                "id": "[resourceId('Microsoft.Network/publicIPAddresses',concat(parameters('lbIPName'),'-','0'))]"
                            }
                            */
                            "subnet" :{
                                "id": "[variables('subnet0Ref')]"
                            },                
                            "privateIPAddress": "[parameters('internalLBAddress')]",
                            "privateIPAllocationMethod": "Static"
                        }
                    }
                ],
                "backendAddressPools": [
                    {
                        "name": "LoadBalancerBEAddressPool",
                        "properties": {}
                    }
                ],
                "loadBalancingRules": [
                    /* Add the AppPort rule, making sure to reference the "-Int" versions of the backendAddressPool, frontendIPConfiguration, and probe variables */
                    {
                        "name": "AppPortLBRule1",
                        "properties": {
                            "backendAddressPool": {
                                "id": "[variables('lbPoolID0-Int')]"
                            },
                            "backendPort": "[parameters('loadBalancedAppPort1')]",
                            "enableFloatingIP": "false",
                            "frontendIPConfiguration": {
                                "id": "[variables('lbIPConfig0-Int')]"
                            },
                            "frontendPort": "[parameters('loadBalancedAppPort1')]",
                            "idleTimeoutInMinutes": "5",
                            "probe": {
                                "id": "[concat(variables('lbID0-Int'),'/probes/AppPortProbe1')]"
                            },
                            "protocol": "tcp"
                        }
                    }
                ],
                "probes": [
                   /* Add the probe for the app port */
                   {
                        "name": "AppPortProbe1",
                        "properties": {
                            "intervalInSeconds": 5,
                            "numberOfProbes": 2,
                            "port": "[parameters('loadBalancedAppPort1')]",
                            "protocol": "tcp"
                        }
                    }
                ],
                "inboundNatPools": [
                ]
            },
            "tags": {
                "resourceType": "Service Fabric",
                "clusterName": "[parameters('clusterName')]"
            }
        },

                                                "loadBalancerBackendAddressPools": [
                                                    {
                                                        "id": "[variables('lbPoolID0')]"
                                                    },
                                                    {
                                                        /* Add internal BE pool */
                                                        "id": "[variables('lbPoolID0-Int')]"
                                                    }
                                                ],

New-AzureRmResourceGroup -Name sfnetworkinginternalexternallb -Location westus

New-AzureRmResourceGroupDeployment -Name deployment -ResourceGroupName sfnetworkinginternalexternallb -TemplateFile C:\SFSamples\Final\template_internalexternalLB.json

• In place upgrade of the cloud service
• Config change of the cloud service that changes the environment, such as adding a new certificate that the guest agent has to install
• Calling RoleEnvironment.RequestRecycle from within the VM
• Stop/Start management operation
• Setting e.Cancel=true in the RoleEnvironment_Changing event
• The role host process exiting the RoleEntryPoint_Run method
• The role host process crashing

• A new version of the Azure Guest Agent
• Host or Guest OS Updates
• Reboot or Reimage management operation
• A hardware issue on the physical server causing service healing and node movement
• A communication failure between the Guest Agent and the Host Agent that lasts longer than 10 minutes
• Unexpected host reboots such as powercycles or bugchecks
• Planned maintenance or hardware decommissioning which will cause Azure to proactively move VMs to new servers (you will receive advance notification emails)

With the older version of the storage client library (version 1.7) you could sign HttpWebRequests using the SignRequestLite function, and there were several examples on the web of how to do this.  The SignRequestLite function has been removed from the 2.0+ versions of the storage client library and so far I have not seen any examples of using the new signing functions.

The new SCL uses Microsoft.WindowsAzure.Storage.Auth.Protocol.SharedKeyAuthenticationHandler to sign the request, and Microsoft.WindowsAzure.Storage.Core.Auth.SharedKeyTableCanonicalizer (for table storage requests) or Microsoft.WindowsAzure.Storage.Core.Auth.SharedKeyCanonicalizer (for blob and queue storage requests) to create the canonicalizer.  In the simplest form, this is the code (using the blob canonicalizer) to sign an httpWebRequest object:

Microsoft.WindowsAzure.Storage.Auth.StorageCredentials credentials = new Microsoft.WindowsAzure.Storage.Auth.StorageCredentials(storageAccountName, storageAccountKey);
Microsoft.WindowsAzure.Storage.Auth.Protocol.SharedKeyAuthenticationHandler auth;
Microsoft.WindowsAzure.Storage.Core.Auth.SharedKeyCanonicalizer canonicalizer = Microsoft.WindowsAzure.Storage.Core.Auth.SharedKeyCanonicalizer.Instance;
auth = new Microsoft.WindowsAzure.Storage.Auth.Protocol.SharedKeyAuthenticationHandler(canonicalizer, credentials, storageAccountName);
auth.SignRequest(httpWebRequest, null);

For a more full featured example (you can largely ignore the MakeRequest function since it is just a generic HTTP request/response handler):

using System.Net;
using Microsoft.WindowsAzure.Storage.Auth;
using Microsoft.WindowsAzure.Storage.Auth.Protocol;
 
        private HttpWebRequest SignRequest(HttpWebRequest request, bool isTable = false)
        {
            // Create a StorageCredentials object with account name and key
            string storageAccountName = "kwillstorage2";
            string storageAccountKey = "S4RUcCvGKBPoFvhyJ0p6Wu0ciJnPTn5+b5MgU5olWhqGABAfvFhMFCbOBSeDgL9VF27TFrYzCQnRHYkbgJgxxg==";
            Microsoft.WindowsAzure.Storage.Auth.StorageCredentials credentials = new Microsoft.WindowsAzure.Storage.Auth.StorageCredentials(storageAccountName, storageAccountKey);
 
            // Create the SharedKeyAuthenticationHandler which is used to sign the HttpWebRequest
            Microsoft.WindowsAzure.Storage.Auth.Protocol.SharedKeyAuthenticationHandler auth;
            if (isTable)
            {
                // Tables use SharedKeyTableCanonicalizer along with storage credentials and storage account name
                Microsoft.WindowsAzure.Storage.Core.Auth.SharedKeyTableCanonicalizer canonicalizertable = Microsoft.WindowsAzure.Storage.Core.Auth.SharedKeyTableCanonicalizer.Instance;
                auth = new Microsoft.WindowsAzure.Storage.Auth.Protocol.SharedKeyAuthenticationHandler(canonicalizertable, credentials, storageAccountName);
                if (request.Headers["MaxDataServiceVersion"] == null)
                {
                    request.Headers.Add("MaxDataServiceVersion", "2.0;NetFx");
                }
            }
            else
            {
                // Blobs and Queues use SharedKeyCanonicalizer
                Microsoft.WindowsAzure.Storage.Core.Auth.SharedKeyCanonicalizer canonicalizer = Microsoft.WindowsAzure.Storage.Core.Auth.SharedKeyCanonicalizer.Instance;
                auth = new Microsoft.WindowsAzure.Storage.Auth.Protocol.SharedKeyAuthenticationHandler(canonicalizer, credentials, storageAccountName);
            }
 
            // Sign the request which will add the Authorization header
            auth.SignRequest(request, null);
            return request;
        }
 
        private void button1_Click(object sender, EventArgs e)
        {
            HttpWebRequest requestBlob, requestTable;
 
            requestBlob = (HttpWebRequest)HttpWebRequest.Create("https://kwillstorage2.blob.core.windows.net/vsdeploy?restype=container&comp=list");
            requestBlob.Method = "GET";
            requestBlob.Headers.Add("x-ms-version", "2014-02-14");
            SignRequest(requestBlob);
            MakeRequest(requestBlob);
 
            requestTable = (HttpWebRequest)HttpWebRequest.Create("https://kwillstorage2.table.core.windows.net/testtable");
            requestTable.Method = "GET";
            requestTable.Headers.Add("x-ms-version", "2014-02-14");
            SignRequest(requestTable, true);
            MakeRequest(requestTable);
        }
 
        private void MakeRequest(HttpWebRequest request)
        {
            HttpWebResponse response = null;
            System.IO.Stream receiveStream;
            System.IO.StreamReader readStream;
            Encoding encode;
            try
            {
                response = (HttpWebResponse)request.GetResponse();
            }
            catch (WebException ex)
            {
                Console.WriteLine(request.Headers.ToString());
                Console.WriteLine(ex.Message + Environment.NewLine + Environment.NewLine + ex.Response.Headers.ToString());
                try
                {
                    receiveStream = ex.Response.GetResponseStream();
                    encode = System.Text.Encoding.GetEncoding("utf-8");
                    // Pipes the stream to a higher level stream reader with the required encoding format. 
                    readStream = new System.IO.StreamReader(receiveStream, encode);
                    Console.WriteLine(readStream.ReadToEnd());

                    // Releases the resources of the response.
                    response.Close();
                    // Releases the resources of the Stream.
                    readStream.Close();
                }
                catch
                {
                }
                return;
            }
 
            Console.WriteLine(request.Method + " " + request.RequestUri + " " + request.ProtocolVersion + Environment.NewLine + Environment.NewLine);
            Console.WriteLine(request.Headers.ToString());
            Console.WriteLine((int)response.StatusCode + " - " + response.StatusDescription + Environment.NewLine + Environment.NewLine);
            Console.WriteLine(response.Headers + Environment.NewLine + Environment.NewLine);
 
            receiveStream = response.GetResponseStream();
            encode = System.Text.Encoding.GetEncoding("utf-8");
            // Pipes the stream to a higher level stream reader with the required encoding format. 
            readStream = new System.IO.StreamReader(receiveStream, encode);
            Console.WriteLine(readStream.ReadToEnd());
 
            // Releases the resources of the response.
            response.Close();
            // Releases the resources of the Stream.
            readStream.Close();
        }

“My certificate provider recently switched to only providing SHA2/SHA256 certificates because SHA-1 certificates are no longer safe.  But Azure only supports SHA1 certificates!  https://msdn.microsoft.com/library/azure/gg465718.aspx says ‘The only thumbprint algorithm currently supported is sha1’”.

Lately I have been seeing this issue more often due to some larger cert providers recently making this change.  The entire industry has been deprecating SHA-1 certificates for a while and Chrome has recently started showing warnings in the browser.  Some references:

• https://www.digicert.com/sha-2-ssl-certificates.htm
• https://knowledge.geotrust.com/support/knowledge-base/index?page=content&id=AD927&actp=LIST&viewlocale=en_US
• https://blog.chromium.org/2014/09/gradually-sunsetting-sha-1.html
• https://blogs.technet.com/b/pki/archive/2013/11/12/sha1-deprecation-policy.aspx
• etc…

Signing algorithm vs. Thumbprint algorithm

The issue stems from confusion between the two types of algorithms used by certificates.

• Signing algorithm.  This is the algorithm used to actually sign the certificate and this is what makes the certificate secure (or in the case of SHA-1, less secure).  The signing algorithm and resulting signature is specified by the certificate authority when it creates the cert and is built into the cert itself.  This algorithm is where SHA1 is being deprecated.  Azure doesn't know or care what this algorithm is.

• Thumbprint algorithm.  This algorithm is used to generate a thumbprint in order to uniquely identify and find a certificate.  This algorithm and value is not built into the certificate but is instead calculated whenever a cert lookup is done.  Multiple thumbprints can be generated using different algorithms all from the same certificate data.  The thumbprint has nothing to do with certificate security since it is just used to identify/find the cert within the cert store.  Windows, .NET, and Azure all use SHA1 algorithm for the thumbprint algorithm, and SHA1 is the only algorithm allowed in the ServiceConfiguration.cscfg file:

<Certificates>
  <Certificate name="Certificate1" thumbprint="69BF333452DAA85E462E33B138F3B65842C8B428" thumbprintAlgorithm="sha1" />
< /Certificates>

Solution

You can use your SHA2/SHA256 signed certificate in Azure, you just have to specify an SHA1 thumbprint.  Your certificate provider should be able to provide you with an SHA1 thumbprint, but it is relatively straightforward to find or calculate the SHA1 thumbprint on your own.  Here are a few options:

1. The easiest option is to simply open the certificate in the Certificate Manager on any Windows OS.  Windows will display the SHA1 thumbprint in the certificate properties window.
2. On a Windows OS you can run ‘certutil –store my’  (replace my with whatever store your cert is in).
3. In Powershell you can call System.Security.Cryptography.SHA1CNG.ComputeHash per https://ig2600.blogspot.com/2010/01/how-do-you-thumbprint-certificate.html
4. There are a few other options including .NET code, openssl, and Apache Commons Codec at https://stackoverflow.com/questions/1270703/how-to-retrieve-compute-an-x509-certificates-thumbprint-in-java.

Thank you to Morgan Simonsen for his excellent blog post Understanding X.509 digital certificate thumbprints which details the different certificate algorithms.

<Update March 2, 2015>

The ability to modify RDP plugin settings via the portal has been restored.  I will leave this blog post up since the password encryption portion can still be valuable for people setting up RDP outside of the Visual Studio tools.

</Update>

There are 2 ways to enable RDP for a PaaS cloud service – via the CSPKG using the RemoteAccess plugin, or via the portal (or Powershell or REST API) using an extension.  There was a recent change to the management portal that disabled the ability to configure the plugin style RDP settings after deployment.  The error you get is:

“Failed to launch remote desktop configuration.”

And if you click Details you get:

“Remote Desktop is enabled for this deployment using the RemoteAccess module, which is specified in the ServiceDefinition.csdef file. To allow configuring Remote Desktop using the management portal, remove the RemoteAccess module and update the deployment. Learn More.”

The best solution is to remove the RDP plugin from the package, rebuild the package, and redeploy.  This will let you enable RDP via the portal after deployment and manage the configuration in the portal.  This also provides a more reliable RDP experience since this shifts the RDP functionality from a static feature in your CSPKG to an extension that can be managed and upgraded by the Azure guest agent.  To disable RDP in your package you can right-click the cloud service project, select ‘Configure Remote Desktop’ and then uncheck the ‘Enable connections for all roles’ option.

However, modifying and then redeploying the package is not always a viable solution.  For the scenarios where you want to update the configuration of an existing service you can use the following steps to manually update the configuration.

Download the configuration

Download the configuration and save a local .CSCFG file.  You can do this via the portal or Powershell.

To do this from the portal go to the Configure tab and click Download:

To do this from Powershell run:

$deployment = Get-AzureDeployment -ServiceName <servicename> –Slot <slot>

([xml]$deployment.Configuration).Save("c:\temp\config.cscfg")

Update the configuration setting

Open the .CSCFG file you saved, make any necessary changes, and then save the .CSCFG.

*See below for how to modify the password.

Upload back to the portal

Upload the modified .CSCFG file back to the portal and wait for the configuration change to complete.

To do this from the portal go to the Configure table and click Upload:

To do this from Powershell run:

Set-AzureDeployment -Config -ServiceName <servicename> -Configuration "c:\temp\config.cscfg" –Slot <slot>

Updating the password

Updating settings such as the expiration date or username is easy, but updating the password is more difficult.  The RDP password is encrypted using a user-provided certificate that is uploaded to the cloud service.  If you are making these RDP changes on the computer that already has that certificate in the cert store (ie. from the dev/deploy machine) then getting a new encrypted password is pretty straightforward:

1. Open a ‘Windows Azure Command Prompt’
2. Execute:
3. csencrypt encrypt-password -copytoclipboard –thumbprint <thumbprint_from_CSCFG>
4. Paste the new password into the AccountEncryptedPassword setting in the .CSCFG

If you don’t have the certificate in the local computer’s cert store you can download it and generate a new encrypted certificate using Powershell:

# Download the cert
$cert = Get-AzureCertificate -ServiceName <servicename> -Thumbprint <thumbprint_from_CSCFG> -ThumbprintAlgorithm "SHA1"

# Save the cert to a file
  "-----BEGIN CERTIFICATE-----" > "C:\Temp\rdpcert.cer"
  $cert.Data >> "C:\Temp\rdpcert.cer"
  "-----END CERTIFICATE-----" >> "C:\Temp\rdpcert.cer"

# Prompt for the new password
  $password = Read-Host -Prompt "Enter the password to encrypt"

#Load the certificate
  [System.Reflection.Assembly]::LoadWithPartialName("System.Security") | Out-Null
  $cert = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2("c:\temp\rdpcert.cer")
  $thumbprint = $cert[0].thumbprint
  $pass = [Text.Encoding]::UTF8.GetBytes($password)

# Encrypt the cert
  $content = new-object Security.Cryptography.Pkcs.ContentInfo -argumentList (,$pass)
  $env = new-object Security.Cryptography.Pkcs.EnvelopedCms $content
  $env.Encrypt((new-object System.Security.Cryptography.Pkcs.CmsRecipient($cert)))

# Write the new password to a file and load the file
  write-host "Writing encrypted password, cut/paste the text below the line to CSCFG file"
  [Convert]::ToBase64String($env.Encode()) > "c:\temp\encrypted_password.txt"
  Invoke-Item "c:\temp\encrypted_password.txt"

A question I see periodically is how to restrict RDP access for PaaS services to specific network IP addresses.  In the past this has always been difficult to do and the typical solution was to use a Startup task to configure firewall rules (ie. using Set-NetFirewallRule or netsh advfirewall per https://msdn.microsoft.com/en-us/library/azure/jj156208.aspx).  This technique generally works fine, but it introduces the extra complexity of a startup task and is not built into the Azure platform itself.

Network ACLs

With the (relatively) recent introduction of network ACLs it becomes much easier to robustly secure an input endpoint on a cloud service.  My colleague Walter Myers has a great blog post about how to enable network ACLs for PaaS roles at https://blogs.msdn.com/b/walterm/archive/2014/04/22/windows-azure-paas-acls-are-here.aspx.  To apply a network ACL to the RDP endpoint it is simply a matter of defining your ACL rules targeting the role which imports the RemoteForwarder plugin, and specifying the name of the RDP endpoint in the endPoint attribute.

Here is the resulting NetworkConfiguration section to add to the CSCFG file:

  <NetworkConfiguration>
    <AccessControls>
      <AccessControl name="RDPRestrict">
        <Rule action="permit" description="PermitRDP" order="100" remoteSubnet="167.220.26.0/24" />
        <Rule action="deny" description="DenyRDP" order="200" remoteSubnet="0.0.0.0/0" />
      </AccessControl>
    </AccessControls>
    <EndpointAcls>
      <EndpointAcl role="WebRole1" endPoint="Microsoft.WindowsAzure.Plugins.RemoteForwarder.RdpInput" accessControl="RDPRestrict" />
    </EndpointAcls>
  </NetworkConfiguration>

Important information:

1. You must enable RDP in the package before publishing your service.  The new model of enabling RDP post-deployment via the management portal or extension APIs will not work.  You can enable RDP in the package using Visual Studio by right-clicking the cloud service project in Solution Explorer and selecting ‘Configure Remote Desktop…’, or in the Publish wizard by checking the ‘Enable Remote Desktop for all roles’ checkbox.
2. The role="WebRole1" attribute must specify the role which imports the RemoteForwarder plugin.  You can look in the CSDEF file and find the role which has <Import moduleName="RemoteForwarder" />.  If you have multiple roles in your service then all of them will import RemoteAccess, but only one of them will import RemoteForwarder and you must specify the role which imports RemoteForwarder.
3. The network configuration defined above will restrict all client’s except for those with IP addresses in the range 167.220.26.1-167.220.26.255.  See Walter’s blog post for more information about how to specify network ACLs, and the MSDN documentation (https://msdn.microsoft.com/en-us/library/azure/dn376541.aspx) for more information about order and precedence of rules.
   

Overview

Windows Azure SDK 2.4 and prior used Windows Azure Diagnostics 1.0 which provided multiple options for configuring diagnostics collection, including code based configuration using the Microsoft.WindowsAzure.Diagnostics classes.  Windows Azure SDK 2.5 introduces Windows Azure Diagnostics 1.2 which streamlines the configuration and adds enhanced capabilities for post-deployment enablement, however it removes the code based configuration capabilities.  This blog post will describe how to convert your older code based diagnostics configuration to the new WAD 1.2 XML based configuration model.

For more information about WAD 1.2 see https://azure.microsoft.com/en-us/documentation/articles/cloud-services-dotnet-diagnostics/.

Specifying diagnostics Configuration in XML

In Azure SDK 2.4 it was possible to configure diagnostics through code as well as xml configuration (the diagnostics.wadcfg file) and developers needed to understand the precedence rules of how the diagnostics agent picks up the configuration settings (see here for more information). Azure SDK 2.5 removes this complexity and the diagnostics agent will now always use the xml configuration. This has some implications when you migrate your Azure SDK 2.4 project to Azure SDK 2.5. If your Azure SDK 2.4 project already uses the xml based diagnostics configuration then when you upgrade the project in Visual Studio to target Azure SDK 2.5, Visual Studio will automatically update the xml based configuration to the new format (diagnostics.wadcfgx). If you project continued to use the code based configuration (for example, using the API in Microsoft.WindowsAzure.Diagnostics and Microsoft.WindowsAzure.Diagnostics.Management) then when its upgraded to SDK 2.5, you will get build warnings which will inform you of the deprecated APIs. The diagnostics data will not be collected unless you configure your diagnostics using the XML file (diagnostics.wadcfgx) or through the Visual Studio diagnostics configuration UI.

Let’s look at an example- In Azure SDK 2.4 you could be using the code below to update the diagnostics configuration for the diagnostics infrastructure logs. In this particular example the code is setting the Loglevel for the diagnostics infrastructure logs to Error and the Transfer Period to 5 minutes. When you migrate this project to SDK 2.5 you will get a build warning around this code with the message that the API is deprecated – “Warning: ‘Microsoft.WindowsAzure.Diagnostics.DiagnosticMonitor’ is obsolete: “This API is deprecated””. The project will still build and deploy but it won’t update the diagnostics configuration.

To configure the diagnostics infrastructure logs in Azure SDK 2.5 you must remove that code based configuration and update the diagnostics configuration xml file (diagnostics.wadcfgx) associated with your role. Visual Studio provides a configuration UI to let you specify these diagnostics settings. To configure the diagnostics through Visual Studio you can Right click on the Role and select Properties to display the Role Designer. On the Configuration tab make sure Enable Diagnostics is selected and click Configure. This will bring up the Diagnostics configuration UI where you can go to the Infrastructure Logs Tab and select Enable transfer of Diagnostics Infrastructure Logs and set the Transfer Period and Log Level Appropriately.

Behind the scenes Visual Studio is updating the diagnostics.wadcfgx file with the appropriate xml to configure the infrastructure logs. See highlighted as an example:

<?xml version="1.0" encoding="utf-8"?>

<DiagnosticsConfiguration xmlns="https://schemas.microsoft.com/ServiceHosting/2010/10/DiagnosticsConfiguration">

  <PublicConfig xmlns="https://schemas.microsoft.com/ServiceHosting/2010/10/DiagnosticsConfiguration">

    <WadCfg>

      <DiagnosticMonitorConfiguration overallQuotaInMB="4096">

        <DiagnosticInfrastructureLogs scheduledTransferPeriod="PT5M" scheduledTransferLogLevelFilter="Error" />

      </DiagnosticMonitorConfiguration>

    </WadCfg>

  </PublicConfig>

  <PrivateConfig xmlns="https://schemas.microsoft.com/ServiceHosting/2010/10/DiagnosticsConfiguration">

    <StorageAccount name="" endpoint="" />

  </PrivateConfig>

  <IsEnabled>true</IsEnabled>
</DiagnosticsConfiguration>

The diagnostics configuration dialog in Visual Studio lets you configure other settings like Application Logs, Windows Event Logs, Performance Counters, Infrastructure Logs, Log Directories, ETW Logs and Crash Dumps.

Let’s look at another example for Performance Counters. With Azure SDK 2.4 (and prior) you would have to do the following in code:

With Azure SDK 2.5 you can use the diagnostics configuration UI to enable the default performance counters:

For Custom Performance counters you still have to instrument your code to create the custom performance category and counters as earlier. Once you have instrumented your code you can enable transferring these custom performance counters to storage by adding them to the list of performance counters in the diagnostics configuration UI. Eg. For a custom performance counter you can enter the category and name “\SampleCustomCategory\Total Button1 Clicks” in the text box and click Add.

Visual Studio will automatically add the performance counters selected in the UI to the diagnostics configuration file diagnostics.wadcfgx:

<?xml version="1.0" encoding="utf-8"?>

<DiagnosticsConfiguration xmlns="https://schemas.microsoft.com/ServiceHosting/2010/10/DiagnosticsConfiguration">

  <PublicConfig xmlns="https://schemas.microsoft.com/ServiceHosting/2010/10/DiagnosticsConfiguration">

    <WadCfg>

      <DiagnosticMonitorConfiguration overallQuotaInMB="4096">

        <PerformanceCounters scheduledTransferPeriod="PT1M">

          <PerformanceCounterConfiguration counterSpecifier="\Processor(_Total)\% Processor Time" sampleRate="PT3M" />

          <PerformanceCounterConfiguration counterSpecifier="\Memory\Available MBytes" sampleRate="PT3M" />

        </PerformanceCounters>

      </DiagnosticMonitorConfiguration>

    </WadCfg>

  </PublicConfig>

  <PrivateConfig xmlns="https://schemas.microsoft.com/ServiceHosting/2010/10/DiagnosticsConfiguration">

    <StorageAccount name="" endpoint="" />

  </PrivateConfig>

  <IsEnabled>true</IsEnabled>
</DiagnosticsConfiguration>

For WindowsEventLog data you can enable transfer of logs for common data sources directly from the diagnostics configuration UI.

For custom event logs that you define in your code for example ->

EventLog.CreateEventSource("GuestBookSource", "Application");

EventLog.WriteEntry("GuestBookSource", "WebRole Started", EventLogEntryType.Error, 9191);

You have to manually add the custom source to the wadcfgx XML file under the WindowsEventLog node. Make sure the name matches the name specified in code:

<WindowsEventLog scheduledTransferPeriod="PT1M">

   <DataSource name="Application!*" />

   <DataSource name="GuestBookSource!*" />

</WindowsEventLog>

Enabling diagnostics extension through PowerShell

Since Azure SDK 2.5 uses the extension model the diagnostics extension, the configuration and the connection string to the diagnostic storage are no longer part of the deployment package and cscfg. All the diagnostics configuration is contained within the wadcfgx. The advantage with this approach is that diagnostics agent and settings are decoupled from the project and can be dynamically enabled and updated even after your application is deployed.

Due to this change some existing workflows need to be rethought – instead of configuring the diagnostics as part of the application that gets deployed to each environment you can first deploy the application to the environment and then apply the diagnostics configuration for it.  When you publish the application from Visual Studio this process is done automatically for you. However if you were deploying your application outside of VS using PowerShell then you have to install the extension separately through PowerShell.

There PowerShell cmdlets for managing the diagnostics extensions on a Cloud Service are -

·         Set-AzureServiceDiagnosticsExtension

·         Get-AzureServiceDiagnosticsExtension

·         Remove-AzureServiceDiagnosticsExtension

You can use the Set-AzureServiceDiagnosticsExtension method to enable diagnostics extension on a cloud service. One of the parameters on this cmdlet is the XML configuration file. This file is slightly different from the diagnostics.wadcfgx file. You can create this file from scratch as described here or you can modify the wadcfgx file and pass in the modified file as a parameter to the powershell cmdlet.

To modify the wadcfgx file –

1.       Make a copy the .wadcfgx.

2.       Remove the following elements from the Copy:

<DiagnosticsConfiguration xmlns="https://schemas.microsoft.com/ServiceHosting/2010/10/DiagnosticsConfiguration">

   <PrivateConfig xmlns="https://schemas.microsoft.com/ServiceHosting/2010/10/DiagnosticsConfiguration">

     <StorageAccount name=" " endpoint="https://core.windows.net/" />

   </PrivateConfig>

   <IsEnabled>false</IsEnabled>

</DiagnosticsConfiguration>

3.       Make sure the top of the file still has xml version and encoding –

<?xml version="1.0" encoding="utf-8"?>

Effectively you are stripping down the Wadcfgx to only contain the <PublicConfig> section and the <?xml> header. You can then call the PowerShell cmdlet along with the appropriate parameters for the staging slots and roles:

$storage_name = ‘<storagename>’

$key= ‘<key>’

$service_name = '<servicename>'

$public_config = '<thepublicconfigfrom_diagnostics.wadcfgx>'

 

$storageContext = New-AzureStorageContext –StorageAccountName $storage_name –StorageAccountKey $key

 

Set-AzureServiceDiagnosticsExtension -StorageContext $storageContext -DiagnosticsConfigurationPath $public_config –ServiceName $service_name -Slot ‘Staging’ -Role ‘WebRole1’

In a previous post we looked at the Windows Azure PaaS SDP package which allows you to quickly and easily gather all of the log data to determine root cause for a variety of PaaS compute issues.  This post will look at a new SDP package which allows you to quickly and easily gather all of the storage analytics logs.

Getting the SDP Package

This package will only work on a Windows 7 or later, or Windows Server 2008 R2 or later computer.

1. Open PowerShell
2. Copy/Paste and Run the following script

md c:\Diagnostics; Import-Module bitstransfer; Start-BitsTransfer https://dsazure.blob.core.windows.net/azuretools/AzureStorageAnalyticsLogs_global.DiagCab c:\Diagnostics\AzureStorageAnalyticsLogs_global.DiagCab; c:\Diagnostics\AzureStorageAnalyticsLogs_global.DiagCab

Alternatively you can download and save the .DiagCab directly from https://dsazure.blob.core.windows.net/azuretools/AzureStorageAnalyticsLogs_global.DiagCab.

Running the SDP Package

1. Enter the storage account name. 
   
    
2. Enter the storage account key.  Note that this key is only temporarily used within the SDP package utility.  It is not saved or transferred.
   
    
3. Enter the starting time and ending time.  The default values will gather logs from the past 24 hours
      .
    
4. Select the analytics logs to gather.
   
    
5. When the tool is finished gathering data click Next and an Explorer window will open showing the latest.cab which is a compressed file containing the most recent set of data, along with folders containing the data from each time the SDP package was run.
   

The Results

There will be several files created as a result of running this SDP package.  The important ones are:

1. ResultReport.xml.  This file lists the data collected and includes the storage account name and time range specified.  In the future we will include intelligent analytics results within this file (ie. “Event <x> found in Blob logs.  This usually indicates <y>.  You can find more information at <link>”).
2. *.csv.  This are the raw data files containing the logs and metrics.  A header line is included in the file to make analysis easy.  The headers correspond to the Logs format and Metrics format.
3. *.xlsx.  If Excel is installed on the computer running the SDP package then these .xlsx files will be created which include pre-built charts showing the most commonly used metrics along with the option to select additional metrics.

Excel charts (*.xlsx)

You can add or remove metrics from the Excel charts using the standard Chart filter tools:

Logs (*.csv)

You can easily filter and sort the .CSV files within Excel.  The following filter can help identify potentially inefficient queries by identifying requests that take longer than X number of milliseconds on the server:

Look for additional blog posts in the future which walk through using the analytics data to identify and solve common issues.

Additional Resources

https://msdn.microsoft.com/en-us/library/windowsazure/hh343270.aspx – In depth documentation about storage analytics and what each field means.

https://www.windowsazure.com/en-us/documentation/articles/storage-monitor-storage-account/ – How to enable and use metrics from the Azure Management Portal.

https://channel9.msdn.com/Series/DIY-Windows-Azure-Troubleshooting/Storage-Analytics – A short 5 and a half minute video showing how to enable and use storage analytics.

• Role Topology – The number of instances, number of internal endpoints, and composition of internal endpoints (ie. internal IP addresses of instances, also known as DIP addresses ).
• Topology Change – Any change in this topology.  Typically a scale up or down of the number of instances, or a service healing event which causes one VM to move to a new physical server and obtain a new internal IP address.
• Rolling Upgrade – The process the fabric controller uses to make changes to a hosted service.  The fabric controller will send the change to Upgrade Domain #0, wait for all instances in UD #0 to return to the Ready state, and then move to UD #1, continuing until all UDs have been walked.  See the Upgrade Domain information at https://msdn.microsoft.com/en-us/library/windowsazure/hh472157.aspx.
• Topology Blast – A new feature to allow topology changes to be sent to all UDs at one time, bypassing the normal UD walk.
• topologyChangeDiscovery – Use this .csdef <ServiceDefinition> attribute to control the type of topology change your service receives.  topologyChangeDiscovery="Blast" will turn on Topology Blast.
• RoleEnvironment.SimultaneousChanged/SimultaneousChanging – These events are raised in your code when a topology change happens and you have set topologyChangeDiscovery="Blast".

1. The server hosting IN_2 (with original DIP 10.31.70.8) has had a failure.  The fabric controller automatically detects this failure and recreates IN_2 on a new server.  IN_2 receives a new DIP of 10.25.18.2.  This constitutes a topology change.
2. The fabric controller begins the rolling upgrade process in order to notify the rest of the instances that there has been a topology change.  Each instance will receive a RoleEnvironment.Changing and RoleEnvironment.Changed event, with the change of type ServiceRuntime.RoleEnvironmentTopologyChange.  The InstanceEndpoints list will be updated with the new DIP(s).
3. In Picture 2 the fabric controller is currently processing UD #0 and IN_0 has a correct InstanceEndpoints list.  When IN_0 attempts to communicate with IN_2 using an InternalEndpoint it will connect to the new DIP 10.25.18.2.
4. IN_1 is in UD #1 and has not yet been notified of the topology change and IN_1 is still using the old incorrect InstanceEndpoints list.  When IN_1 attempts to communicate with IN_2 using an InternalEndpoint it will fail to connect to the old DIP 10.31.70.8.

1. The server hosting IN_2 (with original DIP 10.31.70.8) has had a failure.  The fabric controller automatically detects this failure and recreates IN_2 on a new server.  IN_2 receives a new DIP of 10.25.18.2.  This constitutes a topology change.
2. The fabric controller initiates a topology change.  Because the service has set topologyChangeDiscovery="Blast" the fabric will initiate a topology blast and send the topology change to all instances at the same time.
3. Both IN_0 and IN_1 receive a RoleEnvironment.SimultaneousChanging event at the same time with an updated InstanceEndpoints list.  Both instances are now able to successfully communicate with IN_2.

Turning on Topology Blast

<ServiceDefinition name="waTestFramework" topologyChangeDiscovery="Blast" xmlns="https://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition" schemaVersion="2013-10.2.2">

• With topology blast turned on, only topology changes will fire the Simultaneous* events and all other types of changes will fire the standard Changed/Changing events.
• The SimultaneousChangingEventArgs does not implement a Cancel property.  This is to prevent all role instances from recycling at the same time.

public override bool OnStart()
{
    // For information on handling configuration changes
    // see the MSDN topic at https://go.microsoft.com/fwlink/?LinkId=166357.
    RoleEnvironment.SimultaneousChanged += RoleEnvironment_SimultaneousChanged;
    RoleEnvironment.SimultaneousChanging += RoleEnvironment_SimultaneousChanging;