Get answers to frequently asked questions about Virtual Machine Scale Sets in Azure.
Top frequently asked questions for scale sets
How many VMs can I have in a scale set?
A scale set can have 0 to 1,000 virtual machines (VMs) based on platform images, or 0 to 600 VMs based on custom images.
Are data disks supported within scale sets?
Yes. A scale set can define the configuration of an attached data disk that applies to all VMs in the set. For more information, see Azure scale sets and attached data disks. Other options for storing data include:
- Azure Managed Disks (Premium v2, Premium, Standard, Ultra)
- Azure Files (SMB or NFS shared drives)
- Azure Netapp Files
- Azure shared disks
- Operating system drive
- Temp drive (local, not backed by Azure Storage)
- Azure data service (for example, Azure Table Storage or Azure Blob Storage)
- External data service (for example, a remote database)
Which Azure regions support scale sets?
All regions support scale sets.
Which SKUs are supported for Virtual Machine Scale Sets?
All SKUs are supported for Virtual Machine Scale Sets.
How do I create a scale set by using a custom image?
Create and capture a VM image, and then use that as the source for your scale set. For a tutorial on how to create and use a custom VM image, you can use the Azure CLI or Azure PowerShell.
What is the difference between OS Image Upgrade and Reimage?
OS Image Upgrade is a gradual and non-disruptive process that updates the OS image for the entire Virtual Machine Scale Set over time, ensuring minimal impact on running workloads.
Reimage is a more immediate and disruptive action that affects only the selected VM instance, stopping it temporarily and reinstalling the OS.
Learn more about the difference between OS Image Upgrade and Reimage.
If I reduce my scale set capacity from 20 to 15, which VMs are removed?
By default, virtual machines are removed from the scale set evenly across availability zones (if the scale set is deployed in zonal configuration) and fault domains, to maximize availability. VMs with the highest IDs are removed first.
You can change the order of virtual machine removal by specifying a scale-in policy for the scale set.
What if I then increase the capacity from 15 to 18?
If you increase capacity to 18, then 3 new VMs are created. Each time, the VM instance ID is incremented from the previous highest value (for example, 20, 21, 22). VMs are balanced across fault domains.
When I'm using multiple extensions in a scale set, can I enforce an execution sequence?
Yes, you can use scale set extension sequencing.
Do scale sets work with Azure availability sets?
A regional (non-zonal) scale set uses placement groups, which act as an implicit availability set with five fault domains and five update domains. Scale sets of more than 100 VMs span multiple placement groups. For more information about placement groups, see Working with large Virtual Machine Scale Sets. An availability set of VMs can exist in the same virtual network as a scale set of VMs. A common configuration is to put control node VMs (which often require unique configuration) in an availability set, and put data nodes in the scale set.
Do scale sets work with Azure availability zones?
Yes. For more information, see the scale set zone doc.
Autoscale
What are best practices for Azure autoscale?
Where do I find metric names for autoscaling that uses host-based metrics?
Are there any examples of autoscaling based on an Azure Service Bus topic and queue length?
Yes. For these examples, see Azure Monitor autoscaling common metrics.
For a Service Bus queue, use the following JSON:
"metricName": "MessageCount",
"metricNamespace": "",
"metricResourceUri": "/subscriptions/s1/resourceGroups/rg1/providers/Microsoft.ServiceBus/namespaces/mySB/queues/myqueue"
For a storage queue, use the following JSON:
"metricName": "ApproximateMessageCount",
"metricNamespace": "",
"metricResourceUri": "/subscriptions/s1/resourceGroups/rg1/providers/Microsoft.ClassicStorage/storageAccounts/mystorage/services/queue/queues/mystoragequeue"
Replace example values with your resource Uniform Resource Identifiers (URIs).
Should I autoscale by using host-based metrics or a diagnostics extension?
You can create an autoscale setting on a VM to use host-level metrics, or metrics based on a guest operating system.
For a list of supported metrics, see Azure Monitor autoscaling common metrics.
For a full sample for Virtual Machine Scale Sets, see Advanced autoscale configuration by using Resource Manager templates for Virtual Machine Scale Sets.
The sample uses the host-level CPU metric and a message count metric.
How do I set alert rules on a Virtual Machine Scale Set?
You can create alerts on metrics for Virtual Machine Scale Sets via PowerShell or the Azure CLI. For more information, see Azure Monitor PowerShell quickstart samples and Azure Monitor cross-platform CLI quickstart samples.
The TargetResourceId
of the Virtual Machine Scale Set looks like:
/subscriptions/yoursubscriptionid/resourceGroups/yourresourcegroup/providers/Microsoft.Compute/virtualMachineScaleSets/yourvmssname
You can choose any VM performance counter as the metric to set an alert for. For more information, see Guest operating system metrics for Resource Manager-based Windows VMs and Guest operating system metrics for Linux VMs in the Azure Monitor autoscaling common metrics article.
How do I set up autoscale on a Virtual Machine Scale Set by using PowerShell?
See automatically scale a Virtual Machine Scale Set. You can also configure autoscale with the Azure CLI and Azure templates.
If I have stopped (deallocated) a VM, is that VM started as part of an autoscale operation?
No. If autoscale rules require more VM instances as part of a scale set, a new VM instance is created. VM instances that are stopped (deallocated) aren't started as part of an autoscale event. However, those stopped (deallocated) VMs can be deleted as part of an autoscale event that scales in the number of instances, in the same way that any VM instance can be deleted based on the order of VM instance IDs.
Certificates
How do I securely ship a certificate to the VM?
To securely ship a certificate to the VM, you can install a customer certificate directly into a Windows certificate store from the customer's key vault.
Use the following JSON:
"secrets": [
{
"sourceVault": {
"id": "/subscriptions/{subscriptionid}/resourceGroups/myrg1/providers/Microsoft.KeyVault/vaults/mykeyvault1"
},
"vaultCertificates": [
{
"certificateUrl": "https://mykeyvault1.vault.azure.net/secrets/{secretname}/{secret-version}",
"certificateStore": "certificateStoreName"
}
]
}
]
The code supports Windows and Linux.
For more information, see Create or update a Virtual Machine Scale Set.
How do I use self-signed certificates provisioned for Azure Service Fabric clusters?
For the latest example, in an Azure shell, use the following Azure CLI statement, which will be printed to stdout:
az sf cluster create -h
Self-signed certificates can't be used for distributed trust provided by a certificate authority, and shouldn't be used for any Service Fabric cluster intended to host enterprise production solutions. For more security guidance, review Azure Service Fabric security best practices and Service Fabric cluster security scenarios.
For SSH authentication with a Linux Virtual Machine Scale Set from a Resource Manager template, can I specify an SSH key pair to use?
Yes. The REST API for osProfile
is similar to the standard VM REST API.
Include osProfile
in your template:
"osProfile": {
"computerName": "[variables('vmName')]",
"adminUsername": "[parameters('adminUserName')]",
"linuxConfiguration": {
"disablePasswordAuthentication": "true",
"ssh": {
"publicKeys": [
{
"path": "[variables('sshKeyPath')]",
"keyData": "[parameters('sshKeyData')]"
}
]
}
}
}
This JSON block is used in this Azure Quickstart Template.
For more information, see Create or update a Virtual Machine Scale Set.
How do I remove deprecated certificates?
To remove deprecated certificates, remove the old certificate from the vault certificates list. Leave all the certificates that you want to remain on your computer in the list. This action doesn't remove the certificate from all your VMs. It also doesn't add the certificate to new VMs that are created in the Virtual Machine Scale Set.
To remove the certificate from existing VMs, use a custom script extension to manually remove the certificate from your certificate store.
How do I inject an existing SSH public key into the Virtual Machine Scale Set SSH layer during provisioning?
If you're providing the VMs only with a public SSH key, you don't need to put the public keys in Azure Key Vault. Public keys aren't secret.
You can provide SSH public keys in plain text when you create a Linux VM:
"linuxConfiguration": {
"ssh": {
"publicKeys": [
{
"path": "path",
"keyData": "publickey"
}
]
}
}
linuxConfiguration element name |
Required | Type | Description |
---|---|---|---|
ssh |
No | Collection | Specifies the SSH key configuration for a Linux operating system. |
path |
Yes | String | Specifies the Linux file path where the SSH keys or certificate should be located. |
keyData |
Yes | String | Specifies a base64-encoded SSH public key. |
For an example, see the vm-sshkey GitHub quickstart template.
When I run `Update-AzVmss` after adding more than one certificate from the same key vault, why do I get an error message?
This error can happen if you try to re-add the same vault instead of using a new vault certificate for the existing source vault. The Add-AzVmssSecret
command doesn't work correctly if you're adding more secrets.
To add more secrets from the same key vault, update the following list: $vmss.properties.osProfile.secrets[0].vaultCertificates
.
For the expected input structure, see Create or update a virtual machine set.
Find the secret in the Virtual Machine Scale Set object that is in the key vault. Then, add your certificate reference (the URL and the secret store name) to the list associated with the vault.
Note
Currently, you can't remove certificates from VMs by using the Virtual Machine Scale Set API.
New VMs will not have the old certificate. However, VMs that have the certificate, and that are already deployed, do have the old certificate.
Can I push certificates to the Virtual Machine Scale Set without providing the password, when the certificate is in the secret store?
You don't need to hard-code passwords in scripts. You can dynamically retrieve passwords with the permissions you use to run the deployment script. If you have a script that moves a certificate from the secret store key vault, the secret store get certificate
command also outputs the password of the .pfx file.
How does the `Secrets` property of `virtualMachineProfile.osProfile` for a Virtual Machine Scale Set work? Why do I need the `sourceVault` value when I have to specify the absolute URI for a certificate by using the `certificateUrl` property?
A Windows Remote Management (WinRM) certificate reference must be present in the Secrets
property of the operating system profile.
The purpose of indicating the source vault is to enforce access control list (ACL) policies that exist in a user's Azure Cloud Services model. If the source vault isn't specified, users who don't have permissions to deploy or access secrets to a key vault are able to, through a compute resource provider (CRP). ACLs exist even for resources that don't exist.
If you provide an incorrect source vault ID, but a valid key vault URL, an error is reported when you poll the operation.
If I add secrets to an existing Virtual Machine Scale Set, are the secrets injected into existing VMs, or only into new ones?
Certificates are added to all your VMs, even pre-existing ones. If your Virtual Machine Scale Set upgradePolicy
property is set to manual
, the certificate is added to the VM when you perform a manual update on the VM.
Where do I put certificates for Linux VMs?
To learn how to deploy certificates for Linux VMs, see Deploy certificates to VMs from a customer-managed key vault.
How do I add a new vault certificate to a new certificate object?
To add a vault certificate to an existing secret, see the following PowerShell example. Use only one secret object.
$newVaultCertificate = New-AzVmssVaultCertificateConfig -CertificateStore MY -CertificateUrl https://sansunallapps1.vault.azure.net:443/secrets/dg-private-enc/55fa0332edc44a84ad655298905f1809
$vmss.VirtualMachineProfile.OsProfile.Secrets[0].VaultCertificates.Add($newVaultCertificate)
Update-AzVmss -VirtualMachineScaleSet $vmss -ResourceGroup $rg -Name $vmssName
What happens to certificates if I reimage a VM?
If you reimage a VM, certificates are deleted. Reimaging deletes the entire operating system disk.
What happens if I delete a certificate from the key vault?
If the secret is deleted from the key vault, and then you run stop deallocate
for all your VMs and then start them again, you encounter a failure. The failure occurs because the CRP needs to retrieve the secrets from the key vault, but it can't. In this scenario, you can delete the certificates from the Virtual Machine Scale Set model.
The CRP component doesn't make customer secrets persistent. If you run stop deallocate
for all VMs in the Virtual Machine Scale Set, the cache is deleted. In this scenario, secrets are retrieved from the key vault.
You don't encounter this problem when scaling out, because there's a cached copy of the secret in Azure Service Fabric (in the single-fabric tenant model).
Why do I have to specify the certificate version when I use Key Vault?
The purpose is to make it clear to the user what certificate is deployed on their VMs.
If you create a VM and then update your secret in the key vault, the new certificate isn't downloaded to your VMs. But your VMs appear to reference it, and new VMs get the new secret. To avoid this issue, you're required to reference a secret version.
My team works with several certificates that are distributed to us as .cer public keys. What is the recommended approach for deploying these certificates to a Virtual Machine Scale Set?
To deploy .cer public keys to a Virtual Machine Scale Set, you can generate a .pfx file that contains only .cer files. To do this, use X509ContentType = Pfx
. For example, load the .cer file as an x509Certificate2
object in C# or PowerShell, and then call the method.
For more information, see X509Certificate.Export Method (X509ContentType, String).
How do I pass in certificates as base64 strings?
To emulate passing in a certificate as a base64 string, you can extract the latest versioned URL in a Resource Manager template. Include the following JSON property in your Resource Manager template:
"certificateUrl": "[reference(resourceId(parameters('vaultResourceGroup'), 'Microsoft.KeyVault/vaults/secrets', parameters('vaultName'), parameters('secretName')), '2015-06-01').secretUriWithVersion]"
Do I have to wrap certificates in JSON objects in key vaults?
In Virtual Machine Scale Sets and VMs, certificates must be wrapped in JSON objects.
We also support the content type application/x-pkcs12.
We currently don't support .cer files. To use .cer files, export them into .pfx containers.
Compliance and security
Are Virtual Machine Scale Sets PCI-compliant?
Virtual Machine Scale Sets are a thin API layer on top of the CRP. Both components are part of the compute platform in the Azure service tree.
From a compliance perspective, Virtual Machine Scale Sets are a fundamental part of the Azure compute platform. They share the following with the CRP itself: a team, tools, processes, deployment methodology, security controls, just-in-time (JIT) compilation, monitoring, and alerting. Virtual Machine Scale Sets are Payment Card Industry (PCI)-compliant because the CRP is part of the current PCI Data Security Standard (DSS) attestation.
For more information, see the Microsoft Trust Center.
Do managed identities for Azure resources work with Virtual Machine Scale Sets?
Yes. For more information, see the managed identities overview. You can see some example MSI templates in Azure Quickstart Templates for Linux and Windows.
Deleting
Will the locks I set in place on Virtual Machine Scale Set instances be respected when I'm deleting instances?
In the Azure portal, you have the ability to delete an individual instance, or bulk delete by selecting multiple instances. If you attempt to delete a single instance that has a lock in place, the lock is respected and you can't delete the instance. However, if you bulk select multiple instances, and any of those instances have a lock in place, the locks aren't respected. All of the selected instances will be deleted.
In the Azure CLI, you only have the ability to delete an individual instance. If you attempt to delete a single instance that has a lock in place, the lock is respected and you can't delete that instance.
Extensions
How do I delete a Virtual Machine Scale Set extension?
To delete a Virtual Machine Scale Set extension, use the following PowerShell example:
$vmss = Get-AzVmss -ResourceGroupName "resource_group_name" -VMScaleSetName "vmssName"
$vmss=Remove-AzVmssExtension -VirtualMachineScaleSet $vmss -Name "extensionName"
Update-AzVmss -ResourceGroupName "resource_group_name" -VMScaleSetName "vmssName" -VirtualMacineScaleSet $vmss
You can find the extensionName
value in $vmss
.
Is there a Virtual Machine Scale Set template example that integrates with Azure Monitor Logs?
For a Virtual Machine Scale Set template example that integrates with Azure Monitor Logs, see the second example in Deploy an Azure Service Fabric cluster and enable monitoring by using Azure Monitor Logs.
How do I add an extension to all VMs in my Virtual Machine Scale Set?
If the update policy is set to automatic, redeploying the template with the new extension properties updates all VMs.
If the update policy is set to manual, first update the extension, and then manually update all instances in your VMs.
If the extensions associated with an existing Virtual Machine Scale Set are updated, are existing VMs affected?
If the extension definition in the Virtual Machine Scale Set model is updated, and the upgradePolicy
property is set to automatic
, it updates the VMs. If the upgradePolicy
property is set to manual
, extensions are flagged as not matching the model.
Are extensions run again when an existing machine is service-healed or reimaged?
If an existing VM is service-healed, it appears as a reboot, and the extensions aren't run again. If a VM is reimaged, the process is similar to replacing the operating system drive with the source image. Any specializations from the latest model, such as extensions, are run again.
How do I join a Virtual Machine Scale Set to an Active Directory domain?
To join a Virtual Machine Scale Set to an Active Directory domain, you can define an extension.
To define an extension, use the JsonADDomainExtension
property:
"extensionProfile": {
"extensions": [
{
"name": "joindomain",
"properties": {
"publisher": "Microsoft.Compute",
"type": "JsonADDomainExtension",
"typeHandlerVersion": "1.3",
"settings": {
"Name": "[parameters('domainName')]",
"OUPath": "[variables('ouPath')]",
"User": "[variables('domainAndUsername')]",
"Restart": "true",
"Options": "[variables('domainJoinOptions')]"
},
"protectedsettings": {
"Password": "[parameters('domainJoinPassword')]"
}
}
}
]
}
My Virtual Machine Scale Set extension is trying to install something that requires a reboot. What should I do?
You can use the Azure Automation Desired State Configuration extension. If the operating system is Windows Server 2012 R2, Azure pulls in the Windows Management Framework (WMF) 5.0 setup, reboots, and then continues with the configuration.
How do I run a custom script that's hosted in a private storage account?
Set up protected settings with the storage account key and name. For more information, see Custom script extension.
Passwords
How do I reset the password for VMs in my Virtual Machine Scale Set?
You can:
Change the Virtual Machine Scale Set model directly. This option is only available with API 2017-12-01 and later versions.
Update the admin credentials directly in the scale set model (for example, by using the Azure Resource Explorer, PowerShell or the Azure CLI). After the scale set is updated, all new VMs have the new credentials. Existing VMs only have the new credentials if they are reimaged.
Reset the password by using the VM access extensions. Make sure to follow the password requirements, as described in the FAQ.
Using a VM access extension doesn't require reimaging, because the extension doesn't update the password in the model. The extension runs a script to append the password to the password or to the SSH key file. The extension doesn't remove the original SSH key. Once the extension is updated, upgrade the instances to apply the updates to the username and password on all the VM instances.
Note
If Auto upgrade policy is set to
manual
, manually select the instance to perform an upgrade operation on individual VM instances. If auto upgrade set toAuto
, the extension will automatically upgrade. For more information, see Automatic Extension UpgradesUse the following PowerShell example for a Windows Virtual Machine Scale Set:
$vmssName = "myvmss" $vmssResourceGroup = "myvmssrg" $publicConfig = @{"UserName" = "newuser"} $privateConfig = @{"Password" = "********"} $extName = "VMAccessAgent" $publisher = "Microsoft.Compute" $vmss = Get-AzVmss -ResourceGroupName $vmssResourceGroup -VMScaleSetName $vmssName $vmss = Add-AzVmssExtension -VirtualMachineScaleSet $vmss -Name $extName -Publisher $publisher -Setting $publicConfig -ProtectedSetting $privateConfig -Type $extName -TypeHandlerVersion "2.0" -AutoUpgradeMinorVersion $true Update-AzVmss -ResourceGroupName $vmssResourceGroup -Name $vmssName -VirtualMachineScaleSet $vmss
Use the following Azure CLI example for a Linux Virtual Machine Scale Set:
az vmss extension set \ --resource-group myResouceGroup \ --vmss-name myScaleSet \ --publisher Microsoft.OSTCExtensions \ --name VMAccessForLinux \ --version 1.5 \ --protected-settings "{'username': 'newUser', 'password': 'newPassword'}"
Networking
Is it possible to assign a network security group (NSG) to a scale set, so that it applies to all the VM NICs in the set?
Yes. You can apply an NSG directly to a scale set by referencing it in the networkInterfaceConfigurations
section of the network profile. Here's an example:
"networkProfile": {
"networkInterfaceConfigurations": [
{
"name": "nic1",
"properties": {
"primary": "true",
"ipConfigurations": [
{
"name": "ip1",
"properties": {
"subnet": {
"id": "[concat('/subscriptions/', subscription().subscriptionId,'/resourceGroups/', resourceGroup().name, '/providers/Microsoft.Network/virtualNetworks/', variables('vnetName'), '/subnets/subnet1')]"
},
"loadBalancerInboundNatPools": [
{
"id": "[concat('/subscriptions/', subscription().subscriptionId,'/resourceGroups/', resourceGroup().name, '/providers/Microsoft.Network/loadBalancers/', variables('lbName'), '/inboundNatPools/natPool1')]"
}
],
"loadBalancerBackendAddressPools": [
{
"id": "[concat('/subscriptions/', subscription().subscriptionId,'/resourceGroups/', resourceGroup().name, '/providers/Microsoft.Network/loadBalancers/', variables('lbName'), '/backendAddressPools/addressPool1')]"
}
]
}
}
],
"networkSecurityGroup": {
"id": "[concat('/subscriptions/', subscription().subscriptionId,'/resourceGroups/', resourceGroup().name, '/providers/Microsoft.Network/networkSecurityGroups/', variables('nsgName'))]"
}
}
}
]
}
How do I do a virtual IP (VIP) swap for Virtual Machine Scale Sets in the same subscription and same region?
If you have two Virtual Machine Scale Sets with Azure Load Balancer, and they are in the same subscription and region, you can deallocate the public IP addresses from each one, and assign to the other. For more information, see VIP swap: Blue-green deployment in Azure Resource Manager. You may experience a delay, though, because the resources are deallocated or allocated at the network level. A faster option is to use Azure Application Gateway with two back-end pools, and a routing rule. Alternatively, you can host your application with Azure App Service, which supports fast switching between staging and production slots.
How do I specify a range of private IP addresses to use for static private IP address allocation?
IP addresses are selected from a subnet that you specify.
The allocation method of Virtual Machine Scale Set IP addresses is always dynamic, but that doesn't mean that these IP addresses can change. In this case, dynamic only means that you don't specify the IP address in a PUT
request. Specify the static set by using the subnet.
How do I deploy a Virtual Machine Scale Set to an existing Azure virtual network?
Can I use scale sets with accelerated networking?
Yes. To use accelerated networking, set enableAcceleratedNetworking
to true
in your scale set's networkInterfaceConfigurations
settings. For example:
"networkProfile": {
"networkInterfaceConfigurations": [
{
"name": "niconfig1",
"properties": {
"primary": true,
"enableAcceleratedNetworking" : true,
"ipConfigurations": [
]
}
}
]
}
How can I configure the DNS servers used by a scale set?
To create a Virtual Machine Scale Set with a custom DNS configuration, add a dnsSettings
JSON packet to the scale set networkInterfaceConfigurations
section. Here's an example:
"dnsSettings":{
"dnsServers":["10.0.0.6", "10.0.0.5"]
}
How can I configure a scale set to assign a public IP address to each VM?
To create a Virtual Machine Scale Set that assigns a public IP address to each VM, make sure that the API version of the Microsoft.Compute/virtualMachineScaleSets resource is 2017-03-30, and add a publicipaddressconfiguration
JSON packet to the scale set ipConfigurations
section. Here's an example:
"publicipaddressconfiguration": {
"name": "pub1",
"properties": {
"idleTimeoutInMinutes": 15
}
}
Can I configure a scale set to work with multiple application gateways?
Yes. You can add the resource IDs for multiple application gateway backend address pools to the applicationGatewayBackendAddressPools
list in the ipConfigurations
section of your scale set network profile.
Scale
In what case would I create a Virtual Machine Scale Set with fewer than two VMs?
One reason to create a Virtual Machine Scale Set with fewer than two VMs is to use the elastic properties of a Virtual Machine Scale Set. For example, you can deploy a Virtual Machine Scale Set with zero VMs to define your infrastructure without paying VM running costs. Then, when you're ready to deploy VMs, you can increase the capacity of the Virtual Machine Scale Set to the production instance count.
Another reason you might create a Virtual Machine Scale Set with fewer than two VMs is if you're concerned less with availability than with using an availability set with discrete VMs. Virtual Machine Scale Sets give you a way to work with undifferentiated compute units that are fungible. This uniformity is a key differentiator for Virtual Machine Scale Sets versus availability sets. Many stateless workloads don't track individual units. If the workload drops, you can scale down to one compute unit, and then scale up to many when the workload increases.
How do I change the number of VMs in a Virtual Machine Scale Set?
To change the number of VMs in a Virtual Machine Scale Set in the Azure portal, from the Virtual Machine Scale Set Properties section, select the Scaling pane, and use the slider bar.
How do I define custom alerts for when certain thresholds are reached?
You have some flexibility in how you handle alerts for specified thresholds. For example, you can define customized webhooks. The following webhook example is from a Resource Manager template:
{
"type": "Microsoft.Insights/autoscaleSettings",
"apiVersion": "[variables('insightsApi')]",
"name": "autoscale",
"location": "[parameters('resourceLocation')]",
"dependsOn": [
"[concat('Microsoft.Compute/virtualMachineScaleSets/', parameters('vmSSName'))]"
],
"properties": {
"name": "autoscale",
"targetResourceUri": "[concat('/subscriptions/',subscription().subscriptionId, '/resourceGroups/', resourceGroup().name, '/providers/Microsoft.Compute/virtualMachineScaleSets/', parameters('vmSSName'))]",
"enabled": true,
"notifications": [
{
"operation": "Scale",
"email": {
"sendToSubscriptionAdministrator": true,
"sendToSubscriptionCoAdministrators": true,
"customEmails": [
"youremail@address.com"
]
},
"webhooks": [
{
"serviceUri": "<service uri>",
"properties": {
"key1": "custommetric",
"key2": "scalevmss"
}
}
]
}
]
}
}
Patching and operations
Can I create a scale set in an existing resource group?
Yes, you can.
Can I move a scale set to another resource group?
Yes, you can move scale set resources to a new subscription or resource group.
How do I update my Virtual Machine Scale Set to a new image? How do I manage patching?
To update your Virtual Machine Scale Set to a new image, and to manage patching, see Upgrade a Virtual Machine Scale Set.
Can I use the reimage operation to reset a VM without changing the image? (That is, I want reset a VM to factory settings rather than to a new image.)
Yes, you can use the reimage operation to reset a VM without changing the image. However, if your Virtual Machine Scale Set references a platform image with version = latest
, your VM can update to a later operating system image when you call reimage
.
Is it possible to integrate scale sets with Azure Monitor Logs?
Yes, you can by installing the Azure Monitor extension on the scale set VMs. Here is an example that uses the Azure CLI:
az vmss extension set --name MicrosoftMonitoringAgent --publisher Microsoft.EnterpriseCloud.Monitoring --resource-group Team-03 --vmss-name nt01 --settings "{'workspaceId': '<your workspace ID here>'}" --protected-settings "{'workspaceKey': '<your workspace key here'}"
You can find the required workspaceId
and workspaceKey
in the Log Analytics workspace of the Azure portal. On the Overview page, select the Settings tile. Select the Connected Sources tab at the top.
Note
If your scale set upgradePolicy
is set to manual, you need to apply the extension to the all VMs in the set by calling upgrade on them. In the Azure CLI, this is az vmss update-instances
.
Note
This article was recently updated to use the term Azure Monitor logs instead of Log Analytics. Log data is still stored in a Log Analytics workspace and is still collected and analyzed by the same Log Analytics service. We are updating the terminology to better reflect the role of logs in Azure Monitor. See Azure Monitor terminology changes for details.
Troubleshooting
How do I turn on boot diagnostics?
To turn on boot diagnostics, first, create a storage account. Then, put this JSON block in your Virtual Machine Scale Set virtualMachineProfile
, and update the Virtual Machine Scale Set:
"diagnosticsProfile": {
"bootDiagnostics": {
"enabled": true,
"storageUri": "http://yourstorageaccount.blob.core.windows.net"
}
}
When a new VM is created, the InstanceView
property of the VM shows the details for the screenshot. Here's an example:
"bootDiagnostics": {
"consoleScreenshotBlobUri": "https://o0sz3nhtbmkg6geswarm5.blob.core.windows.net/bootdiagnostics-swarmagen-4157d838-8335-4f78-bf0e-b616a99bc8bd/swarm-agent-9574AE92vmss-0_2.4157d838-8335-4f78-bf0e-b616a99bc8bd.screenshot.bmp",
"serialConsoleLogBlobUri": "https://o0sz3nhtbmkg6geswarm5.blob.core.windows.net/bootdiagnostics-swarmagen-4157d838-8335-4f78-bf0e-b616a99bc8bd/swarm-agent-9574AE92vmss-0_2.4157d838-8335-4f78-bf0e-b616a99bc8bd.serialconsole.log"
}
How do I troubleshoot other issues?
Virtual machine properties
How do I get property information for each VM without making multiple calls? For example, how would I get the fault domain for each of the 100 VMs in my Virtual Machine Scale Set?
You can call ListVMInstanceViews
by doing a REST API GET
on the following resource URI:
/subscriptions/<subscription_id>/resourceGroups/<resource_group_name>/providers/Microsoft.Compute/virtualMachineScaleSets/<scaleset_name>/virtualMachines?$expand=instanceView&$select=instanceView
Note that fault domain is not returned when the scale set uses max spreading (platformFaultDomainCount = 1), because there is no guarantee as to how many fault domains would be used with this setting.
Can I pass different extension arguments to different VMs in a Virtual Machine Scale Set?
No, you can't. However, extensions can act based on the unique properties of the VM they are running on, such as on the machine name. Extensions also can query instance metadata on http://169.254.169.254
to get more information about the VM.
Why are there gaps (for example, 0, 1, 3) between my Virtual Machine Scale Set VM machine names and VM IDs?
Gaps are because your Virtual Machine Scale Set overprovision
property is set to the default value of true
. If overprovisioning is set to true
, more VMs than requested are created. Extra VMs are then deleted. In this case, you gain increased deployment reliability, but at the expense of contiguous naming and contiguous network address translation (NAT) rules.
You can set this property to false
. For small Virtual Machine Scale Sets, deployment reliability won't be significantly affected.
What's the difference between deleting a VM in a Virtual Machine Scale Set and deallocating the VM? When should I choose one over the other?
The main difference is that deallocate
doesn't delete the virtual hard disks (VHDs). There are storage costs associated with running stop deallocate
. You might use one or the other for one of the following reasons:
- You want to stop paying compute costs, but you want to keep the disk state of the VMs.
- You want to start a set of VMs more quickly than you can scale out a Virtual Machine Scale Set.
- Related to this scenario, you might have created your own autoscale engine and want a faster end-to-end scale.
- You have a Virtual Machine Scale Set that is unevenly distributed across fault domains or update domains. This uneven distribution might be because you selectively deleted VMs, or because VMs were deleted after overprovisioning. Running
stop deallocate
followed bystart
on the Virtual Machine Scale Set evenly distributes the VMs across fault domains or update domains.
How do I take a snapshot of a Virtual Machine Scale Set instance?
Create a snapshot from an instance of a Virtual Machine Scale Set. Here's an example:
$rgname = "myResourceGroup"
$vmssname = "myVMScaleSet"
$Id = 0
$location = "East US"
$vmss1 = Get-AzVmssVM -ResourceGroupName $rgname -VMScaleSetName $vmssname -InstanceId $Id
$snapshotconfig = New-AzSnapshotConfig -Location $location -AccountType Standard_LRS -OsType Windows -CreateOption Copy -SourceUri $vmss1.StorageProfile.OsDisk.ManagedDisk.id
New-AzSnapshot -ResourceGroupName $rgname -SnapshotName 'mySnapshot' -Snapshot $snapshotconfig
Create a managed disk from the snapshot. Here's an example:
$snapshotName = "mySnapshot"
$snapshot = Get-AzSnapshot -ResourceGroupName $rgname -SnapshotName $snapshotName
$diskConfig = New-AzDiskConfig -AccountType Premium_LRS -Location $location -CreateOption Copy -SourceResourceId $snapshot.Id
$osDisk = New-AzDisk -Disk $diskConfig -ResourceGroupName $rgname -DiskName ($snapshotName + '_Disk')