Azure Spot Virtual Machines for Virtual Machine Scale Sets

Using Azure Spot Virtual Machines on scale sets allows you to take advantage of our unused capacity at a significant cost savings. At any point in time when Azure needs the capacity back, the Azure infrastructure evicts Azure Spot Virtual Machine instances. Therefore, Azure Spot Virtual Machine instances are great for workloads that can handle interruptions like batch processing jobs, dev/test environments, large compute workloads, and more.

The amount of available capacity can vary based on size, region, time of day, and more. When deploying Azure Spot Virtual Machine instances on scale sets, Azure allocates the instance only if there's capacity available, but there's no SLA for these instances. An Azure Spot Virtual Machine Scale Set is deployed in a single fault domain and offers no high availability guarantees.

Limitations

The following sizes are not supported for Azure Spot Virtual Machines:

• B-series
• Promo versions of any size (like Dv2, NV, NC, H promo sizes)

Azure Spot Virtual Machine can be deployed to any region, except Microsoft Azure operated by 21Vianet.

The following offer types are currently supported:

• Enterprise Agreement
• Pay-as-you-go offer code (003P)
• Sponsored (0036P and 0136P)
• For Cloud Service Provider (CSP), see the Partner Center or contact your partner directly.

Pricing

Pricing for Azure Spot Virtual Machine instances is variable, based on region and SKU. For more information, see pricing for Linux and Windows.

With variable pricing, you have option to set a max price, in US dollars (USD), using up to five decimal places. For example, the value 0.98765would be a max price of $0.98765 USD per hour. If you set the max price to be -1, the instance won't be evicted based on price. The price for the instance will be the current price for Azure Spot Virtual Machine or the price for a standard instance, which ever is less, as long as there's capacity and quota available.

Eviction policy

When creating a scale set using Azure Spot Virtual Machines, you can set the eviction policy to Deallocate (default) or Delete.

The Deallocate policy moves your evicted instances to the stopped-deallocated state allowing you to redeploy evicted instances. However, there's no guarantee that the allocation will succeed. The deallocated VMs counts against your scale set instance quota and you are charged for your underlying disks.

If you would like your instances to be deleted when they are evicted, you can set the eviction policy to delete. With the eviction policy set to delete, you can create new VMs by increasing the scale set instance count property. The evicted VMs are deleted together with their underlying disks, and therefore you won't be charged for the storage. You can also use the autoscaling feature of scale sets to automatically try and compensate for evicted VMs, however, there's no guarantee that the allocation succeeds. It is recommended you only use the autoscale feature on Azure Spot Virtual Machine Scale Sets when you set the eviction policy to delete to avoid the cost of your disks and hitting quota limits.

Users can opt in to receive in-VM notifications through Azure Scheduled Events. This notifies you if your VMs are being evicted and you have 30 seconds to finish any jobs and perform shutdown tasks prior to the eviction.

Eviction history

You can see historical pricing and eviction rates per size in a region in the portal. Select View pricing history and compare prices in nearby regions to see a table or graph of pricing for a specific size. The pricing and eviction rates in the following images are only examples.

Chart:

Table:

Try & restore

This platform-level feature uses AI to automatically try to restore evicted Azure Spot Virtual Machine instances inside a scale set to maintain the target instance count.

Try & restore benefits:

• Attempts to restore Azure Spot Virtual Machines evicted due to capacity.
• Restored Azure Spot Virtual Machines are expected to run for a longer duration with a lower probability of a capacity triggered eviction.
• Improves the lifespan of an Azure Spot Virtual Machine, so workloads run for a longer duration.
• Helps Virtual Machine Scale Sets to maintain the target count for Azure Spot Virtual Machines, similar to maintain target count feature that already exists for Pay-As-You-Go VMs.

Try & restore is disabled in scale sets that use Autoscale. The number of VMs in the scale set is driven by the autoscale rules.

Placement Groups

Placement group is a construct similar to an Azure availability set, with its own fault domains and upgrade domains. By default, a scale set consists of a single placement group with a maximum size of 100 VMs. If the scale set property called singlePlacementGroup is set to false, the scale set can be composed of multiple placement groups and has a range of 0-1,000 VMs.

Important

Unless you are using Infiniband with HPC, it is strongly recommended to set the scale set property singlePlacementGroup to false to enable multiple placement groups for better scaling across the region or zone.

Deploying Azure Spot Virtual Machines in scale sets

To deploy Azure Spot Virtual Machines on scale sets, you can set the new Priority flag to Spot. All VMs in your scale set will be set to Spot. To create a scale set with Azure Spot Virtual Machines, use one of the following methods:

• Azure portal
• Azure CLI
• Azure PowerShell
• Azure Resource Manager templates

Portal

The process to create a scale set that uses Azure Spot Virtual Machines is the same as detailed in the getting started article. When you are deploying a scale set, you can choose to set the Spot flag, eviction type, eviction policy and if you want to try to restore instances:

Azure CLI

Important

Starting November 2023, VM scale sets created using PowerShell and Azure CLI will default to Flexible Orchestration Mode if no orchestration mode is specified. For more information about this change and what actions you should take, go to Breaking Change for VMSS PowerShell/CLI Customers - Microsoft Community Hub

The process to create a scale set with Azure Spot Virtual Machines is the same as detailed in the getting started article. Just add the '--Priority Spot', and add --max-price. In this example, we use -1 for --max-price so the instance won't be evicted based on price.

az vmss create \
    --resource-group myResourceGroup \
    --name myScaleSet \
    --image Ubuntu2204 \
    --orchestration-mode Flexible \
    --single-placement-group false \
    --admin-username azureuser \
    --generate-ssh-keys \
    --priority Spot \
    --eviction-policy Deallocate \
    --max-price -1 \
    --enable-spot-restore True \
    --spot-restore-timeout PT1H

PowerShell

Important

Starting November 2023, VM scale sets created using PowerShell and Azure CLI will default to Flexible Orchestration Mode if no orchestration mode is specified. For more information about this change and what actions you should take, go to Breaking Change for VMSS PowerShell/CLI Customers - Microsoft Community Hub

The process to create a scale set with Azure Spot Virtual Machines is the same as detailed in the getting started article. Just add '-Priority Spot', and supply a -max-price to the New-AzVmssConfig.

$vmssConfig = New-AzVmssConfig `
    -Location "East US 2" `
    -SkuCapacity 2 `
    -OrchestrationMode "Flexible" `
    -SkuName "Standard_DS2" `
    -Priority "Spot" `
    -max-price -1 `
    -EnableSpotRestore `
    -SpotRestoreTimeout 60 `
    -EvictionPolicy delete

Resource Manager templates

The process to create a scale set that uses Azure Spot Virtual Machines is the same as detailed in the getting started article for Linux or Windows.

For Azure Spot Virtual Machine template deployments, use"apiVersion": "2019-03-01" or later.

Add the priority, evictionPolicy, billingProfile and spotRestoryPolicy properties to the "virtualMachineProfile":section and the "singlePlacementGroup": false, property to the "Microsoft.Compute/virtualMachineScaleSets" section in your template:

{
  "type": "Microsoft.Compute/virtualMachineScaleSets",
  },
  "properties": {
    "singlePlacementGroup": false,
    }

        "virtualMachineProfile": {
              "priority": "Spot",
                "evictionPolicy": "Deallocate",
                "billingProfile": {
                    "maxPrice": -1
                },
                "spotRestorePolicy": {
                  "enabled": "bool",
                  "restoreTimeout": "string"
    },
            },

To delete the instance after it has been evicted, change the evictionPolicy parameter to Delete.

Simulate an eviction

You can simulate an eviction of an Azure Spot Virtual Machine to test how well your application responds to a sudden eviction.

Replace the following with your information:

• subscriptionId
• resourceGroupName
• vmName

POST https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Compute/virtualMachines/{vmName}/simulateEviction?api-version=2020-06-01

Response Code: 204 means the simulated eviction was successful.

For more information, see Testing a simulated eviction notification.

FAQ

Q: Once created, is an Azure Spot Virtual Machine instance the same as standard instance?

A: Yes, except there's no SLA for Azure Spot Virtual Machines and they can be evicted at any time.

Q: What to do when you get evicted, but still need capacity?

A: We recommend you use standard VMs instead of Azure Spot Virtual Machines if you need capacity right away.

Q: How is quota managed for Azure Spot Virtual Machine?

A: Azure Spot Virtual Machine instances and standard instances have separate quota pools. Azure Spot Virtual Machine quota is shared between VMs and scale-set instances. For more information, see Azure subscription and service limits, quotas, and constraints.

Q: Can I request for additional quota for Azure Spot Virtual Machine?

A: Yes, you are able to submit the request to increase your quota for Azure Spot Virtual Machines through the standard quota request process.

Q: Can I convert existing scale sets to Azure Spot Virtual Machine Scale Sets?

A: No, setting the Spot flag is only supported at creation time.

Q: If I was using low for low-priority scale sets, do I need to start using Spot instead?

A: For now, both low and Spot will work, but you should start transitioning to using Spot.

Q: Can I create a scale set with both regular VMs and Azure Spot Virtual Machines?

A: No, a scale set cannot support more than one priority type.

Q: Can I use autoscale with Azure Spot Virtual Machine Scale Sets?

A: Yes, you can set autoscaling rules on your Azure Spot Virtual Machine Scale Set. If your VMs are evicted, autoscale can try to create new Azure Spot Virtual Machines. Remember, you are not guaranteed this capacity though.

Q: Does autoscale work with both eviction policies (deallocate and delete)?

A: Yes, however it is recommended that you set your eviction policy to delete when using autoscale. This is because deallocated instances are counted against your capacity count on the scale set. When using autoscale, you will likely hit your target instance count quickly due to the deallocated, evicted instances. Also, your scaling operations could be impacted by spot evictions. For example, Virtual Machine Scale Set instances could fall below the set min count due to multiple spot evictions during scaling operations.

Q: Where can I post questions?

A: You can post and tag your question with azure-spot at Q&A.

Next steps

Check out the Virtual Machine Scale Set pricing page for pricing details.