Applying Spot VM Option

Question

Dear Team,

We have an Azure Databricks-associated VM in our Dev environment and a Windows Server running dev workload. We are looking to apply the Spot VM option for these VMs.

For Databricks VM, we stopped the cluster and waited for more than 30 minutes to allow the VM to be deleted, then restarted the cluster with the Spot VM option. However, in the VM overview, the Spot VM policy is still blank. Is there any other way to ensure that Spot VMs are enabled? What is the correct approach to configure this?

For the Windows VM (Jumphost), should we delete and recreate the VM with Spot VM enabled? If so, can I capture the image from the existing VM and reuse it when spinning up the Spot VM?

Any insights or best practices would be greatly appreciated.

Thanks!

Answer 1

Hi Suki Azure,

Stopping the cluster before enabling Spot instances for the Databricks VM is the right approach, the underlying VM might still be retained by Azure for a while even after the cluster is stopped. Here are a few things to verify:

1. Make sure the cluster is fully terminated before restarting it with Spot enabled.
2. When editing the cluster, ensure that Spot instances are explicitly selected in the worker type settings.
3. You can also check the Azure Resource Explorer to confirm if the Spot policy is applied at the VM scale set level.

If the Spot option is still not reflecting, you might need to create a new cluster from scratch with Spot enabled right from the start.

For the Windows Jump Host, unfortunately, you can’t convert an existing VM to a Spot VM directly. You need to recreate it, but you can definitely capture an image of your current VM and use that to deploy the new one:

1. Go to the Azure portal, navigate to your VM, and use the Capture Image option.
2. Store the image in an Azure Compute Gallery or as a managed image.
3. Create a new Spot VM using that image.

Remember that Spot VMs can be evicted at any time. So, if it’s a critical workload, you may want to have a backup plan in place.

Please refer the document below:

https://learn.microsoft.com/en-us/azure/virtual-machines/spot-vms

https://learn.microsoft.com/en-us/azure/virtual-machines/spot-portal

If you found the information helpful, please click "Upvote" on the post to let us know.

Thank You,