I'm creating a single node compute in Azure Databricks, but it remains at "Finding instances for new nodes, acquiring more instances if necessary"

Hi @alan zhang

I hope the steps provided above were helpful. Please let me know if there's anything else I can assist you with.

Thanks,
Vrishabh

Hi @alan zhang

Here are some recommended actions to address this issue -

Regional Capacity or SKU Availability

• The requested VM size (even for the smallest SKU) may not be available in your selected region due to capacity constraints.
• Action:
  • Check the https://status.azure.com for regional capacity issues.
    • Try changing the cluster location to another region where the SKU is available.

Subscription Quota Limits

• Azure enforces vCPU quotas per subscription, per region, and per VM family. If your subscription has reached its quota, Databricks cannot allocate new instances.
• Action:
  • Navigate to Azure Portal → Suscription → Usage + quotas.
    • Verify quotas for the VM family (e.g., Standard_DSv3).
      • https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/azure-subscription-service-limits#increase-a-quota if needed
        • Reference: CPU core limit prevents cluster creation

Network Configuration Issues

• If your workspace is deployed in a custom VNet, misconfigured routes or firewalls can block Databricks control plane from reaching the compute nodes.
• Action:
  • Ensure outbound connectivity on port 443 to Databricks control plane endpoints.
    • Validate user-defined routes (UDRs) and NAT gateway settings.
      • Reference: Troubleshoot compute issues

Resource Contention or Spot Instances

• If you are using spot instances or a highly demanded SKU (e.g., GPU-enabled), provisioning delays can occur.
• Action:
  • Switch to an on-demand instance type.
    • Reduce cluster size or disable autoscaling temporarily.

Troubleshooting steps -

• Always check event logs in the Databricks UI for detailed error messages.
• If the cluster remains in pending state beyond 30 minutes, terminate and retry after verifying quotas and network settings.
• For persistent issues, raise a support ticket with Azure, including cluster configuration and event logs.

Recommended Microsoft Learn Links

• Troubleshoot compute issues in Azure Databricks
• Azure Databricks documentation
• https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/azure-subscription-service-limits#increase-a-quota

Thanks,
Vrishabh