Azure DataBricks compute limited to 0 - why?

Gerard Church 1 Reputation point
2022-11-26T16:33:33.507+00:00

Iam trying to do a tutorial for DataBricks and am trying to create a single node cluster, however whatever I do it just will not work. I'm pretty sure I'm on a pay-as-you-go subscription and the limit should be 10.

Whichever compute I select, I get this error:
264414-image.png

This account may not have enough CPU cores to satisfy this request  
Estimated available: 0, requested: 4  
Learn more about CPU quota.  

When I've gone to up my limit it says that the limit is 0/10 so I should have 10 avaliable?

Please can someone else me with this. Thanks

Azure Virtual Machines
Azure Virtual Machines
An Azure service that is used to provision Windows and Linux virtual machines.
7,532 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,067 questions
{count} votes

2 answers

Sort by: Most helpful
  1. PRADEEPCHEEKATLA-MSFT 85,031 Reputation points Microsoft Employee
    2022-11-29T06:42:18.487+00:00

    Hello @Gerard Church ,

    As you are using the Node type as Standard_D4a_v4 - please confirm the usage+quota request for Dav4-series in your subscription?

    265046-image.png

    You will experience this error message when you don't have quota for 'Dav4-series` in your subscription.

    To resolve this issue, you need to create the node type which are available in your subscription or raise a request quota increase for the VM series which you are looking for.

    For more details, refer to MS Q&A thread addressing similar issue: Cluster Terminated, Quota Exceeded Expectation while everything is new

    Hope this will help. Please let us know if any further queries.

    ------------------------------

    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
    • If you are interested in joining the VM program and help shape the future of Q&A: Here is jhow you can be part of Q&A Volunteer Moderators
    1 person found this answer helpful.
    0 comments No comments

  2. TP 83,311 Reputation points
    2022-11-26T17:29:34.53+00:00

    Hi,

    In the Azure portal, please navigate to Subscriptions, click on your subscription, then click on Usage + quotas. For the region your Databricks cluster is in, please check the current usage and limits for Total Regional vCPUs as well as Standard DAv4 Family vCPUs (or similar). Your net usage cannot exceed either limit. For example, if your Total Regional vCPUs limit is 5, and your current usage is 2, you would not be able to create your cluster since it needs 4 additional vCPUs.

    Please note that when you delete a cluster it will often take several minutes for Databricks to remove the underlying VMs and thus even though a cluster may appear to be deleted in Databricks the current vCPU usage will still show some vCPUs in use.

    Thanks.

    -TP