Azure DataBricks compute limited to 0 - why?

Question

Azure DataBricks compute limited to 0 - why?

Gerard Church 1

Iam trying to do a tutorial for DataBricks and am trying to create a single node cluster, however whatever I do it just will not work. I'm pretty sure I'm on a pay-as-you-go subscription and the limit should be 10.

Whichever compute I select, I get this error:

This account may not have enough CPU cores to satisfy this request  
Estimated available: 0, requested: 4  
Learn more about CPU quota.

When I've gone to up my limit it says that the limit is 0/10 so I should have 10 avaliable?

Please can someone else me with this. Thanks

PRADEEPCHEEKATLA 90,651 Reputation points Moderator

2022-12-05T10:26:24.667+00:00
Hello @Gerard Church ,

Following up to see if the below suggestion was helpful. And, if you have any further query do let us know.

------------------------------

Please don't forget to click on or upvote button whenever the information provided helps you.

2 answers

Your answer

PRADEEPCHEEKATLA 90,651 Reputation points Moderator

2022-12-05T10:26:24.667+00:00

Hello @Gerard Church ,

Following up to see if the below suggestion was helpful. And, if you have any further query do let us know.

------------------------------

Please don't forget to click on or upvote button whenever the information provided helps you.

Answer 1

Hello @Gerard Church ,

As you are using the Node type as Standard_D4a_v4 - please confirm the usage+quota request for Dav4-series in your subscription?

You will experience this error message when you don't have quota for 'Dav4-series` in your subscription.

To resolve this issue, you need to create the node type which are available in your subscription or raise a request quota increase for the VM series which you are looking for.

For more details, refer to MS Q&A thread addressing similar issue: Cluster Terminated, Quota Exceeded Expectation while everything is new

Hope this will help. Please let us know if any further queries.

------------------------------

Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
Want a reminder to come back and check responses? Here is how to subscribe to a notification
If you are interested in joining the VM program and help shape the future of Q&A: Here is jhow you can be part of Q&A Volunteer Moderators

Answer 2

TP 125.8K Volunteer Moderator

Hi,

In the Azure portal, please navigate to Subscriptions, click on your subscription, then click on Usage + quotas. For the region your Databricks cluster is in, please check the current usage and limits for Total Regional vCPUs as well as Standard DAv4 Family vCPUs (or similar). Your net usage cannot exceed either limit. For example, if your Total Regional vCPUs limit is 5, and your current usage is 2, you would not be able to create your cluster since it needs 4 additional vCPUs.

Please note that when you delete a cluster it will often take several minutes for Databricks to remove the underlying VMs and thus even though a cluster may appear to be deleted in Databricks the current vCPU usage will still show some vCPUs in use.

Thanks.

-TP

Gerard Church 1 Reputation point

2022-11-26T17:35:51.02+00:00

I actually haven't been able to create a cluster at all yet so I won't have used vCPU. Here are the current quotas as per your suggestion:

And:

As you can see there isn't a single usage of any of these. Please advise further, Thanks. @TP

Share via

Azure DataBricks compute limited to 0 - why?

2 answers

Your answer