"SkuNotAvailable" while creating and starting the Azure Databricks Cluster

Prasanna Bandi 0

Hello

I have a Azure Free Trial Subscription. Created Databricks Workspace and tried creating a Cluster in Databrick with following configuration.

Single Node and Single user with 14.3 GB & 4 Cores with 10.4LTS Spark 3.2.1 and Scala 2.12.

After creating the Cluster and tried starting the Cluster, it is taking forever and failed with error "SkuNotAvailable".

Can Someone help me with this issue and get the Cluster started. Thank you.

PRADEEPCHEEKATLA-MSFT 88,471 Reputation points Microsoft Employee

2023-09-21T10:29:23.5533333+00:00

@Prasanna Bandi - Just checking in to see if the below answer helped. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.
PRADEEPCHEEKATLA-MSFT 88,471 Reputation points Microsoft Employee

2023-09-28T08:45:10.6766667+00:00

@Prasanna Bandi - Did you get a chance to click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

2 answers

PRADEEPCHEEKATLA-MSFT 88,471 Reputation points Microsoft Employee

2023-09-20T06:46:19.27+00:00

@Prasanna Bandi - Thanks for the question and using MS Q&A platform.

By default, for any subscription there will be specific limit on the VM cores available per region based on the OfferType. Certain Azure regions require customers to go through a request process in order to gain access. To request access to these regions, a customer or their Cloud Solution Provider may open a support request and work with our Support team to discuss or enable access.

Note: If a SKU isn't available for your subscription in a location or zone that meets your business needs, submit a SKU request to Azure Support (Which is complete at free cost) or try to check the SKU available in you subscription and try to utilize the same to create.

For more details, refer to Resolve errors for SKU not available.

Hope this helps. Do let us know if you any further queries.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.
Please sign in to rate this answer.

1 person found this answer helpful.
Prasanna Bandi 0 Reputation points

2023-09-21T19:46:01.25+00:00

Hi @PRADEEPCHEEKATLA-MSFT - First of all, thanks for your response on my concern. Am trying to submit a SKU Increase request from Help page, but getting below error.

It does not like since mine is Free Trial subscription.

Can you please help me how I can create Databricks workspace and cluster. Thanks.

PRADEEPCHEEKATLA-MSFT 88,471 Reputation points Microsoft Employee

2023-09-25T05:31:37.36+00:00

@Prasanna Bandi - When you create your Azure Databricks workspace, you can select the Trial (Premium - 14-Days Free DBUs) pricing tier to give the workspace access to free Premium Azure Databricks DBUs for 14 days.

For more details, refer to Quickstart: Run a Spark job on Azure Databricks Workspace using the Azure portal.

This is execpted behaviour when you are trying to use more CPU cores which are not available in your subscription (Azure free trial/Student/Pass).

Reason: Azure free trial/Student/Pass subscription has a limit of 4 cores, and you cannot create Databricks cluster multi node using a Student Subscription because it requires more than 8 cores.

You need to upgrade to a Pay-As-You-Go subscription to create Azure Databricks clusters with multi mode.

Note: Azure free trial/Student/Pass subscriptions aren't eligible for limit or quota increases. If you have a Student subscription, you can upgrade to a Pay-As-You-Go subscription.

You can use Azure Free Trial subscription to create a Single node cluster which will have one Driver node with 4 cores.

A Single Node cluster is a cluster consisting of a Spark driver and no Spark workers. Such clusters support Spark jobs and all Spark data sources, including Delta Lake. In contrast, Standard clusters require at least one Spark worker to run Spark jobs.

Single Node clusters are helpful in the following situations:

Running single node machine learning workloads that need Spark to load and save data

Lightweight exploratory data analysis (EDA)

Reference: Azure Databricks - Single Node clusters

To use multi-node cluster in Azure Databricks, you need to have “Pay-As-You-Go” subscription.

Why can't I use Azure Free Trial subscription?

Azure Free Trial has a limit of 4 cores, and you cannot create Azure Databricks cluster using a Free Trial Subscription because to create a spark cluster which requires more than 4 cores i.e (Minimum 4 cores for Driver type and 4 cores for Worker type).

For more details, refer to Quickstart: Run a Spark job on Azure Databricks Workspace using the Azure portal

Hope this helps. Do let us know if you any further queries.

Santhosh Kumar Reddy Mopuru 0 Reputation points

2023-09-26T21:48:07.1166667+00:00

Thanks. I am able to create the Cluster with 14-Day Trial DBUs. I will consider upgrading my Azure account to Pay-As-Go Subscription after 14-Days

PRADEEPCHEEKATLA-MSFT 88,471 Reputation points Microsoft Employee

2023-09-27T12:52:58.17+00:00

@Prasanna Bandi -Glad to know you are able to create cluster with 14-Day Trial DBUs. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.
Luke 45 Reputation points

2023-09-28T09:09:18.3266667+00:00

The issue is related to your subscription type. You are using free Azure Visual Studio (formerly MSDN) benefit, and it has access to a limited set of regions/services.

You can check out this site for more information: Azure Offers by Region | Microsoft Azure. Suggest you use an allowed region instead.

If that will not meet your needs, you can visit the Azure home page Cloud Computing Services | Microsoft Azure to explore expanded access through a paid subscription.
Please sign in to rate this answer.

0 comments No comments
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

"SkuNotAvailable" while creating and starting the Azure Databricks Cluster

2 answers

Your answer