@Mussadiq Welcome to Microsoft Q&A forum and thanks for reaching out here.
As per the error message, the issue occurred due to vCPU usage going over the Azure subscription quota limit as pointed out in the message above. You can increase the limits by going to your Azure Subscription > Usage + Quotas.
How to check Usage + Quotas for your subscription?
Select your subscription => Under Settings => Usage + quotas => Use filter to select "Standard" & "<Your region>" => Check usage of Total Regional vCPUs => If the usage is full, you need to click on Request Increase to increase the limit of cores in the region.
You can also refer to these threads where a similar topic has been discussed:
- Azure DB - Workflow - Delta Live Tables - DataPlaneException: Failed to launch pipeline cluster
- Cluster Terminated, Quota Exceeded Expectation while everything is new
- AZURE_QUOTA_EXCEEDED_EXCEPTION - even with more than vCPUs than Databricks recommends
Here is a youtube video by a community contributor on how to request quota increase: How to Resolve Azure Quota Exceeded Exception On Databricks Cluster
Hope this info helps.
Please don’t forget to
Accept Answer and
Yes for "was this answer helpful" wherever the information provided helps you, this can be beneficial to other community members.