An Apache Spark-based analytics platform optimized for Azure.
Thanks for the clarification. Since you're using a free Azure subscription, that explains why you're hitting the core quota limits and are unable to request a quota increase, free-tier subscriptions come with strict limits (typically 4 vCPUs per region), and quota increase requests are not allowed under this plan.
Here are a few options you can consider:
Option 1 - Use a Paid (Pay-As-You-Go) Subscription
To lift the quota limits and request increases, you'll need to upgrade your subscription to a Pay-As-You-Go model. Once upgraded, you can:
- Request more vCPUs.
- Access a wider range of VM SKUs.
- Deploy larger Databricks clusters.
To upgrade: Go to Subscriptions in Azure Portal → Select your subscription → Click "Upgrade".
Option 2 - Try a Different Region and Smaller Cluster
Some VM sizes are more available in other regions. You can:
- Try regions like East US, West Europe, or Southeast Asia.
- Use a smaller VM SKU (e.g.,
Standard_DS1_v2orStandard_B2s) that fits within the 4-core limit.
However, for Databricks, most clusters require more than 4 cores, so this may still be limiting.
Option 3 - Use Free Tier Services for Learning
If you’re just exploring or doing hands-on learning:
- You can try Azure Machine Learning Notebooks, Azure Synapse Studio, or Visual Studio Code with Azure SDKs, which work with lower resources.
- Or try Databricks Community Edition (outside of Azure) for initial learning: https://community.cloud.databricks.com
I hope this information helps.