Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,694 questions
This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
I have an existing Synapse configuration that has been set from the Synapse studio. Now I have a new spark pool created and want to bind that configuration to the pool. On the portal, it can be set on the sparkpool. But what is the programmatic way to do that, e.g. azure cli, etc.?
First thing, install the Azure CLI, if you haven't already :
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
Then, login to it :
az login
Set the required configuration in a JSON file, for example, create a file named spark-pool-config.json
with your desired configuration:
{
"name": "your-spark-pool-name",
"tags": {},
"autoScale": {
"enabled": true,
"minNodeCount": 3,
"maxNodeCount": 10
},
"nodeSize": "Medium",
"nodeSizeFamily": "MemoryOptimized",
"sessionLevelPackagesEnabled": true,
"sparkEventsFolder": "/events",
"sparkVersion": "3.1"
}
Then, update the Spark pool using the Azure CLI:
az synapse spark pool update --name your-spark-pool-name --workspace-name your-workspace-name --resource-group your-resource-group --file @spark-pool-config.json