Configure Azure Kubernetes Service for SQL Server big data cluster deployments

Article
03/03/2023

Applies to: SQL Server 2019 (15.x)

Important

The Microsoft SQL Server 2019 Big Data Clusters add-on will be retired. Support for SQL Server 2019 Big Data Clusters will end on February 28, 2025. All existing users of SQL Server 2019 with Software Assurance will be fully supported on the platform and the software will continue to be maintained through SQL Server cumulative updates until that time. For more information, see the announcement blog post and Big data options on the Microsoft SQL Server platform.

This article describes how to configure Azure Kubernetes Service (AKS) for SQL Server 2019 Big Data Clusters deployments.

AKS makes it simple to create, configure, and manage a cluster of virtual machines that are preconfigured with a Kubernetes cluster to run containerized applications. This enables you to use your existing skills or draw upon a large and growing body of community expertise, to deploy and manage container-based applications on Microsoft Azure.

This article describes the steps to deploy Kubernetes on AKS using Azure CLI. If you don't have an Azure subscription, create a free account before you begin.

Tip

You can also script the deployment of AKS and a big data cluster in one step. For more information, see how to do this in a python script or an Azure Data Studio notebook.

Prerequisites

Deploy the SQL Server 2019 big data tools:
- Kubectl
- Azure Data Studio
- SQL Server 2019 extension
- Azure CLI
Minimum 1.13 version for Kubernetes server. For AKS, you need to use --kubernetes-version parameter to specify a version different than the default.
To ensure a successful deployment and an optimal experience while validating basic scenarios on AKS, you can use a single node or a multi-node AKS cluster, with these resources available:
- 8 vCPUs across all nodes
- 64 GB of memory per VM
- 24 or more attached disks across all nodes
Tip

Azure infrastructure offers multiple size options for VMs, see here for selections in the region you are planning to deploy.

Create a resource group

An Azure resource group is a logical group in which Azure resources are deployed and managed. The following steps sign into Azure and create a resource group for the AKS cluster.

At the command prompt, run the following command and follow the prompts to login to your Azure subscription:
```
az login
```
If you have multiple subscriptions you can view all of your subscriptions by running the following command:
```
az account list
```
If you want to change to a different subscription you can run this command:
```
az account set --subscription <subscription id>
```
Identify the Azure region where you want to deploy the cluster and the resources by using this command:
```
az account list-locations -o table
```
Create a resource group with the az group create command. The following example creates a resource group named sqlbdcgroup in the westus2 location.
```
az group create --name sqlbdcgroup --location westus2
```

Verify available Kubernetes versions

Use the latest available version of Kubernetes. The latest available version depends on the location where you are deploying the cluster. The following command returns Kubernetes versions available in a specific location.

Before you run the command, update the script. Replace <Azure data center> with the location of your cluster.

bash

az aks get-versions \
--location <Azure data center> \
--query orchestrators \
--o table

PowerShell

az aks get-versions `
--location <Azure data center> `
--query orchestrators `
-o table

Choose the latest available version for your cluster. Record the version number. You will use it in the next step.

Create a Kubernetes cluster

Create a Kubernetes cluster in AKS with the az aks create command. The following example creates a Kubernetes cluster named kubcluster with one Linux agent node of size Standard_L8s.

Before you run the script, replace <version number> with the version number you identified in the previous step.

Make sure you create the AKS cluster in the same resource group that you used in the previous sections.

bash:
```
az aks create --name kubcluster \
--resource-group sqlbdcgroup \
--generate-ssh-keys \
--node-vm-size Standard_L8s \
--node-count 1 \
--kubernetes-version <version number>
```
PowerShell:
```
az aks create --name kubcluster `
--resource-group sqlbdcgroup `
--generate-ssh-keys `
--node-vm-size Standard_L8s `
--node-count 1 `
--kubernetes-version <version number>
```
You can increase or decrease the number of Kubernetes agent nodes by changing the --node-count <n> where <n> is the number of agent nodes you want to use. This does not include the master Kubernetes node, which is managed behind the scenes by AKS. The previous example only uses a single node for evaluation purposes. You can also change the --node-vm-size to select an appropriate virtual machine size that matches your workload requirements. Use the az vm list-sizes --location westus2 -o table command to list available virtual machine sizes in your region.

After several minutes, the command completes and returns JSON-formatted information about the cluster.

Tip

If you get any errors creating the cluster in AKS, see the troubleshooting section of this article.
Save the JSON output from the previous command for later use.

Connect to the cluster

To configure kubectl to connect to your Kubernetes cluster, run the az aks get-credentials command. This step downloads credentials and configures the kubectl CLI to use them.
```
az aks get-credentials --resource-group=sqlbdcgroup --name kubcluster
```
To verify the connection to your cluster, use the kubectl get command to return a list of the cluster nodes. The example below shows the output if you were to have 1 master and 3 agent nodes.
```
kubectl get nodes
```

Troubleshooting

If you have any problems creating an Azure Kubernetes Service with the previous commands, try the following resolutions:

Make sure that you have installed the latest Azure CLI.
Try the same steps using a different resource group and cluster name.
Refer to the detailed troubleshooting documentation for AKS.

Next steps

The steps in this article configured a Kubernetes cluster in AKS. The next step is to deploy a SQL Server 2019 big data cluster on the AKS Kubernetes cluster. For more information on how to deploy big data clusters, see the following article:

How to deploy SQL Server Big Data Clusters on Kubernetes