Get started with SQL Server Big Data Clusters deployment

Applies to: SQL Server 2019 (15.x)

Important

The Microsoft SQL Server 2019 Big Data Clusters add-on will be retired. Support for SQL Server 2019 Big Data Clusters will end on February 28, 2025. All existing users of SQL Server 2019 with Software Assurance will be fully supported on the platform and the software will continue to be maintained through SQL Server cumulative updates until that time. For more information, see the announcement blog post and Big data options on the Microsoft SQL Server platform.

This article provides an overview of how to deploy SQL Server Big Data Clusters. The article introduces you to concepts and provides a framework for understanding the deployment scenarios. Your specific deployment steps vary based on your platform choices for the client and server. For introduction on SQL Server Big Data Clusters, see SQL Server 2019 Big Data Clusters

For other SQL Server deployment scenarios, see:

• Windows
• Linux
• Docker containers

Quick Introduction

Watch this 9-minute video for an overview of how to deploy big data clusters:

Tip

To quickly get an environment with Kubernetes and big data cluster deployed to help you ramp up on its capabilities, use one of the sample scripts pointed to in the scripts section. After deployment, to manage the cluster use the client tools in the following section.

Client tools

Big data clusters require a specific set of client tools. Before you deploy a big data cluster to Kubernetes, you should install the tools required for your deployment. Specific tools are required for different scenarios. Each article should explain the prerequisite tools for performing a specific task. For a full list of tools and installation links, see Install SQL Server 2019 big data tools.

Kubernetes

Big data clusters are deployed as a series of interrelated containers that are managed in Kubernetes. You can host Kubernetes in a variety of ways. Even if you already have an existing Kubernetes environment, you should review the related requirements for big data clusters.

• Azure Kubernetes Service (AKS): AKS allows you to deploy a managed Kubernetes cluster in Azure. You only manage and maintain the agent nodes. With AKS, you don't have to provision your own hardware for the cluster. It is also easy to use a python script or a deployment notebook to create the AKS cluster and deploy the big data cluster in one step. For more information about configuring AKS for a big data cluster deployment, see Configure Azure Kubernetes Service for SQL Server 2019 Big Data Clusters deployments.

• Azure Red Hat OpenShift (ARO): ARO allows you to deploy a managed Red Hat OpenShift cluster in Azure. You only manage and maintain the agent nodes. With ARO, you don't have to provision your own hardware for the cluster. It is also easy to use a python script to create the ARO cluster and deploy the big data cluster in one step. This deployment model is introduced in SQL Server 2019 CU5.

• Multiple machines: You can also deploy Kubernetes to multiple Linux machines, which could be physical servers or virtual machines. The kubeadm tool can be used to create the Kubernetes cluster. You can use a bash script to automate this type of deployment. This method works well if you already have existing infrastructure that you want to use for your big data cluster. For more information about using kubeadm deployments with big data clusters, see Configure Kubernetes on multiple machines for SQL Server 2019 Big Data Clusters deployments.

• Red Hat OpenShift: Deploy to your own Red Hat OpenShift cluster. For information, see Deploy SQL Server Big Data Clusters on OpenShift on-premises and Azure Red Hat OpenShift. This deployment model is introduced in SQL Server 2019 CU5.

Deploy a big data cluster

After configuring Kubernetes, you deploy a big data cluster with the azdata bdc create command. When deploying, you can take several different approaches.

• If you are deploying to a dev-test environment, you can choose to use one of the default configurations provided by azdata.

• To customize your deployment, you can create and use your own deployment configuration files.

• For a completely unattended installation, you can pass all other settings in environment variables. For more information, see unattended deployments.

Deployment scripts

Deployment scripts can help deploy both Kubernetes and big data clusters in a single step. They also often provide default values for big data cluster settings. You can customize any deployment script by creating your own version that configures the big data cluster deployment differently.

The following deployment scripts are currently available:

• Python script -- Deploy a big data cluster on Azure Kubernetes Service (AKS)
• Bash script -- Deploy a big data cluster to a single node kubeadm cluster

Deployment notebooks

You can also deploy a big data cluster by running an Azure Data Studio notebook. For more information on how to use a notebook to deploy on AKS, see the following article:

• Deploy a big data cluster with Azure Data Studio Notebooks.

Next steps

After you successfully deploy a big data cluster, connect to the cluster and consider loading sample data for use with several walkthroughs.