Share via


Create an Azure Data Explorer cluster and database

Azure Data Explorer is a fast, fully managed data analytics service for real-time analysis on large volumes of data streaming from applications, websites, IoT devices, and more. To use Azure Data Explorer, you first create a cluster, and create one or more databases in that cluster. Then, you can ingest (load) data into a database and run queries against it.

In this article, you'll learn how to create a cluster and a database using either C#, Python, Go, the Azure CLI, PowerShell, or an Azure Resource Manager (ARM) template. To learn how to create a cluster and database using the Azure portal, see Quickstart: Create an Azure Data Explorer cluster and database.

Prerequisites

Prerequisites by method of cluster and database creation:

Create an Azure Data Explorer cluster

This section guides you through the process of creating an Azure Data Explorer cluster. Choose the relevant tab for your preferred method to create the cluster.

  1. Create your cluster by using the following code:

    var tenantId = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx"; //Directory (tenant) ID
    var clientId = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx"; //Application ID
    var clientSecret = "PlaceholderClientSecret"; //Client Secret
    var subscriptionId = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx";
    var authClient = ConfidentialClientApplicationBuilder.Create(clientId)
        .WithAuthority($"https://login.microsoftonline.com/{tenantId}")
        .WithClientSecret(clientSecret)
        .Build();
    var result = authClient.AcquireTokenForClient(new[] { "https://management.core.windows.net/" }).ExecuteAsync().Result;
    var credentials = new TokenCredentials(result.AccessToken, result.TokenType);
    var kustoManagementClient = new KustoManagementClient(credentials) { SubscriptionId = subscriptionId };
    var resourceGroupName = "testrg";
    var clusterName = "mykustocluster";
    var skuName = KustoSkuName.StandardE8adsV5;
    var skuTier = KustoSkuTier.Standard;
    var capacity = 5;
    var clusterData = new Cluster(
        location: "Central US",
        sku: new AzureSku(skuName, skuTier, capacity)
    );
    await kustoManagementClient.Clusters.CreateOrUpdateAsync(resourceGroupName, clusterName, clusterData);
    
    Setting Suggested value Field description
    clusterName mykustocluster The desired name of your cluster.
    skuName Standard_E8ads_v5 The SKU that will be used for your cluster.
    tier Standard The SKU tier.
    capacity number The number of instances of the cluster.
    resourceGroupName testrg The resource group name where the cluster will be created.

    Note

    Create a cluster is a long running operation, so it's highly recommended to use CreateOrUpdateAsync, instead of CreateOrUpdate.

  2. Run the following command to check whether your cluster was successfully created:

    clusterData = await kustoManagementClient.Clusters.GetAsync(resourceGroupName, clusterName);
    
  3. Confirm the successful creation of the cluster by verifying the result contains provisioningState as Succeeded.

Create an Azure Data Explorer database

In this section, you'll create a database within the cluster created in the previous section.

  1. Create your database by using the following code:

    var databaseName = "mykustodatabase";
    var softDeletePeriod = TimeSpan.FromDays(3650);
    var hotCachePeriod = TimeSpan.FromDays(3650);
    var databaseData = new ReadWriteDatabase(
        location: clusterData.Location, softDeletePeriod: softDeletePeriod, hotCachePeriod: hotCachePeriod
    );
    await kustoManagementClient.Databases.CreateOrUpdateAsync(resourceGroupName, clusterName, databaseName, databaseData);
    

    Note

    If you are using C# version 2.0.0 or below, use Database instead of ReadWriteDatabase.

    Setting Suggested value Field description
    clusterName mykustocluster The name of your cluster where the database will be created.
    databaseName mykustodatabase The name of your database.
    resourceGroupName testrg The resource group name where the cluster will be created.
    softDeletePeriod 3650:00:00:00 The amount of time that data will be kept available to query.
    hotCachePeriod 3650:00:00:00 The amount of time that data will be kept in cache.
  2. Run the following command to see the database that you created:

    databaseData = await kustoManagementClient.Databases.GetAsync(resourceGroupName, clusterName, databaseName) as ReadWriteDatabase;
    

Next steps