Edit

Azure Data Explorer output bindings for Azure Functions (preview)

  • Article

When a function runs, the Azure Data Explorer output binding ingests data to Azure Data Explorer.

For information on setup and configuration details, see the overview.

Examples

A C# function can be created by using one of the following C# modes:

  • Isolated worker model: Compiled C# function that runs in a worker process that's isolated from the runtime. Isolated worker process is required to support C# functions running on LTS and non-LTS versions .NET and the .NET Framework.
  • In-process model: Compiled C# function that runs in the same process as the Functions runtime.
  • C# script: Used primarily when you create C# functions in the Azure portal.

More samples for the Azure Data Explorer output binding are available in the GitHub repository.

This section contains the following examples:

The examples refer to Product class and a corresponding database table:

public class Product
{
    [JsonProperty(nameof(ProductID))]
    public long ProductID { get; set; }

    [JsonProperty(nameof(Name))]
    public string Name { get; set; }

    [JsonProperty(nameof(Cost))]
    public double Cost { get; set; }
}

.create-merge table Products (ProductID:long, Name:string, Cost:double)

HTTP trigger, write one record

The following example shows a C# function that adds a record to a database. The function uses data provided in an HTTP POST request as a JSON body.

using Microsoft.Azure.Functions.Worker;
using Microsoft.Azure.Functions.Worker.Extensions.Kusto;
using Microsoft.Azure.Functions.Worker.Http;
using Microsoft.Azure.WebJobs.Extensions.Kusto.SamplesOutOfProc.OutputBindingSamples.Common;

namespace Microsoft.Azure.WebJobs.Extensions.Kusto.SamplesOutOfProc.OutputBindingSamples
{
    public static class AddProduct
    {
        [Function("AddProduct")]
        [KustoOutput(Database: "productsdb", Connection = "KustoConnectionString", TableName = "Products")]
        public static async Task<Product> Run(
            [HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = "addproductuni")]
            HttpRequestData req)
        {
            Product? prod = await req.ReadFromJsonAsync<Product>();
            return prod ?? new Product { };
        }
    }
}

HTTP trigger, write records with mapping

The following example shows a C# function that adds a collection of records to a database. The function uses mapping that transforms a Product to Item.

To transform data from Product to Item, the function uses a mapping reference:

.create-merge table Item (ItemID:long, ItemName:string, ItemCost:float)


-- Create a mapping that transforms an Item to a Product

.create-or-alter table Product ingestion json mapping "item_to_product_json" '[{"column":"ProductID","path":"$.ItemID"},{"column":"Name","path":"$.ItemName"},{"column":"Cost","path":"$.ItemCost"}]'

namespace Microsoft.Azure.WebJobs.Extensions.Kusto.SamplesOutOfProc.OutputBindingSamples.Common
{
    public class Item
    {
        public long ItemID { get; set; }

        public string? ItemName { get; set; }

        public double ItemCost { get; set; }
    }
}

using Microsoft.Azure.Functions.Worker;
using Microsoft.Azure.Functions.Worker.Extensions.Kusto;
using Microsoft.Azure.Functions.Worker.Http;
using Microsoft.Azure.WebJobs.Extensions.Kusto.SamplesOutOfProc.OutputBindingSamples.Common;

namespace Microsoft.Azure.WebJobs.Extensions.Kusto.SamplesOutOfProc.OutputBindingSamples
{
    public static class AddProductsWithMapping
    {
        [Function("AddProductsWithMapping")]
        [KustoOutput(Database: "productsdb", Connection = "KustoConnectionString", TableName = "Products", MappingRef = "item_to_product_json")]
        public static async Task<Item> Run(
            [HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = "addproductswithmapping")]
            HttpRequestData req)
        {
            Item? item = await req.ReadFromJsonAsync<Item>();
            return item ?? new Item { };
        }
    }
}

More samples for the Java Azure Data Explorer input binding are available in the GitHub repository.

This section contains the following examples:

The examples refer to a Products class (in a separate file Product.java) and a corresponding database table Products (defined earlier):

package com.microsoft.azure.kusto.common;

import com.fasterxml.jackson.annotation.JsonProperty;

public class Product {
    @JsonProperty("ProductID")
    public long ProductID;
    @JsonProperty("Name")
    public String Name;
    @JsonProperty("Cost")
    public double Cost;

    public Product() {
    }

    public Product(long ProductID, String name, double Cost) {
        this.ProductID = ProductID;
        this.Name = name;
        this.Cost = Cost;
    }
}

HTTP trigger, write a record to a table

The following example shows an Azure Data Explorer output binding in a Java function that adds a product record to a table. The function uses data provided in an HTTP POST request as a JSON body. The function takes another dependency on the com.fasterxml.jackson.core library to parse the JSON body.

<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
    <version>2.13.4.1</version>
</dependency>

package com.microsoft.azure.kusto.outputbindings;

import com.fasterxml.jackson.databind.ObjectMapper;
import com.microsoft.azure.functions.HttpMethod;
import com.microsoft.azure.functions.HttpRequestMessage;
import com.microsoft.azure.functions.HttpResponseMessage;
import com.microsoft.azure.functions.HttpStatus;
import com.microsoft.azure.functions.OutputBinding;
import com.microsoft.azure.functions.annotation.AuthorizationLevel;
import com.microsoft.azure.functions.annotation.FunctionName;
import com.microsoft.azure.functions.annotation.HttpTrigger;
import com.microsoft.azure.functions.kusto.annotation.KustoOutput;
import com.microsoft.azure.kusto.common.Product;

import java.io.IOException;
import java.util.Optional;

import static com.microsoft.azure.kusto.common.Constants.*;

public class AddProduct {
    @FunctionName("AddProduct")
    public HttpResponseMessage run(@HttpTrigger(name = "req", methods = {
            HttpMethod.POST}, authLevel = AuthorizationLevel.ANONYMOUS, route = "addproductuni") HttpRequestMessage<Optional<String>> request,
            @KustoOutput(name = "product", database = "productsdb", tableName = "Products", connection = KUSTOCONNSTR) OutputBinding<Product> product)
            throws IOException {

        if (request.getBody().isPresent()) {
            String json = request.getBody().get();
            ObjectMapper mapper = new ObjectMapper();
            Product p = mapper.readValue(json, Product.class);
            product.setValue(p);
            return request.createResponseBuilder(HttpStatus.OK).header("Content-Type", "application/json").body(product)
                    .build();
        } else {
            return request.createResponseBuilder(HttpStatus.NO_CONTENT).header("Content-Type", "application/json")
                    .build();
        }
    }
}

HTTP trigger, write to two tables

The following example shows an Azure Data Explorer output binding in a Java function that adds records to a database in two different tables (Product and ProductChangeLog). The function uses data provided in an HTTP POST request as a JSON body and multiple output bindings. The function takes another dependency on the com.fasterxml.jackson.core library to parse the JSON body.

<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
    <version>2.13.4.1</version>
</dependency>

The second table, ProductsChangeLog, corresponds to the following definition:

.create-merge table ProductsChangeLog (ProductID:long, CreatedAt:datetime)

and Java class in ProductsChangeLog.java:

package com.microsoft.azure.kusto.common;

import com.fasterxml.jackson.annotation.JsonProperty;

public class ProductsChangeLog {
    @JsonProperty("ProductID")
    public long ProductID;
    @JsonProperty("CreatedAt")
    public String CreatedAt;

    public ProductsChangeLog() {
    }

    public ProductsChangeLog(long ProductID, String CreatedAt) {
        this.ProductID = ProductID;
        this.CreatedAt = CreatedAt;
    }
}

package com.microsoft.azure.kusto.outputbindings;

import com.fasterxml.jackson.databind.ObjectMapper;
import com.microsoft.azure.functions.HttpMethod;
import com.microsoft.azure.functions.HttpRequestMessage;
import com.microsoft.azure.functions.HttpResponseMessage;
import com.microsoft.azure.functions.HttpStatus;
import com.microsoft.azure.functions.OutputBinding;
import com.microsoft.azure.functions.annotation.AuthorizationLevel;
import com.microsoft.azure.functions.annotation.FunctionName;
import com.microsoft.azure.functions.annotation.HttpTrigger;
import com.microsoft.azure.functions.kusto.annotation.KustoOutput;
import com.microsoft.azure.kusto.common.Product;
import com.microsoft.azure.kusto.common.ProductsChangeLog;

import static com.microsoft.azure.kusto.common.Constants.*;

import java.io.IOException;
import java.time.Clock;
import java.time.Instant;
import java.util.Optional;

public class AddMultiTable {
    @FunctionName("AddMultiTable")
    public HttpResponseMessage run(@HttpTrigger(name = "req", methods = {
            HttpMethod.POST}, authLevel = AuthorizationLevel.ANONYMOUS, route = "addmultitable") HttpRequestMessage<Optional<String>> request,
            @KustoOutput(name = "product", database = "productsdb", tableName = "Products", connection = KUSTOCONNSTR) OutputBinding<Product> product,
            @KustoOutput(name = "productChangeLog", database = "productsdb", tableName = "ProductsChangeLog",
                    connection = KUSTOCONNSTR) OutputBinding<ProductsChangeLog> productChangeLog)
            throws IOException {

        if (request.getBody().isPresent()) {
            String json = request.getBody().get();
            ObjectMapper mapper = new ObjectMapper();
            Product p = mapper.readValue(json, Product.class);
            product.setValue(p);
            productChangeLog.setValue(new ProductsChangeLog(p.ProductID, Instant.now(Clock.systemUTC()).toString()));
            return request.createResponseBuilder(HttpStatus.OK).header("Content-Type", "application/json").body(product)
                    .build();
        } else {
            return request.createResponseBuilder(HttpStatus.NO_CONTENT).header("Content-Type", "application/json")
                    .build();
        }
    }
}

More samples for the Azure Data Explorer output binding are available in the GitHub repository.

This section contains the following examples:

The examples refer to a database table.

The examples refer to the tables Products and ProductsChangeLog (defined earlier).

HTTP trigger, write records to a table

The following example shows an Azure Data Explorer output binding in a function.json file and a JavaScript function that adds records to a table. The function uses data provided in an HTTP POST request as a JSON body.

The following example is binding data in the function.json file:

{
  "bindings": [
    {
      "authLevel": "function",
      "name": "req",
      "direction": "in",
      "type": "httpTrigger",
      "methods": [
        "post"
      ],
      "route": "addproduct"
    },
    {
      "name": "$return",
      "type": "http",
      "direction": "out"
    },
    {
      "name": "product",
      "type": "kusto",
      "database": "productsdb",
      "direction": "out",
      "tableName": "Products",
      "connection": "KustoConnectionString"
    }
  ],
  "disabled": false
}

The configuration section explains these properties.

The following snippet is sample JavaScript code:

// Insert the product, which will insert it into the Products table.
module.exports = async function (context, req) {
    // Note that this expects the body to be a JSON object or array of objects which have a property
    // matching each of the columns in the table to insert to.
    context.bindings.product = req.body;
    return {
        status: 201,
        body: req.body
    };
}

HTTP trigger, write to two tables

The following example shows an Azure Data Explorer output binding in a function.json file and a JavaScript function that adds records to a database in two different tables (Products and ProductsChangeLog). The function uses data provided in an HTTP POST request as a JSON body and multiple output bindings.

The second table, ProductsChangeLog, corresponds to the following definition:

.create-merge table ProductsChangeLog (ProductID:long, CreatedAt:datetime)

The following snippet is binding data in the function.json file:

{
  "bindings": [
    {
      "authLevel": "function",
      "name": "req",
      "direction": "in",
      "type": "httpTrigger",
      "methods": [
        "post"
      ],
      "route": "addmultitable"
    },
    {
      "name": "res",
      "type": "http",
      "direction": "out"
    },
    {
      "name": "product",
      "type": "kusto",
      "database": "productsdb",
      "direction": "out",
      "tableName": "Products",
      "connection": "KustoConnectionString"
    },
    {
      "name": "productchangelog",
      "type": "kusto",
      "database": "productsdb",
      "direction": "out",
      "tableName": "ProductsChangeLog",
      "connection": "KustoConnectionString"
    }
  ],
  "disabled": false
}

The configuration section explains these properties.

The following snippet is sample JavaScript code:

module.exports = async function (context, req) {
    context.log('JavaScript HTTP trigger and Kusto output binding function processed a request.');
    context.log(req.body);

    if (req.body) {
        var changeLog = {ProductID:req.body.ProductID, CreatedAt: new Date().toISOString()};
        context.bindings.product = req.body;
        context.bindings.productchangelog = changeLog;
        context.res = {
            body: req.body,
            mimetype: "application/json",
            status: 201
        }
    } else {
        context.res = {
            status: 400,
            body: "Error reading request body"
        }
    }
}

More samples for the Azure Data Explorer output binding are available in the GitHub repository.

This section contains the following examples:

The examples refer to the tables Products and ProductsChangeLog (defined earlier).

HTTP trigger, write records to a table

The following example shows an Azure Data Explorer output binding in a function.json file and a Python function that adds records to a table. The function uses data provided in an HTTP POST request as a JSON body.

The following snippet is binding data in the function.json file:

{
  "scriptFile": "__init__.py",
  "bindings": [
    {
      "authLevel": "Anonymous",
      "type": "httpTrigger",
      "direction": "in",
      "name": "req",
      "methods": [
        "post"
      ],
      "route": "addproductuni"
    },
    {
      "type": "http",
      "direction": "out",
      "name": "$return"
    },
    {
      "name": "product",
      "type": "kusto",
      "database": "sdktestsdb",
      "direction": "out",
      "tableName": "Products",
      "connection": "KustoConnectionString"
    }
  ]
}

The configuration section explains these properties.

The following snippet is sample Python code:

import azure.functions as func
from Common.product import Product


def main(req: func.HttpRequest, product: func.Out[str]) -> func.HttpResponse:
    body = str(req.get_body(),'UTF-8')
    product.set(body)
    return func.HttpResponse(
        body=body,
        status_code=201,
        mimetype="application/json"
    )

HTTP trigger, write to two tables

The following example shows an Azure Data Explorer output binding in a function.json file and a JavaScript function that adds records to a database in two different tables (Products and ProductsChangeLog). The function uses data provided in an HTTP POST request as a JSON body and multiple output bindings. The second table, ProductsChangeLog, corresponds to the following definition:

.create-merge table ProductsChangeLog (ProductID:long, CreatedAt:datetime)

The following snippet is binding data in the function.json file:

{
  "scriptFile": "__init__.py",
  "bindings": [
    {
      "authLevel": "Anonymous",
      "type": "httpTrigger",
      "direction": "in",
      "name": "req",
      "methods": [
        "post"
      ],
      "route": "addmultitable"
    },
    {
      "type": "http",
      "direction": "out",
      "name": "$return"
    },
    {
      "name": "product",
      "type": "kusto",
      "database": "sdktestsdb",
      "direction": "out",
      "tableName": "Products",
      "connection": "KustoConnectionString"
    },
    {
      "name": "productchangelog",
      "type": "kusto",
      "database": "sdktestsdb",
      "direction": "out",
      "tableName": "ProductsChangeLog",
      "connection": "KustoConnectionString"
    }
  ]
}

The configuration section explains these properties.

The following snippet is sample Python code:

import json
from datetime import datetime

import azure.functions as func
from Common.product import Product


def main(req: func.HttpRequest, product: func.Out[str],productchangelog: func.Out[str]) -> func.HttpResponse:
    body = str(req.get_body(),'UTF-8')
    # parse x:
    product.set(body)
    id = json.loads(body)["ProductID"]

    changelog = {
        "ProductID": id,
        "CreatedAt": datetime.now().isoformat(),
    }
    productchangelog.set(json.dumps(changelog))
    return func.HttpResponse(
        body=body,
        status_code=201,
        mimetype="application/json"
    )

Attributes

The C# library uses the KustoAttribute attribute to declare the Azure Data Explorer bindings on the function, which has the following properties.

Attribute property Description
Database Required. The database against which the query must be executed.
Connection Required. The name of the variable that holds the connection string, which is resolved through environment variables or through function app settings. Defaults to look up on the variable KustoConnectionString. At runtime, this variable is looked up against the environment. Documentation on the connection string is at Kusto connection strings. For example: "KustoConnectionString": "Data Source=https://your_cluster.kusto.windows.net;Database=your_Database;Fed=True;AppClientId=your_AppId;AppKey=your_AppKey;Authority Id=your_TenantId.
TableName Required. The table to ingest the data into.
MappingRef Optional. Attribute to pass a mapping ref that's already defined in the cluster.
ManagedServiceIdentity Optional. A managed identity can be used to connect to Azure Data Explorer. To use a system managed identity, use "system." Any other identity names are interpreted as a user managed identity.
DataFormat Optional. The default data format is multijson/json. It can be set to text formats supported in the datasource format enumeration. Samples are validated and provided for CSV and JSON formats.

Annotations

The Java functions runtime library uses the @KustoInput annotation (com.microsoft.azure.functions.kusto.annotation.KustoOutput).

Element Description
name Required. The name of the variable that represents the query results in function code.
database Required. The database against which the query must be executed.
connection Required. The name of the variable that holds the connection string, which is resolved through environment variables or through function app settings. Defaults to look up on the variable KustoConnectionString. At runtime, this variable is looked up against the environment. Documentation on the connection string is at Kusto connection strings. For example: "KustoConnectionString": "Data Source=https://your_cluster.kusto.windows.net;Database=your_Database;Fed=True;AppClientId=your_AppId;AppKey=your_AppKey;Authority Id=your_TenantId.
tableName Required. The table to ingest the data into.
mappingRef Optional. Attribute to pass a mapping ref that's already defined in the cluster.
dataFormat Optional. The default data format is multijson/json. It can be set to text formats supported in the datasource format enumeration. Samples are validated and provided for CSV and JSON formats.
managedServiceIdentity A managed identity can be used to connect to Azure Data Explorer. To use a system managed identity, use "system." Any other identity names are interpreted as a user managed identity.

Configuration

The following table explains the binding configuration properties that you set in the function.json file.

function.json property Description
type Required. Must be set to kusto.
direction Required. Must be set to out.
name Required. The name of the variable that represents the query results in function code.
database Required. The database against which the query must be executed.
connection Required. The name of the variable that holds the connection string, resolved through environment variables or through function app settings. Defaults to look up on the variable KustoConnectionString. At runtime, this variable is looked up against the environment. Documentation on the connection string is at Kusto connection strings. For example: "KustoConnectionString": "Data Source=https://your_cluster.kusto.windows.net;Database=your_Database;Fed=True;AppClientId=your_AppId;AppKey=your_AppKey;Authority Id=your_TenantId.
tableName Required. The table to ingest the data into.
mappingRef Optional. Attribute to pass a mapping ref that's already defined in the cluster.
dataFormat Optional. The default data format is multijson/json. It can be set to text formats supported in the datasource format enumeration. Samples are validated and provided for CSV and JSON formats.
managedServiceIdentity A managed identity can be used to connect to Azure Data Explorer. To use a system managed identity, use "system." Any other identity names are interpreted as a user managed identity.

When you're developing locally, add your application settings in the local.settings.json file in the Values collection.

Usage

The attribute's constructor takes the database and the attributes TableName, MappingRef, and DataFormat and the connection setting name. The KQL command can be a KQL statement or a KQL function. The connection string setting name corresponds to the application setting (in local.settings.json for local development) that contains the Kusto connection strings. For example:"KustoConnectionString": "Data Source=https://your_cluster.kusto.windows.net;Database=your_Database;Fed=True;AppClientId=your_AppId;AppKey=your_AppKey;Authority Id=your_TenantId. Queries executed by the input binding are parameterized. The values provided in the KQL parameters are used at runtime.

Next steps

Read data from a table (input binding)