Edit

Azure OpenAI embeddings input binding for Azure Functions

Important

The Azure OpenAI extension for Azure Functions is currently in preview.

The Azure OpenAI embeddings input binding allows you to generate embeddings for inputs. The binding can generate embeddings from files or raw text inputs.

For information on setup and configuration details of the Azure OpenAI extension, see Azure OpenAI extensions for Azure Functions. To learn more about embeddings in Azure OpenAI Service, see Understand embeddings in Azure OpenAI Service.

Note

References and examples are only provided for the Node.js v4 model.

Note

References and examples are only provided for the Python v2 model.

Note

While both C# process models are supported, only isolated worker model examples are provided.

Example

Go support isn't currently available for this binding.

This example shows how to generate embeddings for a raw text string.

internal class EmbeddingsRequest
{
    [JsonPropertyName("rawText")]
    public string? RawText { get; set; }

    [JsonPropertyName("filePath")]
    public string? FilePath { get; set; }

    [JsonPropertyName("url")]
    public string? Url { get; set; }
}

/// <summary>
/// Example showing how to use the <see cref="EmbeddingsAttribute"/> input binding to generate embeddings 
/// for a raw text string.
/// </summary>
[Function(nameof(GenerateEmbeddings_Http_RequestAsync))]
public async Task GenerateEmbeddings_Http_RequestAsync(
    [HttpTrigger(AuthorizationLevel.Function, "post", Route = "embeddings")] HttpRequestData req,
    [EmbeddingsInput("{rawText}", InputType.RawText, EmbeddingsModel = "%EMBEDDING_MODEL_DEPLOYMENT_NAME%")] EmbeddingsContext embeddings)
{
    using StreamReader reader = new(req.Body);
    string request = await reader.ReadToEndAsync();

    EmbeddingsRequest? requestBody = JsonSerializer.Deserialize<EmbeddingsRequest>(request);

    this.logger.LogInformation(
        "Received {count} embedding(s) for input text containing {length} characters.",
        embeddings.Count,
        requestBody?.RawText?.Length);

    // TODO: Store the embeddings into a database or other storage.
}

This example shows how to retrieve embeddings stored at a specified file that is accessible to the function.

[Function(nameof(GetEmbeddings_Http_FilePath))]
public async Task GetEmbeddings_Http_FilePath(
    [HttpTrigger(AuthorizationLevel.Function, "post", Route = "embeddings-from-file")] HttpRequestData req,
    [EmbeddingsInput("{filePath}", InputType.FilePath, MaxChunkLength = 512, EmbeddingsModel = "%EMBEDDING_MODEL_DEPLOYMENT_NAME%")] EmbeddingsContext embeddings)
{
    using StreamReader reader = new(req.Body);
    string request = await reader.ReadToEndAsync();

    EmbeddingsRequest? requestBody = JsonSerializer.Deserialize<EmbeddingsRequest>(request);
    this.logger.LogInformation(
        "Received {count} embedding(s) for input file '{path}'.",
        embeddings.Count,
        requestBody?.FilePath);

    // TODO: Store the embeddings into a database or other storage.
}

This example shows how to generate embeddings for a raw text string.

@FunctionName("GenerateEmbeddingsHttpRequest")
public HttpResponseMessage generateEmbeddingsHttpRequest(
        @HttpTrigger(
            name = "req", 
            methods = {HttpMethod.POST},
            authLevel = AuthorizationLevel.FUNCTION,
            route = "embeddings")
        HttpRequestMessage<EmbeddingsRequest> request,
        @EmbeddingsInput(name = "Embeddings", input = "{RawText}", inputType = InputType.RawText, embeddingsModel = "%EMBEDDING_MODEL_DEPLOYMENT_NAME%") String embeddingsContext,
        final ExecutionContext context) {

    if (request.getBody() == null) 
    {
        throw new IllegalArgumentException(
                "Invalid request body. Make sure that you pass in {\"rawText\": value } as the request body.");
    }

    JSONObject embeddingsContextJsonObject = new JSONObject(embeddingsContext);

    context.getLogger().info(String.format("Received %d embedding(s) for input text containing %s characters.",
            embeddingsContextJsonObject.get("count"),
            request.getBody().getRawText().length()));

    // TODO: Store the embeddings into a database or other storage.
    return request.createResponseBuilder(HttpStatus.ACCEPTED)
            .header("Content-Type", "application/json")
            .build();
}

This example shows how to retrieve embeddings stored at a specified file that is accessible to the function.

@FunctionName("GenerateEmbeddingsHttpFilePath")
public HttpResponseMessage generateEmbeddingsHttpFilePath(
    @HttpTrigger(
        name = "req", 
        methods = {HttpMethod.POST},
        authLevel = AuthorizationLevel.FUNCTION,
        route = "embeddings-from-file")
    HttpRequestMessage<EmbeddingsRequest> request,
    @EmbeddingsInput(name = "Embeddings", input = "{FilePath}", inputType = InputType.FilePath, maxChunkLength = 512, embeddingsModel = "%EMBEDDING_MODEL_DEPLOYMENT_NAME%") String embeddingsContext,
    final ExecutionContext context) {

    if (request.getBody() == null) 
    {
        throw new IllegalArgumentException(
                "Invalid request body. Make sure that you pass in {\"filePath\": value } as the request body.");
    }

    JSONObject embeddingsContextJsonObject = new JSONObject(embeddingsContext);

    context.getLogger().info(String.format("Received %d embedding(s) for input file %s.",
            embeddingsContextJsonObject.get("count"),
            request.getBody().getFilePath()));

    // TODO: Store the embeddings into a database or other storage.
    return request.createResponseBuilder(HttpStatus.ACCEPTED)
            .header("Content-Type", "application/json")
            .build();
}

This example shows how to generate embeddings for a raw text string.

const embeddingsHttpInput = input.generic({
    input: '{rawText}',
    inputType: 'RawText',
    type: 'embeddings',
    embeddingsModel: '%EMBEDDING_MODEL_DEPLOYMENT_NAME%'
})

app.http('generateEmbeddings', {
    methods: ['POST'],
    route: 'embeddings',
    authLevel: 'function',
    extraInputs: [embeddingsHttpInput],
    handler: async (request, context) => {
        let requestBody = await request.json();
        let response = context.extraInputs.get(embeddingsHttpInput);

        context.log(
            `Received ${response.count} embedding(s) for input text containing ${requestBody.rawText?.length ?? 0} characters.`
        );
        
        // TODO: Store the embeddings into a database or other storage.

        return {status: 202}
    }
});

interface EmbeddingsHttpRequest {
    rawText?: string;
}

const embeddingsHttpInput = input.generic({
    input: '{rawText}',
    inputType: 'RawText',
    type: 'embeddings',
    embeddingsModel: '%EMBEDDING_MODEL_DEPLOYMENT_NAME%'
})

app.http('generateEmbeddings', {
    methods: ['POST'],
    route: 'embeddings',
    authLevel: 'function',
    extraInputs: [embeddingsHttpInput],
    handler: async (request, context) => {
        let requestBody: EmbeddingsHttpRequest = await request.json();
        let response: any = context.extraInputs.get(embeddingsHttpInput);

        context.log(
            `Received ${response.count} embedding(s) for input text containing ${requestBody.rawText?.length ?? 0} characters.`
        );
        
        // TODO: Store the embeddings into a database or other storage.

        return {status: 202}
    }
});

This example shows how to generate embeddings for a raw text string.

const embeddingsFilePathInput = input.generic({
    input: '{filePath}',
    inputType: 'FilePath',
    type: 'embeddings',
    maxChunkLength: 512,
    embeddingsModel: '%EMBEDDING_MODEL_DEPLOYMENT_NAME%'
})

app.http('getEmbeddingsFilePath', {
    methods: ['POST'],
    route: 'embeddings-from-file',
    authLevel: 'function',
    extraInputs: [embeddingsFilePathInput],
    handler: async (request, context) => {
        let requestBody = await request.json();
        let response = context.extraInputs.get(embeddingsFilePathInput);

        context.log(
            `Received ${response.count} embedding(s) for input file ${requestBody.filePath}.`
        );
        
        // TODO: Store the embeddings into a database or other storage.

        return {status: 202}
    }
});

interface EmbeddingsFilePath {
    filePath?: string;
}

const embeddingsFilePathInput = input.generic({
    input: '{filePath}',
    inputType: 'FilePath',
    type: 'embeddings',
    maxChunkLength: 512,
    embeddingsModel: '%EMBEDDING_MODEL_DEPLOYMENT_NAME%'
})

app.http('getEmbeddingsFilePath', {
    methods: ['POST'],
    route: 'embeddings-from-file',
    authLevel: 'function',
    extraInputs: [embeddingsFilePathInput],
    handler: async (request, context) => {
        let requestBody: EmbeddingsFilePath = await request.json();
        let response: any = context.extraInputs.get(embeddingsFilePathInput);

        context.log(
            `Received ${response.count} embedding(s) for input file ${requestBody.filePath}.`
        );
        
        // TODO: Store the embeddings into a database or other storage.

        return {status: 202}
    }
});

This example shows how to generate embeddings for a raw text string.

Here's the function.json file for generating the embeddings:

{
  "bindings": [
    {
      "authLevel": "function",
      "type": "httpTrigger",
      "direction": "in",
      "name": "Request",
      "route": "embeddings",
      "methods": [
        "post"
      ]
    },
    {
      "type": "http",
      "direction": "out",
      "name": "Response"
    },
    {
      "name": "Embeddings",
      "type": "embeddings",
      "direction": "in",
      "inputType": "RawText",
      "input": "{rawText}",
      "embeddingsModel": "%EMBEDDING_MODEL_DEPLOYMENT_NAME%"
    }
  ]
}

For more information about function.json file properties, see the Configuration section.

using namespace System.Net

param($Request, $TriggerMetadata, $Embeddings)

$input = $Request.Body.RawText

Write-Host "Received $($Embeddings.Count) embedding(s) for input text containing $($input.Length) characters."

Push-OutputBinding -Name Response -Value ([HttpResponseContext]@{
        StatusCode = [HttpStatusCode]::Accepted
})

This example shows how to generate embeddings for a raw text string.

@app.function_name("GenerateEmbeddingsHttpRequest")
@app.route(route="embeddings", methods=["POST"])
@app.embeddings_input(
    arg_name="embeddings",
    input="{rawText}",
    input_type="rawText",
    embeddings_model="%EMBEDDING_MODEL_DEPLOYMENT_NAME%",
)
def generate_embeddings_http_request(
    req: func.HttpRequest, embeddings: str
) -> func.HttpResponse:
    user_message = req.get_json()
    embeddings_json = json.loads(embeddings)
    embeddings_request = {"raw_text": user_message.get("rawText")}
    logging.info(
        f'Received {embeddings_json.get("count")} embedding(s) for input text '
        f'containing {len(embeddings_request.get("raw_text"))} characters.'
    )
    # TODO: Store the embeddings into a database or other storage.
    return func.HttpResponse(status_code=200)

Attributes

Apply the EmbeddingsInput attribute to define an embeddings input binding, which supports these parameters:

Parameter Description
Input The input string for which to generate embeddings.
AIConnectionName Optional. Gets or sets the name of the configuration section for AI service connectivity settings. For Azure OpenAI: If specified, looks for "Endpoint" and "Key" values in this configuration section. If not specified or the section doesn't exist, falls back to environment variables: AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_KEY. For user-assigned managed identity authentication, this property is required. For OpenAI service (non-Azure), set the OPENAI_API_KEY environment variable.
EmbeddingsModel Optional. The ID of the model to use, which defaults to text-embedding-ada-002. You shouldn't change the model for an existing database. For more information, see Usage.
MaxChunkLength Optional. The maximum number of characters used for chunking the input. For more information, see Usage.
MaxOverlap Optional. Gets or sets the maximum number of characters to overlap between chunks.
InputType Optional. Gets the type of the input.

Annotations

The EmbeddingsInput annotation enables you to define an embeddings input binding, which supports these parameters:

Element Description
name Gets or sets the name of the input binding.
input The input string for which to generate embeddings.
aiConnectionName Optional. Gets or sets the name of the configuration section for AI service connectivity settings. For Azure OpenAI: If specified, looks for "Endpoint" and "Key" values in this configuration section. If not specified or the section doesn't exist, falls back to environment variables: AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_KEY. For user-assigned managed identity authentication, this property is required. For OpenAI service (non-Azure), set the OPENAI_API_KEY environment variable.
embeddingsModel Optional. The ID of the model to use, which defaults to text-embedding-ada-002. You shouldn't change the model for an existing database. For more information, see Usage.
maxChunkLength Optional. The maximum number of characters used for chunking the input. For more information, see Usage.
maxOverlap Optional. Gets or sets the maximum number of characters to overlap between chunks.
inputType Optional. Gets the type of the input.

Decorators

During the preview, define the input binding as a generic_input_binding binding of type embeddings, which supports these parameters: embeddings decorator supports these parameters:

Parameter Description
arg_name The name of the variable that represents the binding parameter.
input The input string for which to generate embeddings.
ai_connection_name Optional. Gets or sets the name of the configuration section for AI service connectivity settings. For Azure OpenAI: If specified, looks for "Endpoint" and "Key" values in this configuration section. If not specified or the section doesn't exist, falls back to environment variables: AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_KEY. For user-assigned managed identity authentication, this property is required. For OpenAI service (non-Azure), set the OPENAI_API_KEY environment variable.
embeddings_model Optional. The ID of the model to use, which defaults to text-embedding-ada-002. You shouldn't change the model for an existing database. For more information, see Usage.
maxChunkLength Optional. The maximum number of characters used for chunking the input. For more information, see Usage.
max_overlap Optional. Gets or sets the maximum number of characters to overlap between chunks.
input_type Gets the type of the input.

Configuration

The binding supports these configuration properties that you set in the function.json file.

Property Description
type Must be EmbeddingsInput.
direction Must be in.
name The name of the input binding.
input The input string for which to generate embeddings.
aiConnectionName Optional. Gets or sets the name of the configuration section for AI service connectivity settings. For Azure OpenAI: If specified, looks for "Endpoint" and "Key" values in this configuration section. If not specified or the section doesn't exist, falls back to environment variables: AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_KEY. For user-assigned managed identity authentication, this property is required. For OpenAI service (non-Azure), set the OPENAI_API_KEY environment variable.
embeddingsModel Optional. The ID of the model to use, which defaults to text-embedding-ada-002. You shouldn't change the model for an existing database. For more information, see Usage.
maxChunkLength Optional. The maximum number of characters used for chunking the input. For more information, see Usage.
maxOverlap Optional. Gets or sets the maximum number of characters to overlap between chunks.
inputType Optional. Gets the type of the input.

Configuration

The binding supports these properties, which are defined in your code:

Property Description
input The input string for which to generate embeddings.
aiConnectionName Optional. Gets or sets the name of the configuration section for AI service connectivity settings. For Azure OpenAI: If specified, looks for "Endpoint" and "Key" values in this configuration section. If not specified or the section doesn't exist, falls back to environment variables: AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_KEY. For user-assigned managed identity authentication, this property is required. For OpenAI service (non-Azure), set the OPENAI_API_KEY environment variable.
embeddingsModel Optional. The ID of the model to use, which defaults to text-embedding-ada-002. You shouldn't change the model for an existing database. For more information, see Usage.
maxChunkLength Optional. The maximum number of characters used for chunking the input. For more information, see Usage.
maxOverlap Optional. Gets or sets the maximum number of characters to overlap between chunks.
inputType Optional. Gets the type of the input.

See the Example section for complete examples.

Usage

Changing the default embeddings model changes the way that embeddings are stored in the vector database. Changing the default model can cause the lookups to start misbehaving when they don't match the rest of the data that was previously ingested into the vector database. The default model for embeddings is text-embedding-ada-002.

When calculating the maximum character length for input chunks, consider that the maximum input tokens allowed for second-generation input embedding models like text-embedding-ada-002 is 8191. A single token is approximately four characters in length (in English), which translates to roughly 32,000 (English) characters of input that can fit into a single chunk.

Connections

To use the Azure OpenAI binding extension, you need to specify a connection to an OpenAI model definition. Set the OpenAI model connection in your bindings by using one of these approaches:

  • Use the AIConnectionName binding property (preferred for Azure OpenAI).
  • Set AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_KEY in app settings (for Azure OpenAI).
  • Set only Open_API_Key in app settings (for https://api.openai.com).

The way you set the connection depends on both the model API and the authentication method, as indicated by the following table:

Authentication/Model API Azure OpenAI OpenAI (https://api.openai.com)
Managed identity connection AIConnectionName Not supported
Key Vault reference AZURE_OPENAI_ENDPOINT
AZURE_OPENAI_KEY		 Open_API_Key
App Configuration reference AZURE_OPENAI_ENDPOINT
AZURE_OPENAI_KEY		 Open_API_Key
Shared secret AZURE_OPENAI_ENDPOINT
AZURE_OPENAI_KEY		 Open_API_Key

Use managed identity-based connections and the AIConnectionName property.

When you use AIConnectionName, the value of this property setting depends on the type of connection:

  • Managed identity connection: The AIConnectionName property is a <CONNECTION_NAME_PREFIX> shared by a group of settings that together define an identity-based connection to Azure OpenAI. For more information, see Define identity connections.
  • Key Vault reference: The AIConnectionName property setting returns an Azure Key Vault reference to the location where the API key is centrally maintained. For more information, see Define Key Vault connections.
  • App Configuration reference: The AIConnectionName property setting returns an Azure App Configuration reference that returns an API key or a Key Vault reference. For more information, see Azure App Configuration in the connections article.
  • API key: The AIConnectionName property setting resolves to app settings containing the endpoint and key directly. Because shared keys can be compromised, use managed identity connections when possible. For more information, see Define connections.

To learn more about bindings connections, see Manage connections in Azure Functions.

The OpenAI bindings include an AIConnectionName property that you can use to specify the <ConnectionNamePrefix> for the group of app settings that define the connection to Azure OpenAI:

Setting name Description
<CONNECTION_NAME_PREFIX>__endpoint Sets the URI endpoint of the Azure OpenAI service. This setting is always required.
<CONNECTION_NAME_PREFIX>__clientId Sets the specific user-assigned identity to use when obtaining an access token. Requires that <CONNECTION_NAME_PREFIX>__credential is set to managedidentity. The property accepts a client ID corresponding to a user-assigned identity assigned to the application. It's invalid to specify both a Resource ID and a client ID. If you don't specify this property, the system-assigned identity is used. This property is used differently in local development scenarios, when credential shouldn't be set.
<CONNECTION_NAME_PREFIX>__credential Defines how an access token is obtained for the connection. Use managedidentity for managed identity authentication. This value is only valid when a managed identity is available in the hosting environment.
<CONNECTION_NAME_PREFIX>__managedIdentityResourceId When credential is set to managedidentity, set this property to specify the resource Identifier to use when obtaining a token. The property accepts a resource identifier corresponding to the resource ID of the user-defined managed identity. It's invalid to specify both a resource ID and a client ID. If you don't specify either, the system-assigned identity is used. This property is used differently in local development scenarios, when credential shouldn't be set.
<CONNECTION_NAME_PREFIX>__key Sets the shared secret key required to access the endpoint of the Azure OpenAI service by using key-based authentication. As a security best practice, always use Microsoft Entra ID with managed identities for authentication.

Consider these managed identity connection settings when you set the AIConnectionName property to myAzureOpenAI:

  • myAzureOpenAI__endpoint=https://contoso.openai.azure.com/
  • myAzureOpenAI__credential=managedidentity
  • myAzureOpenAI__clientId=aaaaaaaa-bbbb-cccc-1111-222222222222

At runtime, the host interprets these settings as a single myAzureOpenAI setting:

"myAzureOpenAI":
{
    "endpoint": "https://contoso.openai.azure.com/",
    "credential": "managedidentity",
    "clientId": "aaaaaaaa-bbbb-cccc-1111-222222222222"
}

When you use managed identities, make sure to add your identity to the Cognitive Services OpenAI User role.

When running locally, add these settings to the local.settings.json project file. For more information, see Local development with identity-based connections.

For more information, see Work with application settings.

Related content

  • Last updated on