Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Important
The Azure OpenAI extension for Azure Functions is currently in preview.
The Azure OpenAI text completion input binding allows you to bring the results text completion APIs into your code executions. You can define the binding to use both predefined prompts with parameters or pass through an entire prompt.
For information on setup and configuration details of the Azure OpenAI extension, see Azure OpenAI extensions for Azure Functions. To learn more about Azure OpenAI completions, see Learn how to generate or manipulate text.
Note
References and examples are only provided for the Node.js v4 model.
Note
References and examples are only provided for the Python v2 model.
Note
While both C# process models are supported, only isolated worker model examples are provided.
Example
Go support isn't currently available for this binding.
This example demonstrates the templating pattern, where the HTTP trigger function takes a name parameter and embeds it into a text prompt, which is then sent to the Azure OpenAI completions API by the extension. The response to the prompt is returned in the HTTP response.
[Function(nameof(WhoIs))]
public static IActionResult WhoIs(
[HttpTrigger(AuthorizationLevel.Function, Route = "whois/{name}")] HttpRequestData req,
[TextCompletionInput("Who is {name}?", ChatModel = "%CHAT_MODEL_DEPLOYMENT_NAME%")] TextCompletionResponse response)
{
return new OkObjectResult(response.Content);
}
This example takes a prompt as input, sends it directly to the completions API, and returns the response as the output.
[Function(nameof(GenericCompletion))]
public static IActionResult GenericCompletion(
[HttpTrigger(AuthorizationLevel.Function, "post")] HttpRequestData req,
[TextCompletionInput("{Prompt}", ChatModel = "%CHAT_MODEL_DEPLOYMENT_NAME%")] TextCompletionResponse response,
ILogger log)
{
string text = response.Content;
return new OkObjectResult(text);
}
This example demonstrates the templating pattern, where the HTTP trigger function takes a name parameter and embeds it into a text prompt, which is then sent to the Azure OpenAI completions API by the extension. The response to the prompt is returned in the HTTP response.
@FunctionName("WhoIs")
public HttpResponseMessage whoIs(
@HttpTrigger(
name = "req",
methods = {HttpMethod.GET},
authLevel = AuthorizationLevel.FUNCTION,
route = "whois/{name}")
HttpRequestMessage<Optional<String>> request,
@BindingName("name") String name,
@TextCompletion(prompt = "Who is {name}?", chatModel = "%CHAT_MODEL_DEPLOYMENT_NAME%", name = "response", isReasoningModel = false) TextCompletionResponse response,
final ExecutionContext context) {
return request.createResponseBuilder(HttpStatus.OK)
.header("Content-Type", "application/json")
.body(response.getContent())
.build();
}
This example takes a prompt as input, sends it directly to the completions API, and returns the response as the output.
@FunctionName("GenericCompletion")
public HttpResponseMessage genericCompletion(
@HttpTrigger(
name = "req",
methods = {HttpMethod.POST},
authLevel = AuthorizationLevel.FUNCTION)
HttpRequestMessage<Optional<String>> request,
@TextCompletion(prompt = "{prompt}", chatModel = "%CHAT_MODEL_DEPLOYMENT_NAME%", name = "response", isReasoningModel = false) TextCompletionResponse response,
final ExecutionContext context) {
return request.createResponseBuilder(HttpStatus.OK)
.header("Content-Type", "application/json")
.body(response.getContent())
.build();
}
This example demonstrates the templating pattern, where the HTTP trigger function takes a name parameter and embeds it into a text prompt, which is then sent to the Azure OpenAI completions API by the extension. The response to the prompt is returned in the HTTP response.
const { app, input } = require("@azure/functions");
// This OpenAI completion input requires a {name} binding value.
const openAICompletionInput = input.generic({
prompt: 'Who is {name}?',
maxTokens: '100',
type: 'textCompletion',
chatModel: '%CHAT_MODEL_DEPLOYMENT_NAME%'
})
app.http('whois', {
methods: ['GET'],
route: 'whois/{name}',
authLevel: 'function',
extraInputs: [openAICompletionInput],
handler: async (_request, context) => {
var response = context.extraInputs.get(openAICompletionInput)
return { body: response.content.trim() }
}
});
This example demonstrates the templating pattern, where the HTTP trigger function takes a name parameter and embeds it into a text prompt, which is then sent to the Azure OpenAI completions API by the extension. The response to the prompt is returned in the HTTP response.
import { app, input } from "@azure/functions";
// This OpenAI completion input requires a {name} binding value.
const openAICompletionInput = input.generic({
prompt: 'Who is {name}?',
maxTokens: '100',
type: 'textCompletion',
chatModel: '%CHAT_MODEL_DEPLOYMENT_NAME%'
})
app.http('whois', {
methods: ['GET'],
route: 'whois/{name}',
authLevel: 'function',
extraInputs: [openAICompletionInput],
handler: async (_request, context) => {
var response: any = context.extraInputs.get(openAICompletionInput)
return { body: response.content.trim() }
}
});
This example demonstrates the templating pattern, where the HTTP trigger function takes a name parameter and embeds it into a text prompt, which is then sent to the Azure OpenAI completions API by the extension. The response to the prompt is returned in the HTTP response.
Here's the function.json file for TextCompletionResponse:
{
"bindings": [
{
"authLevel": "function",
"type": "httpTrigger",
"direction": "in",
"name": "Request",
"route": "whois/{name}",
"methods": [
"get"
]
},
{
"type": "http",
"direction": "out",
"name": "Response"
},
{
"type": "textCompletion",
"direction": "in",
"name": "TextCompletionResponse",
"prompt": "Who is {name}?",
"maxTokens": "100",
"chatModel": "%CHAT_MODEL_DEPLOYMENT_NAME%"
}
]
}
For more information about function.json file properties, see the Configuration section.
The code simply returns the text from the completion API as the response:
using namespace System.Net
param($Request, $TriggerMetadata, $TextCompletionResponse)
Push-OutputBinding -Name Response -Value ([HttpResponseContext]@{
StatusCode = [HttpStatusCode]::OK
Body = $TextCompletionResponse.Content
})
This example demonstrates the templating pattern, where the HTTP trigger function takes a name parameter and embeds it into a text prompt, which is then sent to the Azure OpenAI completions API by the extension. The response to the prompt is returned in the HTTP response.
@app.route(route="whois/{name}", methods=["GET"])
@app.text_completion_input(
arg_name="response",
prompt="Who is {name}?",
max_tokens="100",
chat_model="%CHAT_MODEL_DEPLOYMENT_NAME%",
)
def whois(req: func.HttpRequest, response: str) -> func.HttpResponse:
response_json = json.loads(response)
return func.HttpResponse(response_json["content"], status_code=200)
This example takes a prompt as input, sends it directly to the completions API, and returns the response as the output.
@app.route(route="genericcompletion", methods=["POST"])
@app.text_completion_input(
arg_name="response",
prompt="{Prompt}",
chat_model="%CHAT_MODEL_DEPLOYMENT_NAME%",
)
def genericcompletion(
req: func.HttpRequest,
response: str
) -> func.HttpResponse:
response_json = json.loads(response)
return func.HttpResponse(response_json["content"], status_code=200)
Attributes
The specific attribute you apply to define a text completion input binding depends on your C# process mode.
In the isolated worker model, apply TextCompletionInput to define a text completion input binding.
The attribute supports these parameters:
| Parameter | Description |
|---|---|
| Prompt | Gets or sets the prompt to generate completions for, encoded as a string. |
| AIConnectionName | Optional. Gets or sets the name of the configuration section for AI service connectivity settings. For Azure OpenAI: If specified, looks for "Endpoint" and "Key" values in this configuration section. If not specified or the section doesn't exist, falls back to environment variables: AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_KEY. For user-assigned managed identity authentication, this property is required. For OpenAI service (non-Azure), set the OPENAI_API_KEY environment variable. |
| ChatModel | Optional. Gets or sets the ID of the model to use as a string, with a default value of gpt-3.5-turbo. |
| Temperature | Optional. Gets or sets the sampling temperature to use, as a string between 0 and 2. Higher values (0.8) make the output more random, while lower values like (0.2) make output more focused and deterministic. You should use either Temperature or TopP, but not both. |
| TopP | Optional. Gets or sets an alternative to sampling with temperature, called nucleus sampling, as a string. In this sampling method, the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. You should use either Temperature or TopP, but not both. |
| MaxTokens | Optional. Gets or sets the maximum number of tokens to generate in the completion, as a string with a default of 100. The token count of your prompt plus max_tokens can't exceed the model's context length. Most models have a context length of 2,048 tokens (except for the newest models, which support 4096). |
| IsReasoningModel | Optional. Gets or sets a value indicating whether the chat completion model is a reasoning model. This option is experimental and associated with the reasoning model until all models have parity in the expected properties, with a default value of false. |
Annotations
The TextCompletion annotation enables you to define a text completion input binding, which supports these parameters:
| Element | Description |
|---|---|
| name | Gets or sets the name of the input binding. |
| prompt | Gets or sets the prompt to generate completions for, encoded as a string. |
| aiConnectionName | Optional. Gets or sets the name of the configuration section for AI service connectivity settings. For Azure OpenAI: If specified, looks for "Endpoint" and "Key" values in this configuration section. If not specified or the section doesn't exist, falls back to environment variables: AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_KEY. For user-assigned managed identity authentication, this property is required. For OpenAI service (non-Azure), set the OPENAI_API_KEY environment variable. |
| chatModel | Gets or sets the ID of the model to use as a string, with a default value of gpt-3.5-turbo. |
| temperature | Optional. Gets or sets the sampling temperature to use, as a string between 0 and 2. Higher values (0.8) make the output more random, while lower values like (0.2) make output more focused and deterministic. You should use either Temperature or TopP, but not both. |
| topP | Optional. Gets or sets an alternative to sampling with temperature, called nucleus sampling, as a string. In this sampling method, the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. You should use either Temperature or TopP, but not both. |
| maxTokens | Optional. Gets or sets the maximum number of tokens to generate in the completion, as a string with a default of 100. The token count of your prompt plus max_tokens can't exceed the model's context length. Most models have a context length of 2,048 tokens (except for the newest models, which support 4096). |
| isReasoningModel | Optional. Gets or sets a value indicating whether the chat completion model is a reasoning model. This option is experimental and associated with the reasoning model until all models have parity in the expected properties, with a default value of false. |
Decorators
During the preview, define the input binding as a generic_input_binding binding of type textCompletion, which supports these parameters:
| Parameter | Description |
|---|---|
| arg_name | The name of the variable that represents the binding parameter. |
| prompt | Gets or sets the prompt to generate completions for, encoded as a string. |
| ai_connection_name | Optional. Gets or sets the name of the configuration section for AI service connectivity settings. For Azure OpenAI: If specified, looks for "Endpoint" and "Key" values in this configuration section. If not specified or the section doesn't exist, falls back to environment variables: AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_KEY. For user-assigned managed identity authentication, this property is required. For OpenAI service (non-Azure), set the OPENAI_API_KEY environment variable. |
| chat_model | Gets or sets the ID of the model to use as a string, with a default value of gpt-3.5-turbo. |
| temperature | Optional. Gets or sets the sampling temperature to use, as a string between 0 and 2. Higher values (0.8) make the output more random, while lower values like (0.2) make output more focused and deterministic. You should use either Temperature or TopP, but not both. |
| top_p | Optional. Gets or sets an alternative to sampling with temperature, called nucleus sampling, as a string. In this sampling method, the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. You should use either Temperature or TopP, but not both. |
| max_tokens | Optional. Gets or sets the maximum number of tokens to generate in the completion, as a string with a default of 100. The token count of your prompt plus max_tokens can't exceed the model's context length. Most models have a context length of 2,048 tokens (except for the newest models, which support 4096). |
| is_reasoning _model | Optional. Gets or sets a value indicating whether the chat completion model is a reasoning model. This option is experimental and associated with the reasoning model until all models have parity in the expected properties, with a default value of false. |
Configuration
The binding supports these configuration properties that you set in the function.json file.
| Property | Description |
|---|---|
| type | Must be textCompletion. |
| direction | Must be in. |
| name | The name of the input binding. |
| prompt | Gets or sets the prompt to generate completions for, encoded as a string. |
| aiConnectionName | Optional. Gets or sets the name of the configuration section for AI service connectivity settings. For Azure OpenAI: If specified, looks for "Endpoint" and "Key" values in this configuration section. If not specified or the section doesn't exist, falls back to environment variables: AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_KEY. For user-assigned managed identity authentication, this property is required. For OpenAI service (non-Azure), set the OPENAI_API_KEY environment variable. |
| chatModel | Gets or sets the ID of the model to use as a string, with a default value of gpt-3.5-turbo. |
| temperature | Optional. Gets or sets the sampling temperature to use, as a string between 0 and 2. Higher values (0.8) make the output more random, while lower values like (0.2) make output more focused and deterministic. You should use either Temperature or TopP, but not both. |
| topP | Optional. Gets or sets an alternative to sampling with temperature, called nucleus sampling, as a string. In this sampling method, the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. You should use either Temperature or TopP, but not both. |
| maxTokens | Optional. Gets or sets the maximum number of tokens to generate in the completion, as a string with a default of 100. The token count of your prompt plus max_tokens can't exceed the model's context length. Most models have a context length of 2,048 tokens (except for the newest models, which support 4096). |
| isReasoningModel | Optional. Gets or sets a value indicating whether the chat completion model is a reasoning model. This option is experimental and associated with the reasoning model until all models have parity in the expected properties, with a default value of false. |
Configuration
The binding supports these properties, which are defined in your code:
| Property | Description |
|---|---|
| prompt | Gets or sets the prompt to generate completions for, encoded as a string. |
| aiConnectionName | Optional. Gets or sets the name of the configuration section for AI service connectivity settings. For Azure OpenAI: If specified, looks for "Endpoint" and "Key" values in this configuration section. If not specified or the section doesn't exist, falls back to environment variables: AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_KEY. For user-assigned managed identity authentication, this property is required. For OpenAI service (non-Azure), set the OPENAI_API_KEY environment variable. |
| chatModel | Gets or sets the ID of the model to use as a string, with a default value of gpt-3.5-turbo. |
| temperature | Optional. Gets or sets the sampling temperature to use, as a string between 0 and 2. Higher values (0.8) make the output more random, while lower values like (0.2) make output more focused and deterministic. You should use either Temperature or TopP, but not both. |
| topP | Optional. Gets or sets an alternative to sampling with temperature, called nucleus sampling, as a string. In this sampling method, the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. You should use either Temperature or TopP, but not both. |
| maxTokens | Optional. Gets or sets the maximum number of tokens to generate in the completion, as a string with a default of 100. The token count of your prompt plus max_tokens can't exceed the model's context length. Most models have a context length of 2,048 tokens (except for the newest models, which support 4096). |
| isReasoningModel | Optional. Gets or sets a value indicating whether the chat completion model is a reasoning model. This option is experimental and associated with the reasoning model until all models have parity in the expected properties, with a default value of false. |
Usage
See the Example section for complete examples.
Connections
To use the Azure OpenAI binding extension, you need to specify a connection to an OpenAI model definition. Set the OpenAI model connection in your bindings by using one of these approaches:
- Use the
AIConnectionNamebinding property (preferred for Azure OpenAI). - Set
AZURE_OPENAI_ENDPOINTandAZURE_OPENAI_KEYin app settings (for Azure OpenAI). - Set only
Open_API_Keyin app settings (forhttps://api.openai.com).
The way you set the connection depends on both the model API and the authentication method, as indicated by the following table:
| Authentication/Model API | Azure OpenAI | OpenAI (https://api.openai.com) |
|---|---|---|
| Managed identity connection | AIConnectionName |
Not supported |
| Key Vault reference | AZURE_OPENAI_ENDPOINTAZURE_OPENAI_KEY |
Open_API_Key |
| App Configuration reference | AZURE_OPENAI_ENDPOINTAZURE_OPENAI_KEY |
Open_API_Key |
| Shared secret | AZURE_OPENAI_ENDPOINTAZURE_OPENAI_KEY |
Open_API_Key |
Use managed identity-based connections and the AIConnectionName property.
When you use AIConnectionName, the value of this property setting depends on the type of connection:
- Managed identity connection: The
AIConnectionNameproperty is a<CONNECTION_NAME_PREFIX>shared by a group of settings that together define an identity-based connection to Azure OpenAI. For more information, see Define identity connections. - Key Vault reference: The
AIConnectionNameproperty setting returns an Azure Key Vault reference to the location where the API key is centrally maintained. For more information, see Define Key Vault connections. - App Configuration reference: The
AIConnectionNameproperty setting returns an Azure App Configuration reference that returns an API key or a Key Vault reference. For more information, see Azure App Configuration in the connections article. - API key: The
AIConnectionNameproperty setting resolves to app settings containing the endpoint and key directly. Because shared keys can be compromised, use managed identity connections when possible. For more information, see Define connections.
To learn more about bindings connections, see Manage connections in Azure Functions.
The OpenAI bindings include an AIConnectionName property that you can use to specify the <ConnectionNamePrefix> for the group of app settings that define the connection to Azure OpenAI:
| Setting name | Description |
|---|---|
<CONNECTION_NAME_PREFIX>__endpoint |
Sets the URI endpoint of the Azure OpenAI service. This setting is always required. |
<CONNECTION_NAME_PREFIX>__clientId |
Sets the specific user-assigned identity to use when obtaining an access token. Requires that <CONNECTION_NAME_PREFIX>__credential is set to managedidentity. The property accepts a client ID corresponding to a user-assigned identity assigned to the application. It's invalid to specify both a Resource ID and a client ID. If you don't specify this property, the system-assigned identity is used. This property is used differently in local development scenarios, when credential shouldn't be set. |
<CONNECTION_NAME_PREFIX>__credential |
Defines how an access token is obtained for the connection. Use managedidentity for managed identity authentication. This value is only valid when a managed identity is available in the hosting environment. |
<CONNECTION_NAME_PREFIX>__managedIdentityResourceId |
When credential is set to managedidentity, set this property to specify the resource Identifier to use when obtaining a token. The property accepts a resource identifier corresponding to the resource ID of the user-defined managed identity. It's invalid to specify both a resource ID and a client ID. If you don't specify either, the system-assigned identity is used. This property is used differently in local development scenarios, when credential shouldn't be set. |
<CONNECTION_NAME_PREFIX>__key |
Sets the shared secret key required to access the endpoint of the Azure OpenAI service by using key-based authentication. As a security best practice, always use Microsoft Entra ID with managed identities for authentication. |
Consider these managed identity connection settings when you set the AIConnectionName property to myAzureOpenAI:
myAzureOpenAI__endpoint=https://contoso.openai.azure.com/myAzureOpenAI__credential=managedidentitymyAzureOpenAI__clientId=aaaaaaaa-bbbb-cccc-1111-222222222222
At runtime, the host interprets these settings as a single myAzureOpenAI setting:
"myAzureOpenAI":
{
"endpoint": "https://contoso.openai.azure.com/",
"credential": "managedidentity",
"clientId": "aaaaaaaa-bbbb-cccc-1111-222222222222"
}
When you use managed identities, make sure to add your identity to the Cognitive Services OpenAI User role.
When running locally, add these settings to the local.settings.json project file. For more information, see Local development with identity-based connections.
For more information, see Work with application settings.