Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Use an OpenAI-compatible client to send requests to a model deployment through the namespace endpoint. This article uses the OpenAI Python client and a namespace API key. To use Microsoft Entra ID instead, see Connect an application to an AI Manager model endpoint by using Microsoft Entra ID. For Azure RBAC role assignment, see Grant access to AI Manager.
Important
AKS preview features are available on a self-service, opt-in basis. Previews are provided "as is" and "as available," and they're excluded from the service-level agreements and limited warranty. AKS previews are partially covered by customer support on a best-effort basis. As such, these features aren't meant for production use. For more information, see the following support articles:
Before you begin
- A model deployment whose
provisioningStateisSucceededand that has at least one ready replica (currentReplicasgreater than0). To check readiness, see the quickstart. For state meanings, see Provisioning states. - Azure CLI with access to the AI Manager CLI extension for the public preview.
- An Azure sign-in for a subscription that contains the AI Manager.
- Python and the
openaipackage. - Permission to list namespace API keys. Azure AIManager Inference User grants this permission with the least privilege. Azure AIManager User and Azure AIManager Contributor also include it. Assign the role at the namespace or a parent scope.
Get the endpoint and key
Set variables for your resource group, AI Manager, and namespace.
az login az account set --subscription "<subscription-id>" export RESOURCE_GROUP="<resource-group>" export MANAGER_NAME="<ai-manager-name>" export NAMESPACE_NAME="<namespace-name>"Retrieve the namespace endpoint and the primary namespace API key (
primaryKey). The namespace also has asecondaryKeythat works the same way.export AI_MANAGER_ENDPOINT="$(az aimanager namespace list-accesskeys \ --resource-group "$RESOURCE_GROUP" \ --aimanager "$MANAGER_NAME" \ --name "$NAMESPACE_NAME" \ --query endpoint \ --output tsv)" export AI_MANAGER_API_KEY="$(az aimanager namespace list-accesskeys \ --resource-group "$RESOURCE_GROUP" \ --aimanager "$MANAGER_NAME" \ --name "$NAMESPACE_NAME" \ --query primaryKey \ --output tsv)"Confirm that both values are set. The command prints a message only if neither value is empty.
test -n "$AI_MANAGER_ENDPOINT" && test -n "$AI_MANAGER_API_KEY" && echo "Endpoint and key retrieved."
The endpoint value is the OpenAI-compatible base URL. Use it as returned, including any path component.
Send a request with the OpenAI Python client
Install the
openaipackage if it isn't already installed.python -m pip install openaiSet the
modelvalue to your model deployment name. Then useAI_MANAGER_ENDPOINTandAI_MANAGER_API_KEYfrom Get the endpoint and key to send a chat completion request.export AI_MANAGER_DEPLOYMENT="<model-deployment-name>"import os from openai import OpenAI client = OpenAI( api_key=os.environ["AI_MANAGER_API_KEY"], base_url=os.environ["AI_MANAGER_ENDPOINT"], ) response = client.chat.completions.create( model=os.environ["AI_MANAGER_DEPLOYMENT"], messages=[ {"role": "user", "content": "What does Kubernetes do?"}, ], ) print(response.choices[0].message.content)
The request uses the following values:
modelfield: The model deployment name. Don't use the model catalog'smodelIdvalue.Authorization: Bearerheader: Carries the namespace API key. The OpenAI client sends it automatically.api-keyheader: Carries the namespace API key for clients that set headers directly.
A successful request prints the response text from choices[0].message.content. The text varies by model. For a request using curl, see the AI Manager quickstart.