Secure access to Azure OpenAI from Azure Kubernetes Service (AKS)

Article
10/12/2023

In this article, you learn how to secure access to Azure OpenAI from Azure Kubernetes Service (AKS) using Microsoft Entra Workload ID. You learn how to:

Enable workload identities on an AKS cluster.
Create an Azure user-assigned managed identity.
Create a Microsoft Entra ID federated credential.
Enable workload identity on a Kubernetes Pod.

Note

We recommend using Microsoft Entra Workload ID and managed identities on AKS for Azure OpenAI access because it enables a secure, passwordless authentication process for accessing Azure resources.

Before you begin

You need an Azure account with an active subscription. If you don't have one, create an account for free.
This article builds on Deploy an application that uses OpenAI on AKS. You should complete that article before you begin this one.
You need a custom domain name enabled on your Azure OpenAI account to use for Microsoft Entra authorization. For more information, see Custom subdomain names for Azure AI services.

Prerequisites

Use the Bash environment in Azure Cloud Shell. For more information, see Quickstart for Bash in Azure Cloud Shell.
If you prefer to run CLI reference commands locally, install the Azure CLI. If you're running on Windows or macOS, consider running Azure CLI in a Docker container. For more information, see How to run the Azure CLI in a Docker container.
- If you're using a local installation, sign in to the Azure CLI by using the az login command. To finish the authentication process, follow the steps displayed in your terminal. For other sign-in options, see Sign in with the Azure CLI.
- When you're prompted, install the Azure CLI extension on first use. For more information about extensions, see Use extensions with the Azure CLI.
- Run az version to find the version and dependent libraries that are installed. To upgrade to the latest version, run az upgrade.

Enable Microsoft Entra Workload ID on an AKS cluster

The Microsoft Entra Workload ID and OIDC Issuer Endpoint features aren't enabled on AKS by default. You must enable them on your AKS cluster before you can use them.

Set the resource group name and AKS cluster resource group name variables.

# Set the resource group variable
RG_NAME=myResourceGroup

# Set the AKS cluster resource group variable
AKS_NAME=$(az resource list --resource-group $RG_NAME --resource-type Microsoft.ContainerService/managedClusters --query "[0].name" -o tsv)

Enable the Microsoft Entra Workload ID and OIDC Issuer Endpoint features on your existing AKS cluster using the az aks update command.
```
az aks update \
    --resource-group $RG_NAME \
    --name $AKS_NAME \
    --enable-workload-identity \
    --enable-oidc-issuer
```

Get the AKS OIDC Issuer Endpoint URL using the az aks show command.

AKS_OIDC_ISSUER=$(az aks show --resource-group $RG_NAME --name $AKS_NAME --query "oidcIssuerProfile.issuerUrl" -o tsv)

Create an Azure user-assigned managed identity

Create an Azure user-assigned managed identity using the az identity create command.

# Set the managed identity name variable
MANAGED_IDENTITY_NAME=myIdentity

# Create the managed identity
az identity create \
    --resource-group $RG_NAME \
    --name $MANAGED_IDENTITY_NAME

Get the managed identity client ID and object ID using the az identity show command.

# Get the managed identity client ID
MANAGED_IDENTITY_CLIENT_ID=$(az identity show --resource-group $RG_NAME --name $MANAGED_IDENTITY_NAME --query clientId -o tsv)

# Get the managed identity object ID
MANAGED_IDENTITY_OBJECT_ID=$(az identity show --resource-group $RG_NAME --name $MANAGED_IDENTITY_NAME --query principalId -o tsv)

Get the Azure OpenAI resource ID using the az resource list command.

AOAI_RESOURCE_ID=$(az resource list --resource-group $RG_NAME --resource-type Microsoft.CognitiveServices/accounts --query "[0].id" -o tsv)

Grant the managed identity access to the Azure OpenAI resource using the az role assignment create command.

az role assignment create \
    --role "Cognitive Services OpenAI User" \
    --assignee-object-id $MANAGED_IDENTITY_OBJECT_ID \
    --assignee-principal-type ServicePrincipal \
    --scope $AOAI_RESOURCE_ID

Create a Microsoft Entra ID federated credential

Set the federated credential, namespace, and service account variables.

# Set the federated credential name variable
FEDERATED_CREDENTIAL_NAME=myFederatedCredential

# Set the namespace variable
SERVICE_ACCOUNT_NAMESPACE=default

# Set the service account variable
SERVICE_ACCOUNT_NAME=ai-service-account

Create the federated credential using the az identity federated-credential create command.

az identity federated-credential create \
    --name ${FEDERATED_CREDENTIAL_NAME} \
    --resource-group ${RG_NAME} \
    --identity-name ${MANAGED_IDENTITY_NAME} \
    --issuer ${AKS_OIDC_ISSUER} \
    --subject system:serviceaccount:${SERVICE_ACCOUNT_NAMESPACE}:${SERVICE_ACCOUNT_NAME}

Use Microsoft Entra Workload ID on AKS

To use Microsoft Entra Workload ID on AKS, you need to make a few changes to the ai-service deployment manifest.

Create a ServiceAccount

Get the kubeconfig for your cluster using the az aks get-credentials command.

az aks get-credentials \
    --resource-group $RG_NAME \
    --name $AKS_NAME

Create a Kubernetes ServiceAccount using the kubectl apply command.

kubectl apply -f - <<EOF
apiVersion: v1
kind: ServiceAccount
metadata:
  annotations:
    azure.workload.identity/client-id: ${MANAGED_IDENTITY_CLIENT_ID}
  name: ${SERVICE_ACCOUNT_NAME}
  namespace: ${SERVICE_ACCOUNT_NAMESPACE}
EOF

Enable Microsoft Entra Workload ID on the Pod

Set the Azure OpenAI resource name, endpoint, and deployment name variables.

# Get the Azure OpenAI resource name
AOAI_NAME=$(az resource list \
  --resource-group $RG_NAME \
  --resource-type Microsoft.CognitiveServices/accounts \
  --query "[0].name" -o tsv)

# Get the Azure OpenAI endpoint
AOAI_ENDPOINT=$(az cognitiveservices account show \
  --resource-group $RG_NAME \
  --name $AOAI_NAME \
  --query properties.endpoint -o tsv)

# Get the Azure OpenAI deployment name
AOAI_DEPLOYMENT_NAME=$(az cognitiveservices account deployment list  \
  --resource-group $RG_NAME \
  --name $AOAI_NAME \
  --query "[0].name" -o tsv)

Redeploy the ai-service with the ServiceAccount and the azure.workload.identity/use annotation set to true using the kubectl apply command.

kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ai-service
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ai-service
  template:
    metadata:
      labels:
        app: ai-service
        azure.workload.identity/use: "true"
    spec:
      serviceAccountName: $SERVICE_ACCOUNT_NAME
      nodeSelector:
        "kubernetes.io/os": linux
      containers:
      - name: ai-service
        image: ghcr.io/azure-samples/aks-store-demo/ai-service:latest
        ports:
        - containerPort: 5001
        env:
        - name: USE_AZURE_OPENAI
          value: "True"
        - name: USE_AZURE_AD
          value: "True"
        - name: AZURE_OPENAI_DEPLOYMENT_NAME
          value: "${AOAI_DEPLOYMENT_NAME}"
        - name: AZURE_OPENAI_ENDPOINT
          value: "${AOAI_ENDPOINT}"
        resources:
          requests:
            cpu: 20m
            memory: 50Mi
          limits:
            cpu: 50m
            memory: 128Mi
EOF

Test the application

Verify the new pod is running using the kubectl get pods command.
```
kubectl get pods --selector app=ai-service -w
```
Get the pod logs using the kubectl logs command. It may take a few minutes for the pod to initialize.
```
kubectl logs --selector app=ai-service -f
```
The following example output shows the app has initialized and is ready to accept requests. The first line suggests the code is missing configuration variables. However, the Azure Identity SDK handles this process and sets the AZURE_CLIENT_ID and AZURE_TENANT_ID variables.
```
Incomplete environment configuration. These variables are set: AZURE_CLIENT_ID, AZURE_TENANT_ID
INFO:     Started server process [1]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:5001 (Press CTRL+C to quit)
```
Get the pod environment variables using the kubectl describe pod command. The output demonstrates that the Azure OpenAI API key no longer exists in the Pod's environment variables.
```
kubectl describe pod --selector app=ai-service
```

Open a new terminal and get the IP of the store admin service using the following echo command.

echo "http://$(kubectl get svc/store-admin -o jsonpath='{.status.loadBalancer.ingress[0].ip}')"

Open a web browser and navigate to the IP address from the previous step.
Select Products. You should be able to add a new product and get a description for it using Azure OpenAI.

Next steps

In this article, you learned how to secure access to Azure OpenAI from Azure Kubernetes Service (AKS) using Microsoft Entra Workload ID.

For more information on Microsoft Entra Workload ID, see Microsoft Entra Workload ID.