Tutorial: Use code interpreter sessions in Semantic Kernel with Azure Container Apps
Article
Semantic Kernel is an open-source AI framework created by Microsoft for .NET, Python, and Java developers working with Large Language Models (LLMs). When you build an AI agent with Semantic Kernel, an LLM interprets user input and generates a response. The AI agent often struggles when it needs to perform mathematical and symbolic reasoning to produce a response. By integrating Azure Container Apps dynamic sessions with Semantic Kernel, you give the agent a code interpreter to use to perform specialized tasks.
In this tutorial, you learn how to run a Semantic Kernel AI agent in a web API. The API accepts user input and returns a response generated by the AI agent. The agent uses a code interpreter in dynamic sessions to perform calculations.
The sample app in this quickstart uses an LLM from Azure OpenAI. It also uses Azure Container Apps sessions to run code generated by the LLM.
Update the Azure CLI to the latest version.
az upgrade
Remove the Azure Container Apps extension if it's already installed and install a preview version the Azure Container Apps extension containing commands for sessions:
az extension remove --name containerapp
az extension add \
--name containerapp \
--allow-preview true -y
Replace <AZURE_OPENAI_ENDPOINT> with the Azure OpenAI account endpoint and <SESSION_POOL_MANAGEMENT_ENDPOINT> with the session pool management endpoint.
The app uses DefaultAzureCredential to authenticate with Azure services. On your local machine, it uses your current Azure CLI login credentials. You must give yourself the Cognitive Services OpenAI User role on the Azure OpenAI account for the app to access the model endpoints, and the Azure ContainerApps Session Executor role on the session pool for the app to access the session pool.
Retrieve your Azure CLI user name:
az account show --query user.name --output tsv
Run the following commands to retrieve the Azure OpenAI account resource ID:
az cognitiveservices account show --name $AZURE_OPENAI_NAME --resource-group $RESOURCE_GROUP_NAME --query id --output tsv
Assign the Cognitive Services OpenAI User role to your Azure CLI user on the Azure OpenAI account:
az role assignment create --role "Cognitive Services OpenAI User" --assignee <CLI_USERNAME> --scope <AZURE_OPENAI_RESOURCE_ID>
Replace <CLI_USERNAME> with your Azure CLI user name and <AZURE_OPENAI_RESOURCE_ID> with the Azure OpenAI account resource ID.
Run the following commands to retrieve the session pool resource ID:
az containerapp sessionpool show --name $SESSION_POOL_NAME --resource-group $RESOURCE_GROUP_NAME --query id --output tsv
Assign the Azure ContainerApps Session Executor role using its ID to your Azure CLI user on the session pool:
az role assignment create \
--role "Azure ContainerApps Session Executor" \
--assignee <CLI_USERNAME> \
--scope <SESSION_POOL_RESOURCE_ID>
Replace <CLI_USERNAME> with your Azure CLI user name and <SESSION_POOL_RESOURCE_ID> with the session pool resource ID.
Run the app
Before running the sample app, open main.py in an editor and review the code. The app uses FastAPI to create a web API that accepts a user message in the query string.
The following lines of code instantiate a SessionsPythonTool and provide it to the Semantic Kernel agent:
When it needs to perform calculations, the kernel uses the code interpreter in SessionsPythonTool to run the code. The code is executed in a session in the session pool. By default, a random session identifier is generated when you instantiate the tool. If the kernel uses the tool to run multiple Python code snippets, it uses the same session. To ensure each end user has a unique session, use a separate kernel and tool for each user.
SessionsPythonTool is available in version 0.9.8b1 or later of the semantic-kernel package.
Run the sample app:
fastapi dev main.py
Open a browser and navigate to http://localhost:8000/docs. You see the Swagger UI for the sample app.
Expand the /chat endpoint and select Try it out.
Enter What time is it right now? in the message field and select Execute.
The agent responds with the current time. In the terminal, you see the logs showing the agent generated Python code to get the current time and ran it in a code interpreter session.
To stop the app, enter Ctrl+C in the terminal.
Optional: Deploy the sample app to Azure Container Apps
To deploy the FastAPI app to Azure Container Apps, you need to create a container image and push it to a container registry. Then you can deploy the image to Azure Container Apps. The az containerapp up command combines these steps into a single command.
You then need to configure managed identity for the app and assign it the proper roles to access Azure OpenAI and the session pool.
Set the variables for the Container Apps environment and app name: