Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Azure Container Apps dynamic sessions provides fast and scalable access to a code interpreter. Each code interpreter session is fully isolated by a Hyper-V boundary and is designed to run untrusted code.
Code interpreter sessions are ideal for scenarios where you need to run code that is potentially malicious or could cause harm to the host system or other users, such as:
- Code generated by a large language model (LLM).
- Code submitted by an end user in a web or SaaS application.
For popular LLM frameworks such as LangChain, LlamaIndex, or Semantic Kernel, you can use tools and plugins to integrate AI apps with code interpreter sessions.
Your applications can also integrate with code interpreter session using a REST API. The API allows you to:
Execute code in a session and retrieve results
Upload and download files to and from the session.
You can upload and download executable code files, or data files that your code can process.
The built-in code interpreter sessions support the most common code execution scenarios without the need to manage infrastructure or containers.
If you need full control over the code execution environment or have a different scenario that requires isolated sandboxes, you can use custom code interpreter sessions.
To use code interpreter sessions, you need an Azure resource called a session pool that defines the configuration for code interpreter sessions.
In the session pool, you can specify settings such as the maximum number of concurrent sessions and how long a session can be idle before the session is terminated.
You can create a session pool using the Azure portal, Azure CLI, or Azure Resource Manager templates. After you create a session pool, you can use the pool's management API endpoints to manage and execute code inside a session.
For more information about how to create and configure a session pool, see Use session pools.
After you create a session pool, your application can interact with sessions in the pool using an integration with an LLM framework or by using the pool's management API endpoints directly.
Important
The session identifier is sensitive information which requires you to use a secure process to manage its value. Part of this process requires that your application ensures each user or tenant only has access to their own sessions.
Failure to secure access to sessions could result in misuse or unauthorized access to data stored in your users' sessions. For more information, see Session identifiers.
When you interact with sessions in a pool, you use a session identifier to reference each session A session identifier is a string that you define that is unique within the session pool. If you're building a web application, you can use the user's ID. If you're building a chatbot, you can use the conversation ID.
If there's a running session with the identifier, the session is reused. If there's no running session with the identifier, a new session is automatically created.
Authentication is handled using Microsoft Entra tokens. Valid Microsoft Entra tokens are generated by an identity belonging to the Azure ContainerApps Session Executor and Contributor roles on the session pool.
If you're using an LLM framework integration, the framework handles the token generation and management for you. Ensure that the application is configured with a managed identity with the necessary role assignments on the session pool.
If you're using the pool's management API endpoints directly, you must generate a token and include it in the Authorization header of your HTTP requests. In addition to the role assignments previously mentioned, token needs to contain an audience (aud) claim with the value https://dynamicsessions.io.
To learn more, see Authentication and authorization.
You can upload and download files, and list all the files in a code interpreter session.
To upload a file to a session, send a POST request to the uploadFile endpoint in a multipart form data request. Include the file data in the request body. The file must include a filename.
Uploaded files are stored in the session's file system under the /mnt/data directory.
The following example shows how to upload a file to a session.
Before you send the request, replace the placeholders between the <> brackets with values specific to your request.
POST https://<REGION>.dynamicsessions.io/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/sessionPools/<SESSION_POOL_NAME>/files/upload?api-version=2024-02-02-preview&identifier=<SESSION_ID>
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW
Authorization: Bearer <token>
------WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="file"; filename="<FILE_NAME_AND_EXTENSION>"
Content-Type: application/octet-stream
(data)
------WebKitFormBoundary7MA4YWxkTrZu0gW--
To download a file from a session's /mnt/data directory, send a GET request to the file/content/{filename} endpoint. The response includes the file data.
The following example demonstrates how to format a GET request to download a file.
Before you send the request, replace the placeholders between the <> brackets with values specific to your request.
GET https://<REGION>.dynamicsessions.io/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/sessionPools/<SESSION_POOL_NAME>/files/content/<FILE_NAME_AND_EXTENSION>?api-version=2024-02-02-preview&identifier=<SESSION_ID>
Authorization: Bearer <TOKEN>
To list the files in a session's /mnt/data directory, send a GET request to the files endpoint.
The following example shows you how to list the files in a session's directory.
Before you send the request, replace the placeholders between the <> brackets with values specific to your request.
GET https://<REGION>.dynamicsessions.io/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/sessionPools/<SESSION_POOL_NAME>/files?api-version=2024-02-02-preview&identifier=<SESSION_ID>
Authorization: Bearer <TOKEN>
The response contains a list of files in the session.
The following listing shows a sample of the type of response you can expect from requesting session contents.
{
"$id": "1",
"value": [
{
"$id": "2",
"properties": {
"$id": "3",
"filename": "test1.txt",
"size": 16,
"lastModifiedTime": "2024-05-02T07:21:07.9922617Z"
}
},
{
"$id": "4",
"properties": {
"$id": "5",
"filename": "test2.txt",
"size": 17,
"lastModifiedTime": "2024-05-02T07:21:08.8802793Z"
}
}
]
}
Code interpreter sessions are designed to run untrusted code in isolated environments, ensuring that your applications and data remain protected.
A managed identity from Microsoft Entra ID allows your container session pools and their sessions to access other Microsoft Entra protected resources. Both system-assigned and user-assigned managed identities are supported in a session pool.
For more about managed identities in Microsoft Entra ID, see Managed identities for Azure resources.
There are two ways to use managed identities with custom container session pools:
Image pull authentication: Use the managed identity to authenticate with the container registry to pull the container image.
Resource access: Use the session pool's managed identity in a session to access other Microsoft Entra protected resources. Due to its security implications, this capability is disabled by default.
Important
If you enable access to managed identity in a session, any code or programs running in the session can create Microsoft Entra tokens for the pool's managed identity. Since sessions typically run untrusted code, use this feature with extreme caution.
To enable managed identity for a custom container session pool, use Azure Resource Manager.
Instead of using the session pool management API directly, the following LLM frameworks provide integrations with code interpreter sessions:
| Framework | Package | Tutorial |
|---|---|---|
| LangChain | Python: langchain-azure-dynamic-sessions |
Tutorial |
| LlamaIndex | Python: llama-index-tools-azure-code-interpreter |
Tutorial |
| Semantic Kernel | Python: semantic-kernel (version 0.9.8-b1 or later) |
Tutorial |
If you're not using an LLM framework integration, you can interact with the session pool directly using the management API endpoints.
To execute code in a session, send a POST request to the code/execute endpoint with the code to run in the request body.
The following example prints Hello, world! in Python.
Before you send the request, replace the placeholders between the <> brackets with the appropriate values for your session pool and session identifier.
POST https://<REGION>.dynamicsessions.io/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/sessionPools/<SESSION_POOL_NAME>/code/execute?api-version=2024-02-02-preview&identifier=<SESSION_ID>
Content-Type: application/json
Authorization: Bearer <token>
{
"properties": {
"codeInputType": "inline",
"executionType": "synchronous",
"code": "print('Hello, world!')"
}
}
To reuse a session, specify the same session identifier in subsequent requests.
To upload a file to a session, send a POST request to the uploadFile endpoint in a multipart form data request. Include the file data in the request body. The file must include a filename.
Uploaded files are stored in the session's file system under the /mnt/data directory.
Before you send the request, replace the placeholders between the <> brackets with values specific to your request.
POST https://<REGION>.dynamicsessions.io/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/sessionPools/<SESSION_POOL_NAME>/files/upload?api-version=2024-02-02-preview&identifier=<SESSION_ID>
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW
Authorization: Bearer <token>
------WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="file"; filename="<FILE_NAME_AND_EXTENSION>"
Content-Type: application/octet-stream
(data)
------WebKitFormBoundary7MA4YWxkTrZu0gW--
Note
The file upload limit is 128MB. If this is exceeded a HTTP 413 may be returned.
To download a file from a session's /mnt/data directory, send a GET request to the file/content/{filename} endpoint. The response includes the file data.
Before you send the request, replace the placeholders between the <> brackets with values specific to your request.
GET https://<REGION>.dynamicsessions.io/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/sessionPools/<SESSION_POOL_NAME>/files/content/<FILE_NAME_AND_EXTENSION>?api-version=2024-02-02-preview&identifier=<SESSION_ID>
Authorization: Bearer <TOKEN>
To list the files in a session's /mnt/data directory, send a GET request to the files endpoint.
Before you send the request, replace the placeholders between the <> brackets with values specific to your request.
GET https://<REGION>.dynamicsessions.io/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/sessionPools/<SESSION_POOL_NAME>/files?api-version=2024-02-02-preview&identifier=<SESSION_ID>
Authorization: Bearer <TOKEN>
The response contains a list of files in the session.
The following listing shows a sample of the type of response you can expect from requesting session contents.
{
"$id": "1",
"value": [
{
"$id": "2",
"properties": {
"$id": "3",
"filename": "test1.txt",
"size": 16,
"lastModifiedTime": "2024-05-02T07:21:07.9922617Z"
}
},
{
"$id": "4",
"properties": {
"$id": "5",
"filename": "test2.txt",
"size": 17,
"lastModifiedTime": "2024-05-02T07:21:08.8802793Z"
}
}
]
}
Python code interpreter sessions include popular Python packages such as NumPy, pandas, and scikit-learn.
To output the list of preinstalled packages, call the code/execute endpoint with the following code.
Before you send the request, replace the placeholders between the <> brackets with values specific to your request.
POST https://<REGION>.dynamicsessions.io/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/sessionPools/<SESSION_POOL_NAME>/identifier/<SESSION_ID>/code/execute?api-version=2024-02-02-preview&identifier=<SESSION_ID>
Content-Type: application/json
Authorization: Bearer <TOKEN>
{
"properties": {
"codeInputType": "inline",
"executionType": "synchronous",
"code": "import pkg_resources\n[(d.project_name, d.version) for d in pkg_resources.working_set]"
}
}
Code interpreter sessions don't support logging directly. Your application that's interacting with the sessions can log requests to the session pool management API and its responses.
Code interpreter sessions are billed based on the duration of each session. For more information, see Billing.