Edit

Foundry Hosted Agents

Hosted agents in Microsoft Foundry Agent Service let you deploy Agent Framework agents as containerized applications to Microsoft-managed infrastructure. The platform handles scaling, session state persistence, security, and lifecycle management so you can focus on your agent's logic. Microsoft Foundry Hosted Agents is generally available.

With the Agent Framework hosting integration, you can expose an Agent, including a workflow wrapped with Workflow.as_agent(), through the Foundry Responses or Invocations protocol with minimal code.

When to use hosted agents

Choose Foundry hosted agents when you want:

  • Managed infrastructure — no need to configure containers, web servers, or scaling rules yourself.
  • Built-in session management — the platform persists $HOME and uploaded files across turns and idle periods.
  • Dedicated agent identity — every deployed agent gets its own Entra identity for secure access to models, tools, and downstream services.
  • OpenAI-compatible endpoints — clients can interact with your agent using any OpenAI-compatible SDK through the Responses protocol.

Note

The Python agent-framework-foundry-hosting integration is prerelease. Microsoft Foundry Hosted Agents, the managed hosting service, is generally available.

Prerequisites

For local testing, you also need:

Install the hosting NuGet package:

dotnet add package Microsoft.Agents.AI.Foundry.Hosting --prerelease
dotnet add package Azure.AI.Projects --prerelease
  • Python 3.10 or later

Install the prerelease hosting package, Foundry client, and Azure authentication package:

pip install --pre agent-framework-foundry agent-framework-foundry-hosting azure-identity

In Foundry, the platform supplies the caller's user context and call context; the hosting infrastructure uses them to isolate state per user and forward request context to Foundry services. Local runs don't receive that platform context, so applications must supply their own identity and state controls when needed.

Responses protocol

The Responses protocol is the recommended starting point for most agents. It exposes an OpenAI-compatible /responses endpoint, and the platform manages conversation history, streaming, and session lifecycle automatically.

using Azure.AI.AgentServer.Core;
using Azure.AI.Projects;
using Azure.Identity;
using Microsoft.Agents.AI;
using Microsoft.Agents.AI.Foundry.Hosting;

var projectEndpoint = new Uri(Environment.GetEnvironmentVariable("FOUNDRY_PROJECT_ENDPOINT")
    ?? throw new InvalidOperationException("FOUNDRY_PROJECT_ENDPOINT is not set."));
var deployment = Environment.GetEnvironmentVariable("AZURE_AI_MODEL_DEPLOYMENT_NAME") ?? "gpt-4o";

AIAgent agent = new AIProjectClient(projectEndpoint, new DefaultAzureCredential())
    .AsAIAgent(
        model: deployment,
        instructions: "You are a helpful AI assistant.",
        name: "my-agent");

var builder = AgentHost.CreateBuilder(args);
builder.Services.AddFoundryResponses(agent);
builder.RegisterProtocol("responses", endpoints => endpoints.MapFoundryResponses());

var app = builder.Build();
app.Run();

The AgentHost.CreateBuilder creates an application host preconfigured for the Foundry hosting environment. AddFoundryResponses registers your agent with the Responses protocol handler, and MapFoundryResponses maps the /responses HTTP endpoint.

import os

from agent_framework import Agent
from agent_framework.foundry import FoundryChatClient
from agent_framework_foundry_hosting import ResponsesHostServer
from azure.identity import DefaultAzureCredential

client = FoundryChatClient(
    project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
    model=os.environ["AZURE_AI_MODEL_DEPLOYMENT_NAME"],
    credential=DefaultAzureCredential(),
)

agent = Agent(
    client=client,
    instructions="You are a helpful AI assistant.",
)

server = ResponsesHostServer(agent)
server.run()

The ResponsesHostServer wraps your agent and exposes it through the Foundry Responses protocol. For a non-workflow agent, the default history_source="agent_server" uses the configured Agent Server response provider as the model's history source. The host prevents the downstream model service from retaining a second copy when the client stores history by default.

Don't combine the default history source with a HistoryProvider that has load_messages=True. Also don't set the conversation_id, previous_response_id, or conversation downstream service continuation options. The host rejects these configurations to prevent duplicate history.

Use ResponsesHostServer(agent, history_source="agent") when the agent's history provider or downstream model service must manage conversation history. This mode passes only the current request input from Agent Server and preserves the agent's history and service storage behavior. Custom SupportsAgentRun implementations must use this mode. The store parameter remains separate: it selects the response provider that persists Responses API inputs and outputs in both modes.

The host owns the supplied agent and might add hosting-specific context providers. Don't reuse the agent with another host or invoke it directly after host construction.

Persist state and handle long-running conversations

ResponsesHostServer configures Foundry-backed stores by default. For non-workflow agents, AgentSessionStoreProvider supplies a FoundryAgentSessionStore. For workflow agents, CheckpointStoreProvider supplies a FoundryCheckpointStore. FunctionApprovalStoreProvider supplies a FoundryFunctionApprovalStore for pending approvals. These stores use Foundry State Store when hosted and local Agent Server state when you run locally.

With history_source="agent", the configured session store persists provider state carried by AgentSession, including messages from InMemoryHistoryProvider.

To customize storage, pass a StoreProvider to agent_session_store_provider or function_approval_store_provider. Pass a ContextScopedStoreProvider to checkpoint_store_provider. For example, implement SessionStore and StoreProvider[SessionStore] to use your own non-workflow agent session store.

Import ResponsesServerOptions from azure.ai.agentserver.responses, and pass it to ResponsesHostServer through the options parameter. The available long-running conversation options depend on the agent type:

Capability Agent type Requirements and behavior
Resilient background responses Workflow only Set ResponsesServerOptions(resilient_background=True). Send the Responses request with store=true and background=true. After a restart, the host resumes the latest durable workflow checkpoint or replays the original input if no checkpoint exists. Don't configure checkpoint storage on the workflow because the host manages it. Make external side effects idempotent because work after the last durable checkpoint might repeat.
Steerable conversations Non-workflow only Set ResponsesServerOptions(steerable_conversations=True) and send Responses requests with store=true. Keep turns on one linear chain by reusing the same conversation value. Alternatively, send the immediately preceding previous_response_id and preserve the resolved agent_session_id. The host rejects stale predecessors that would create a fork.

ResponsesHostServer raises RuntimeError if you enable resilient background responses for a non-workflow agent or steerable conversations for a workflow agent. For complete implementations, see the custom storage, resilient long-running workflow, and steerable long-running agent samples.

When a Foundry-hosted MCP tool requires user consent, ResponsesHostServer returns an incomplete response with an oauth_consent_request output item. Present its consent_link to the user, then continue with the incomplete response's ID as previous_response_id after the user completes consent. The host preserves the agent session for this retry and exposes only absolute HTTPS consent links.

Invocations protocol

The Invocations protocol gives you full control over the HTTP request and response. Use it when you need custom payloads, non-conversational processing, or streaming protocols that aren't OpenAI-compatible.

With the Invocations protocol in C#, you implement a custom InvocationHandler to process incoming requests:

using Azure.AI.AgentServer.Core;
using Azure.AI.AgentServer.Invocations;
using Microsoft.Agents.AI;

var builder = AgentHost.CreateBuilder(args);

builder.Services.AddSingleton<AIAgent, MyAgent>();
builder.Services.AddInvocationsServer();
builder.Services.AddScoped<InvocationHandler, MyInvocationHandler>();

builder.RegisterProtocol("invocations", endpoints => endpoints.MapInvocationsServer());

var app = builder.Build();
app.Run();

The AddInvocationsServer method registers the Invocations protocol services. You implement InvocationHandler to define how your agent processes each request.

For a lightweight setup, use InvocationsHostServer from the agent_framework_foundry_hosting package. It wraps your agent similarly to ResponsesHostServer and handles session management automatically:

import os

from agent_framework import Agent
from agent_framework.foundry import FoundryChatClient
from agent_framework_foundry_hosting import InvocationsHostServer
from azure.identity import DefaultAzureCredential

client = FoundryChatClient(
    project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
    model=os.environ["AZURE_AI_MODEL_DEPLOYMENT_NAME"],
    credential=DefaultAzureCredential(),
)

agent = Agent(
    client=client,
    instructions="You are a friendly assistant. Keep your answers brief.",
    default_options={"store": False},
)

server = InvocationsHostServer(agent)
server.run()

For full control over request handling, use InvocationAgentServerHost from the azure.ai.agentserver.invocations package directly and implement your own invoke handler:

import os
from collections.abc import AsyncGenerator

from agent_framework import Agent, AgentSession
from agent_framework.foundry import FoundryChatClient
from azure.ai.agentserver.invocations import InvocationAgentServerHost
from azure.identity import DefaultAzureCredential
from starlette.requests import Request
from starlette.responses import JSONResponse, Response, StreamingResponse

_sessions: dict[str, AgentSession] = {}

client = FoundryChatClient(
    project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
    model=os.environ["AZURE_AI_MODEL_DEPLOYMENT_NAME"],
    credential=DefaultAzureCredential(),
)

agent = Agent(
    client=client,
    instructions="You are a friendly assistant. Keep your answers brief.",
    default_options={"store": False},
)

app = InvocationAgentServerHost()


@app.invoke_handler
async def handle_invoke(request: Request):
    """Handle streaming multi-turn chat."""
    data = await request.json()
    session_id = request.state.session_id
    stream = data.get("stream", False)
    user_message = data.get("message", None)

    if user_message is None:
        return Response(content="Missing 'message' in request", status_code=400)

    session = _sessions.setdefault(session_id, AgentSession(session_id=session_id))

    if stream:

        async def stream_response() -> AsyncGenerator[str]:
            async for update in agent.run(user_message, session=session, stream=True):
                yield update.text

        return StreamingResponse(
            stream_response(),
            media_type="text/event-stream",
            headers={"Cache-Control": "no-cache", "Connection": "keep-alive"},
        )

    response = await agent.run([user_message], session=session, stream=stream)
    return JSONResponse({"response": response.text})


if __name__ == "__main__":
    app.run()

Warning

The in-memory session store in the custom handler example is lost on restart. Use durable storage (for example, Cosmos DB) in production.

For a complete Invocations deployment, see the Foundry-hosted Telegram sample. It places API Management in front of the hosted agent webhook and uses managed identities, Key Vault, and Cosmos DB for durable conversation history.

Note

Go support for Foundry hosted agents is coming soon. See the Agent Framework Go repository for the latest status.

Tip

Refer the Python samples or the C# samples for examples of a hosted agent project. Or use the azd ai agent init command to scaffold a new hosted agent project from scratch. Refer to this quickstart guide for step-by-step instructions.

Running locally

The Azure Developer CLI (azd) provides the easiest way to run and test your hosted agent locally.

Initialize a project

Create a new folder and initialize from a sample manifest:

mkdir my-hosted-agent && cd my-hosted-agent
azd ai agent init -m <path-to-agent.manifest.yaml>

Tip

The manifest can be a path to a local YAML file or a URL to a remote manifest.

Set environment variables

export FOUNDRY_PROJECT_ENDPOINT="https://<account>.services.ai.azure.com/api/projects/<project>"
export AZURE_AI_MODEL_DEPLOYMENT_NAME="<your-model-deployment>"

Run the agent host

azd ai agent run

The agent host starts on http://localhost:8088.

Invoke the agent

azd ai agent invoke --local "Hello!"

Or use curl:

curl -X POST http://localhost:8088/responses \
  -H "Content-Type: application/json" \
  -d '{"input": "Hello!"}'

Or in PowerShell:

(Invoke-WebRequest -Uri http://localhost:8088/responses -Method POST -ContentType "application/json" -Body '{"input": "Hello!"}').Content

Deploying to Foundry

Once you've verified your agent locally, deploy it to Microsoft Foundry:

  1. Provision resources (if you don't already have a Foundry project):

    azd provision

    This creates a resource group with a Foundry instance, project, model deployment, Application Insights, and a container registry.

  2. Deploy the agent:

    azd deploy

    This packages your agent as a container image, pushes it to Azure Container Registry, and deploys it to Foundry Agent Service.

The Foundry hosting infrastructure automatically injects the following environment variables into your agent container at runtime:

Variable Description
FOUNDRY_PROJECT_ENDPOINT The endpoint URL for the Foundry project.
AZURE_AI_MODEL_DEPLOYMENT_NAME The model deployment name (configured during azd ai agent init).
APPLICATIONINSIGHTS_CONNECTION_STRING The Application Insights connection string for telemetry.

Once deployed, your agent is accessible through its dedicated Foundry endpoint and can also be tested from the Foundry portal.

Next steps

Hosted agents concepts

  • Last updated on