Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Use langchain-azure-ai[hosting] to expose a compiled LangGraph graph as a Microsoft Foundry
hosted agent. This integration
lets you keep your LangChain and LangGraph agent logic in code while
Foundry manages the hosted runtime, sessions, scale, identity, and protocol
endpoints.
In this article, you create a minimal LangGraph agent, expose it through either the Responses or Invocations protocol, test it through HTTP, and deploy it to Foundry with the Azure Developer CLI or the Foundry Toolkit Visual Studio Code extension.
You also learn how to host an existing LangGraph graph by using the hosting CLI.
Note
The langchain-azure-ai package is open source in the langchain-ai/langchain-azure repository. For questions or feedback, open an issue.
Prerequisites
- An Azure subscription. Create one for free.
- A Foundry project.
- A deployed chat model, such as
gpt-4.1orgpt-5-mini. - Python 3.11 or later.
- Azure CLI signed in (
az login) soDefaultAzureCredentialcan authenticate.
Install the package
Install langchain-azure-ai version 1.3.0 or later with the hosting extra:
pip install -U "langchain-azure-ai[hosting]>=1.3.0" azure-identity
The hosting extra installs the Foundry protocol libraries used by the host
servers:
azure-ai-agentserver-responsesfor the OpenAI-compatible/responsesendpoint.azure-ai-agentserver-invocationsfor the generic/invocationsendpoint.
Choose a hosting protocol
Hosted agents can expose one or more protocols. Start with Responses for most conversational agents.
| Protocol | Host class | Endpoint | Use when |
|---|---|---|---|
| Responses | ResponsesHostServer |
/responses |
You want OpenAI-compatible chat, streaming, response history, and conversation threading. |
| Invocations | InvocationsHostServer |
/invocations |
You want a custom JSON shape, a webhook-style endpoint, or non-conversational processing. |
For background on protocol behavior and sessions, see Hosted agents and Manage Hosted agent sessions.
Configure environment variables
Set the project endpoint and model deployment name for local development:
export FOUNDRY_PROJECT_ENDPOINT="https://<resource>.services.ai.azure.com/api/projects/<project>"
export AZURE_AI_MODEL_DEPLOYMENT_NAME="gpt-4.1"
For Python hosted deployments, add PYTHONNOUSERSITE: "1" to the agent service's env map in azure.yaml, or set ENV PYTHONNOUSERSITE=1 in the Python Dockerfile. This setting disables user-site package loading. Install dependencies in a virtual environment or the image, not with pip install --user. See Configure Python startup.
When the same code runs as a hosted agent in Foundry, the platform injects FOUNDRY_PROJECT_ENDPOINT. The sample's azure.yaml configures AZURE_AI_MODEL_DEPLOYMENT_NAME for the selected model deployment.
Responses protocol
Use the Responses protocol when you want an OpenAI-compatible chat endpoint with streaming, response history, and conversation threading.
Create a Responses host
Create a file named main.py with a minimal LangGraph agent that uses a
Foundry model. This pattern matches the basic Responses sample in the
langchain-azure-ai source repository.
import os
from azure.ai.projects import AIProjectClient
from azure.identity import DefaultAzureCredential, get_bearer_token_provider
from langchain.agents import create_agent
from langchain_openai import ChatOpenAI
from langchain_azure_ai.agents.hosting import ResponsesHostServer
_AZURE_AI_SCOPE = "https://ai.azure.com/.default"
def build_chat_model() -> ChatOpenAI:
project_endpoint = os.environ["FOUNDRY_PROJECT_ENDPOINT"].rstrip("/")
deployment = os.environ.get("AZURE_AI_MODEL_DEPLOYMENT_NAME", "gpt-4.1")
credential = DefaultAzureCredential()
project = AIProjectClient(endpoint=project_endpoint, credential=credential)
openai_client = project.get_openai_client()
token_provider = get_bearer_token_provider(credential, _AZURE_AI_SCOPE)
return ChatOpenAI(
model=deployment,
base_url=str(openai_client.base_url),
api_key=token_provider,
)
def main() -> None:
graph = create_agent(build_chat_model(), tools=[])
port = int(os.environ.get("PORT", "8088"))
ResponsesHostServer(graph).run(port=port)
if __name__ == "__main__":
main()
What this snippet does: Creates a LangGraph agent with LangChain's
create_agent, connects it to the Foundry project's OpenAI-compatible model
endpoint, and passes the compiled graph to ResponsesHostServer. The host
starts an HTTP server and exposes the graph through POST /responses. By
default, the server binds to port 8088, or to the value of the PORT
environment variable when one is set.
Note
Deep Agents are hosted in the same way as other LangGraph agents. Pass the
agent directly to ResponsesHostServer.
agent = create_deep_agent(...)
ResponsesHostServer(agent).run(port=port)
Run the app locally:
python main.py
Test the Responses endpoint
Send a non-streaming Responses request to the local server.
Bash:
curl -sS -H "Content-Type: application/json" \
-X POST http://localhost:8088/responses \
-d '{"input":"Give me one practical tip for testing hosted agents.","stream":false}'
PowerShell:
$body = @{
input = "Give me one practical tip for testing hosted agents."
stream = $false
} | ConvertTo-Json
Invoke-RestMethod `
-Uri http://localhost:8088/responses `
-Method Post `
-Body $body `
-ContentType "application/json"
For streaming responses, set stream to true. The host emits Responses API
server-sent events, such as response.created, response.output_text.delta,
and response.completed.
Conversations
ResponsesHostServer supports two conversation-state patterns. The pattern it
uses depends on whether your compiled graph has a LangGraph checkpointer.
| Graph configuration | Conversation source | What the host sends to the graph on later turns |
|---|---|---|
| Graph without a checkpointer | Responses history from the protocol runtime | Prior response history plus the current request input |
| Graph compiled with a checkpointer | LangGraph checkpoint state keyed by the conversation or response thread | Current request input only |
Use a checkpointer when your graph needs LangGraph runtime state, interrupts, or node-local state across turns. For local testing, you can use an in-memory checkpointer:
from langgraph.checkpoint.memory import MemorySaver
graph = create_agent(
build_chat_model(),
tools=[],
checkpointer=MemorySaver(),
)
For production Hosted agents, use a durable checkpointer instead of an in-memory checkpointer so graph state survives container restarts.
Use FoundryCheckpointSaver from the
langchain-azure-ai package to
store checkpoints in Foundry State Store. To store checkpoints in Azure Cosmos DB
for NoSQL, use the asynchronous CosmosDBSaver from the
langchain-azure-cosmosdb
package.
Note
Starting with langchain-azure-ai version 1.3.0, Responses requests to a graph
configured with a checkpointer must omit store or set it to true.
Setting store to false returns HTTP 400, including for streaming requests.
Clients continue a Responses conversation by passing previous_response_id or
a conversation ID. For local testing, chain the previous response ID in the
next request:
POST http://localhost:8088/responses
Content-Type: application/json
{
"input": "Can you make that more concise?",
"previous_response_id": "<previous-response-id>",
"stream": false
}
When the agent runs in Foundry, the same pattern works through the Hosted agent
Responses endpoint. If later turns also need the same hosted sandbox filesystem,
include agent_session_id or use a conversation ID. For details, see
Manage Hosted agent sessions.
To branch, rewind, or regenerate from an earlier turn's exact graph checkpoint, configure a checkpointer on the graph and enable response branching on the host:
ResponsesHostServer(graph, enable_response_branching=True).run(port=port)
With response branching enabled, reuse any completed, stored response ID (store=True by default) as previous_response_id:
- Branch or rewind: Pass an earlier turn's response ID with a new prompt to start an independent branch from that checkpoint without modifying later turns.
- Regenerate: Resend a turn's prompt with the same parent
previous_response_idto create a fresh response from that parent checkpoint.
Human-in-the-loop
If your graph uses LangGraph interrupt() calls, ResponsesHostServer surfaces
pending interrupts through standard Responses API output items:
- A
function_callitem named__hosted_agent_adapter_interrupt__. - An
mcp_approval_requestitem withserver_labelset tolanggraph.
Clients can resume the graph by sending one of the following input items:
- A
function_call_outputitem. Set itscall_idto thecall_idof the returnedfunction_callitem. - An
mcp_approval_responseitem. Set itsapproval_request_idto theidof the returnedmcp_approval_requestitem.
Use mcp_approval_response for approve or reject decisions. When a graph uses LangChain's
HumanInTheLoopMiddleware, the host converts the choice into middleware
decisions. For custom interrupt() calls, approval resumes the graph with
the original interrupt value. Rejection fails the response without resuming
the graph.
Use function_call_output to provide a custom resume value or a LangGraph
Command payload with resume, update, or goto fields.
Invocations protocol
Use InvocationsHostServer when your callers can't use the Responses API
request shape or when your scenario isn't a chat conversation. The default
Invocations host accepts a message string and an optional stream flag.
Create an Invocations host
Use the same model-building function from the Responses example, but start
InvocationsHostServer instead of ResponsesHostServer.
import os
from langchain.agents import create_agent
from langgraph.checkpoint.memory import MemorySaver
from langchain_azure_ai.agents.hosting import InvocationsHostServer
def main() -> None:
graph = create_agent(
build_chat_model(),
tools=[],
checkpointer=MemorySaver(),
)
port = int(os.environ.get("PORT", "8088"))
InvocationsHostServer(graph).run(port=port)
if __name__ == "__main__":
main()
What this snippet does: Hosts the LangGraph agent through POST /invocations. The MemorySaver checkpointer gives local multi-turn continuity
for a given session ID. For production, use a durable checkpointer so state
survives container restarts.
Note
Deep Agents are hosted in the same way as other LangGraph agents. Pass the
agent directly to InvocationsHostServer.
agent = create_deep_agent(...)
InvocationsHostServer(agent).run(port=port)
Test the Invocations endpoint
Send a non-streaming request:
curl -i -X POST http://localhost:8088/invocations \
-H "Content-Type: application/json" \
-d '{"message":"My name is Alice.","stream":false}'
Non-streaming requests return JSON in this shape:
{
"response": "Assistant text"
}
For multi-turn conversations, reuse the x-agent-session-id response header as
the agent_session_id query parameter on the next request:
curl -X POST "http://localhost:8088/invocations?agent_session_id=<session-id>" \
-H "Content-Type: application/json" \
-d '{"message":"What is my name?"}'
Streaming requests return text/event-stream events with token payloads:
curl -N -X POST http://localhost:8088/invocations \
-H "Content-Type: application/json" \
-d '{"message":"Count to 5.","stream":true}'
The stream contains token events followed by a terminal done event:
data: {"token": "..."}
event: done
data: {}
Customize the request schema
To customize the request body, subclass InvocationsHostServer and override
parse_request. You can also override build_input to map the parsed data to a
custom graph state.
from starlette.requests import Request
from langchain_azure_ai.agents.hosting import InvocationsHostServer
class TicketHostServer(InvocationsHostServer):
async def parse_request(self, request: Request) -> tuple[str, bool]:
data = await request.json()
ticket_id = data["ticket_id"]
description = data["description"]
stream = bool(data.get("stream", False))
return f"Summarize ticket {ticket_id}: {description}", stream
if __name__ == "__main__":
TicketHostServer(graph).run()
What this snippet does: Accepts a custom ticket payload and converts it to a
single user message before the host invokes the graph. For more complex graph
state, override build_input instead of flattening the request to text.
Deploy
You can deploy by using the Azure Developer CLI or the Foundry Toolkit Visual Studio Code extension. The Azure Developer CLI flow uses a sample azure.yaml project. The extension flow provides a guided deployment experience in Visual Studio Code.
Hosted agent deployment requires the Foundry Project Manager role on the project. For details, see Deploy a Hosted agent.
Deploy with Azure Developer CLI
The LangGraph hosted chat agent sample in the Foundry samples repository includes the project and deployment configuration. Its main.py exports a graph through langgraph.json, and azure.yaml starts it with langchain_azure_ai.agents.hosting.run. For details about the hosted-agent configuration, see Author azure.yaml for hosted agents.
Install the AI agent extension and sign in before you initialize a sample:
azd ext install azure.ai.agents
azd auth login
The sample targets the Python 3.13 hosted runtime. To run it locally, install Python 3.13 or later and uv. azd ai agent run starts the application's configured startupCommand. See Run a hosted agent locally.
Initialize from a sample azure.yaml
Create a new folder and initialize it from a sample azure.yaml. Replace the azure.yaml URL with the sample you want to use.
mkdir my-langchain-agent
cd my-langchain-agent
azd ai agent init -t https://github.com/microsoft-foundry/foundry-samples/blob/main/samples/python/hosted-agents/langgraph/responses/01-langgraph-chat/azure.yaml
Follow the prompts from azd ai agent init. If you don't already have a
Foundry project and model deployment, the initialization flow can guide you
through creating them.
Run the agent locally
Run the agent host locally through azd:
azd ai agent run
The host serves on http://127.0.0.1:8088. In another terminal, invoke the
local protocol endpoint directly:
curl -X POST http://127.0.0.1:8088/responses \
-H "Content-Type: application/json" \
-d '{"input": "Hello!"}'
PowerShell equivalent:
(Invoke-WebRequest -Uri http://127.0.0.1:8088/responses `
-Method POST -ContentType 'application/json' `
-Body '{"input": "Hello!"}').Content
You can also invoke the local agent through azd:
azd ai agent invoke --local "Hello!"
Deploy to Foundry
If the initialized project uses a new Foundry project and model deployment, provision the Azure resources first:
azd provision
Deploy the agent:
azd deploy
The sample uses source-code deployment. azd deploy uploads the source for
remote dependency resolution and deploys the agent to the Foundry hosted
runtime.
The sample reads the following runtime environment variables:
FOUNDRY_PROJECT_ENDPOINT: The endpoint URL for the Foundry project where the agent is deployed.AZURE_AI_MODEL_DEPLOYMENT_NAME: The model deployment name configured inazure.yaml.APPLICATIONINSIGHTS_CONNECTION_STRING: The connection string for the project's Application Insights instance, when configured.
For complete deployment concepts, permissions, and management details, see Deploy a Hosted agent and Manage Hosted agent lifecycle.
Deploy with Foundry Toolkit Visual Studio Code extension
To start from a LangGraph sample, follow Start from a sample in Visual Studio Code. For Framework, select LangGraph, and choose a sample for your protocol.
After the toolkit creates your project, follow the dependency installation, local testing, and deployment steps. Use your LangGraph project for these steps.
Host an existing agent
Use the langchain_azure_ai.agents.hosting.run module to host a graph
referenced in your project's langgraph.json file. The graph entry can
point to an exported compiled graph or a factory that returns a compiled graph.
The CLI calls the factory without arguments.
From the project root, start a Responses host:
python -m langchain_azure_ai.agents.hosting.run --protocol responses
To expose the same graph through the Invocations protocol, set --protocol to
invocations. If langgraph.json defines multiple graphs, pass the graph name
as the first argument. Use --config <path> if the configuration file
isn't at the default langgraph.json path. For example:
python -m langchain_azure_ai.agents.hosting.run agent --protocol invocations
Use the same module command as the container entry point when you deploy the existing application to Foundry.
For example, configure the command in azure.yaml.
The key setting is the entry point.
services:
my-agent:
host: azure.ai.agent
kind: hosted
env:
PYTHONNOUSERSITE: "1"
codeConfiguration:
runtime: python_3_13
entryPoint: '-m langchain_azure_ai.agents.hosting.run --protocol responses'
...
...
Troubleshooting
Use this checklist to diagnose common issues while developing Hosted agents
with langchain-azure-ai[hosting].
Graph schema validation fails
The default hosts expect a compiled LangGraph graph whose state has a
messages field, such as MessagesState. If your graph uses a custom state
schema, subclass the host and override build_input. For Responses, override
handle_create when you need full control over request parsing, graph
execution, and emitted Responses events.
Conversation state doesn't continue
For the Responses protocol, pass previous_response_id or a conversation ID
on later turns. If your graph uses a checkpointer, make sure the checkpointer is
configured and durable for the environment where the agent runs.
For the Invocations protocol, the platform doesn't store conversation history.
Use an agent_session_id query parameter to route later calls to the same
hosted sandbox and use your own state store or LangGraph checkpointer for
conversation state.
The model can't be reached in the hosted container
Confirm that the hosted agent version includes AZURE_AI_MODEL_DEPLOYMENT_NAME,
and that the agent identity has permission to call the Foundry project. The
platform sets FOUNDRY_PROJECT_ENDPOINT; your code should read that variable
when running in Foundry.
Next step
Related content
- LangGraph hosted-agent samples in Foundry samples for complete Foundry projects and deployment configuration.
- LangGraph hosting samples in langchain-azure-ai for protocol, workflow, and integration examples.
- Get started with LangChain and LangGraph with Foundry
- Use Foundry Agent Service with LangGraph
- Trace LangChain and LangGraph apps with Microsoft Foundry and Azure Monitor
- Manage hosted agent sessions