Foundry Local

Foundry Local lets you run supported Microsoft Foundry models on your local machine while still using the standard Agent Framework Python Agent experience.

Warning

Treat model or embedding output, and model-generated tool calls when supported, as untrusted application input. Validate them before code performs side effects or renders content. If you configure a custom endpoint, verify that you trust its URL and operator.

Note

Foundry Local is not currently supported in .NET.

Prerequisites

Install Foundry Local and its local runtime components. Use Python 3.11 through 3.14 on Apple Silicon macOS, Linux x86-64 or ARM64, or Windows x64 or ARM64. Agent Framework uses Foundry Local SDK 2.3.1 or later within the 2.x line. The first request can take longer while the selected model is downloaded and loaded.

Installation

pip install agent-framework-foundry-local --pre

Configuration

Set the default local model with:

FOUNDRY_LOCAL_MODEL="phi-4-mini"

You can also pass the model explicitly with FoundryLocalClient(model="phi-4-mini").

Note

FoundryLocalClient lives in the agent_framework.foundry namespace. It is a local chat client, so you typically pair it with a standard Agent.

Create a local agent

import asyncio

from agent_framework import Agent
from agent_framework.foundry import FoundryLocalClient

async def main():
    agent = Agent(
        client=FoundryLocalClient(model="phi-4-mini"),
        name="LocalAgent",
        instructions="You are a helpful local assistant.",
    )
    result = await agent.run("What's the weather like in Seattle?")
    print(result)

asyncio.run(main())

Tools

FoundryLocalClient is a local chat client paired with a standard Agent, so the supported tools are the ones the chosen local model can actually call — they are not provided by a hosted runtime. Hosted Foundry tool types (get_code_interpreter_tool, get_web_search_tool, etc.) are not available on FoundryLocalClient.

Tool Status Notes
Function Tools ⚠️ Supported only if the selected local model supports function calling. Use FoundryLocalClient.manager to inspect model capabilities.
Tool Approval ✅ Provided by the framework's function-invoking chat client; works with any function-tool call.
Code Interpreter ❌ No hosted runtime.
File Search ❌ No hosted runtime.
Web Search ❌ No hosted runtime.
Hosted MCP Tools ❌ Not exposed by the local runtime.
Local MCP Tools ✅ Runs in your process and works with any chat client.

Model capabilities

Not every local model supports the same features. Function calling and structured outputs depend on the selected model. The FoundryLocalClient.manager helper can be used to inspect the local catalog and supported capabilities before you run an agent.

The shared SDK manager discovers registered local models and prepares the selected model before the first inference. Configure the manager before creating the first client when you need custom runtime settings.

For additional runtime controls, FoundryLocalClient supports options such as device, bootstrap, and prepare_model. bootstrap=False requires an initialized manager and a running service. prepare_model=False defers model loading until the first request; it doesn't disable loading. Unsupported generation options such as do_sample and early_stopping must be removed. Close client HTTP resources before closing the shared manager once at application shutdown.

Note

Go support for this feature is coming soon. See the Agent Framework Go repository for the latest status.

Next steps