How to use tool calling with Foundry Local

Foundry Local can make use of tool calling, a technique where you prompt the model with definitions of available tools that together with a text prompt, allow the model to work out which tools should be called and with what input data. The application then calls those tools and adds the results to a subsequent model prompt to answer the user's query.

The tools can perform functions that the model doesn't have access to, such as getting the current weather, or reading files on the local file system, or accessing a user's address book (providing the application has permission to do so).

This guide shows you how to use this feature of Foundry Local.

Models that support tool calling

Using the Foundry Local CLI, you can run the foundry model list command to see which models support tool calling.

In the Task column, you can see that the tools task indicates that tool calling is supported.

Prerequisites

.NET 8.0 SDK or later installed.

Samples repository

You can find the complete sample code for this article in the Foundry samples GitHub repository. To clone the repository and navigate to the sample, use:

git clone https://github.com/microsoft-foundry/foundry-samples.git
cd foundry-samples/samples/csharp/foundry-local/tool-calling-foundry-local-sdk

Install packages

dotnet add package Microsoft.AI.Foundry.Local
dotnet add package OpenAI

The C# samples in the GitHub repository are preconfigured projects. If you're building from scratch, you should read the Foundry Local SDK reference for more details on how to set up your C# project with Foundry Local.

Understanding tool choice settings

The tool choice parameter controls whether and how the model invokes the tools you provide. Tool choice is sent as part of the chat completion request alongside your tool definitions.

Different models have different capabilities when it comes to tool calling, but in general you can expect the following behavior for each option:

Option	Value	Behavior	Reliability
Auto	`"auto"`	The model decides whether to call a tool or respond directly, based on the user's message and the available tool definitions.	Reliable across all tool-calling models
None	`"none"`	The model won't call any tools, even if tools are provided in the request.	Reliable across all tool-calling models
Required	`"required"`	The model must call at least one tool.	Best-effort
Specific function	`{"type": "function", "function": {"name": "my_function"}}`	The model must call the specified function.	Best-effort

Use native chat completions with tool calling

Copy and paste the following code into a C# file named Program.cs:

using Microsoft.AI.Foundry.Local;
using System.Text.Json;

CancellationToken ct = new CancellationToken();

var config = new Configuration
{
    AppName = "foundry_local_samples",
    LogLevel = Microsoft.AI.Foundry.Local.LogLevel.Information
};


// Initialize the singleton instance.
await FoundryLocalManager.CreateAsync(config, Utils.GetAppLogger());
var mgr = FoundryLocalManager.Instance;


// Ensure that any Execution Provider (EP) downloads run and are completed.
// Download and register all execution providers.
var currentEp = "";
await mgr.DownloadAndRegisterEpsAsync((epName, percent) =>
{
    if (epName != currentEp)
    {
        if (currentEp != "") Console.WriteLine();
        currentEp = epName;
    }
    Console.Write($"\r  {epName.PadRight(30)}  {percent,6:F1}%");
});
if (currentEp != "") Console.WriteLine();


// Get the model catalog
var catalog = await mgr.GetCatalogAsync();


// Get a model using an alias.
var model = await catalog.GetModelAsync("qwen2.5-0.5b") ?? throw new Exception("Model not found");


// Download the model (the method skips download if already cached)
await model.DownloadAsync(progress =>
{
    Console.Write($"\rDownloading model: {progress:F2}%");
    if (progress >= 100f)
    {
        Console.WriteLine();
    }
});


// Load the model
Console.Write($"Loading model {model.Id}...");
await model.LoadAsync();
Console.WriteLine("done.");


// Create a stateful native chat session.
using var chatSession = new ChatSession(model);
chatSession.SetStreaming(true);
chatSession.SetOptions(new RequestOptions { ToolChoice = ToolChoice.Required });


chatSession.AddToolDefinition(
    "multiply_numbers",
    "A tool for multiplying two numbers.",
    """
    {
      "type": "object",
      "properties": {
        "first": { "type": "integer", "description": "The first number in the operation" },
        "second": { "type": "integer", "description": "The second number in the operation" }
      },
      "required": ["first", "second"]
    }
    """);


// Get a streaming chat completion response.
using var firstRequest = new Request();
firstRequest.AddItem(MessageItem.System(
    "You are a helpful AI assistant. Use the provided tool to answer the question."));
firstRequest.AddItem(MessageItem.User("What is the answer to 7 multiplied by 6?"));

var toolCalls = new List<(string CallId, string Name, string Arguments)>();
Console.WriteLine("Chat completion response:");
await using var streamingResponse = chatSession.ProcessStreamingRequestAsync(firstRequest, ct);
await foreach (var item in streamingResponse)
{
    using (item)
    {
        if (item is TextItem { Type: TextItemType.Default } text)
        {
            Console.Write(text.Text);
            Console.Out.Flush();
        }
        else if (item is ToolCallItem toolCall)
        {
            toolCalls.Add((toolCall.CallId, toolCall.Name, toolCall.Arguments));
        }
    }
}
using var firstResponse = await streamingResponse.FinalResponse;
Console.WriteLine();


// Invoke tools called and add their results to the next request.
using var toolResultRequest = new Request();
foreach (var toolCall in toolCalls)
{
    if (toolCall.Name == "multiply_numbers")
    {
        var arguments = JsonSerializer.Deserialize<Dictionary<string, int>>(toolCall.Arguments)!;
        var first = arguments["first"];
        var second = arguments["second"];

        Console.WriteLine($"\nInvoking tool: {toolCall.Name} with arguments {first} and {second}");
        var result = Utils.MultiplyNumbers(first, second);
        Console.WriteLine($"Tool response: {result}");
        toolResultRequest.AddItem(new ToolResultItem(toolCall.CallId, result.ToString()));
    }
}
Console.WriteLine("\nTool calls completed. Prompting model to continue conversation...\n");

// Set tool calling back to auto so that the model can decide whether to call
// the tool again or continue the conversation based on the new user prompt
chatSession.SetOptions(new RequestOptions { ToolChoice = ToolChoice.Auto });
toolResultRequest.AddItem(MessageItem.System("Respond only with the answer generated by the tool."));


// Run the next turn of the conversation
Console.WriteLine("Chat completion response:");
await using var finalStream = chatSession.ProcessStreamingRequestAsync(toolResultRequest, ct);
await foreach (var item in finalStream)
{
    using (item)
    {
        if (item is TextItem { Type: TextItemType.Default } text)
        {
            Console.Write(text.Text);
            Console.Out.Flush();
        }
    }
}
using var finalResponse = await finalStream.FinalResponse;
Console.WriteLine();


// Tidy up - unload the model
finalResponse.Dispose();
await finalStream.DisposeAsync();
toolResultRequest.Dispose();
firstResponse.Dispose();
await streamingResponse.DisposeAsync();
firstRequest.Dispose();
chatSession.Dispose();
await model.UnloadAsync();

Run the native chat completions example

dotnet run

Use OpenAI Web server for tool calling

If you prefer to use the OpenAI SDKs to call the Foundry Local web service, use the following example that demonstrates how to handle tool calling in that scenario.

Tip

Use options.ToolChoice = ChatToolChoice.CreateAutoChoice(); (the default) for the most reliable behavior. Write clear tool names and descriptions so the model calls the correct tool on its own.

using Microsoft.AI.Foundry.Local;
using OpenAI;
using OpenAI.Chat;
using System.ClientModel;
using System.Text.Json;

var config = new Configuration
{
    AppName = "foundry_local_samples",
    LogLevel = Microsoft.AI.Foundry.Local.LogLevel.Information,
    Web = new Configuration.WebService
    {
        Urls = "http://127.0.0.1:52495"
    }
};


// Initialize the singleton instance.
await FoundryLocalManager.CreateAsync(config, Utils.GetAppLogger());
var mgr = FoundryLocalManager.Instance;


// Download and register all execution providers.
var currentEp = "";
await mgr.DownloadAndRegisterEpsAsync((epName, percent) =>
{
    if (epName != currentEp)
    {
        if (currentEp != "") Console.WriteLine();
        currentEp = epName;
    }
    Console.Write($"\r  {epName.PadRight(30)}  {percent,6:F1}%");
});
if (currentEp != "") Console.WriteLine();


// Get the model catalog
var catalog = await mgr.GetCatalogAsync();


// Get a model using an alias
var model = await catalog.GetModelAsync("qwen2.5-0.5b") ?? throw new Exception("Model not found");
// Download the model (the method skips download if already cached)
await model.DownloadAsync(progress =>
{
    Console.Write($"\rDownloading model: {progress:F2}%");
    if (progress >= 100f)
    {
        Console.WriteLine();
    }
});


// Load the model
Console.Write($"Loading model {model.Id}...");
await model.LoadAsync();
Console.WriteLine("done.");


// Start the web service
Console.Write($"Starting web service on {config.Web.Urls}...");
await mgr.StartWebServiceAsync();
Console.WriteLine("done.");


// <<<<<< OPEN AI SDK USAGE >>>>>>
// Use the OpenAI SDK to call the local Foundry web service

ApiKeyCredential key = new ApiKeyCredential("notneeded");
OpenAIClient client = new OpenAIClient(key, new OpenAIClientOptions
{
    Endpoint = new Uri(config.Web.Urls + "/v1"),
});


// Get chat client
var chatClient = client.GetChatClient(model.Id);


// Prepare messages
var messages = new List<ChatMessage>
{
    ChatMessage.CreateSystemMessage("You are a helpful AI assistant. If necessary, you can use any provided tools to answer the question."),
    ChatMessage.CreateUserMessage("What is the answer to 7 multiplied by 6?")
};


// Prepare tools
var tools = new List<ChatTool>
{
    ChatTool.CreateFunctionTool(
        functionName: "multiply_numbers",
        functionDescription: "A tool for multiplying two numbers.",
        functionParameters: BinaryData.FromString("""
        {
            "type": "object",
            "properties": {
                "first": { "type": "number", "description": "The first number in the operation" },
                "second": { "type": "number", "description": "The second number in the operation" }
            },
            "required": ["first", "second"]
        }
        """)
    )
};


// Prepare chat completion options
var options = new ChatCompletionOptions
{
    ToolChoice = ChatToolChoice.CreateRequiredChoice()  // Force the model to make a tool call
};
foreach (var tool in tools)
{
    options.Tools.Add(tool);
}


// Get a streaming chat completion response
var completionUpdates = chatClient.CompleteChatStreaming(messages, options);
var toolCalls = new List<StreamingChatToolCallUpdate>();
Console.Write($"[ASSISTANT]: ");
foreach (var completionUpdate in completionUpdates)
{
    if (completionUpdate.ContentUpdate.Count > 0)
    {
        Console.Write(completionUpdate.ContentUpdate[0].Text);
    }

    if (completionUpdate.FinishReason == ChatFinishReason.ToolCalls)
    {
        foreach (var toolCall in completionUpdate.ToolCallUpdates)
        {
            toolCalls.Add(toolCall);
        }
    }
}
Console.WriteLine();


// Invoke tools called and append responses to the chat
foreach (var toolCall in toolCalls)
{
    if (toolCall.FunctionName == "multiply_numbers")
    {
        var arguments = JsonDocument.Parse(toolCall.FunctionArgumentsUpdate.ToString()).RootElement;
        var first = arguments.GetProperty("first").GetInt32();
        var second = arguments.GetProperty("second").GetInt32();

        Console.WriteLine($"\nInvoking tool: {toolCall.FunctionName} with arguments {first} and {second}");
        var result = Utils.MultiplyNumbers(first, second);
        Console.WriteLine($"Tool response: {result.ToString()}");

        messages.Add(ChatMessage.CreateToolMessage(toolCallId: "abcd1234", content: result.ToString()));
    }
}
Console.WriteLine("\nTool calls completed. Prompting model to continue conversation...\n");


// Prompt the model to continue the conversation after the tool call
messages.Add(ChatMessage.CreateSystemMessage("Respond only with the answer generated by the tool."));


// Set tool calling back to auto so that the model can decide whether to call
// the tool again or continue the conversation based on the new user prompt
options.ToolChoice = ChatToolChoice.CreateAutoChoice();


// Run the next turn of the conversation
Console.WriteLine("Chat completion response:");
completionUpdates = chatClient.CompleteChatStreaming(messages, options);
Console.Write($"[ASSISTANT]: ");
foreach (var completionUpdate in completionUpdates)
{
    if (completionUpdate.ContentUpdate.Count > 0)
    {
        Console.Write(completionUpdate.ContentUpdate[0].Text);
    }
}
Console.WriteLine();

// <<<<<< END OPEN AI SDK USAGE >>>>>>


// Tidy up
// Stop the web service and unload model
await mgr.StopWebServiceAsync();
await model.UnloadAsync();

Run the OpenAI web service example

dotnet run

Prerequisites

Node.js 20 or later installed.

Samples repository

The complete sample code for this article is available in the foundry-samples GitHub repository. To clone the repository and navigate to the sample use:

git clone https://github.com/microsoft-foundry/foundry-samples.git
cd foundry-samples/samples/javascript/foundry-local/tool-calling-foundry-local

Install packages

npm install foundry-local-sdk openai

Understanding tool choice settings

The tool choice parameter controls whether and how the model invokes the tools you provide. Tool choice is sent as part of the chat completion request alongside your tool definitions.

Different models have different capabilities when it comes to tool calling, but in general you can expect the following behavior for each option:

Option	Value	Behavior	Reliability
Auto	`"auto"`	The model decides whether to call a tool or respond directly, based on the user's message and the available tool definitions.	Reliable across all tool-calling models
None	`"none"`	The model doesn't invoke any tools, even if tools are provided in the request.	Reliable across all tool-calling models
Required	`"required"`	The model must call at least one tool.	Best-effort
Specific function	`{"type": "function", "function": {"name": "my_function"}}`	The model must call the specified function.	Best-effort

Use chat completions with tool calling

Copy and paste the following code into a JavaScript file named app.js:

// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.

import { OpenAI } from "openai";
import { FoundryLocalManager } from "foundry-local-sdk";

// By using an alias, the most suitable model will be downloaded 
// to your end-user's device.
// TIP: You can find a list of available models by running the 
// following command in your terminal: `foundry model list`.
const alias = "qwen2.5-0.5b";

function multiplyNumbers(first, second) {
  return first * second;
}

async function runToolCallingExample() {
  let manager = null;
  let model = null;

  try {
    console.log("Initializing Foundry Local SDK...");
    manager = FoundryLocalManager.create({
      appName: "foundry_local_samples",
      webServiceUrls: "http://localhost:5000",
      logLevel: "info"
    });

    // Download and register all execution providers.
    let currentEp = '';
    await manager.downloadAndRegisterEps((epName, percent) => {
      if (epName !== currentEp) {
        if (currentEp !== '') process.stdout.write('\n');
        currentEp = epName;
      }
      process.stdout.write(`\r  ${epName.padEnd(30)}  ${percent.toFixed(1).padStart(5)}%`);
    });
    if (currentEp !== '') process.stdout.write('\n');

    const catalog = manager.catalog;
    model = await catalog.getModel(alias);
    if (!model) {
      throw new Error(`Model ${alias} not found`);
    }

    console.log(`Loading model ${model.id}...`);
    await model.download();
    await model.load();
    console.log('✓ Model loaded');

    manager.startWebService();
    const endpoint = manager.urls[0];
    if (!endpoint) {
      throw new Error("Foundry Local web service did not return an endpoint.");
    }

    const openai = new OpenAI({
      baseURL: `${endpoint.replace(/\/$/, "")}/v1`,
      apiKey: "local"
    });

    // Prepare messages
    const messages = [
      {
        role: "system",
        content: "You are a helpful AI assistant. If necessary, you can use any provided tools to answer the question."
      },
      { role: "user", content: "What is the answer to 7 multiplied by 6?" }
    ];

    // Prepare tools
    const tools = [
      {
        type: "function",
        function: {
          name: "multiply_numbers",
          description: "A tool for multiplying two numbers.",
          parameters: {
            type: "object",
            properties: {
              first: {
                type: "integer",
                description: "The first number in the operation"
              },
              second: {
                type: "integer",
                description: "The second number in the operation"
              }
            },
            required: ["first", "second"]
          }
        }
      }
    ];

    // Start the conversation
    console.log("Chat completion response:");
    const firstResponse = await openai.chat.completions.create({
      model: model.id,
      messages,
      tools,
      tool_choice: "required",
      stream: false
    });

    const assistantMessage = firstResponse.choices[0]?.message;
    if (!assistantMessage) {
      throw new Error("The model returned no assistant message.");
    }
    messages.push(assistantMessage);

    // Invoke tools called and append responses to the chat
    for (const toolCall of assistantMessage.tool_calls ?? []) {
      if (toolCall.function?.name === "multiply_numbers") {
        const args = JSON.parse(toolCall.function.arguments || "{}");
        const first = args.first;
        const second = args.second;

        console.log(`\nInvoking tool: ${toolCall.function.name} with arguments ${first} and ${second}`);
        const result = multiplyNumbers(first, second);
        console.log(`Tool response: ${result}`);

        messages.push({
          role: "tool",
          tool_call_id: toolCall.id,
          content: result.toString()
        });
      }
    }

    console.log("\nTool calls completed. Prompting model to continue conversation...\n");

    // Prompt the model to continue the conversation after the tool call
    messages.push({
      role: "system",
      content: "Respond only with the answer generated by the tool."
    });

    // Run the next turn of the conversation
    console.log("Chat completion response:");
    const secondStream = await openai.chat.completions.create({
      model: model.id,
      messages,
      tools,
      tool_choice: "auto",
      stream: true
    });

    for await (const chunk of secondStream) {
      const content = chunk.choices?.[0]?.delta?.content;
      if (content) {
        process.stdout.write(content);
      }
    }

    console.log();
  } finally {
    if (model) {
      try {
        if (await model.isLoaded()) {
          await model.unload();
        }
      } catch (cleanupError) {
        console.warn("Cleanup warning while unloading model:", cleanupError);
      }
    }

    if (manager) {
      try {
        manager.stopWebService();
      } catch (cleanupError) {
        console.warn("Cleanup warning while stopping service:", cleanupError);
      }
    }
  }
}

await runToolCallingExample().catch((error) => {
  console.error("Error running sample:", error);
  process.exitCode = 1;
});

To run the application, execute the following command in your terminal:

node app.js

Prerequisites

Python 3.11 or later installed.

Samples repository

The complete sample code for this article is available in the foundry-samples GitHub repository. To clone the repository and navigate to the sample use:

git clone https://github.com/microsoft-foundry/foundry-samples.git
cd foundry-samples/samples/python/foundry-local/tool-calling

Install packages

pip install foundry-local-sdk openai

Install the OpenAI SDK:

pip install openai

Understanding tool choice settings

The tool choice parameter controls whether and how the model invokes the tools you provide. The parameter is sent as part of the chat completion request alongside your tool definitions.

Different models have different capabilities when it comes to tool calling, but in general you can expect the following behavior for each option:

Option	Value	Behavior	Reliability
Auto	`"auto"`	The model decides whether to call a tool or respond directly, based on the user's message and the available tool definitions.	Reliable across all tool-calling models
None	`"none"`	The model won't call any tools, even if tools are provided in the request.	Reliable across all tool-calling models
Required	`"required"`	The model must call at least one tool.	Best-effort
Specific function	`{"type": "function", "function": {"name": "my_function"}}`	The model must call the specified function.	Best-effort

Use chat completions with tool calling

Copy and paste the following code into a Python file named app.py:

import json

from foundry_local_sdk import (
    ChatSession,
    Configuration,
    FoundryLocalManager,
    MessageItem,
    Request,
    RequestOptions,
    SearchOptions,
    TextItem,
    ToolCallItem,
    ToolResultItem,
)



# --- Tool definitions ---
tools = [
    {
        "name": "get_weather",
        "description": "Get the current weather for a location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "The city or location",
                },
                "unit": {
                    "type": "string",
                    "enum": ["celsius", "fahrenheit"],
                    "description": "Temperature unit",
                },
            },
            "required": ["location"],
        },
    },
    {
        "name": "calculate",
        "description": "Perform a math calculation",
        "parameters": {
            "type": "object",
            "properties": {
                "expression": {
                    "type": "string",
                    "description": "The math expression to evaluate",
                }
            },
            "required": ["expression"],
        },
    },
]


# --- Tool implementations ---
def get_weather(location, unit="celsius"):
    """Simulate a weather lookup."""
    return {
        "location": location,
        "temperature": 22 if unit == "celsius" else 72,
        "unit": unit,
        "condition": "Sunny",
    }


def calculate(expression):
    """Evaluate a math expression safely."""
    allowed = set("0123456789+-*/(). ")
    if not all(character in allowed for character in expression):
        return {"error": "Invalid expression"}
    try:
        result = eval(expression, {"__builtins__": {}}, {})
        return {"expression": expression, "result": result}
    except Exception as error:
        return {"error": str(error)}


tool_functions = {"get_weather": get_weather, "calculate": calculate}


def response_content(response):
    text_parts = []
    tool_calls = []
    for item in response:
        parts = item.parts if isinstance(item, MessageItem) else [item]
        for part in parts:
            if isinstance(part, TextItem):
                text_parts.append(part.text)
            elif isinstance(part, ToolCallItem):
                tool_calls.append((part.call_id, part.name, part.arguments))
    return "".join(text_parts), tool_calls


def process_tool_calls(session, answer, pending_calls):
    """Handle tool calls until the stateful session produces a final answer."""
    while pending_calls:
        with Request() as follow_up:
            for call_id, function_name, arguments_json in pending_calls:
                arguments = json.loads(arguments_json)
                print(f"  Tool call: {function_name}({arguments})")
                result = tool_functions[function_name](**arguments)
                follow_up.add_item(
                    ToolResultItem(call_id=call_id, result=json.dumps(result))
                )

            with session.process_request(follow_up) as next_response:
                answer, pending_calls = response_content(next_response)

    return answer




def main():
    # Initialize the Foundry Local SDK
    config = Configuration(app_name="foundry_local_samples")
    FoundryLocalManager.initialize(config)
    manager = FoundryLocalManager.instance

    # Discover and register execution providers that are not already available.
    missing_eps = [ep.name for ep in manager.discover_eps() if not ep.is_registered]
    if missing_eps:
        manager.download_and_register_eps(missing_eps)

    # Select and load a tool-capable model.
    model = manager.catalog.get_model("qwen2.5-0.5b")
    if model is None:
        raise RuntimeError("Model 'qwen2.5-0.5b' was not found in the catalog.")
    if not model.is_cached:
        model.download(
            lambda progress: print(
                f"\rDownloading model: {progress:.2f}%", end="", flush=True
            )
        )
        print()
    model.load()
    print("Model loaded and ready.")

    try:
        with ChatSession(model) as session:
            session.set_options(
                RequestOptions(
                    search=SearchOptions(temperature=0.0, max_output_tokens=512)
                )
            )
            for tool in tools:
                session.add_tool_definition(
                    tool["name"],
                    tool["description"],
                    json.dumps(tool["parameters"]),
                )

            with (
                Request()
                .add_item(
                    MessageItem.system(
                        "You are a helpful assistant with access to tools. "
                        "Use them when needed to answer questions accurately."
                    )
                )
                .add_item(
                    MessageItem.user(
                        "What is the weather in Seattle and what is 42 * 17?"
                    )
                ) as request
            ):
                print("Sending request with tools...")
                with session.process_request(request) as response:
                    answer, pending_calls = response_content(response)
                answer = process_tool_calls(session, answer, pending_calls)

            print(f"\nAssistant: {answer}")
    finally:
        # Clean up
        if model.is_loaded:
            model.unload()
        manager.close()
        print("Model unloaded.")




if __name__ == "__main__":
    main()

To run the application, execute the following command in your terminal:

python app.py

Prerequisites

Rust and Cargo installed (Rust 1.70.0 or later).

Samples repository

The complete sample code for this article is available in the foundry-samples GitHub repository. To clone the repository and navigate to the sample use:

git clone https://github.com/microsoft-foundry/foundry-samples.git
cd foundry-samples/samples/rust/foundry-local/tool-calling-foundry-local

Install packages

cargo add foundry-local-sdk
cargo add tokio --features full
cargo add tokio-stream anyhow

Add extra dependencies for HTTP and JSON:

cargo add anyhow reqwest --features reqwest/json
cargo add serde serde_json --features serde/derive

Understanding tool choice settings

The tool choice parameter controls whether and how the model invokes the tools you provide. Tool choice is sent as part of the chat completion request alongside your tool definitions.

Different models have different capabilities when it comes to tool calling, but in general you can expect the following behavior for each option:

Option	Value	Behavior	Reliability
Auto	`"auto"`	The model decides whether to call a tool or respond directly, based on the user's message and the available tool definitions.	Reliable across all tool-calling models
None	`"none"`	The model doesn't call any tools, even if tools are provided in the request.	Reliable across all tool-calling models
Required	`"required"`	The model must call at least one tool.	Best effort (tool call could be ignored by smaller models)
Specific function	`{"type": "function", "function": {"name": "my_function"}}`	The model must call the specified function.	Best-effort (tool call could be ignored by smaller models)

Use chat completions with tool calling

Replace the contents of src/main.rs with the following code:

// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.

use std::io::{self, Write};

use foundry_local_sdk::{
    ChatSession, FoundryLocalConfig, FoundryLocalManager, Item, Request, RequestOptions,
    SearchOptions, TextKind, ToolChoice, ToolDefinition,
};
use serde_json::{json, Value};
use tokio_stream::StreamExt;

const ALIAS: &str = "qwen2.5-0.5b";

fn multiply_numbers(first: f64, second: f64) -> f64 {
    first * second
}

fn invoke_tool(name: &str, args: &Value) -> String {
    match name {
        "multiply_numbers" => {
            let first = args.get("first").and_then(Value::as_f64).unwrap_or(0.0);
            let second = args.get("second").and_then(Value::as_f64).unwrap_or(0.0);
            multiply_numbers(first, second).to_string()
        }
        _ => format!("Unknown tool: {name}"),
    }
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    println!("Tool Calling with Foundry Local");
    println!("===============================\n");

    // ── 1. Initialise the manager ────────────────────────────────────────
    let manager = FoundryLocalManager::create(FoundryLocalConfig::new("foundry_local_samples"))?;

    manager
        .download_and_register_eps_with_progress(None, {
            let mut current_ep = String::new();
            move |ep_name: &str, percent: f64| {
                if ep_name != current_ep {
                    if !current_ep.is_empty() {
                        println!();
                    }
                    current_ep = ep_name.to_string();
                }
                print!("\r  {ep_name:<30}  {percent:5.1}%");
                io::stdout().flush().ok();
            }
        })
        .await?;
    println!();

    // ── 2. Load a model ─────────────────────────────────────────────────
    let model = manager.catalog().get_model(ALIAS).await?;
    println!("Model: {} (id: {})", model.alias(), model.id());

    if !model.is_cached().await? {
        println!("Downloading model...");
        model
            .download(Some(|progress: f64| {
                print!("\r  {progress:.1}%");
                io::stdout().flush().ok();
            }))
            .await?;
        println!();
    }

    println!("Loading model...");
    model.load().await?;
    println!("✓ Model loaded\n");

    let session = ChatSession::new(&model).await?;

    let parameters = json!({
        "type": "object",
        "properties": {
            "first": {
                "type": "number",
                "description": "The first number in the operation"
            },
            "second": {
                "type": "number",
                "description": "The second number in the operation"
            }
        },
        "required": ["first", "second"]
    });
    session
        .add_tool_definition(
            ToolDefinition::new("multiply_numbers", parameters.to_string())
                .with_description("Multiply two numbers."),
        )
        .await?;

    let request = Request::from_items(vec![
        Item::system_message(vec![Item::text(
            "You are a helpful assistant. Use the provided tool when multiplication is requested.",
        )]),
        Item::user_message(vec![Item::text("What is the answer to 7 multiplied by 6?")]),
    ])
    .with_options(RequestOptions {
        search: SearchOptions {
            max_output_tokens: Some(512),
            ..Default::default()
        },
        tool_choice: Some(ToolChoice::Required),
        ..Default::default()
    });

    println!("Chat completion response:");
    let mut tool_results = Vec::new();
    let mut stream = session.process_streaming_request(request);
    while let Some(item) = stream.next().await {
        let item = item?;
        if let Some(call) = item.as_tool_call() {
            let args: Value = serde_json::from_str(&call.arguments).unwrap_or_else(|_| json!({}));
            println!("\nInvoking tool: {} with arguments {args}", call.name);
            let result = invoke_tool(&call.name, &args);
            println!("Tool response: {result}");
            tool_results.push(Item::tool_result(call.call_id.clone(), result));
        } else if let Item::Text {
            text,
            kind: TextKind::Default,
        } = item
        {
            print!("{text}");
            io::stdout().flush().ok();
        }
    }

    if !tool_results.is_empty() {
        println!("\nTool calls completed. Prompting model to continue conversation...\n");
        let follow_up = Request::from_items(tool_results).with_options(RequestOptions {
            search: SearchOptions {
                max_output_tokens: Some(512),
                ..Default::default()
            },
            tool_choice: Some(ToolChoice::None),
            ..Default::default()
        });

        print!("Chat completion response: ");
        let mut stream = session.process_streaming_request(follow_up);
        while let Some(item) = stream.next().await {
            if let Item::Text {
                text,
                kind: TextKind::Default,
            } = item?
            {
                print!("{text}");
                io::stdout().flush().ok();
            }
        }
        println!("\n");
    }

    // ── 7. Clean up ─────────────────────────────────────────────────────
    drop(session);
    println!("Unloading model...");
    model.unload().await?;
    println!("Done.");

    Ok(())
}

To run the application, execute the following command in your terminal:

cargo run

Prerequisites

CMake 3.20 or later.
A C++20 compiler.
The NuGet CLI.

Samples repository

Clone the Foundry samples GitHub repository and go to the C++ sample:

git clone https://github.com/microsoft-foundry/foundry-samples.git
cd foundry-samples/samples/cpp/foundry-local/tool-calling

Install packages

The C++ samples require:

CMake 3.20 or later.
A C++20 compiler.
The NuGet CLI on your PATH.

Configure and build the sample:

cmake -S . -B build
cmake --build build --config Release

During configuration, the sample restores version 2.2.1 of the Microsoft.AI.Foundry.Local.Runtime NuGet package to build/_packages. The package contains the C++ headers and native libraries for Windows, Linux, and macOS. You don't need a separate WinML package.

The CMake configuration selects the runtime assets for your operating system and architecture and copies the required native libraries beside the executable. On macOS and Linux, run the executable from build. On Windows, run the .exe file from build.

Use chat completions with tool calling

The sample defines a local tool, sends its definition with a chat request, executes the requested function, and returns the tool result to the model.

#include <foundry_local/foundry_local_cpp.h>

#include <iostream>
#include <string>

using namespace foundry_local;

constexpr const char* kWeatherSchema = R"({
  "type": "object",
  "properties": {
    "location": {"type": "string", "description": "City name"}
  },
  "required": ["location"]
})";

void PrintAssistantMessage(const Item& item) {
  for (const Item& part : item.GetMessage().parts) {
    if (part.GetType() == FOUNDRY_LOCAL_ITEM_TEXT) {
      std::cout << part.GetText().text;
    }
  }
  std::cout << "\n";
}

int main() {
  try {
    Configuration config("foundry_local_cpp_tool_calling");
    Manager manager(std::move(config));
    auto model = manager.GetCatalog().GetModel("qwen2.5-0.5b");
    if (!model) {
      throw std::runtime_error("Model 'qwen2.5-0.5b' was not found in the catalog.");
    }

    if (!model->IsCached()) {
      model->Download([](float progress) {
        std::cout << "\rDownloading model: " << static_cast<int>(progress) << "%" << std::flush;
        return 0;
      });
      std::cout << "\n";
    }
    model->Load();

    {
      ChatSession session(*model);
      session.AddToolDefinition(ToolDefinition{
          "get_weather",
          "Get the current weather for a city.",
          kWeatherSchema,
      });

      Request request{UserMessage("What is the weather in Paris?")};
      request.SetOptions({
          .search = {.temperature = 0.0F, .max_output_tokens = 256, .seed = 42},
          .tool_choice = FOUNDRY_LOCAL_TOOL_CHOICE_REQUIRED,
      });

      Response response = session.ProcessRequest(request);
      for (int iteration = 0; iteration < 8; ++iteration) {
        bool handled_tool = false;
        for (const Item& item : response.GetItems()) {
          if (item.GetType() == FOUNDRY_LOCAL_ITEM_MESSAGE) {
            std::cout << "Assistant: ";
            PrintAssistantMessage(item);
            handled_tool = false;
            break;
          }
          if (item.GetType() == FOUNDRY_LOCAL_ITEM_TOOL_CALL) {
            const auto call = item.GetToolCall();
            std::cout << "Tool call: " << call.name << "(" << call.arguments << ")\n";
            Request result;
            result.AddItem(Item::ToolResult(
                std::string(call.call_id),
                R"({"temperature":22,"unit":"celsius","condition":"sunny"})"));
            result.SetOptions({
                .search = {.temperature = 0.0F, .max_output_tokens = 256, .seed = 42},
                .tool_choice = FOUNDRY_LOCAL_TOOL_CHOICE_NONE,
            });
            response = session.ProcessRequest(result);
            handled_tool = true;
            break;
          }
        }
        if (!handled_tool) {
          break;
        }
      }
    }

    model->Unload();
    manager.Shutdown();
  } catch (const Error& error) {
    std::cerr << "Foundry Local error [" << error.Code() << "]: " << error.what() << "\n";
    return 1;
  } catch (const std::exception& error) {
    std::cerr << "Error: " << error.what() << "\n";
    return 1;
  }

  return 0;
}

Run the sample:

./build/tool-calling

Feedback

Was this page helpful?

Last updated on 2026-11-03

How to use tool calling with Foundry Local

Models that support tool calling

Prerequisites

Samples repository

Install packages

Understanding tool choice settings

Use native chat completions with tool calling

Run the native chat completions example

Use OpenAI Web server for tool calling

Run the OpenAI web service example

Prerequisites

Samples repository

Install packages

Understanding tool choice settings

Use chat completions with tool calling

Prerequisites

Samples repository

Install packages

Understanding tool choice settings

Use chat completions with tool calling

Prerequisites

Samples repository

Install packages

Understanding tool choice settings

Use chat completions with tool calling

Prerequisites

Samples repository

Install packages

Use chat completions with tool calling

Related content

Feedback

Additional resources