Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
ONNX Runtime GenAI lets a .NET Agent Framework application run a compatible model locally. Use it for offline development, on-device inference, or deployments where model execution must stay on the host.
Note
The current ONNX client doesn't support function calling. Function tools passed to the agent are ignored.
Prerequisites
- .NET 8 or later.
- A model exported for ONNX Runtime GenAI.
- Sufficient local memory and a compatible execution provider for the selected model.
Install the packages
dotnet add package Microsoft.ML.OnnxRuntimeGenAI
dotnet add package Microsoft.Agents.AI --prerelease
Configuration
ONNX_MODEL_PATH="<path-to-onnx-runtime-genai-model-directory>"
Create an ONNX-backed agent
Download a model exported for ONNX Runtime GenAI and point ONNX_MODEL_PATH to the model directory.
// E.g. C:\repos\Phi-4-mini-instruct-onnx\cpu_and_mobile\cpu-int4-rtn-block-32-acc-level-4
var modelPath = Environment.GetEnvironmentVariable("ONNX_MODEL_PATH") ?? throw new InvalidOperationException("ONNX_MODEL_PATH is not set.");
// Get a chat client for ONNX and use it to construct an AIAgent.
using OnnxRuntimeGenAIChatClient chatClient = new(modelPath);
AIAgent agent = chatClient.AsAIAgent(instructions: "You are good at telling jokes.", name: "Joker");
// Invoke the agent and output the text result.
Console.WriteLine(await agent.RunAsync("Tell me a joke about a pirate."));
The model files, execution provider, quantization, and available memory determine hardware compatibility and performance. Review the model license before redistributing it.
Tools
The current ONNX client doesn't support function calling or provider-hosted tools.