Distributed AI inference on Windows Server documentation

Explore distributed AI inference on Windows Server. Compare runtimes, connect clients to endpoints, build with SQL Server, and plan GPU infrastructure.

Understand AI inference approaches

Overview

• Plan distributed AI inference on Windows Server

Concept

• What is Windows ML?

Plan GPU acceleration in Windows Server

Concept

• Plan for GPU acceleration in Windows Server

Explore inference runtimes

Get started

• Get started with Foundry Local for on-device inference

Connect apps and tools to model endpoints

How-To Guide

• Connect GitHub Copilot CLI to a Windows Server inference endpoint
• Connect Visual Studio Code to local and remote model endpoints
• Connect Intelligent Terminal Preview to a Windows Server model endpoint

Build AI solutions with SQL Server

Overview

• Intelligent applications and AI in SQL Server

Configure GPUs for clustered VMs

How-To Guide

• Use GPUs with Discrete Device Assignment in clustered VMs