Edit

Generate text embeddings with Foundry Local

The Foundry Local SDK provides an embedding API that converts text into numerical vectors on-device. Use these vectors for similarity search, classification, clustering, and retrieval-augmented generation (RAG).

The SDK supports both single-input and batch embedding generation through a dedicated embedding client.

Prerequisites

Samples repository

The complete sample code for this article is available in the foundry-samples GitHub repository. To clone the repository and navigate to the sample use:

git clone https://github.com/microsoft-foundry/foundry-samples.git
cd foundry-samples/samples/python/foundry-local/embeddings

Install packages

If you're developing or shipping on Windows, select the Windows tab. The Windows package integrates with the Windows ML runtime — it provides the same API surface area with a wider breadth of hardware acceleration.

pip install foundry-local-sdk-winml openai

Generate text embeddings

Copy and paste the following code into a Python file named app.py:

from foundry_local_sdk import Configuration, FoundryLocalManager



def main():
    # Initialize the Foundry Local SDK
    config = Configuration(app_name="foundry_local_samples")
    FoundryLocalManager.initialize(config)
    manager = FoundryLocalManager.instance

    # Select and load an embedding model from the catalog
    model = manager.catalog.get_model("qwen3-embedding-0.6b")
    model.download(
        lambda progress: print(
            f"\rDownloading model: {progress:.2f}%",
            end="",
            flush=True,
        )
    )
    print()
    model.load()
    print("Model loaded and ready.")

    # Get an embedding client
    client = model.get_embedding_client()

    # Generate a single embedding
    print("\n--- Single Embedding ---")
    response = client.generate_embedding("The quick brown fox jumps over the lazy dog")
    embedding = response.data[0].embedding
    print(f"Dimensions: {len(embedding)}")
    print(f"First 5 values: {embedding[:5]}")

    # Generate embeddings for multiple inputs
    print("\n--- Batch Embeddings ---")
    batch_response = client.generate_embeddings(
        [
            "Machine learning is a subset of artificial intelligence",
            "The capital of France is Paris",
            "Rust is a systems programming language",
        ]
    )

    print(f"Number of embeddings: {len(batch_response.data)}")
    for i, data in enumerate(batch_response.data):
        print(f"  [{i}] Dimensions: {len(data.embedding)}")

    # Clean up
    model.unload()
    print("\nModel unloaded.")


if __name__ == "__main__":
    main()

Run the code by using the following command:

python app.py

Troubleshooting

  • ModuleNotFoundError: No module named 'foundry_local_sdk': Install the SDK by running pip install foundry-local-sdk.
  • Model not found: Run the optional model listing snippet to find an alias available on your device, then update the alias passed to get_model.
  • Slow first run: Model downloads can take time the first time you run the app.

Prerequisites

Samples repository

The complete sample code for this article is available in the foundry-samples GitHub repository. To clone the repository and navigate to the sample use:

git clone https://github.com/microsoft-foundry/foundry-samples.git
cd foundry-samples/samples/csharp/foundry-local/embeddings

Install packages

If you're developing or shipping on Windows, select the Windows tab. The Windows package integrates with the Windows ML runtime — it provides the same API surface area with a wider breadth of hardware acceleration.

dotnet add package Microsoft.AI.Foundry.Local.WinML
dotnet add package OpenAI

The C# samples in the GitHub repository are preconfigured projects. If you're building from scratch, you should read the Foundry Local SDK reference for more details on how to set up your C# project with Foundry Local.

Generate text embeddings

Copy and paste the following code into a C# file named Program.cs:

using Microsoft.AI.Foundry.Local;

var config = new Configuration
{
    AppName = "foundry_local_samples",
    LogLevel = Microsoft.AI.Foundry.Local.LogLevel.Information
};

// Initialize the singleton instance.
await FoundryLocalManager.CreateAsync(config, Utils.GetAppLogger());
var mgr = FoundryLocalManager.Instance;

// Get the model catalog
var catalog = await mgr.GetCatalogAsync();

// Get an embedding model
var model = await catalog.GetModelAsync("qwen3-embedding-0.6b") ?? throw new Exception("Embedding model not found");

// Download the model (the method skips download if already cached)
await model.DownloadAsync(progress =>
{
    Console.Write($"\rDownloading model: {progress:F2}%");
    if (progress >= 100f)
    {
        Console.WriteLine();
    }
});

// Load the model
Console.Write($"Loading model {model.Id}...");
await model.LoadAsync();
Console.WriteLine("done.");

// Get an embedding client
var embeddingClient = await model.GetEmbeddingClientAsync();

// Generate a single embedding
Console.WriteLine("\n--- Single Embedding ---");
var response = await embeddingClient.GenerateEmbeddingAsync("The quick brown fox jumps over the lazy dog");
var embedding = response.Data[0].Embedding;
Console.WriteLine($"Dimensions: {embedding.Count}");
Console.WriteLine($"First 5 values: [{string.Join(", ", embedding.Take(5).Select(v => v.ToString("F6")))}]");

// Generate embeddings for multiple inputs
Console.WriteLine("\n--- Batch Embeddings ---");
var batchResponse = await embeddingClient.GenerateEmbeddingsAsync([
    "Machine learning is a subset of artificial intelligence",
    "The capital of France is Paris",
    "Rust is a systems programming language"
]);

Console.WriteLine($"Number of embeddings: {batchResponse.Data.Count}");
for (var i = 0; i < batchResponse.Data.Count; i++)
{
    Console.WriteLine($"  [{i}] Dimensions: {batchResponse.Data[i].Embedding.Count}");
}

// Tidy up - unload the model
await model.UnloadAsync();
Console.WriteLine("\nModel unloaded.");

Run the code by using the following command:

dotnet run

Troubleshooting

  • Build errors referencing net8.0: Install the .NET 8.0 SDK, then rebuild the app.
  • Model not found: Run the optional model listing snippet to find an alias available on your device, then update the alias passed to GetModelAsync.
  • Slow first run: Model downloads can take time the first time you run the app.

Prerequisites

Samples repository

The complete sample code for this article is available in the foundry-samples GitHub repository. To clone the repository and navigate to the sample use:

git clone https://github.com/microsoft-foundry/foundry-samples.git
cd foundry-samples/samples/javascript/foundry-local/embeddings

Install packages

If you're developing or shipping on Windows, select the Windows tab. The Windows package integrates with the Windows ML runtime — it provides the same API surface area with a wider breadth of hardware acceleration.

npm install foundry-local-sdk-winml openai

Generate text embeddings

Copy and paste the following code into a JavaScript file named app.js:

import { FoundryLocalManager } from 'foundry-local-sdk';

// Initialize the Foundry Local SDK
console.log('Initializing Foundry Local SDK...');

const manager = FoundryLocalManager.create({
    appName: 'foundry_local_samples',
    logLevel: 'info'
});
console.log('✓ SDK initialized successfully');

// Get an embedding model
const modelAlias = 'qwen3-embedding-0.6b';
const model = await manager.catalog.getModel(modelAlias);

// Download the model
console.log(`\nDownloading model ${modelAlias}...`);
await model.download((progress) => {
    process.stdout.write(`\rDownloading... ${progress.toFixed(2)}%`);
});
console.log('\n✓ Model downloaded');

// Load the model
console.log(`\nLoading model ${modelAlias}...`);
await model.load();
console.log('✓ Model loaded');

// Create embedding client
console.log('\nCreating embedding client...');
const embeddingClient = model.createEmbeddingClient();
console.log('✓ Embedding client created');

// Generate a single embedding
console.log('\n--- Single Embedding ---');
const response = await embeddingClient.generateEmbedding(
    'The quick brown fox jumps over the lazy dog'
);

const embedding = response.data[0].embedding;
console.log(`Dimensions: ${embedding.length}`);
console.log(`First 5 values: [${embedding.slice(0, 5).map(v => v.toFixed(6)).join(', ')}]`);

// Generate embeddings for multiple inputs
console.log('\n--- Batch Embeddings ---');
const batchResponse = await embeddingClient.generateEmbeddings([
    'Machine learning is a subset of artificial intelligence',
    'The capital of France is Paris',
    'Rust is a systems programming language'
]);

console.log(`Number of embeddings: ${batchResponse.data.length}`);
for (let i = 0; i < batchResponse.data.length; i++) {
    console.log(`  [${i}] Dimensions: ${batchResponse.data[i].embedding.length}`);
}

// Unload the model
console.log('\nUnloading model...');
await model.unload();
console.log('✓ Model unloaded');

Run the code by using the following command:

node app.js

Troubleshooting

  • Cannot find module 'foundry-local-sdk': Run npm install foundry-local-sdk to install the SDK.
  • Model not found: Verify the model alias is correct. Use manager.catalog.getModels() to list available models.
  • Slow first run: Model downloads can take time the first time you run the app.

Prerequisites

Samples repository

The complete sample code for this article is available in the foundry-samples GitHub repository. To clone the repository and navigate to the sample use:

git clone https://github.com/microsoft-foundry/foundry-samples.git
cd foundry-samples/samples/rust/foundry-local/embeddings

Install packages

If you're developing or shipping on Windows, select the Windows tab. The Windows package integrates with the Windows ML runtime — it provides the same API surface area with a wider breadth of hardware acceleration.

cargo add foundry-local-sdk --features winml
cargo add tokio --features full
cargo add tokio-stream anyhow

Generate text embeddings

Replace the contents of main.rs with the following code:

// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.

use foundry_local_sdk::{FoundryLocalConfig, FoundryLocalManager};

const ALIAS: &str = "qwen3-embedding-0.6b";

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    println!("Native Embeddings");
    println!("=================\n");

    // ── 1. Initialise the manager ────────────────────────────────────────
    let manager = FoundryLocalManager::create(FoundryLocalConfig::new("foundry_local_samples"))?;

    // ── 2. Pick a model and ensure it is downloaded ─────────────────────
    let model = manager.catalog().get_model(ALIAS).await?;
    println!("Model: {} (id: {})", model.alias(), model.id());

    if !model.is_cached().await? {
        println!("Downloading model...");
        model
            .download(Some(|progress: f64| {
                print!("\r  {progress:.1}%");
                std::io::Write::flush(&mut std::io::stdout()).ok();
            }))
            .await?;
        println!();
    }

    println!("Loading model...");
    model.load().await?;
    println!("✓ Model loaded\n");

    // ── 3. Create an embedding client ───────────────────────────────────
    let client = model.create_embedding_client();

    // ── 4. Single embedding ─────────────────────────────────────────────
    println!("--- Single Embedding ---");
    let response = client
        .generate_embedding("The quick brown fox jumps over the lazy dog")
        .await?;

    let embedding = &response.data[0].embedding;
    println!("Dimensions: {}", embedding.len());
    println!(
        "First 5 values: {:?}",
        &embedding[..5]
    );

    // ── 5. Batch embeddings ─────────────────────────────────────────────
    println!("\n--- Batch Embeddings ---");
    let batch_response = client
        .generate_embeddings(&[
            "Machine learning is a subset of artificial intelligence",
            "The capital of France is Paris",
            "Rust is a systems programming language",
        ])
        .await?;

    println!("Number of embeddings: {}", batch_response.data.len());
    for (i, data) in batch_response.data.iter().enumerate() {
        println!("  [{i}] Dimensions: {}", data.embedding.len());
    }

    // ── 6. Unload the model ─────────────────────────────────────────────
    println!("\nUnloading model...");
    model.unload().await?;
    println!("Done.");

    Ok(())
}

Run the code by using the following command:

cargo run

Troubleshooting

  • Build errors: Ensure you have Rust 1.70.0 or later installed. Run rustup update to get the latest version.
  • Model not found: Verify the model alias is correct. Use manager.catalog().get_models().await? to list available models.
  • Slow first run: Model downloads can take time the first time you run the app.

Related content

  • Last updated on