Tutorial: Membangun aplikasi RAG

Bangun aplikasi retrieval-augmented generation (RAG) yang dapat menjawab pertanyaan seputar kumpulan dokumen secara langsung di perangkat Anda. RAG menggabungkan pencarian berbasis penyematan dengan model percakapan sehingga jawaban didasarkan pada data Anda sendiri daripada hanya mengandalkan pengetahuan pelatihan model.

Dalam tutorial ini, Anda mempelajari cara:

  • Menyiapkan proyek dan menginstal SDK Lokal Foundry
  • Membuat pangkalan pengetahuan dokumen teks
  • Membuat penyematan untuk dokumen
  • Mencari dokumen yang relevan menurut kesamaan
  • Berdasarkan konteks yang diambil, hasilkan jawaban
  • Membersihkan sumber daya

Prasyarat

  • Komputer Windows, macOS, atau Linux dengan RAM minimal 8 GB.
  • Python 3.11 atau yang lebih baru terinstal.

Menginstal paket

Jika Anda mengembangkan atau mengirim di Windows, pilih tab Windows. Paket Windows terintegrasi dengan runtime Windows ML — ini menyediakan area permukaan API yang sama dengan luas akselerasi perangkat keras yang lebih luas.

pip install foundry-local-sdk-winml openai

Membuat pangkalan pengetahuan

Aplikasi RAG memerlukan kumpulan dokumen untuk dicari. Tentukan daftar string teks yang berfungsi sebagai pangkalan pengetahuan. Dalam aplikasi produksi, daftarnya bisa berupa paragraf dari file, rekaman database, atau sumber teks lainnya.

Buat file yang disebut main.py dan tambahkan kode berikut:

import math
from foundry_local_sdk import Configuration, FoundryLocalManager

# Knowledge base — each string represents a document
documents = [
    "Foundry Local runs AI models directly on your device without cloud connectivity.",
    "The Foundry Local SDK supports Python, C#, JavaScript, and Rust.",
    "Embedding models convert text into numerical vectors for similarity search.",
    "Foundry Local uses ONNX Runtime for efficient model inference on CPUs and GPUs.",
    "The model catalog provides pre-optimized models that you can download and run locally.",
    "Retrieval-augmented generation grounds model responses in your own data.",
    "Vector similarity search finds documents that are semantically close to a query.",
    "Chat completions generate natural language responses from a prompt and context.",
]

Membuat penyematan dokumen

Inisialisasi SDK, muat model penyematan, dan konversi setiap dokumen menjadi vektor numerik. Vektor ini mewakili arti semantik dari setiap dokumen dan mengaktifkan pencarian kesamaan.

Tambahkan kode berikut ke main.py:

def main():
    # Initialize the SDK
    config = Configuration(app_name="foundry_local_rag")
    FoundryLocalManager.initialize(config)
    manager = FoundryLocalManager.instance

    # Load the embedding model
    embedding_model = manager.catalog.get_model("qwen3-embedding-0.6b")
    embedding_model.download(
        lambda p: print(f"\rDownloading embedding model: {p:.1f}%", end="", flush=True)
    )
    print()
    embedding_model.load()
    embedding_client = embedding_model.get_embedding_client()

    # Embed all documents in a single batch call
    response = embedding_client.generate_embeddings(documents)
    doc_embeddings = [item.embedding for item in response.data]
    print(f"Indexed {len(doc_embeddings)} documents.")

Metode generate_embeddings ini menerima daftar string dan mengembalikan satu vektor per input. Setiap vektor menangkap arti semantik teks, sehingga dokumen serupa menghasilkan vektor yang berdekatan di ruang penyematan.

Mencari dokumen yang relevan

Untuk menemukan dokumen yang terkait dengan kueri, bandingkan penyematan kueri terhadap setiap penyematan dokumen menggunakan kesamaan kosinus. Kesamaan kosinus mengukur seberapa dekat dua vektor dalam arah, terlepas dari besarnya. Nilai di dekat 1,0 menunjukkan kesamaan tinggi.

Tambahkan fungsi pembantu berikut di atas main() di main.py:

def cosine_similarity(a, b):
    """Compute cosine similarity between two vectors."""
    dot = sum(x * y for x, y in zip(a, b))
    norm_a = math.sqrt(sum(x * x for x in a))
    norm_b = math.sqrt(sum(x * x for x in b))
    return dot / (norm_a * norm_b) if norm_a and norm_b else 0.0


def find_relevant(query_embedding, doc_embeddings, top_k=2):
    """Return the indices and scores of the top-k most similar documents."""
    scores = []
    for i, doc_emb in enumerate(doc_embeddings):
        score = cosine_similarity(query_embedding, doc_emb)
        scores.append((i, score))
    scores.sort(key=lambda x: x[1], reverse=True)
    return scores[:top_k]

Fungsi ini find_relevant memberi peringkat semua dokumen berdasarkan kesamaan dan mengembalikan kecocokan teratas. Pendekatan ini berfungsi dengan baik untuk koleksi kecil. Untuk himpunan data yang lebih besar, pertimbangkan database vektor khusus.

Hasilkan jawaban dasar

Muat model obrolan dan gabungkan dokumen yang diambil dengan pertanyaan pengguna dalam permintaan sistem. Model obrolan menggunakan konteks yang disediakan untuk menghasilkan jawaban yang berdasarkan dokumen Anda.

Tambahkan kode berikut ke fungsi main() setelah bagian penyematan:

    # Load the chat model
    chat_model = manager.catalog.get_model("qwen2.5-0.5b")
    chat_model.download(
        lambda p: print(f"\rDownloading chat model: {p:.1f}%", end="", flush=True)
    )
    print()
    chat_model.load()
    chat_client = chat_model.get_chat_client()

    print("\nModels loaded. Ready for questions.")
    print("\nThe knowledge base contains information about:")
    print("  - Foundry Local features and architecture")
    print("  - Supported programming languages")
    print("  - Embedding models and vector search")
    print("  - ONNX Runtime inference")
    print("  - The model catalog")
    print("  - RAG and chat completions")
    print("\nExample questions:")
    print('  "What programming languages does the SDK support?"')
    print('  "How does Foundry Local run models?"')
    print('  "What is retrieval-augmented generation?"')
    print('\nType "quit" to exit.\n')

    # Interactive query loop
    while True:
        query = input("Question: ").strip()
        if not query or query.lower() == "quit":
            break

        # Embed the query
        query_response = embedding_client.generate_embedding(query)
        query_embedding = query_response.data[0].embedding

        # Retrieve the most relevant documents
        results = find_relevant(query_embedding, doc_embeddings, top_k=2)
        context = "\n".join(f"- {documents[i]}" for i, _ in results)

        # Build the prompt with retrieved context
        messages = [
            {
                "role": "system",
                "content": (
                    "Answer the user's question using only the provided context. "
                    "If the context doesn't contain enough information, say so.\n\n"
                    f"Context:\n{context}"
                ),
            },
            {"role": "user", "content": query},
        ]

        # Stream the response
        print("Answer: ", end="", flush=True)
        for chunk in chat_client.complete_streaming_chat(messages):
            content = chunk.choices[0].delta.content
            if content:
                print(content, end="", flush=True)
        print("\n")

    # Clean up
    embedding_model.unload()
    chat_model.unload()
    print("Models unloaded. Done!")


if __name__ == "__main__":
    main()

Perintah sistem menginstruksikan model untuk menjawab hanya menggunakan konteks yang diambil. Permintaan sistem menjaga respons tetap berlandaskan pada dokumen Anda dan mengurangi jawaban yang salah. Output streaming mencetak setiap token saat dihasilkan, membuat respons terasa interaktif.

Kode lengkap

Berikut adalah aplikasi lengkap yang menggabungkan semua langkah:

import math
from foundry_local_sdk import Configuration, FoundryLocalManager

# Knowledge base
documents = [
    "Foundry Local runs AI models directly on your device without cloud connectivity.",
    "The Foundry Local SDK supports Python, C#, JavaScript, and Rust.",
    "Embedding models convert text into numerical vectors for similarity search.",
    "Foundry Local uses ONNX Runtime for efficient model inference on CPUs and GPUs.",
    "The model catalog provides pre-optimized models that you can download and run locally.",
    "Retrieval-augmented generation grounds model responses in your own data.",
    "Vector similarity search finds documents that are semantically close to a query.",
    "Chat completions generate natural language responses from a prompt and context.",
]


def cosine_similarity(a, b):
    """Compute cosine similarity between two vectors."""
    dot = sum(x * y for x, y in zip(a, b))
    norm_a = math.sqrt(sum(x * x for x in a))
    norm_b = math.sqrt(sum(x * x for x in b))
    return dot / (norm_a * norm_b) if norm_a and norm_b else 0.0


def find_relevant(query_embedding, doc_embeddings, top_k=2):
    """Return the indices and scores of the top-k most similar documents."""
    scores = []
    for i, doc_emb in enumerate(doc_embeddings):
        score = cosine_similarity(query_embedding, doc_emb)
        scores.append((i, score))
    scores.sort(key=lambda x: x[1], reverse=True)
    return scores[:top_k]


def main():
    # Initialize the SDK
    config = Configuration(app_name="foundry_local_rag")
    FoundryLocalManager.initialize(config)
    manager = FoundryLocalManager.instance

    # Load the embedding model
    embedding_model = manager.catalog.get_model("qwen3-embedding-0.6b")
    embedding_model.download(
        lambda p: print(f"\rDownloading embedding model: {p:.1f}%", end="", flush=True)
    )
    print()
    embedding_model.load()
    embedding_client = embedding_model.get_embedding_client()

    # Embed all documents
    response = embedding_client.generate_embeddings(documents)
    doc_embeddings = [item.embedding for item in response.data]
    print(f"Indexed {len(doc_embeddings)} documents.")

    # Load the chat model
    chat_model = manager.catalog.get_model("qwen2.5-0.5b")
    chat_model.download(
        lambda p: print(f"\rDownloading chat model: {p:.1f}%", end="", flush=True)
    )
    print()
    chat_model.load()
    chat_client = chat_model.get_chat_client()

    print("\nModels loaded. Ready for questions.")
    print("\nThe knowledge base contains information about:")
    print("  - Foundry Local features and architecture")
    print("  - Supported programming languages")
    print("  - Embedding models and vector search")
    print("  - ONNX Runtime inference")
    print("  - The model catalog")
    print("  - RAG and chat completions")
    print("\nExample questions:")
    print('  "What programming languages does the SDK support?"')
    print('  "How does Foundry Local run models?"')
    print('  "What is retrieval-augmented generation?"')
    print('\nType "quit" to exit.\n')

    # Interactive query loop
    while True:
        query = input("Question: ").strip()
        if not query or query.lower() == "quit":
            break

        # Embed the query
        query_response = embedding_client.generate_embedding(query)
        query_embedding = query_response.data[0].embedding

        # Retrieve the most relevant documents
        results = find_relevant(query_embedding, doc_embeddings, top_k=2)
        context = "\n".join(f"- {documents[i]}" for i, _ in results)

        # Build the prompt with retrieved context
        messages = [
            {
                "role": "system",
                "content": (
                    "Answer the user's question using only the provided context. "
                    "If the context doesn't contain enough information, say so.\n\n"
                    f"Context:\n{context}"
                ),
            },
            {"role": "user", "content": query},
        ]

        # Stream the response
        print("Answer: ", end="", flush=True)
        for chunk in chat_client.complete_streaming_chat(messages):
            content = chunk.choices[0].delta.content
            if content:
                print(content, end="", flush=True)
        print("\n")

    # Clean up
    embedding_model.unload()
    chat_model.unload()
    print("Models unloaded. Done!")


if __name__ == "__main__":
    main()

Jalankan aplikasi:

python main.py

Anda melihat output yang mirip dengan:

Downloading embedding model: 100.0%
Indexed 8 documents.
Downloading chat model: 100.0%

Models loaded. Ready for questions.

The knowledge base contains information about:
  - Foundry Local features and architecture
  - Supported programming languages
  - Embedding models and vector search
  - ONNX Runtime inference
  - The model catalog
  - RAG and chat completions

Example questions:
  "What programming languages does the SDK support?"
  "How does Foundry Local run models?"
  "What is retrieval-augmented generation?"

Type "quit" to exit.

Question: What programming languages does the SDK support?
Answer: The Foundry Local SDK supports Python, C#, JavaScript, and Rust.

Question: quit
Models unloaded. Done!

Membersihkan sumber daya

Bobot model tetap berada di cache lokal Anda setelah Anda membongkar model. Lain kali Anda menjalankan aplikasi, langkah unduhan dilewati dan model dimuat lebih cepat. Tidak diperlukan pembersihan tambahan kecuali Anda ingin mengklaim kembali ruang disk.

Konten terkait

  • Terakhir diperbarui pada