Events
Take the Microsoft Learn Challenge
Nov 19, 11 PM - Jan 10, 11 PM
Ignite Edition - Build skills in Microsoft Azure and earn a digital badge by January 10!
Register nowThis browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
This article offers you examples on how to create an RAG pipeline. For advanced scenarios, you can build your own custom Azure Machine Learning pipelines from code (typically notebooks) that allows you granular control of the RAG workflow. Azure Machine Learning provides several in-built pipeline components for data chunking, embeddings generation, test data creation, automatic prompt generation, prompt evaluation. These components can be used as per your needs using notebooks. You can even use the Vector Index created in Azure Machine Learning in LangChain.
Important
This feature is currently in public preview. This preview version is provided without a service-level agreement, and we don't recommend it for production workloads. Certain features might not be supported or might have constrained capabilities.
For more information, see Supplemental Terms of Use for Microsoft Azure Previews.
An Azure subscription. If you don't have an Azure subscription, create a free account.
Access to Azure OpenAI.
Enable prompt flow in your Azure Machine Learning workspace
In your Azure Machine Learning workspace, you can enable prompt flow by turn-on Build AI solutions with Prompt flow in the Manage preview features panel.
Azure Machine Learning offers notebook tutorials for several use cases with prompt flow pipelines.
QA Data Generation
QA Data Generation can be used to get the best prompt for RAG and to evaluation metrics for RAG. This notebook shows you how to create a QA dataset from your data (Git repo).
Test Data Generation and Auto Prompt
Use vector indexes to build a retrieval augmented generation model and to evaluate prompt flow on a test dataset.
Create a FAISS based Vector Index
Set up an Azure Machine Learning Pipeline to pull a Git Repo, process the data into chunks, embed the chunks and create a langchain compatible FAISS Vector Index.
How to create vector index in Azure Machine Learning prompt flow (preview)
Use Vector Stores with Azure Machine Learning (preview)
Events
Take the Microsoft Learn Challenge
Nov 19, 11 PM - Jan 10, 11 PM
Ignite Edition - Build skills in Microsoft Azure and earn a digital badge by January 10!
Register nowTraining
Module
Implement Retrieval Augmented Generation (RAG) with Azure Databricks - Training
Implement Retrieval Augmented Generation (RAG) with Azure Databricks
Certification
Microsoft Certified: Azure Data Scientist Associate - Certifications
Manage data ingestion and preparation, model training and deployment, and machine learning solution monitoring with Python, Azure Machine Learning and MLflow.