RAG local option

Question

hi team, happy to learn the option to build my RAG in local. What’s the current pro and cons. Please share any insight

Answer 1

Hello @Aiden Drey

Thanks for reaching out to us, you can do Retrieval Augmented Generation using Azure Machine Learning prompt flow (preview)

https://learn.microsoft.com/en-us/azure/machine-learning/concept-retrieval-augmented-generation?view=azureml-api-2

This feature is currently in public preview. This preview version is provided without a service-level agreement, and we don't recommend it for production workloads. Certain features might not be supported or might have constrained capabilities.

For more information, see Supplemental Terms of Use for Microsoft Azure Previews.

In Azure Machine Learning, you can now implement RAG in a prompt flow. Support for RAG is currently in public preview.

RAG in Azure Machine Learning is enabled by integration with Azure OpenAI Service for large language models and vectorization, with support for Faiss and Azure AI Search (formerly Cognitive Search) as vector stores, and support for open source offerings tools and frameworks such as LangChain for data chunking.

To implement RAG, a few key requirements must be met. First, data should be formatted in a manner that allows efficient searchability before sending it to the LLM, which ultimately reduces token consumption. To ensure the effectiveness of RAG, it's also important to regularly update your data on a periodic basis. Furthermore, having the capability to evaluate the output from the LLM using your data enables you to measure the efficacy of your techniques. Azure Machine Learning not only allows you to get started easily on these aspects, but also enables you to improve and productionize RAG. Azure Machine Learning offers:

• Samples for starting RAG-based Q&A scenarios.
• Wizard-based UI experience to create and manage data and incorporate it into prompt flows.
• Ability to measure and enhance RAG workflows, including test data generation, automatic prompt creation, and visualized prompt evaluation metrics.
• Advanced scenarios with more control using the new built-in RAG components for creating custom pipelines in notebooks.
• Code experience, which allows utilization of data created with open source offerings like LangChain.
• Seamless integration of RAG workflows into MLOps workflows using pipelines and jobs.

I hope this helps! Please have a try on this feature and let us know how it works.

Regards,

Yutong

-Please kindly accept the answer if you feel helpful to support the community, thanks a lot.