Advanced RAG and Agentic Workflow Use Cases Azure AI Foundry

Question

Advanced RAG and Agentic Workflow Use Cases Azure AI Foundry

MelDev 0

I have a project where I want to build an Advanced RAG and Agentic Workflow for proposal response , an AI solution that ingests incoming document, retrieves relevant information from our internal documents, and drafts grounded responses for our proposals team to review. My approach is to utilize Azure AI Foundry Hub as the central platform and create multiple projects under the Hub to accommodate different use cases.

For a reliable enterprise solution, which Azure services should I use?

Pavan Kumar Purilla 90 Reputation points Microsoft External Staff Moderator

2026-07-22T04:23:37.07+00:00

Hello MelDev,

Thank you for reaching out to Microsoft Q&A.

For your proposal response use case, where you need to ingest incoming documents, retrieve relevant information from internal knowledge sources, and generate grounded draft responses for your proposal team, your approach of using a single Azure AI Foundry Hub with multiple Projects is a good design choice. This allows you to centrally manage governance, security, models, and monitoring while keeping individual use cases organized within separate projects.

For a reliable enterprise implementation, I recommend using Azure AI Search as the knowledge retrieval layer for your RAG workflow, combined with Azure AI Document Intelligence to process and extract content from incoming proposal documents. Azure OpenAI models can then be used to generate grounded responses based on the retrieved content, while Azure AI Foundry Agents can help automate tasks such as requirement analysis, knowledge retrieval, draft generation, and response validation.

For enterprise readiness, consider using Azure Blob Storage or SharePoint as the content repository, Microsoft Entra ID and Managed Identities for secure access control, Azure Key Vault for secret management, and Azure Monitor/Application Insights for operational monitoring and troubleshooting. Depending on your workflow requirements, Azure Functions or Logic Apps can also be used to integrate the solution with existing business processes and approval workflows.

Overall, this combination of services provides a scalable and secure foundation for building an Advanced RAG and Agentic AI solution in Azure AI Foundry.

References:

Quickstart: Set up Microsoft Foundry resources

Azure AI Search for RAG

Azure AI Document Intelligence

Create and deploy an Azure OpenAI in Microsoft Foundry Models

I hope this helps. Please let us know if you have any further questions. Thank you!
Pavan Kumar Purilla 90 Reputation points Microsoft External Staff Moderator

2026-07-23T02:08:47.1033333+00:00

Hello MelDev,
Did you get any chance to review the above response. Do let me know if you have any further queries.

3 answers

Your answer

Pavan Kumar Purilla 90 Reputation points Microsoft External Staff Moderator

2026-07-23T02:08:47.1033333+00:00

Hello MelDev,
Did you get any chance to review the above response. Do let me know if you have any further queries.

Answer 1

Thank you for the detailed response, the service list is very helpful, and I agree with most of it (AI Search for hybrid retrieval, Document Intelligence, Managed Identities, etc.).

However, I'd like to challenge the core architectural recommendation. You've endorsed the hub-based (Hub + Projects) model, but according to Microsoft's current documentation, hub-based projects are now the classic/legacy path:

Microsoft's guidance (https://learn.microsoft.com/azure/ai-foundry/what-is-azure-ai-foundry) now recommends Foundry projects (hosted directly on a Foundry resource, no hub) as the default for generative AI apps and agents.

Could you please clarify in which scenarios we need to use Hub-based and when to use Foundry projects hosted directly on a Foundry resource?

New agent and model capabilities (Foundry Agent Service features, newer SDKs, model catalog additions) are shipping on the Foundry resource type, not on hub-based projects. Building a new enterprise platform on the resource type that no longer receives new agent features seems hard to justify for a solution whose core is agents.

Given that, could you clarify:

Is there a specific technical reason you'd still choose a Hub for this use case in 2026?
Are there capabilities in my scenario (advanced RAG, Agent Service, Document Intelligence, Private Link) that you believe require a hub-based project?
If not, would you agree the recommendation should be updated to a Foundry resource with multiple Foundry projects?

I want to make sure we're not building a new platform on an architecture that's already in maintenance mode , migration later (recreating agents, connections, and vector stores) would be costly. Appreciate your thoughts.

Pavan Kumar Purilla 90 Reputation points Microsoft External Staff Moderator

2026-07-24T11:00:31.8533333+00:00

Hello MelDev,
Thank you for your follow-up and for sharing the latest Microsoft guidance.

You are correct that Microsoft now recommends Azure AI Foundry resources with Foundry projects for building new generative AI and agent-based applications.

My earlier recommendation was based on the traditional Hub + Projects architecture, which is still supported and is commonly used by organizations with existing deployments. However, for a new solution like your proposal response platform, using Azure AI Foundry resources with Foundry projects aligns with Microsoft's current guidance.

Based on the requirements you've described, including Advanced RAG, Azure AI Search, Agent Service, Document Intelligence, Private Link, and enterprise security, there is no specific requirement to use a Hub-based project. These capabilities can be implemented using Foundry projects hosted directly on an Azure AI Foundry resource.

Therefore, for a new deployment, using an Azure AI Foundry resource with multiple Foundry projects would align with Microsoft's current guidance and allow you to take advantage of the latest platform capabilities and future enhancements.

Answer 2

Deleted

This answer has been deleted due to a violation of our Code of Conduct. The answer was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.

Comments have been turned off. Learn more

Answer 3

Jose Benjamin Solis Nolasco 10,651 Volunteer Moderator

Welcome to Microsoft Q&A

Hello @MelDev , I hope you are doing well.

For an enterprise-grade Advanced RAG and agentic workflow in Azure AI Foundry, your proposed architecture (one Hub with multiple Projects for different use cases) is a good approach. It provides logical separation while allowing centralized governance and shared resources.

For this scenario, a typical architecture includes in others organizations:

Azure AI Foundry Hub & Projects to organize AI resources, models, connections, and environments for different business scenarios.
Azure AI Search as the retrieval layer with vector search and hybrid search to index your internal proposal documents and knowledge base.

Azure Blob Storage (or SharePoint/OneDrive via ingestion pipelines) as the primary document repository.

Azure AI Document Intelligence to extract structured content from PDFs, Word documents, and scanned documents before indexing.

Azure OpenAI models (for example, GPT models) for grounded response generation using retrieved context.

Azure AI Foundry Agents (or agent orchestration) to coordinate tasks such as document retrieval, proposal drafting, validation, and citation generation.

Azure Logic Apps, Azure Functions, or Azure Durable Functions to orchestrate business workflows, approvals, and integrations with downstream systems.

Microsoft Entra ID together with Managed Identities for secure authentication between Azure services.

Azure Key Vault to securely store secrets, API keys, and connection strings.

Azure Monitor and Application Insights to monitor agent executions, latency, failures, and usage.

Azure AI Content Safety (where appropriate) to help filter harmful inputs and outputs before presenting responses to users.

References

Azure AI Foundry documentation: https://learn.microsoft.com/azure/ai-foundry/

Azure AI Search (RAG): https://learn.microsoft.com/azure/search/retrieval-augmented-generation-overview

Azure AI Foundry Agents: https://learn.microsoft.com/azure/ai-foundry/agents/

Azure AI Document Intelligence: https://learn.microsoft.com/azure/ai-services/document-intelligence/

Azure OpenAI on Azure AI Foundry: https://learn.microsoft.com/azure/ai-foundry/model-inference/

If my answer helped you, please consider marking it as accepted. This can help other community members facing a similar scenario

MelDev 0 Reputation points

2026-07-23T08:19:59.9866667+00:00
Thank you for the detailed response — the service list is helpful, and I agree with most of it (AI Search for hybrid retrieval, Document Intelligence, Managed Identities, etc.).

However, I'd like to challenge the core architectural recommendation. You've endorsed the hub-based (Hub + Projects) model, but according to Microsoft's current documentation, hub-based projects are now the classic/legacy path:

Microsoft's guidance (https://learn.microsoft.com/azure/ai-foundry/what-is-azure-ai-foundry) now recommends Foundry projects (hosted directly on a Foundry resource, no hub) as the default for generative AI apps and agents. Hub-based projects are positioned mainly for Azure ML scenarios, training, fine-tuning, custom model hosting, none of which apply to a RAG + agent workload like this one.

New agent and model capabilities (Foundry Agent Service features, newer SDKs, model catalog additions) are shipping on the Foundry resource type, not on hub-based projects. Building a new enterprise platform on the resource type that no longer receives new agent features seems hard to justify for a solution whose core is agents.

The new model also consolidates RBAC, Azure Policy, and networking under a single resource provider, which simplifies exactly the enterprise governance concerns you raise elsewhere in your answer.

Given that, could you clarify:

Is there a specific technical reason you'd still choose a Hub for this use case in 2026?

Are there capabilities in my scenario (advanced RAG, Agent Service, Document Intelligence, Private Link) that you believe require a hub-based project?

If not, would you agree the recommendation should be updated to a Foundry resource with multiple Foundry projects?

I want to make sure we're not building a new platform on an architecture that's already in maintenance mode , migration later (recreating agents, connections, and vector stores) would be costly. Appreciate your thoughts.
Jose Benjamin Solis Nolasco 10,651 Reputation points Volunteer Moderator

2026-07-29T13:25:01.71+00:00

Hello @MelDev , Thank you for raising this point. You're correct that the current Azure AI Foundry documentation recommends Foundry resources with Foundry projects as the default architecture for new generative AI applications and Agent Service scenarios.

For your use case, which includes Advanced RAG, Agent Service, Document Intelligence, Azure AI Search, and Private Link, I don't see a requirement that specifically necessitates a Hub-based project. If you're building a new solution in 2026, Foundry resources with multiple Foundry projects would generally align better with the current platform direction and provide access to the latest agent capabilities as they become available.

Hub-based projects continue to be appropriate for scenarios that rely on Azure Machine Learning capabilities, such as model training, fine-tuning, custom model management, or other Azure ML-centric workflows. However, for a solution centered on RAG and agents, the newer Foundry resource model is the recommended approach unless you have a dependency on those Azure ML features.

References:

What is Azure AI Foundry?

Azure AI Foundry Agent Service

If my answer helped you, please consider marking it as accepted. This helps others in the community find similar solutions.