Foundry IQ

Foundry IQ grounds Agent Framework agents with content from a search index. In Python, AzureAISearchContextProvider supports semantic and agentic retrieval. In .NET, connect a Foundry IQ client to TextSearchProvider.

This integration uses the RAG pattern: it retrieves relevant external content before model invocation without treating that content as conversational memory.

Connect Foundry IQ to TextSearchProvider

Create a SearchClient, map search hits to TextSearchProvider.TextSearchResult, and attach the provider through AIContextProviders.

The sample hosts the resulting agent in Foundry, but the search adapter works with a regular ChatClientAgent.

Install the packages

pip install agent-framework-azure-ai-search agent-framework-foundry --pre

Use semantic retrieval

Semantic mode performs search against an existing index and can combine keyword and vector retrieval.

Use agentic retrieval

Agentic mode uses a Foundry IQ Knowledge Base for query planning and multi-hop retrieval.

Some agentic output and reasoning options require the preview azure-search-documents package.

With retrieval_reasoning_effort="low" or "medium", agentic retrieval accepts image-only user and assistant messages that contain supported image data or image URI content. These messages participate in agentic_message_history_count after unsupported content is filtered. Semantic retrieval and agentic "minimal" reasoning remain text-only. System messages, non-image attachments, whitespace-only text, and empty messages don't trigger retrieval.

Configure per-source retrieval

Pass knowledge_source_params to AzureAISearchContextProvider in agentic mode when a knowledge source needs options such as filter_add_on. Each entry replaces the provider-generated parameters for the matching knowledge_source_name; entries for other source names are appended.

When you don't set include_reference_source_data, the provider sets it to True on a copy so references retain their source data. An explicit False is preserved. Duplicate knowledge_source_name values raise ValueError, and knowledge_source_params isn't accepted in semantic mode.

Forward caller identity for permission-aware retrieval

Set query_source_credential=caller_credential when a knowledge source uses document-level permissions and retrieval results must be security trimmed for each caller. Pass the signed-in caller's sync or async Azure token credential separately from the application credential that connects to Foundry IQ.

For each agentic retrieval, the provider requests a token for https://search.azure.com/.default. It forwards the caller's Microsoft Entra identity in the x-ms-query-source-authorization header so Foundry IQ can enforce the indexed permissions.

This option requires a preview azure-search-documents build, version 12.1.0b1 or later within the supported 12.x range:

pip install --pre "azure-search-documents>=12.1.0b1,<13"

Agent Framework fails closed if the installed SDK doesn't support query-source authorization. It raises a ValueError before sending a retrieval request. Token acquisition failures also stop retrieval; the provider doesn't retry without the caller's identity.

Go provides a Foundry IQ vector-store connector through provider/azureprovider. Create it with azureprovider.NewAISearch, supply an Azure token credential or API-key credential, and use vectordata.GetCollection for typed records.

The connector supports vector and keyword/vector hybrid search, portable filters, and generated embeddings. Use embedding.Bind to provide a Foundry or OpenAI embedding generator. For RAG, adapt the search results to textsearch.Result and register an agent/textsearch context provider.

The runnable Foundry IQ sample requires AZURE_AI_SEARCH_ENDPOINT, FOUNDRY_PROJECT_ENDPOINT, and FOUNDRY_EMBEDDING_MODEL, plus an Azure credential. Its schema expects 3,072-dimensional embeddings. The sample creates and deletes a uniquely named temporary index, so the credential needs permission to manage indexes and documents.

String queries without a configured embedding generator require an existing index with a Foundry IQ vectorizer. The Go connector doesn't provision vectorizers or implement the Python agentic-retrieval context provider shown on this page.

See Vector store integrations and the Go connector guide for configuration and limitations.

Production considerations

  • Prefer Microsoft Entra authentication or managed identity over search keys.
  • Apply tenant-aware filters and index isolation.
  • Treat retrieved content as untrusted input and mitigate indirect prompt injection.
  • Preserve source metadata when the agent should cite documents.

Next steps