Issues with knowledge bases or knowledge sources with a Foundry agent

Question

Hi there,

I’m testing Microsoft Foundry Agent Service with the Azure AI Search tool against an existing Azure AI Search index. Agent creation works, but the response fails exactly when the Azure AI Search tool is invoked.

The runtime error is : openai.APIError: Invalid endpoint or connection failed

What I’ve already verified:

• Foundry project authentication works
• the Azure AI Search connection resolves correctly in the SDK
• the agent is created successfully
• the response starts and reaches early streaming events
• the failure happens only when the Search tool is actually used

I tested both:

• a keyless/project-managed-identity Azure AI Search connection
• an API-key Azure AI Search connection

Both fail with the same error.

I also verified:

• Azure AI Search networking is public
• firewall allows all networks
• auth mode allows both key and RBAC access
• the index exists and has:
  • a searchable/retrievable text field
    • a vector field
      • a retrievable URL field for citations

So at this point it does not look like:

• a Python SDK issue
• a basic Foundry auth issue
• an obvious RBAC issue
• an index-name typo
• a managed identity only issue

It looks more like either:

• a Foundry Azure AI Search tool/runtime issue
• a hidden compatibility requirement for the target index schema
• a bad/stale connection object even though it resolves correctly

Has anyone seen this specific behavior where:

• agent creation succeeds
• response starts
• Azure AI Search tool fails only at execution time

and the same happens with both MI and API-key connections?

Thank you for your help !

Answer 1

Hello Emmanuel Jackson,

Welcome to Microsoft Q&A and Thanks again for the detailed troubleshooting,

Based on everything you’ve shared, this does not look like a client-side or basic configuration issue. The fact that:

• Agent creation succeeds
• Streaming starts normally
• The failure happens only when the Azure AI Search tool is invoked
• And it reproduces with both Managed Identity and API key authentication

strongly points to either a runtime-level issue in the Foundry Agent → Azure AI Search integration, or a connection/endpoint resolution issue that only surfaces at execution time.

This pattern is typically seen in a few specific scenarios:

1. Endpoint resolution issue at runtime

Even if the connection validates during setup, the agent runtime may:

• Resolve an incorrect or malformed endpoint
• Fail to reach the Search service internally

This results in:

openai.APIError: Invalid endpoint or connection failed

2. Connection object inconsistency

Sometimes the connection Looks valid in the SDK But is stale or not fully propagated to the runtime

This explains why Creation works, Execution fails

3. Schema / tool compatibility edge case

Even with a valid index, the agent runtime expects:

A clearly defined searchable text field (primary content)

Proper separation of text vs vector fields

Fields explicitly marked as:

• searchable = true
• retrievable = true

If mapping is ambiguous, failure can occur only at execution time.

4. Backend limitation

Given Same failure across multiple resources, Same behavior with both auth types

This may indicate a service-side issue in Foundry Agent’s Search tool execution layer.

Additional checks

Please go through the following:

1. Confirm Search endpoint format

Make sure your connection uses exactly:

https://<your-search-name>.search.windows.net

• No trailing /
• No /indexes/...
• Correct service name and region

If using a Foundry knowledge base, ensure search_service_endpoint matches this exactly

2. Validate authentication and roles

If using Managed Identity:

Go to Azure Portal → Search resource → Access Control (IAM)

Assign Search Index Data Reader

If using API Key:

• Go to Keys tab in Search resource
• Ensure “API Key” or “Both” is enabled
• Re-copy the key and update the Foundry connection

3. Networking and TLS checks

Even if Azure shows “Allow all networks”:

• Corporate firewall / proxy may interfere
• TLS inspection can break connectivity

Ensure *.search.windows.net is allowlisted and No SSL interception is happening

4. Recreate the connection

• Delete the existing Azure AI Search connection
• Recreate it fresh
• Rebind it to the agent

This resolves many hidden/stale config issues.

5. Test connectivity outside the agent

Try a direct call using the same auth:

• List indexes
• Run a simple query

If this fails → issue is Search / network layer If this works → issue is Foundry runtime layer

6. Minimal index test

Create a simple test index with One text field (content) and One vector field

Ensure both are Searchable and Retrievable

This helps isolate schema-related issues.

7. Region alignment

Confirm Foundry project region, Azure AI Search region are compatible. Cross-region calls can fail at runtime.

8. Streaming vs non-streaming behavior

Since you’re using streaming Try a non-streaming call via SDK Or increase timeout

In rare cases Streaming sessions fail under longer execution paths

Please refer this

Connect a Foundry IQ knowledge base to Foundry Agent Service (Endpoint & 400/404 errors):

https://learn.microsoft.com/en-us/azure/foundry/agents/how-to/foundry-iq-connect?wt.mc_id=knowledgesearch_inproduct_azure-cxp-community-insider&tabs=foundry%2Cpython#troubleshooting

Foundry IQ FAQ (Auth methods & permission roles): https://learn.microsoft.com/en-us/azure/foundry/agents/concepts/foundry-iq-faq?wt.mc_id=knowledgesearch_inproduct_azure-cxp-community-insider

I Hope this helps. Do let me know if you have any further queries.

---

If this answers your query, please do click Accept Answer and Yes for was this answer helpful.

Thank you!

Answer 2

Based on the described symptoms, focus on the Azure AI Search tool connection and the known limitations and troubleshooting items for Foundry agents with Azure AI Search:

1. Verify the connection endpoint and index name
   • Confirm the project connection is pointing to the correct Azure AI Search service endpoint (for example, https://<name>.search.windows.net) and not another URL.
   • Confirm the index name in the tool configuration exactly matches the index name in Azure AI Search, including case sensitivity. A mismatch can cause runtime failures when the tool executes, even if the agent is created successfully.
2. Re-check authentication mode vs. networking
   • Since networking is public and both key and RBAC are allowed, either keyless (managed identity) or API key should be valid in principle.
   • However, if at any point the service is moved behind a private virtual network or public access is disabled, key-based authentication will fail at runtime. In that case, the connection must use Microsoft Entra project managed identity (keyless authentication) instead of an API key.
   • Ensure the current connection configuration matches the actual networking setup of the Azure AI Search resource.
3. Validate RBAC roles for keyless (managed identity) connections
   • For keyless authentication, ensure the Foundry project’s managed identity has the required roles on the Azure AI Search resource:
     • Search Index Data Contributor and Search Service Contributor for the Azure AI Search tool.
   • Missing these roles can cause 401/403 errors when the tool executes, which can surface as a generic connection or endpoint failure.
4. Check for hidden connection issues
   • Even if the connection object resolves in the SDK, it can still be misconfigured for runtime use. Recreate the Azure AI Search project connection using the supported commands/SDKs for Foundry projects (Microsoft.CognitiveServices), not the older ML workspace tooling:
     • Use az cognitiveservices account project connection create or the azure-mgmt-cognitiveservices SDK rather than az ml / azure-ai-ml, which target Microsoft.MachineLearningServices and do not support new Foundry projects.
   • A connection created with the wrong resource provider can appear to exist but fail when the tool actually calls Azure AI Search.
5. Confirm the tool can actually access the index
   • Use Azure AI Search’s test query feature directly on the index to confirm that:
     • The index is queryable.
     • The fields used by the tool (searchable text, vector field, URL field) are present and retrievable.
   • If Azure AI Search itself returns no results or errors, fix those issues first.
6. Check for streaming/citation behavior
   • Since the response starts and then fails when the tool runs, verify that:
     • The agent instructions request citations only if the index has a retrievable URL field and the tool is configured to return citations.
     • If streaming is used, confirm that url_citation annotations are expected and that the client code can handle them. While this usually affects citation display rather than connectivity, mis-handling streaming annotations can surface as runtime errors on the client side.

If, after confirming the endpoint, index name, roles, and connection creation method, the error persists identically for both keyless and API-key connections, it aligns with a potential Foundry Azure AI Search tool/runtime issue. In that case, capture:

• The exact connection definition (redacting secrets),
• The Azure AI Search service name and region,
• The time and correlation IDs of failing calls,

and open a support ticket so the product team can inspect the backend tool invocation.

---

References:

• Connect an Azure AI Search index to Foundry agents (python)
• Connect an Azure AI Search index to Foundry agents (csharp)
• Connect an Azure AI Search index to Foundry agents (typescript)
• Connect a Foundry IQ knowledge base to Foundry Agent Service