Knowledge base sync shows Completed, checking if I'm missing an explicit attach step for retrieval

Question

I understand the general RAG pattern from other platforms but I'm newer to how Foundry wires a knowledge source into an agent's retrieval path specifically.

My setup:

from azure.ai.projects import AIProjectClient

agent = project_client.agents.create_agent(
    model="gpt-4o",
    name="support-agent",
    instructions="Answer using the connected knowledge base only.",
    tools=[{"type": "file_search"}],
    tool_resources={
        "file_search": {
            "vector_store_ids": [vector_store.id]
        }
    }
)

The portal shows the SharePoint sync as Completed with a normal timestamp. Querying the agent with something directly answerable from the source docs still returns a generic non-grounded answer, or "I don't have that information."

What I checked:

• Confirmed the connected account has explicit read permission on the target folder, not just the site root.
• Reduced the test case to a single plain text file to rule out size/format issues.
• Queried using near exact phrasing copied from the test doc to rule out a semantic mismatch.
• Re-ran the sync manually three times, consistently Completed, no warnings anywhere in the portal.

Given that Completed doesn't seem to guarantee retrieval, is there a required step to explicitly link the vector_store_id to the agent's tool_resources after ingestion, separate from the sync itself? Is there an API call like:

vector_store.file_counts

or similar that would show me actual indexed chunk counts, so I can confirm ingestion produced usable embeddings rather than trusting the sync status label alone?

Answer 1

Hi @Dipendu Singh , thank you for your question!

The most likely gap here is that Completed on the SharePoint sync only confirms the crawl/ingestion job finished, not that it fed into the same vector_store your agent is querying those can be two separate resources under the hood. A SharePoint-connected knowledge source in Foundry typically indexes into a managed Azure AI Search index as part of the sync, which is a different object from an OpenAI-style vector store populated via direct file uploads. If vector_store.id in your code isn't the exact store the SharePoint sync wrote into, file_search will run cleanly against a store with zero relevant (or zero total) chunks and return a non-grounded answer, with nothing surfacing as an error since the tool call itself succeeds.

To confirm this before changing anything else:

• Call client.vector_stores.files.list(vector_store_id=vector_store.id) (or the equivalent in azure-ai-projects) and check the actual file count and per-file status a file that's completed at the sync/portal level can still show in_progress or failed at the embedding level if you look at the vector store's own file status rather than the portal's sync label.
• Separately check vector_store.file_counts (as you mentioned), if completed is 0 and total is 0, the SharePoint sync never actually wrote into this vector store, confirming a mismatch between the sync target and what your agent references.
• If the counts do show indexed files with nonzero completed, the issue is more likely retrieval-side (e.g., file_search chunking/ranking not surfacing the doc for your test query), which would point toward testing with azure_ai_search as the tool type instead if the underlying knowledge source is Search-backed rather than vector-store-backed.

So the explicit "attach" step isn't missing from your code (you are correctly passing vector_store_ids at agent creation), the more likely fix is confirming that ID is actually the one the SharePoint sync populated, since Foundry's SharePoint connector and the Assistants-style file_search vector store aren't guaranteed to be the same object unless explicitly wired together.

Please upvote or accept answer if this is helpful!