Foundry Agent Service + Knowledge Base: correlating citations back to source chunks — anyone found a reliable mapping?

Question

With Azure OpenAI On Your Data being retired and Microsoft pushing customers toward Foundry IQ / Knowledge Bases, I'm evaluating a migration path based on:

• Azure AI Foundry SDK for C# (Azure.AI.Projects) (latest stable)

Foundry Agent Service

• A Foundry IQ Knowledge Base connected through the knowledge_base_retrieve MCP tool for grounding

What I'm seeing:

The agent's response includes citation markers like this:

【6:0†source】 ← the answer synthesis text, containing inline references like [ref_id:0], [ref_id:1], [ref_id:2]
【6:1†source】 ← internal chunk identifiers, i.e. { "id": "a1b2c3", "document_id": "doc_042" }
【6:2†source】 ← { "id": "d4e5f6", "document_id": "doc_042" }

Where the citation markers are in the format [message_idx:search_idx†source]

What I've worked out so far (and where it breaks):

At first glance it looks like search_idx on each metadata block lines up with the ref_id number cited inline, e.g., 【6:1†source】 corresponds to [ref_id:1], 【6:2†source】 corresponds to [ref_id:2].

But this can't be the whole story: search_idx:0 is already used by the synthesis text block itself, so there's no 【6:0†source】 metadata slot available. Yet the text also cites [ref_id:0]. That leaves a few possibilities I can't distinguish between yet:

• Metadata blocks are offset by +1 from ref_id (since slot 0 is reserved for text), so ref_id:0 → 6:1, ref_id:1 → 6:2, etc. (a simple shift, not a direct match).
• ref_id and search_idx are actually independent numbering schemes (e.g., ref_id indexes across all retrieved documents, while search_idx only numbers the metadata blocks that got included), meaning there's no fixed arithmetic relationship at all.

Something else I haven't considered.

I haven't been able to validate any of this against the full payload yet (still working through a fairly large tool-output blob), and there's zero documentation I can find on any of it.

What I'm looking for:

• Has anyone actually confirmed how ref_id and search_idx relate? Is it a fixed offset, independent numbering, or something else?

Is there an officially supported way to correlate a specific in-text citation to its underlying source chunk? Preferably similar to the On Your Data Citations collection (ChatMessageContext.Citations) which gave you Content, Title, Url, and similar fields directly per citation, rather than reverse-engineering text markers.

Any pointers appreciated!

Answer 1

Hello @Brenda Griggs

Thank you for reaching out and for providing a detailed explanation of your observations.

From your description, I understand that you're trying to determine whether there's a deterministic relationship between the inline citation references (for example, ref_id:0, ref_id:1) and the metadata returned by the knowledge_base_retrieve MCP tool (such as id and document_id), similar to the structured ChatMessageContext.Citations object that was available with Azure OpenAI On Your Data.

After reviewing the current Azure AI Foundry documentation and SDK behavior, there is currently no officially documented or supported mapping that explains how the following elements relate to one another:

• Inline citation markers (for example, 【message_idx:search_idx†source】)
• Inline references such as ref_id:0, ref_id:1
• The retrieved source metadata (id, document_id, and other chunk information)

Although these values may appear to correlate in some responses, Microsoft has not published a contract that defines a deterministic relationship between them. Therefore, assumptions such as:

• ref_id corresponding directly to search_idx
• A fixed offset (for example, ref_id + 1 = search_idx)
• Any other arithmetic relationship

should not be relied upon, as the numbering appears to be an implementation detail rather than a supported API contract and may change in future service updates.

From the current documentation, what is documented is the overall retrieval pipeline used by Foundry Agent Service when working with a Knowledge Base. The service:

• Plans and decomposes the user query.
• Retrieves relevant content using keyword, vector, or hybrid search.
• Applies reranking to the retrieved results.
• Synthesizes the final response with source references.

However, the documentation does not describe how the internal numbering (ref_id, search_idx, or message_idx) is generated, nor does it document an equivalent to the ChatMessageContext.Citations collection that exposes structured citation metadata for each individual citation.

This differs from the Azure OpenAI On Your Data experience, where applications could directly consume structured citation information such as content, title, filepath, URL, and other metadata without having to interpret inline markers.

If your application requires deterministic traceability between an answer and the exact source chunk that was retrieved, we recommend relying on the structured output returned by the knowledge_base_retrieve tool rather than attempting to parse or infer relationships from the inline citation markers. The retrieved documents include identifiers such as document_id and chunk id, which are intended for programmatic processing, whereas the inline citation markers are primarily designed for presentation within the generated response.

Since the citation numbering itself isn't currently documented, the recommended approach for understanding the retrieval and grounding process is to use Azure AI Foundry Tracing.

By enabling tracing, you can inspect the complete execution flow, including:

• Query planning
• Tool invocations
• Retrieval operations
• Intermediate processing steps
• Final answer synthesis

While tracing does not currently provide a direct mapping between each inline citation marker and its corresponding source chunk, it is the supported mechanism for understanding the retrieval pipeline and diagnosing grounding behavior end-to-end.

Please refer this

ChatMessageContext.Citations Property

Set up tracing in Microsoft Foundry

Agent tracing overview

https://learn.microsoft.com/azure/ai-foundry/observability/how-to/trace-agent-setup?view=foundry https://learn.microsoft.com/azure/ai-foundry/observability/how-to/trace-agent-framework?view=foundry

Connect a Foundry IQ knowledge base to Foundry Agent Service (MCP pipeline, source references, API version note): https://learn.microsoft.com/azure/foundry/agents/how-to/foundry-iq-connect?wt.mc_id=knowledgesearch_inproduct_azure-cxp-community-insider

(Classic example of structured citations as message annotations) Grounding with Bing Custom Search (preview) (classic): https://learn.microsoft.com/azure/foundry-classic/agents/how-to/tools-classic/bing-custom-search?wt.mc_id=knowledgesearch_inproduct_azure-cxp-community-insider#understand-citations-in-the-agent-response

I Hope this helps. Do let me know if you have any further queries.

---

If this answers your query, please do click Accept Answer and Yes for was this answer helpful.

Thank you!

Answer 2

Hi Brenda Griggs,

Thanks for the detailed write-up this is a sharp observation, and the confusion here is understandable because you're actually looking at two independent numbering systems, not one.

1. ref_id and search_idx are not directly related (no fixed offset)

The ref_id values you see inline ([ref_id:0], [ref_id:1], etc.) come from the knowledge base's own grounding payload the JSON-encoded string returned in the retrieve action's response (content.text), where each chunk is serialized with its own ref_id field alongside title, terms, and content. This numbering is generated by the knowledge base itself, based on the order/rank of retrieved chunks.

The 【message_idx:search_idx†source】 markers, by contrast, come from the underlying model/Assistants-style response format (the same annotation convention used across Foundry Agent Service tool integrations, including Azure AI Search grounding). search_idx there just indexes annotation slots within that specific message it's a rendering artifact of how the model API structures citations, not something the knowledge base coordinates with ref_id. That's why you correctly ruled out a simple 1:1 match, and why a fixed "+1 offset" theory won't hold up reliably either it may look consistent in one run and break in the next, since the two systems don't share a common source of truth.

2. There's no way to correlate them via arithmetic, but there is a supported way to get chunk-level mapping

Rather than reverse-engineering the marker text, use the references array from the retrieve response (set includeReferences: true / includeReferenceSourceData: true on your knowledgeSourceParams). Each entry has:

• id — the actual reference ID to use for citation purposes (this is the one meant for correlating citations to chunks, not search_idx)
• activitySource — cross-references back to the activity array entry that produced it, useful for citation linking
• sourceData — the title/terms/content fields when includeReferenceSourceData is on
• docKey — the underlying document key (not the same as id)

This is the closest equivalent to the old ChatMessageContent.Citations model you're used to from On Your Data it gives you a stable, documented id → content mapping instead of parsing marker strings.

3. Practical recommendation

If you need reliable citation-to-chunk correlation in production, treat ref_id (from the grounding JSON) as your authoritative key, and match it against the references array's id field for that same request both are scoped to the retrieve/agent call and are documented as intended for citation purposes. Avoid relying on search_idx from the 【】 markers for this purpose; it's not documented as a stable identifier and appears to be a byproduct of the model's own response formatting rather than something the knowledge base guarantees alignment with.

Hope this helps clarify the two systems. If you find this answer is helpful kindly accept the answer.

Best Reagrds,

Jerald Felix