How to prevent Azure AI Foundry agent in voicelive mode not to say out citations?

Question

Hi!
I'm using an agent with Bing Grounding, Ai Azure Search and an endpoint DB call, but it always puts the citations stuff after the generated text, resulting in the agent reading out the URLs, or vector references.
Those in the "Knowledge" part e.g the grounding with bing and AI search put these marks in the final text, but the DB doesn't. Is it possible to bypass this? Prompting doesn't work, it is like appended after the final LLM response has been generated.

Answer 1

Hi ,

Thanks for reaching out to Microsoft Q&A.

 In Azure AI Foundry, you cannot remove grounding citations just by prompting because they are appended after the LLM finishes its answer. Bing and Azure Search add them as part of the retrieval pipeline. To stop them being spoken in voice mode, you must either disable citation output in the grounding component, or intercept the response and send only the “answer text” to text-to-speech. The correct fix is to drop the citation fields programmatically before TTS, or filter them out with a simple regex. Prompting alone will never prevent them, because the grounding step enforces citations after generation.

Please 'Upvote'(Thumbs-up) and 'Accept' as answer if the reply was helpful. This will be benefitting other community members who face the same issue.

Answer 2

Hi Farsang Bence

It looks like you’re trying to figure out how to stop your Azure AI Foundry agent from reading out citations like URLs or vector references after the generated text. This is definitely a common scenario when working with AI agents.

Unfortunately, at the moment, there isn't a straightforward option or prompt that allows you to suppress those citation details if they are being automatically appended after the agent's response. However, here are a few things you can consider trying:

1. Check Response Formatting: Make sure that your responses are formatted correctly in the response structure from your source, especially when pulling in knowledge from Bing or Azure Search.
2. Manipulate the Response: If your agent's logic allows it, you might manipulate the response after it is generated but before it's read out. This might include stripping out certain elements like citations if applicable.
3. Review Grounding Settings: Look into how your grounding with Bing and Azure Search is set up, as the citation appendage seems to originate from there.
4. Utilize Custom Logic: If you're managing the responses through code, consider implementing additional logic to handle or filter out outputs that contain citations.

   Reference List

• Cognitive Speech Official Documentation
• Azure Speech Services Documentation
• Quickstart: Create a voice live real-time voice agent with Microsoft Foundry Agent Service (Python)
• Voice live API quickstart

I Hope this helps. Do let me know if you have any further queries.

Thankyou!