Microsoft Copilot Agent - struggle with hallucination

Question

Hi Community

I'm working on a Copilot Agent for our HR department and the chat bor consist o 30 documents and are only allowed to browse these for knowledge.

But when you for ex ask it about insurance companies it mentions old companies we dont cooperate with anymore, and they are not mentioned in the documents I have attached.

These files are attached through SharePoint and I suspect the version history to be the issue for the hallucination - can anyone approve it, or could the issue be elsewhere?

Answer 1

Hi John,

Copilot Studio does not read or use SharePoint version history, so older file versions are not causing the hallucinations you’re seeing. When the agent brings up outdated insurance companies or information not present in your documents, it usually happens because the model is filling gaps or generalizing beyond the approved HR knowledge set. This occurs when the content is not explicit enough, when instructions are too open-ended, or when fallback behavior allows the model to rely on general world knowledge.

To prevent this, make sure your HR documents contain precise, up-to-date information and clearly state which vendors or policies are approved. In Copilot Studio, enable “Restrict responses to knowledge,” tighten your system instructions, and turn off any fallback behaviors that let the agent answer outside the SharePoint knowledge base. If needed, break large documents into smaller, well-structured files and add explicit guardrail instructions telling the agent not to reference companies or details that are not part of your knowledge sources.

Relevant Official Documentation

1. Restrict responses to knowledge sources
2. Optimize knowledge content for accuracy
3. How Copilot uses knowledge sources

Thank you.

Karan Shewale.

If this response resolves your issue, please Accept the answer and, if helpful, click the “Upvote” button. Your feedback helps us improve and assist others more effectively.