Copilot Studio Agent Works in Test but Not After Publishing

Question

Issue Summary

I am using Copilot Studio with a standard corporate account and have developed an AI agent integrated with SharePoint and Microsoft Teams.

The agent is configured with:

• SharePoint knowledge sources
• Topics created directly in Copilot Studio
• Quick replies and guided conversation flows
• Additional knowledge configured directly within the agent

In the Agent Test environment, everything works as expected:

• Topics trigger correctly.
• Quick replies are displayed correctly.
• Knowledge sources return the expected answers.
• The configured conversation flow works properly.

However, after publishing:

• Quick replies do not appear.
• Configured topics are not triggered as expected.
• The agent generates answers that were never configured or trained.
• In some cases, Copilot does not generate a response at all.
• The published behavior differs significantly from the Agent Test experience.

Because the issue affects not only SharePoint knowledge but also topics and knowledge configured directly within the agent, it does not appear to be solely related to SharePoint permissions or authentication.

Questions

1. Is this a known issue related to publishing, runtime synchronization, or channel deployment?
2. How can I verify that the latest published version is actually being executed in Copilot and Teams?
3. Are there any logs, diagnostics, or telemetry tools available to determine why the published agent behaves differently from the Agent Test environment?
4. Could you advise how to verify whether the published agent is using general AI knowledge instead of the configured knowledge sources and topics?
5. Are there any known issues where published agents ignore configured topics, quick replies, or knowledge sources and instead generate responses based on general AI knowledge?
6. What additional troubleshooting steps would you recommend to identify the root cause of this behavior? Issue Summary I am using Copilot Studio with a standard corporate account and have developed an AI agent integrated with SharePoint and Microsoft Teams. The agent is configured with:
   • SharePoint knowledge sources
   • Topics created directly in Copilot Studio
   • Quick replies and guided conversation flows
   • Additional knowledge configured directly within the agent
   In the Agent Test environment, everything works as expected:
   • Topics trigger correctly.
   • Quick replies are displayed correctly.
   • Knowledge sources return the expected answers.
   • The configured conversation flow works properly.
   However, after publishing:
   • Quick replies do not appear.
   • Configured topics are not triggered as expected.
   • The agent generates answers that were never configured or trained.
   • In some cases, Copilot does not generate a response at all.
   • The published behavior differs significantly from the Agent Test experience.
   Because the issue affects not only SharePoint knowledge but also topics and knowledge configured directly within the agent, it does not appear to be solely related to SharePoint permissions or authentication. Questions
   1. Is this a known issue related to publishing, runtime synchronization, or channel deployment?
   2. How can I verify that the latest published version is actually being executed in Copilot and Teams?
   3. Are there any logs, diagnostics, or telemetry tools available to determine why the published agent behaves differently from the Agent Test environment?
   4. Could you advise how to verify whether the published agent is using general AI knowledge instead of the configured knowledge sources and topics?
   5. Are there any known issues where published agents ignore configured topics, quick replies, or knowledge sources and instead generate responses based on general AI knowledge?
   6. What additional troubleshooting steps would you recommend to identify the root cause of this behavior?

Answer 1

Hi,
Based on your description, since the issue affects topics, quick replies, and knowledge sources after publishing, this does not appear to be related solely to SharePoint permissions or authentication.

As a first step, please verify that you are testing the latest published version of the agent. According to Microsoft documentation, published content may take some time to propagate across channels, and existing Teams conversations may continue using older content until a new session is started. You can try starting a new conversation or entering "start over" in the conversation to ensure the latest published version is loaded.

Reference: Key concepts - Publish and deploy your agent - Microsoft Copilot Studio | Microsoft Learn

Could you also confirm the following?

1/ Does the issue occur only in Microsoft Teams, or across all published channels?

2/ After republishing, have you tested using a completely new chat session?

3/ Have you reviewed the conversation history and analytics in Copilot Studio to determine whether the published conversations are matching the expected topics?

4/ If you create a simple test topic with a unique trigger phrase, does the published agent invoke that topic correctly?

If the behavior persists after these checks, the issue may require investigation of backend publishing, synchronization, or runtime telemetry that is not accessible through the Microsoft Community. In that case, I recommend contacting your Microsoft 365 administrator to open a support request with Microsoft Support so they can review the agent's diagnostic data and service-side logs.
Hope this will helps.

Answer 2

1. Yes, there are known issues where the published agent or channel doesn’t immediately use the latest configuration. In particular, for agents published to Microsoft Teams, a known behavior is that Teams can keep using a previously published version of the agent, which can cause unexpected behavior or generic errors at runtime. For example, when Teams doesn’t refresh the updated agent configuration after a publish/republish, users can see a generic SystemError during execution. This is explicitly documented as a configuration refresh issue on the Teams channel, and the same underlying behavior (old configuration being cached) can explain why topics, quick replies, or knowledge sources appear to be ignored after publishing.
2. To verify that the latest published version is being executed in Teams, use the documented refresh steps for the Teams channel:
   • In the Microsoft Teams admin center, locate the agent app, disable it, wait a few seconds, and then re-enable it.
   • In Copilot Studio, turn off the Teams channel for the agent, save the changes, and then turn the channel back on.
   • Republish the agent.
   After these steps, test again in Teams. If the behavior now matches the Test your agent experience, this confirms that the issue was due to Teams using an older cached configuration.
3. For logs, diagnostics, or telemetry to understand differences between Test and published behavior:
   • Use the Publish page in Copilot Studio to review publish status and any error details when publishing. Error codes and messages here help identify configuration or dependency issues that might affect runtime behavior.
   • For Teams-specific runtime issues (for example, SystemError), use the Teams admin center to manage and refresh the app as described above, and capture the Conversation ID, Agent ID, and time of the error. These are the key diagnostics to provide if escalation to Microsoft Support is required.
   • For agents used with Dynamics 365 Customer Service (if applicable), the Copilot Service admin center exposes configuration and publishing status for knowledge sources and plugins, which can help verify that the correct knowledge configuration is active.
4. To verify whether the published agent is using general AI knowledge instead of configured knowledge sources and topics:
   • For SharePoint-based generative answers, confirm that the Create generative answers node is correctly configured to point to the intended SharePoint locations and that the content is searchable. If SharePoint search returns no results, the generative answers node won’t provide a response and the agent may fall back to generic behavior.
   • Ensure that only supported document formats and modern SharePoint pages are used, and that content has been indexed by SharePoint search.
   • In scenarios where Dynamics 365 Customer Service is using Copilot Studio knowledge sources, verify in the Copilot Service admin center that the Use knowledge sources configured in Copilot Studio (preview) option is enabled, the knowledge sources are added and published, and that the Publish knowledge sources step has completed successfully.
   If, after confirming these, the agent still produces answers clearly unrelated to the configured knowledge or topics, this strongly suggests fallback to general AI behavior or to an older configuration, and the Teams refresh steps plus republish should be applied.
5. Known issues where published agents ignore configured topics, quick replies, or knowledge sources and instead behave differently include:
   • Teams continuing to use a previously published version of the agent after a publish/republish, causing a mismatch between the Test your agent behavior and the Teams behavior.
   • Published agents (in other channels such as Microsoft 365 Copilot via Azure AI Foundry) not supporting certain capabilities like streaming responses or citations, which can make responses appear different from the test environment.
   While the documentation doesn’t list a specific bug where topics and quick replies are “ignored,” the documented configuration refresh issue in Teams is a concrete, known cause of published behavior diverging from test behavior.
6. Additional troubleshooting steps to identify the root cause:
   • Confirm publish success and configuration:
     • On the Publish page, verify that the latest publish completed successfully and review any error codes or messages.
     • Ensure all required components (topics, flows, connectors, data sources, knowledge sources) are present and correctly configured before publishing.
   • Refresh the Teams channel configuration:
     • Disable and re-enable the agent app in the Teams admin center.
     • Turn the Teams channel off and on again in Copilot Studio, then republish.
     • Retest in Teams and compare behavior with Test your agent.
   • Validate SharePoint generative answers:
     • Confirm that the Create generative answers node points to the correct SharePoint site/library and that documents are in supported formats and indexed.
     • If no search results are returned from SharePoint, adjust content or search settings so that relevant items appear in search.
   • Validate knowledge sources for Dynamics 365 Customer Service (if used):
     • In Copilot Service admin center, ensure Use knowledge sources configured in Copilot Studio (preview) is enabled.
     • In Copilot Studio, verify that the intended knowledge sources (SharePoint, public website, file upload, Dataverse, enterprise data) are added and then published via the Publish knowledge sources dialog.
   • If the problem persists after these steps:
     • Capture Conversation ID, Agent ID, and timestamps for problematic conversations.
     • Open a support case with Microsoft Support, providing these identifiers and a description of the difference between Test and published behavior.

---

References:

• Key concepts - Publish and deploy your agent
• Connect and configure an agent for Teams and Microsoft 365
• Generative answers pointing to SharePoint sources don't return results in Copilot Studio
• Enable knowledge sources from Microsoft Copilot Studio (preview)
• Design a testing strategy for your agents