Inconsistent Output Behavior in Generative AI Action within Copilot Studio Topic

Question

Hi Team,

I am encountering an issue with output generation in a Copilot Studio Topic that I have developed.

The topic is designed to extract acronyms from a presentation to support my Copilot Studio agent. It analyzes the user query, identifies the presentation name, locates the corresponding PowerPoint file in SharePoint, processes the entire presentation, and extracts all acronyms from each slide. This functionality is implemented using a Generative AI action.

However, I am observing inconsistent behavior during execution. While the response is being generated, the system correctly lists the acronyms along with their respective slide numbers. But once the generation process completes, the final message returned is: "No information was found that could help answer this."

This is confusing because the relevant information is clearly identified and displayed during the generation phase. Additionally, this issue occurs most of the time, although occasionally I do receive the expected output.

I would appreciate any insights into the possible causes of this behavior and recommendations for ensuring more consistent and reliable outcomes.

Thank you for your support.

Regards,
Jahnavi

Answer 1

Hi Kalla, Jahnavi,

This issue in Copilot Studio happens when a Generative AI action correctly extracts data (like acronyms and slide numbers), but the Generative Orchestration layer overrides the result and returns a fallback message like “No information was found that could help answer this.” This usually occurs because the AI action output is not stored as a structured topic output variable, or because the orchestrator re-evaluates and discards the tool result during final response generation. It can also happen when outputs are too large, unstructured, or when topic-level generation and orchestrator generation conflict in the same turn. The recommended fix is to store the AI action result in a structured output variable (preferably JSON), avoid double generation between the topic and orchestrator, keep outputs concise, and let the orchestrator use the captured result directly instead of re-generating the response. In short, the extraction step works correctly, but the final orchestration layer is replacing or ignoring the output during response composition.

References

https://learn.microsoft.com/en-us/microsoft-copilot-studio/generative-orchestration

https://learn.microsoft.com/en-us/microsoft-copilot-studio/faqs-generative-orchestration

Thanks and Regards,

Karan Shewale

Answer 2

The behavior described matches how generative orchestration and generative actions can sometimes conflict with, override, or discard intermediate results when the final response is composed.

Key points from the platform behavior:

1. Generative orchestration can override topic-level generative outputs
   • When generative orchestration is enabled, the orchestrator automatically calls the Universal Search Tool (UST) and composes a final answer.
   • If the topic also uses a Generative AI action (or generative answers node) to produce its own response, both the orchestrator and the topic can generate answers for the same user input.
   • As documented, this can lead to duplicate or overlapping responses, or the orchestrator deciding that “no information was found” even though the topic-level generative action produced content.
   • This explains why acronyms and slide numbers appear during generation (the tool/action is working) but the final message is sometimes replaced with "No information was found that could help answer this."
2. Orchestration depends heavily on topic inputs/outputs
   • In generative orchestration mode, the orchestrator expects topics and actions to expose structured inputs and outputs, not just free-form text.
   • If the Generative AI action returns text but the topic doesn’t clearly expose that result as an output variable for the orchestrator, the planner may treat the tool call as non‑useful and fall back to a generic “no information” message.
   • Poorly specified or missing output parameters can cause the orchestrator to discard the intermediate result when compiling the final answer.
3. Content and size limits can cause the orchestrator to drop results
   • The final OpenAI request includes user input, conversation history, tool outputs, and other context. If this becomes too large, the system can hit TooMuchDataToHandle or related internal limits and trim or ignore some content when forming the final answer.
   • Large slide decks or verbose extracted content can increase the risk that the orchestrator deems the tool output unusable or exceeds internal limits, leading again to a fallback message.
4. Responsible AI and filters can block or alter final responses
   • Payload inspection, classifiers, and content filters run on tool outputs and knowledge content as well as user input.
   • If the extracted text or prompt pattern is flagged (for example, as an indirect attack or other restricted content), the orchestrator can block or alter the final answer, even if the intermediate tool output looked correct.

To improve consistency and reliability for this scenario:

1. Avoid overlapping generative sources for the same turn
   • If generative orchestration is enabled, avoid also using a generative answers node in the same topic for the same user input, as documented under “Mixing generative orchestration with topics that include generative answers nodes”.
   • Either:
     • Let the orchestrator handle knowledge retrieval and summarization, or
     • Use a more deterministic pattern where the topic calls the Generative AI action and then returns a structured output that the orchestrator simply formats.
2. Design clear topic inputs and outputs for the acronym extractor
   • Define explicit input parameters (for example, PresentationName) with clear names and descriptions so the orchestrator reliably passes the correct file name and context.
   • Capture the result of the Generative AI action as a topic output variable (for example, ExtractedAcronyms) instead of only sending a message directly from the topic.
   • Let the orchestrator use that output when composing the final answer, rather than re‑asking the model to “figure out” what to say from scratch.
3. Avoid “double-handling” the tool output in prompts
   • If the Generative AI action already returns a structured or summarized result, don’t re‑embed that result as open‑ended context in another prompt (for example, “The result of the action says {summary}”).
   • Pass the result as a structured output and let the orchestrator include it directly. This reduces the chance that the final LLM call ignores or overwrites the tool output.
4. Keep tool outputs concise and scoped
   • If the extracted acronym list is very long, consider:
     • Limiting to unique acronyms and their slide numbers.
     • Truncating or batching very large decks.
   • This reduces the risk of hitting request-size limits or having the planner decide the content is too unwieldy to use.
5. Use orchestration controls to manage the final response
   • Use the “A plan completes” topic trigger and Response formatting settings (Settings → Generative AI → Responses → Response formatting) to control whether the orchestrator’s final response is sent as-is or whether the topic should take over final messaging.
   • This can prevent the orchestrator from replacing a good topic-level result with a generic fallback.
6. Monitor for throttling and rate limits
   • If the agent intermittently hits GenAISearchandSummarizeRateLimitReached, GenAIToolPlannerRateLimitReached, or OpenAIRateLimitReached, the orchestrator may fall back to generic messages.
   • Check for these error codes in the activity map and adjust usage patterns or capacity if needed.

In practice, the most effective pattern for this acronym-extraction topic is:

1. Topic receives the user query and presentation name as an input parameter.
2. Topic calls a deterministic action (connector/flow) to locate the PowerPoint in SharePoint.
3. Topic calls a Generative AI action to extract acronyms and slide numbers, then stores the result in a structured output variable.
4. The topic does not call another generative answers node on the same turn; instead, the orchestrator uses the structured output to compose the final answer.
5. If generative orchestration is enabled, avoid additional overlapping generative answers in the same flow.

This reduces the chance that the orchestrator discards the intermediate result and returns "No information was found that could help answer this." even when the tool has produced the correct data.

---

References:

• FAQ for using generative orchestration
• FAQ for generative answers
• Explore AI capabilities in Copilot Studio
• Apply generative orchestration capabilities
• Use the Fallback topic
• Understand error codes