Share via

Integrating Azure OpenAI GPT with Azure AI Search in Power Automate

Power Automate 20 Reputation points
2025-07-07T14:18:10.96+00:00

Hi everyone,

I'm currently working on partially automating our customer service workflow by generating draft replies to incoming emails. The goal is to create helpful, well-formulated responses based on the email content, which can then be manually reviewed and sent.

My current setup:

  • An Azure OpenAI deployment using GPT-4o
  • An Azure AI Search resource connected to a Blob Storage with .txt files that serve as the knowledge base
  • A Power Automate flow that:
    • Triggers when an email is received
    • Uses the email body as input
    • Builds a prompt
    • Sends it via HTTP to the Azure OpenAI API
    • Saves the output as a draft reply

The challenge:

I want to use GPT-4o in combination with Azure AI Search, so that:

  1. Relevant content from the knowledge base is retrieved
  2. GPT uses that content to generate a fluent, friendly, and accurate response

Right now:

When I use GPT only, the reply is nicely phrased but lacks accurate, fact-based support.

When I use only AI Search, the result is correct but too dry and robotic.

In the Azure OpenAI Playground, this works great: GPT automatically uses context retrieved via AI Search to generate a solid, customer-friendly reply.

What I need:

I would like to understand how to get this combination (GPT + AI Search) working outside the Playground – ideally in Power Automate, but I’m open to other approaches too.

Specifically:

What is the best way to combine GPT and AI Search in a workflow like this?

  • Is there a way to do this via API, or should I structure the flow differently?

Any guidance, examples, or documentation links would be very helpful. Thanks in advance!

Azure AI Search
Azure AI Search

An Azure search service with built-in artificial intelligence capabilities that enrich information to help identify and explore relevant content at scale.

0 comments

Answer accepted by question author

santoshkc 15,615 Reputation points Microsoft External Staff Moderator
2025-07-09T14:59:44.05+00:00

Hi Power Automate,

Thanks for sharing your setup and use case and this is a great example of how to combine Azure OpenAI and Azure AI Search to streamline customer service workflows.

What you're trying to achieve is known as the "Retrieve-then-Generate" (RAG) pattern, where relevant content is first retrieved from a knowledge base (in your case, via Azure AI Search), and then GPT-4o uses that content to generate a well-written response. While the Azure OpenAI Playground handles this integration behind the scenes, doing it outside, such as in Power Automate which requires connecting the pieces manually.

To replicate this in Power Automate, you can add an HTTP action to first query Azure AI Search using the incoming email content as the search term. Then, take the top results and construct a prompt by embedding that content along with the original email. Finally, send this constructed prompt to the Azure OpenAI API (chat completions endpoint) and save the response as your draft reply.

Here’s a helpful Microsoft guide that explains how to use your own data with Azure OpenAI:

Use your data with Azure OpenAI

And for detailed API integration, refer to:

Azure AI Search REST API and Azure OpenAI REST reference

I hope this helps! Thank you

If this answers your query, please do click on 'Accept answer' and 'upvote.

Was this answer helpful?

1 person found this answer helpful.

0 additional answers

Sort by: Most helpful
Most helpful Newest Oldest

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.