API call to RAG-based service

Question

API call to RAG-based service

Steven Kent 40

I built a RAG-based OpenAI service that uses data I uploaded to AI Portal. It works when I am testing from the chat playground.

Now I want to make the same exact thing callable from Postman. I am able to call the service from Postman. The problem I am having is the API call does not take advantage of the RAG pattern I set up.

How do I get the API that I call to do exactly what the chat playground is already doing?

Accepted answer

1 additional answer

Your answer

Answer 1

Hi Steven Kent

Welcome to Microsoft Q&A Forum, thank you for posting your query here!

To ensure that your API call takes advantage of the RAG pattern you have set up, please make sure that you are sending AI Search index and key along OpenAI key.

Here is an example of how to make an API call.

curl -i -X POST $AZURE_OPENAI_ENDPOINT/openai/deployments/$AZURE_OPENAI_DEPLOYMENT_ID/chat/completions?api-version=2024-10-21 \
-H "Content-Type: application/json" \
-H "api-key: $AZURE_OPENAI_API_KEY" \
-d \
'
{
"data_sources": [
{
"type": "azure_search",
"parameters": {
"endpoint": "'$AZURE_AI_SEARCH_ENDPOINT'",
"index_name": "'$AZURE_AI_SEARCH_INDEX'",
"authentication": {
"type": "api_key",
"key": "'$AZURE_AI_SEARCH_API_KEY'"
}
}
}
],
"messages": [
{
"role": "user",
"content": "What are my available health plans?"
}
]
}
'

Kindly refer below link: rest-api

Hope this helps. Do let us know if you any further queries.

If this answers your query, do click Accept Answer and Yes for was this answer helpful.

Thank You.

Steven Kent 40 Reputation points

2025-01-29T15:48:23.8233333+00:00

Thank you @saideep anchuri
Adding the data_sources array and populating the values has gotten the API call to start using the RAG pattern.

Answer 2

Hello Steven Kent,

Welcome to the Microsoft Q&A and thank you for posting your questions here.

I understand that your RAG-based OpenAI service works in the chat playground but does not function as expected when called via API (Postman).

By following the below steps, you will be able to resolve it.

Step 1: Compare the API Request in the Playground and Postman

Open the browser’s DevTools (F12 in Chrome) or any of your browser and go to the Network tab.
Make a request in the chat playground and find the actual API request being sent to OpenAI.
Copy the full request payload, including headers, and compare it with your Postman request.

Step 2: Ensure RAG-Specific Parameters Are Present

If you are using Azure OpenAI’s on-your-data feature, the API request must include:

"data_sources": [
  {
    "type": "azure_search",
    "parameters": {
      "endpoint": "YOUR_AZURE_SEARCH_ENDPOINT",
      "index_name": "YOUR_INDEX_NAME",
      "authentication": {
        "type": "api_key",
        "key": "YOUR_SEARCH_API_KEY"
      }
    }
  }
]

Ensure that your payload in Postman includes this section.

Step 3: Check for Missing System Messages or Parameters

Some RAG-based services require system messages or specific parameters. For an example:

"messages": [
  {"role": "system", "content": "You are an AI assistant retrieving relevant knowledge."},
  {"role": "user", "content": "What are my available health plans?"}
]

NOTE: Copy the exact structure of messages from the playground.

Step 4: Validate Headers and API Version

Make sure you are using the correct API version (e.g., api-version=2024-10-21) and include all required headers:

{
  "Authorization": "Bearer YOUR_OPENAI_API_KEY",
  "Content-Type": "application/json"
}

Step 5: Debug the API Response

Compare the API responses from Postman and the chat playground.
If the API response is different, check for missing retrieval evidence (e.g., lack of "context" or "documents" in the response).
If the response does not contain retrieved documents, your RAG setup might not be properly configured.

I hope this is helpful! Do not hesitate to let me know if you have any other questions.

Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful.

Share via

API call to RAG-based service

1 additional answer

Your answer