Issue While Implementing Knowledge Agents – Retrieve API Errors

Question

Issue While Implementing Knowledge Agents – Retrieve API Errors

Jalpa Chudasama | CHECKER 0

We attempted to implement Knowledge Agents using the official documentation provided here: 🔗 https://learn.microsoft.com/en-us/azure/search/search-agentic-retrieval-how-to-create

We are currently facing issues with the Retrieve API. Below is the step-by-step implementation and the problems encountered.  

Step 1: Create Knowledge Agent


$searchUrl = 'test.search.windows.net';
$agentName = 'agentName';
$indexName = 'indexName';
$modelProviderUrl = 'https://test.openai.azure.com';
$searchApiKey = '*********';
$modelApiKey = '************';
$endpoint = "https://{$searchUrl}/agents/{$agentName}?api-version=2025-05-01-preview";
$data = [
    'name' => $agentName,
    'targetIndexes' => [[
        'indexName' => $indexName,
        'defaultRerankerThreshold' => 2.5,
        'defaultIncludeReferenceSourceData' => true,
        'defaultMaxDocsForReranker' => 200
    ]],
    'models' => [[
        'kind' => 'azureOpenAI',
        'azureOpenAIParameters' => [
        'resourceUri' => $modelProviderUrl,
        'apiKey' => $modelApiKey,
        'deploymentId' => 'text-embedding-3-large',
        'modelName' => 'gpt-4o'
        ]
    ]],
    'requestLimits' => [
        'maxOutputSize' => 5000,
        'maxRuntimeInSeconds' => 60
    ],
    'encryptionKey' => null
];
$ch = curl_init($endpoint);
$headers = [
'Content-Type: application/json',
'api-key: ' . $searchApiKey
];
curl_setopt_array($ch, [
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_CUSTOMREQUEST => 'PUT',
    CURLOPT_POSTFIELDS => json_encode($data),
    CURLOPT_HTTPHEADER => $headers
]);
$response = curl_exec($ch);
$httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
if (curl_errno($ch)) {
    echo 'cURL Error: ' . curl_error($ch) . "\n";
} else {
    echo "HTTP Status Code: " . $httpCode . "\n";
    echo "Response: " . $response . "\n";
}
curl_close($ch);


Step 2: Retrieve API Implementation


$searchUrl = 'test.search.windows.net';
$agentName = 'agentName';
$indexName = 'indexName';
$searchApiKey = '***********************';
$endpoint = "https://{$searchUrl}/agents/{$agentName}/retrieve?api-version=2025-05-01-preview";
$data = [
    'messages' => [
        [
        'role' => 'assistant',
            'content' => [[
            'type' => 'text',
            'text' => "You are a helpful assistant for Contoso Human Resources. You have access to a search index containing guidelines about health care coverage for Washington state. If you can't find the answer in the search, say you don't know."
            ]]
        ],
        [
        'role' => 'user',
            'content' => [[
            'type' => 'text',
            'text' => 'What are my vision benefits?'
            ]]
        ]
    ],
    'targetIndexParams' => [[
        'indexName' => $indexName,
        'filterAddOn' => "State eq 'WA'",
        'IncludeReferenceSourceData' => true,
        'rerankerThreshold' => 2.5,
        'maxDocsForReranker' => 250
    ]]
];
$ch = curl_init($endpoint);
$headers = [
    'Content-Type: application/json',
    // Note: We tried both of the following in different tests
    // 'api-key: ' . $searchApiKey
    'Authorization: Bearer ' . $searchApiKey
];
curl_setopt_array($ch, [
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_CUSTOMREQUEST => 'POST',
    CURLOPT_POSTFIELDS => json_encode($data),
    CURLOPT_HTTPHEADER => $headers
]);
$response = curl_exec($ch);
$httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
if (curl_errno($ch)) {
    echo 'cURL Error: ' . curl_error($ch) . "\n";
} else {
    echo "HTTP Status Code: " . $httpCode . "\n";
    echo "<pre>";
    print_r(json_decode($response));
    echo "</pre>";
}
curl_close($ch);


Step 3: Generate Access Token


$tenant = '*************';
$client = '*************';
$secret = '*************';
$url = "https://login.microsoftonline.com/$tenant/oauth2/v2.0/token";
$data = http_build_query([
    'grant_type' => 'client_credentials',
    'client_id' => $client,
    'client_secret' => $secret,
    'scope' => 'https://search.azure.com/.default'
]);
$ch = curl_init($url);
curl_setopt_array($ch, [
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_POST => true,
    CURLOPT_HTTPHEADER => ['Content-Type: application/x-www-form-urlencoded'],
    CURLOPT_POSTFIELDS => $data
]);
$resp = curl_exec($ch);
curl_close($ch);
echo "<pre>";
print_r(json_decode($resp)->access_token);
echo "</pre>";

Issues Encountered with Retrieve API

Using Access Token:
- Status Code: 403 Forbidden
- Issue: No response returned.
Using Search API Key as Bearer Token:
- Status Code: 401 Unauthorized
- Issue: No response returned.
Using Search API Key with api-key Header:
- Status Code: 404 Not Found
- Response: Could not complete model action. The model endpoint returned status code '404' (NotFound). Resource not found

Laxman Reddy Revuri 5,395 Reputation points Microsoft External Staff Moderator

2025-06-13T06:19:14.0933333+00:00

Hi @Jalpa Chudasama | CHECKER
It seems you are experiencing issues with the Retrieve API for your Knowledge Agents.

A 401 Unauthorized error occurs when the Search API key is incorrectly used in the Authorization: Bearer header. Instead, the correct approach is to pass the API key using the api-key header. This is the expected authentication method for Azure AI Search endpoints when using an API key.

A 403 Forbidden error typically means that you're using an Azure Active Directory (AAD) token, but your app registration doesn't have the required permissions. To resolve this, you need to add the “Azure Cognitive Search” API to your app registration under API permissions, assign the necessary application permissions, and make sure to grant admin consent in Entra ID (Azure AD portal).

Lastly, a 404 Not Found error suggests that the Knowledge Agent cannot reach your Azure OpenAI model endpoint. This can happen if the resourceUri, deploymentId, or region configuration is incorrect. Double-check that the OpenAI resource URI is valid the deployment ID matches your deployed model, and that both the Azure Search and Azure OpenAI services are in the same Azure region to ensure proper connectivity.

Ensure both Azure Search and Azure OpenAI have public access or are in the same VNet

If using private endpoints, configure VNet &DNS correctly

Use Diagnostic Settings to send logs to Log Analytics for better error tracing.
Laxman Reddy Revuri 5,395 Reputation points Microsoft External Staff Moderator

2025-06-16T05:06:35.9133333+00:00

Hi @Jalpa Chudasama | CHECKER
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.

Mamta Mavani 0

Hello @Laxman Reddy Revuri
Based on your suggestions, I have tried all the recommended steps, including assigning the required permissions and generating an access token, but I am still encountering the same error.
User's image

And the code like

<?php

// Configuration
$searchUrl = 'test.search.windows.net';
$agentName = 'test-agent';
$indexName = 'test_vector';
$searchApiKey = '**************************';

// API endpoint
$endpoint = "https://{$searchUrl}/agents/{$agentName}/retrieve?api-version=2025-05-01-preview";
// Request payload
$data = [
    'messages' => [
        [
            'role' => 'assistant',
            'content' => [
                [
                    'type' => 'text',
                    'text' => 'You are a helpful assistant for Contoso Human Resources. You have access to a search index containing guidelines about health care coverage for Washington state. If you can\'t find the answer in the search, say you don\'t know.'
                ]
            ]
        ],
        [
            'role' => 'user',
            'content' => [
                [
                    'type' => 'text',
                    'text' => 'What are my vision benefits?'
                ]
            ]
        ]
    ],
    'targetIndexParams' => [
        [
            'indexName' => $indexName,
            'filterAddOn' => "State eq 'WA'",
            'IncludeReferenceSourceData' => true,
            'rerankerThreshold' => 2.5,
            'maxDocsForReranker' => 250
        ]
    ]
];

$ch = curl_init($endpoint);

$headers = [
    'Content-Type: application/json',
    'api-key: ' . $searchApiKey
];

curl_setopt_array($ch, [
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_CUSTOMREQUEST => 'POST',
    CURLOPT_POSTFIELDS => json_encode($data),
    CURLOPT_HTTPHEADER => $headers
]);

// Execute the request
$response = curl_exec($ch);
$httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);

curl_close($ch);

Mamta Mavani 0 Reputation points

2025-06-18T04:34:23.22+00:00

Hello, @Laxman Reddy Revuri
Could you please check our last comment when you get a chance?
We're currently waiting for your response to proceed further. Let us know if any clarification is needed.
Suresh Chikkam 2,135 Reputation points Microsoft External Staff Moderator

2025-06-20T07:35:48.1766667+00:00

Hi Jalpa Chudasama | CHECKER,

I looked into your issue, and you're almost there just a few key things need to be corrected to make the Retrieve API work.

First, don’t use the Authorization: Bearer header with the Search API key. That’s why you’re getting the 401 error. Azure Search only accepts the key in the api-key header. Once you switch to that, the call will go through.

Next, the 403 error with the access token means either the token doesn’t have the right permissions or admin consent is missing. But for now, to keep things simple, just stick with the api-key method that works well unless you need RBAC.

The 404 is the important one. It means the agent can't talk to your model. This usually happens when the resourceUri doesn’t match the Azure OpenAI endpoint, or the deploymentId doesn’t match the actual deployment name in your Azure OpenAI resource. Also, both Azure OpenAI and Azure Cognitive Search must be in the same region if they’re not, this call will always fail.

Chcek that your index name is correct and already exists. You can test it quickly using a simple GET request to list the indexes. Once you’ve fixed the model details, region, and headers, your Retrieve API should work as expected.
Jalpa Chudasama | CHECKER 0 Reputation points

2025-06-23T11:01:39.23+00:00
Hello @Suresh Chikkam, I reviewed your response, but I’m still encountering the same issue.

When I use the searchApiKey, I receive the following error from the Agent Retrieve API:

{ "status": "Error", "http_code": 500, "error": "{\"error\":{\"code\":\"\",\"message\":\"An error has occurred.\"}}" }

When I use the Authorization Bearer token, I get this error:

{ "status": "Error", "http_code": 403, "error": "" }

I’ve already added the necessary API permissions to the token. This token is being used for authentication with the API.
Could you please verify if there’s anything wrong—perhaps the token permissions are not configured correctly?
Suresh Chikkam 2,135 Reputation points Microsoft External Staff Moderator

2025-06-25T17:19:29.13+00:00
Hi Jalpa, I dug into both your 500 and 403 and here’s what I’d try next.

For the 500 when using your api-key header, it usually means the service can’t even deserialize your retrieve request. The preview Retrieve API expects each message’s content to be a simple string, not a nested array of objects.

try this:

"messages":[ {"role":"assistant","content":"You are a helpful assistant for Contoso HR…"}, {"role":"user","content":"What are my vision benefits?"} ]

That change alone often moves you past a silent internal error. Once you’ve simplified the payload, give the call another shot. If you still see a 500, turn on Diagnostic Settings on your Search service, route logs to Log Analytics, and filter on the “SearchAgent” category you’ll get the stack trace of what’s really blowing up.

For the 403 with your AAD token, the problem is that your service principal only has a delegated “user_impersonation” scope. In client-credentials flows there is no user to impersonate, so the token never carries that permission. Instead, you need to grant the SP an Azure RBAC role on your Search resource Search Index Data Reader is enough for Retrieve. Once that role assignment is in place, request a token for the scope https://search.azure.com/.default and call with Authorization: Bearer <token>. The 403 will vanish and you’ll either get a successful grounding response or a clear error you can act on.

Give those two tweaks a try simplify your message payload and swap your AAD setup to use RBAC and you should see your Retrieve call start returning real results instead of errors.

1 answer

Your answer

Laxman Reddy Revuri 5,395 Reputation points Microsoft External Staff Moderator

2025-06-13T06:19:14.0933333+00:00

Hi @Jalpa Chudasama | CHECKER
It seems you are experiencing issues with the Retrieve API for your Knowledge Agents.

A 401 Unauthorized error occurs when the Search API key is incorrectly used in the Authorization: Bearer header. Instead, the correct approach is to pass the API key using the api-key header. This is the expected authentication method for Azure AI Search endpoints when using an API key.

A 403 Forbidden error typically means that you're using an Azure Active Directory (AAD) token, but your app registration doesn't have the required permissions. To resolve this, you need to add the “Azure Cognitive Search” API to your app registration under API permissions, assign the necessary application permissions, and make sure to grant admin consent in Entra ID (Azure AD portal).

Lastly, a 404 Not Found error suggests that the Knowledge Agent cannot reach your Azure OpenAI model endpoint. This can happen if the resourceUri, deploymentId, or region configuration is incorrect. Double-check that the OpenAI resource URI is valid the deployment ID matches your deployed model, and that both the Azure Search and Azure OpenAI services are in the same Azure region to ensure proper connectivity.

Ensure both Azure Search and Azure OpenAI have public access or are in the same VNet

If using private endpoints, configure VNet &DNS correctly

Use Diagnostic Settings to send logs to Log Analytics for better error tracing.
Laxman Reddy Revuri 5,395 Reputation points Microsoft External Staff Moderator

2025-06-16T05:06:35.9133333+00:00

Hi @Jalpa Chudasama | CHECKER
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Mamta Mavani 0 Reputation points

2025-06-16T12:26:04.61+00:00

Hello @Laxman Reddy Revuri
Based on your suggestions, I have tried all the recommended steps, including assigning the required permissions and generating an access token, but I am still encountering the same error.

And the code like

<?php // Configuration $searchUrl = 'test.search.windows.net'; $agentName = 'test-agent'; $indexName = 'test_vector'; $searchApiKey = '**************************'; // API endpoint $endpoint = "https://{$searchUrl}/agents/{$agentName}/retrieve?api-version=2025-05-01-preview"; // Request payload $data = [ 'messages' => [ [ 'role' => 'assistant', 'content' => [ [ 'type' => 'text', 'text' => 'You are a helpful assistant for Contoso Human Resources. You have access to a search index containing guidelines about health care coverage for Washington state. If you can\'t find the answer in the search, say you don\'t know.' ] ] ], [ 'role' => 'user', 'content' => [ [ 'type' => 'text', 'text' => 'What are my vision benefits?' ] ] ] ], 'targetIndexParams' => [ [ 'indexName' => $indexName, 'filterAddOn' => "State eq 'WA'", 'IncludeReferenceSourceData' => true, 'rerankerThreshold' => 2.5, 'maxDocsForReranker' => 250 ] ] ]; $ch = curl_init($endpoint); $headers = [ 'Content-Type: application/json', 'api-key: ' . $searchApiKey ]; curl_setopt_array($ch, [ CURLOPT_RETURNTRANSFER => true, CURLOPT_CUSTOMREQUEST => 'POST', CURLOPT_POSTFIELDS => json_encode($data), CURLOPT_HTTPHEADER => $headers ]); // Execute the request $response = curl_exec($ch); $httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE); curl_close($ch);
Mamta Mavani 0 Reputation points

2025-06-18T04:34:23.22+00:00

Hello, @Laxman Reddy Revuri
Could you please check our last comment when you get a chance?
We're currently waiting for your response to proceed further. Let us know if any clarification is needed.
Suresh Chikkam 2,135 Reputation points Microsoft External Staff Moderator

2025-06-20T07:35:48.1766667+00:00

Hi Jalpa Chudasama | CHECKER,

I looked into your issue, and you're almost there just a few key things need to be corrected to make the Retrieve API work.

First, don’t use the Authorization: Bearer header with the Search API key. That’s why you’re getting the 401 error. Azure Search only accepts the key in the api-key header. Once you switch to that, the call will go through.

Next, the 403 error with the access token means either the token doesn’t have the right permissions or admin consent is missing. But for now, to keep things simple, just stick with the api-key method that works well unless you need RBAC.

The 404 is the important one. It means the agent can't talk to your model. This usually happens when the resourceUri doesn’t match the Azure OpenAI endpoint, or the deploymentId doesn’t match the actual deployment name in your Azure OpenAI resource. Also, both Azure OpenAI and Azure Cognitive Search must be in the same region if they’re not, this call will always fail.

Chcek that your index name is correct and already exists. You can test it quickly using a simple GET request to list the indexes. Once you’ve fixed the model details, region, and headers, your Retrieve API should work as expected.
Jalpa Chudasama | CHECKER 0 Reputation points

2025-06-23T11:01:39.23+00:00

Hello @Suresh Chikkam, I reviewed your response, but I’m still encountering the same issue.

When I use the searchApiKey, I receive the following error from the Agent Retrieve API:

{ "status": "Error", "http_code": 500, "error": "{\"error\":{\"code\":\"\",\"message\":\"An error has occurred.\"}}" }

When I use the Authorization Bearer token, I get this error:

{ "status": "Error", "http_code": 403, "error": "" }

I’ve already added the necessary API permissions to the token. This token is being used for authentication with the API.
Could you please verify if there’s anything wrong—perhaps the token permissions are not configured correctly?
Suresh Chikkam 2,135 Reputation points Microsoft External Staff Moderator

2025-06-25T17:19:29.13+00:00

Hi Jalpa, I dug into both your 500 and 403 and here’s what I’d try next.

For the 500 when using your api-key header, it usually means the service can’t even deserialize your retrieve request. The preview Retrieve API expects each message’s content to be a simple string, not a nested array of objects.

try this:

"messages":[ {"role":"assistant","content":"You are a helpful assistant for Contoso HR…"}, {"role":"user","content":"What are my vision benefits?"} ]

That change alone often moves you past a silent internal error. Once you’ve simplified the payload, give the call another shot. If you still see a 500, turn on Diagnostic Settings on your Search service, route logs to Log Analytics, and filter on the “SearchAgent” category you’ll get the stack trace of what’s really blowing up.

For the 403 with your AAD token, the problem is that your service principal only has a delegated “user_impersonation” scope. In client-credentials flows there is no user to impersonate, so the token never carries that permission. Instead, you need to grant the SP an Azure RBAC role on your Search resource Search Index Data Reader is enough for Retrieve. Once that role assignment is in place, request a token for the scope https://search.azure.com/.default and call with Authorization: Bearer <token>. The 403 will vanish and you’ll either get a successful grounding response or a clear error you can act on.

Give those two tweaks a try simplify your message payload and swap your AAD setup to use RBAC and you should see your Retrieve call start returning real results instead of errors.

Answer 1

Hello @Suresh Chikkam ,

I passed the JSON object you suggested in the payload; however, I’m still receiving the following error:

{
    "status": "Error",
    "http_code": 400,
    "error": "{\"error\":{\"code\":\"\",\"message\":\"The request is invalid. Details: An unexpected 'PrimitiveValue' node was found when reading from the JSON reader. A 'StartObject' node was expected.\"}}"
}

Enabled logs under Diagnostic Settings in the Azure portal.
Generated a valid access token using the https://search.azure.com/.default scope.
Assigned the Search Index Data Reader role to the identity used to generate the token.
Double-checked that the token is correctly attached in the Authorization: Bearer {token} header.

Despite this, I’m also getting 403 error when used token and the same 400 error mentioned above.

If possible, could we arrange a call?

Thanks,
Jalpa Chudasama

Share via

Issue While Implementing Knowledge Agents – Retrieve API Errors

1 answer

Your answer