Error code: 400 when running General purpose Fluency evaluators

Question

Error code: 400 when running General purpose Fluency evaluators

Justin Obanor 20

Using the sample sample_fluency.py from azure-sdk-for-python at sdk/ai/azure-ai-projects/samples/evaluations/agentic_evaluators/sample_fluency.py, client.evals.create() succeeds but client.evals.runs.create() fails with:

BadRequestError: Error code: 400 - {'error': {'code': 'UserError', 'severity': None, 'message': 'The action cannnot be finished with reason Forbidden', 'messageFormat': 'The action cannnot be finished with reason {error}', 'messageParameters': {'error': 'Forbidden'}, 'referenceCode': None, 'detailsUri': None, 'target': None, 'details': [], 'innerError': {'code': 'UnauthorizedUserAction', 'innerError': None}, 'debugInfo': None, 'additionalInfo': None}, 'correlation': {'operation': 'e9a6cfb5f254b4298cc505acf31ed58c', 'request': '57fc69d1782fbf76'}, 'environment': 'eastus2', 'location': 'eastus2', 'time': '2026-06-22T18:35:59.0328447+00:00', 'componentName': 'raisvc', 'statusCode': 400}

The failure occurs only when creating the evaluation run.

The identity executing the code has the Foundry User role assigned on the Azure AI Foundry resource. I also assigned Foundry User to the project managed identity on the project level.

Inference works successfully using:

client.chat.completions.create(...)

with the same project and model deployment.

Is there additional RBAC required specifically for Evaluation Runs? Or any thing else that might be cauing the RequestError

Anshika Varshney 13,675 Reputation points Microsoft External Staff Moderator

2026-06-22T20:13:18.21+00:00
Hello @Justin Obanor

Based on the error details and Microsoft's documentation, the issue is likely network restrictions on your Azure AI Foundry account, not missing RBACroles.

Primary Cause: Network Restrictions Blocking raisvc

The error mentions raisvc (the evaluation backend service) and shows "Forbidden" - Microsoft has confirmed this exact issue occurs when the Foundry account uses Selected Networks network restrictions: [learn.microsoft]

Evaluation works when networkAcls.defaultAction = Allow

Evaluation fails when networkAcls.defaultAction = Deny (even with bypass = AzureServices)

The evaluation backend service cannot reach your restricted resources, even though inference works (your direct chat.completions.create call uses a different path).

Solution Steps

1. Check Network Configuration

bash az cognitive services account show \ --resource-group <your-resource-group> \ --name <your-foundry-account-name> \ --query "networkAcls"

2. Temporary Test: Enable Public Access

Temporarily set defaultAction to Allow to confirm this is the issue:

bash az cognitive services account update \ --resource-group <your-resource-group> \ --name <your-foundry-account-name> \ --bypass AzureServices \ --default-action Allow

If evaluations work after this, network restrictions are confirmed as the cause.

3. Permanent Fix: Add Required Service Endpoints

Instead of fully disabling restrictions, add the evaluation service to your permitted networks:

Find the evaluation backend service endpoint

Add it to your network ACLs' allowed IPs/vnets

Or use a private endpoint configuration for evaluations

Secondary Checks (if network isn't the issue)

RBAC Requirements for Evaluation Runs:

While Foundry User should work, verify these minimum roles on the project:

Azure AI Foundry User (you have this)

Cognitive Services OpenAI User (if using Azure OpenAI)

Storage Blob Data Contributor on the storage account (for Entra ID auth)

Storage Account Configuration:

If using Entra ID authentication for storage:

bash # Verify managed identity has Storage Blob Data Contributor

The storage account must also have publicNetworkAccess = Enabled.

Start with the network restriction check, this matches your exact error pattern (create succeeds, runs.create fails with raisvc/Forbidden) and is the most common cause per Microsoft's recent reports.

I Hope this helps. Do let me know if you have any further queries.
Thankyou!

Share via

Error code: 400 when running General purpose Fluency evaluators

Your answer