Error when creating agent evaluation from Foundry UI

Question

Error when creating agent evaluation from Foundry UI

Ipsita Dutta 0

User's image

I’m getting the attached error while creating an agent evaluation in Microsoft Foundry. I’ve checked the dataset, and it looks correct from my side. Could you please help me understand what might be causing this issue or verify if I’m missing any permissions?

SRILAKSHMI C 19,635 Reputation points Microsoft External Staff Moderator

2026-07-14T11:06:30.9066667+00:00
Hello @Ipsita Dutta

Thank you for reaching out to Microsoft Q&A.

From the screenshot, the failure occurs while you're in the Create new evaluation wizard in Microsoft Foundry, and the error displayed is Request failed with status code 400: The action cannot be finished with reason Forbidden

Since you've already confirmed that the dataset itself looks correct, this error is more indicative of an authorization or access issue rather than a problem with the dataset formatting.

Here are a few things we recommend verifying:

Verify your Azure AI Foundry project role

Ensure your account has the Azure AI Foundry User role (or an equivalent role such as Contributor/Owner) on the Foundry project.

Evaluation and observability features require appropriate project permissions, and insufficient access can result in a Forbidden error.

Verify storage account permissions (if your dataset is stored in Azure Storage)

If the evaluation uses a user-provided dataset stored in Azure Storage, confirm that:

The storage account is correctly connected to your Azure AI Foundry project.

Your account has the Storage Blob Data Contributor role on the connected storage account.

Missing storage permissions can prevent the evaluation from being created.

Confirm access to the evaluator model deployment

If you're using AI-assisted evaluators, ensure the required judge model deployment exists and is accessible within the project.

You can verify this under Build > Deployments in Azure AI Foundry.

Verify access to the target agent and other connected resources

Ensure you have access to the selected agent (documentextractoragent:v1) as well as any datasets, storage accounts, or Azure AI resources referenced by the evaluation.

Try creating the evaluation using another account with Owner permissions to determine whether the issue is permission-related.

Try using an InPrivate/Incognito browser session or a different browser to rule out any client-side issues.

Please refer this

Quickstart: Evaluate your hosted agent (Create evaluation wizard steps + troubleshooting notes) https://learn.microsoft.com/azure/foundry/observability/quickstarts/quickstart-evaluate-hosted-agent

Troubleshoot evaluation and observability issues (includes permission/access checks + Application Insights delay notes) https://learn.microsoft.com/azure/foundry/observability/how-to/troubleshooting

Run evaluations from the Microsoft Foundry portal (evaluation creation / configure agents / user prompt & mapping concepts) https://learn.microsoft.com/azure/foundry/how-to/evaluate-generative-ai-app

I Hope this helps. Do let me know if you have any further queries.

Thank you!

1 answer

Your answer

Answer 1

Hi @Ipsita Dutta ,

Could you please check the below storage account and access permissions.

Storage Account:

Azure AI Foundry must read your uploaded evaluation dataset from, and write logs to, the Workspace's default Storage Account.
Open the Azure Portal and navigate to the Storage Account linked to your AI Foundry Workspace/Project.
In the left menu under Security + networking, click Networking. If your firewall is set to 'Selected networks', scroll down to Exceptions and check the box for: 'Allow Azure services on the trusted services list to access this storage account'.
Save the changes and wait 2–3 minutes before retrying the evaluation.

Missing RBAC on the Project's Managed Identity:

When evaluating tool calls, the platform uses the system's Managed Identity to read and parse the dataset—not your personal user account.

Find the Managed Identity name associated with your Azure AI Foundry Project.
Navigate to your Storage Account -> Access Control (IAM) in the Azure Portal.
Click Add -> Add role assignment. Select 'Storage Blob Data Contributor'.
Assign it to your Project's Managed Identity.

Please 'Upvote'(Thumbs-up) and 'Accept' as answer if the reply was helpful. This will be benefitting other community members who face the same issue.

Thank you!