Edit

Set up permissions for Microsoft Foundry evaluation workflows

Use this article to find the roles required for your Microsoft Foundry evaluation workflow. You don't need every role listed here. Start with the common evaluation role, and then add roles only if your workflow uses scheduled or continuous evaluation, traces, or your own storage account.

Tip

For most portal and SDK evaluation workflows, your signed-in account needs the Foundry User role on the Foundry project. The extra roles in this article apply to specific workflows and resources.

Important

The Foundry RBAC roles were recently renamed. Foundry User, Foundry Owner, Foundry Account Owner, and Foundry Project Manager were previously named Azure AI User, Azure AI Owner, Azure AI Account Owner, and Azure AI Project Manager. You might still see the previous names in some places while the rename rolls out. The role IDs and core permissions are unchanged by the rename.

Before you begin

To assign Azure roles, you need Microsoft.Authorization/roleAssignments/write permission at the target scope. For example, the Role Based Access Control Administrator and User Access Administrator roles include this permission. If you can't create role assignments, send the relevant rows from this article to your administrator.

Evaluation workflows can use more than one identity:

  • Your identity is the account that you use to sign in to the Foundry portal, Azure Developer CLI, or SDK.
  • An application identity is the workload identity or service principal that runs evaluations in an application or CI/CD pipeline.
  • The project managed identity is the identity that Foundry uses to access connected resources, such as traces or storage.

Find the roles for your workflow

First, assign the role in this table for the evaluation task you want to perform.

I want to Assign the role to Role Assign at
Create continuous or scheduled evaluation rules The project managed identity Foundry User The Foundry resource or project used for the evaluation rule
Run or review portal or SDK evaluations, including agent, synthetic dataset, admin-connected model, and benchmark workflows Your identity or application identity Foundry User, or a role with equivalent permissions The Foundry resource or project that contains the evaluation
Run evaluations in CI/CD The workload identity or service principal used by the pipeline Foundry User, or a role with equivalent permissions The Foundry resource or project that contains the evaluation
Run agent evaluations with the Azure Developer CLI Your identity Foundry User The Foundry resource

Evaluation jobs use Microsoft Entra ID. If an evaluation invokes a model deployment, assign Foundry User at the Foundry account scope. A project-only role assignment doesn't authorize model inference. For more information, see Deployment type-specific permissions.

For the permissions included in Foundry roles and guidance for custom roles, see Role-based access control for Microsoft Foundry.

Add permissions for trace-based workflows

Use this section if you run one-time, continuous, or scheduled evaluations over Application Insights traces, create a dataset from traces, or view log-based monitoring data.

The identity that submits a trace evaluation or trace-based dataset job first needs the Foundry User role from Find the roles for your workflow. Then assign the following roles:

I want to Assign the role to Role Assign at
Run trace evaluations or create datasets from traces The project managed identity Reader The connected Application Insights resource
Read protected trace tables during evaluation or dataset generation The project managed identity Privileged Monitoring Data Reader, in addition to Reader The connected Application Insights resource
View traces or log-based monitoring data Your identity Log Analytics Reader The connected Application Insights resource and, for workspace-scoped queries, its linked Log Analytics workspace
View protected trace content Your identity Privileged Monitoring Data Reader, in addition to Log Analytics Reader The Application Insights resource for resource-scoped queries or the Log Analytics workspace for workspace-scoped queries

Trace evaluation and dataset generation use a resource-context query against the connected Application Insights resource. The Reader role provides the resource read permission required for this query.

If the linked Log Analytics workspace is configured to Require workspace permissions, also assign Log Analytics Reader to the project managed identity on that workspace. For protected tables, assign Privileged Monitoring Data Reader on the workspace too. If the workspace uses DataActionsOnly mode, use Log Analytics Data Reader instead of a control-plane reader role.

For information about protected trace content, see Protect sensitive content in traces.

Add permissions for your own storage account

Use this section only if your Foundry project connects to your own Azure Storage account by using Microsoft Entra ID authentication.

Assign the role to Role Assign at
The project managed identity Storage Blob Data Contributor The storage account connected to the Foundry project

This role lets the evaluation service read and write datasets and evaluation results in blob storage. If the connection uses an account key instead, this managed-identity role isn't used. Microsoft recommends Microsoft Entra ID authentication for granular access control.

For storage and network requirements, see Bring your own storage.

Verify your setup

After you assign the roles:

  1. In the Azure portal, open each resource where you assigned a role.
  2. Select Access control (IAM) > Check access.
  3. Search for the user, application identity, or project managed identity.
  4. Confirm that the expected role appears at the required scope.
  5. Wait several minutes for new role assignments to take effect, and then retry the workflow.

If an evaluation still fails with 401 Unauthorized or 403 Forbidden, see Troubleshoot evaluation and observability issues.

Next steps

  • Last updated on