Edit

Use Risks & Safety monitoring in Azure OpenAI Studio (preview)

  • Article

When you use an Azure OpenAI model deployment with a content filter, you may want to check the results of the filtering activity. You can use that information to further adjust your filter configuration to serve your specific business needs and Responsible AI principles.

Azure OpenAI Studio provides a Risks & Safety monitoring dashboard for each of your deployments that uses a content filter configuration.

Access Risks & Safety monitoring

To access Risks & Safety monitoring, you need an Azure OpenAI resource in one of the supported Azure regions: East US, Switzerland North, France Central, Sweden Central, Canada East. You also need a model deployment that uses a content filter configuration.

Go to Azure OpenAI Studio and sign in with the credentials associated with your Azure OpenAI resource. Select the Deployments tab on the left and then select your model deployment from the list. On the deployment's page, select the Risks & Safety tab at the top.

Content detection

The Content detection pane shows information about content filter activity. Your content filter configuration is applied as described in the Content filtering documentation.

Report description

Content filtering data is shown in the following ways:

  • Total blocked request count and block rate: This view shows a global view of the amount and rate of content that is filtered over time. This helps you understand trends of harmful requests from users and see any unexpected activity.
  • Blocked requests by category: This view shows the amount of content blocked for each category. This is an all-up statistic of harmful requests across the time range selected. It currently supports the harm categories hate, sexual, self-harm, and violence.
  • Block rate over time by category: This view shows the block rate for each category over time. It currently supports the harm categories hate, sexual, self-harm, and violence.
  • Severity distribution by category: This view shows the severity levels detected for each harm category, across the whole selected time range. This is not limited to blocked content but rather includes all content that was flagged by the content filters.
  • Severity rate distribution over time by category: This view shows the rates of detected severity levels over time, for each harm category. Select the tabs to switch between supported categories.

Screenshot of the content detection pane in the Risks & Safety monitoring page.

Adjust your content filter configuration to further align with business needs and Responsible AI principles.

Potentially abusive user detection

The Potentially abusive user detection pane leverages user-level abuse reporting to show information about users whose behavior has resulted in blocked content. The goal is to help you get a view of the sources of harmful content so you can take responsive actions to ensure the model is being used in a responsible way.

Report description

The potentially abusive user detection relies on the user information that customers send with their Azure OpenAI API calls, together with the request content. The following insights are shown:

  • Total potentially abusive user count: This view shows the number of detected potentially abusive users over time. These are users for whom a pattern of abuse was detected and who might introduce high risk.

Next steps

Next, create or edit a content filter configuration in Azure OpenAI Studio.