Cuir in eagar

Comhroinn trí


Use a blocklist

Caution

The sample data in this guide might contain offensive content. User discretion is advised.

The default AI classifiers are sufficient for most content moderation needs. However, you might need to screen for items that are specific to your use case. Blocklists let you add custom terms to the AI classifiers. You can use blocklists to screen for specific terms or phrases that you want to flag in your content.

Prerequisites

  • An Azure subscription - Create one for free
  • Once you have your Azure subscription, create a Content Safety resource in the Azure portal to get your key and endpoint. Enter a unique name for your resource, select your subscription, and select a resource group, supported region (see Region availability), and supported pricing tier. Then select Create.
    • The resource takes a few minutes to deploy. After it finishes, Select go to resource. In the left pane, under Resource Management, select Subscription Key and Endpoint. The endpoint and either of the keys are used to call APIs.
  • One of the following installed:
    • cURL for REST API calls.
    • Python 3.x installed
      • Your Python installation should include pip. You can check if you have pip installed by running pip --version on the command line. Get pip by installing the latest version of Python.
      • If you're using Python, you'll need to install the Azure AI Content Safety client library for Python. Run the command pip install azure-ai-contentsafety in your project directory.
    • .NET Runtime installed.
      • .NET Core SDK installed.
      • If you're using .NET, you'll need to install the Azure AI Content Safety client library for .NET. Run the command dotnet add package Azure.AI.ContentSafety --prerelease in your project directory.

Create environment variables

In this example, you'll write your credentials to environment variables on the local machine running the application.

To set the environment variable for your key and endpoint, open a console window and follow the instructions for your operating system and development environment.

  • To set the CONTENT_SAFETY_KEY environment variable, replace YOUR_CONTENT_SAFETY_KEY with one of the keys for your resource.
  • To set the CONTENT_SAFETY_ENDPOINT environment variable, replace YOUR_CONTENT_SAFETY_ENDPOINT with the endpoint for your resource.

Important

If you use an API key, store it securely somewhere else, such as in Azure Key Vault. Don't include the API key directly in your code, and never post it publicly.

For more information about AI services security, see Authenticate requests to Azure AI services.

setx CONTENT_SAFETY_KEY 'YOUR_CONTENT_SAFETY_KEY'
setx CONTENT_SAFETY_ENDPOINT 'YOUR_CONTENT_SAFETY_ENDPOINT'

After you add the environment variables, you might need to restart any running programs that will read the environment variables, including the console window.

Analyze text with a blocklist

You can create blocklists to use with the Text API. The following steps help you get started.

Create or modify a blocklist

Copy the cURL command below to a text editor and make the following changes:

  1. Replace <endpoint> with your endpoint URL.
  2. Replace <enter_your_key_here> with your key.
  3. Replace <your_list_name> (in the URL) with a custom name for your list. Also replace the last term of the REST URL with the same name. Allowed characters: 0-9, A-Z, a-z, - . _ ~.
  4. Optionally replace the value of the "description" field with a custom description.
curl --location --request PATCH '<endpoint>/contentsafety/text/blocklists/<your_list_name>?api-version=2024-09-01' \
--header 'Ocp-Apim-Subscription-Key: <enter_your_key_here>' \
--header 'Content-Type: application/json' \
--data-raw '{
    "description": "This is a violence list"
}'

The response code should be 201(created a new list) or 200(updated an existing list).

Add blocklistItems to the list

Note

There is a maximum limit of 10,000 terms in total across all lists. You can add at most 100 blocklistItems in one request.

Copy the cURL command below to a text editor and make the following changes:

  1. Replace <endpoint> with your endpoint URL.
  2. Replace <enter_your_key_here> with your key.
  3. Replace <your_list_name> (in the URL) with the name you used in the list creation step.
  4. Optionally replace the value of the "description" field with a custom description.
  5. Replace the value of the "text" field with the item you'd like to add to your blocklist. The maximum length of a blocklistItem is 128 characters.
curl --location --request POST '<endpoint>/contentsafety/text/blocklists/<your_list_name>:addOrUpdateBlocklistItems?api-version=2024-09-01' \
--header 'Ocp-Apim-Subscription-Key: <enter_your_key_here>' \
--header 'Content-Type: application/json' \
--data-raw '"blocklistItems": [{
    "description": "string",
    "text": "bleed"
}]'

Tip

You can add multiple blocklistItems in one API call. Make the request body a JSON array of data groups:

{
   "blocklistItems": [
       {
           "description": "string",
           "text": "bleed"
       },
       {
           "description": "string",
           "text": "blood"
       }
   ]
}

The response code should be 200.

{
"blocklistItems:"[
  {
  "blocklistItemId": "string",
  "description": "string",
  "text": "bleed"
  
   }
 ]
}

Note

There will be some delay after you add or edit a blockItem before it takes effect on text analysis, usually not more than five minutes.

Analyze text with a blocklist

Copy the cURL command below to a text editor and make the following changes:

  1. Replace <endpoint> with your endpoint URL.
  2. Replace <enter_your_key_here> with your key.
  3. Replace <your_list_name> with the name you used in the list creation step. The "blocklistNames" field can contain an array of multiple list IDs.
  4. Optionally change the value of "breakByBlocklists". true indicates that once a blocklist is matched, the analysis will return immediately without model output. false will cause the model to continue to do analysis in the default categories.
  5. Optionally change the value of the "text" field to whatever text you want to analyze.
curl --location --request POST '<endpoint>/contentsafety/text:analyze?api-version=2024-09-01&' \
--header 'Ocp-Apim-Subscription-Key: <enter_your_key_here>' \
--header 'Content-Type: application/json' \
--data-raw '{
  "text": "I want to beat you till you bleed",
  "categories": [
    "Hate",
    "Sexual",
    "SelfHarm",
    "Violence"
  ],
  "blocklistNames":["<your_list_name>"],
  "haltOnBlocklistHit": false,
  "outputType": "FourSeverityLevels"
}'

The JSON response will contain a "blocklistMatchResults" that indicates any matches with your blocklist. It reports the location in the text string where the match was found.

{
  "blocklistsMatch": [
    {
      "blocklistName": "string",
      "blocklistItemId": "string",
      "blocklistItemText": "bleed"
    }
  ],
  "categoriesAnalysis": [
    {
      "category": "Hate",
      "severity": 0
    }
  ]
}

Other blocklist operations

This section contains more operations to help you manage and use the blocklist feature.

List all blocklistItems in a list

Copy the cURL command below to a text editor and make the following changes:

  1. Replace <endpoint> with your endpoint URL.
  2. Replace <enter_your_key_here> with your key.
  3. Replace <your_list_name> (in the request URL) with the name you used in the list creation step.
curl --location --request GET '<endpoint>/contentsafety/text/blocklists/<your_list_name>/blocklistItems?api-version=2024-09-01' \
--header 'Ocp-Apim-Subscription-Key: <enter_your_key_here>' \
--header 'Content-Type: application/json'

The status code should be 200 and the response body should look like this:

{
 "values": [
  {
   "blocklistItemId": "string",
   "description": "string",
   "text": "bleed",
  }
 ]
}

List all blocklists

Copy the cURL command below to a text editor and make the following changes:

  1. Replace <endpoint> with your endpoint URL.
  2. Replace <enter_your_key_here> with your key.
curl --location --request GET '<endpoint>/contentsafety/text/blocklists?api-version=2024-09-01' \
--header 'Ocp-Apim-Subscription-Key: <enter_your_key_here>' \
--header 'Content-Type: application/json'

The status code should be 200. The JSON response looks like this:

"value": [
  {
    "blocklistName": "string",
    "description": "string"
  }
]

Get a blocklist by blocklistName

Copy the cURL command below to a text editor and make the following changes:

  1. Replace <endpoint> with your endpoint URL.
  2. Replace <enter_your_key_here> with your key.
  3. Replace <your_list_name> (in the request URL) with the name you used in the list creation step.
cURL --location '<endpoint>contentsafety/text/blocklists/<your_list_name>?api-version=2024-09-01' \
--header 'Ocp-Apim-Subscription-Key: <enter_your_key_here>' \
--data ''

The status code should be 200. The JSON response looks like this:

{
  "blocklistName": "string",
  "description": "string"
}

Get a blocklistItem by blocklistName and blocklistItemId

Copy the cURL command below to a text editor and make the following changes:

  1. Replace <endpoint> with your endpoint URL.
  2. Replace <enter_your_key_here> with your key.
  3. Replace <your_list_name> (in the request URL) with the name you used in the list creation step.
  4. Replace <your_item_id> with the ID value for the blocklistItem. This is the value of the "blocklistItemId" field from the Add blocklistItem or Get all blocklistItems API calls.
cURL --location '<endpoint>contentsafety/text/blocklists/<your_list_name>/blocklistItems/<your_item_id>?api-version=2024-09-01' \
--header 'Ocp-Apim-Subscription-Key: <enter_your_key_here>' \
--data ''

The status code should be 200. The JSON response looks like this:

{
  "blocklistItemId": "string",
  "description": "string",
  "text": "string"
}
  1. Replace <your_list_name> with the name you used in the list creation step.
  2. Replace <your_block_item_id> with the ID of the item you want to get.
  3. Run the script.

Remove blocklistItems from a blocklist.

Note

There will be some delay after you delete an item before it takes effect on text analysis, usually not more than five minutes.

Copy the cURL command below to a text editor and make the following changes:

  1. Replace <endpoint> with your endpoint URL.
  2. Replace <enter_your_key_here> with your key.
  3. Replace <your_list_name> (in the request URL) with the name you used in the list creation step.
  4. Replace <item_id> with the ID value for the blocklistItem. This is the value of the "blocklistItemId" field from the Add blocklistItem or Get all blocklistItems API calls.
curl --location --request POST '<endpoint>/contentsafety/text/blocklists/<your_list_name>:removeBlocklistItems?api-version=2024-09-01' \
--header 'Ocp-Apim-Subscription-Key: <enter_your_key_here>' \
--header 'Content-Type: application/json'
--data-raw '"blocklistItemIds":[
    "<item_id>"
]'

Tip

You can delete multiple blocklistItems in one API call. Make the request body an array of blocklistItemId values.

The response code should be 204.

Delete a list and all of its contents

Note

There will be some delay after you delete a list before it takes effect on text analysis, usually not more than five minutes.

Copy the cURL command below to a text editor and make the following changes:

  1. Replace <endpoint> with your endpoint URL.
  2. Replace <enter_your_key_here> with your key.
  3. Replace <your_list_name> (in the request URL) with the name you used in the list creation step.
curl --location --request DELETE '<endpoint>/contentsafety/text/blocklists/<your_list_name>?api-version=2024-09-01' \
--header 'Ocp-Apim-Subscription-Key: <enter_your_key_here>' \
--header 'Content-Type: application/json' \

The response code should be 204.

Next steps

See the API reference documentation to learn more about the APIs used in this guide.