The sample data in this guide might contain offensive content. User discretion is advised.
The default AI classifiers are sufficient for most content moderation needs. However, you might need to screen for items that are specific to your use case.
Once you have your Azure subscription, create a Content Safety resource in the Azure portal to get your key and endpoint. Enter a unique name for your resource, select the subscription you entered on the application form, and select a resource group, supported region, and supported pricing tier. Then select Create.
The resource takes a few minutes to deploy. After it finishes, Select go to resource. In the left pane, under Resource Management, select Subscription Key and Endpoint. The endpoint and either of the keys are used to call APIs.
Your Python installation should include pip. You can check if you have pip installed by running pip --version on the command line. Get pip by installing the latest version of Python.
If you're using Python, you'll need to install the Azure AI Content Safety client library for Python. Run the command pip install azure-ai-contentsafety in your project directory.
If you're using .NET, you'll need to install the Azure AI Content Safety client library for .NET. Run the command dotnet add package Azure.AI.ContentSafety --prerelease in your project directory.
Create environment variables
In this example, you'll write your credentials to environment variables on the local machine running the application.
Tip
Don't include the key directly in your code, and never post it publicly. See the Azure AI services security article for more authentication options like Azure Key Vault.
To set the environment variable for your key and endpoint, open a console window and follow the instructions for your operating system and development environment.
To set the CONTENT_SAFETY_KEY environment variable, replace YOUR_CONTENT_SAFETY_KEY with one of the keys for your resource.
To set the CONTENT_SAFETY_ENDPOINT environment variable, replace YOUR_CONTENT_SAFETY_ENDPOINT with the endpoint for your resource.
After you add the environment variables, you might need to restart any running programs that will read the environment variables, including the console window.
Copy the cURL command below to a text editor and make the following changes:
Replace <endpoint> with your endpoint URL.
Replace <enter_your_key_here> with your key.
Replace <your_list_name> (in the URL) with a custom name for your list. Also replace the last term of the REST URL with the same name. Allowed characters: 0-9, A-Z, a-z, - . _ ~.
Optionally replace the value of the "description" field with a custom description.
Create a new C# console app and open it in your preferred editor or IDE. Paste in the following code.
string endpoint = Environment.GetEnvironmentVariable("CONTENT_SAFETY_ENDPOINT");
string key = Environment.GetEnvironmentVariable("CONTENT_SAFETY_KEY");
ContentSafetyClient client = new ContentSafetyClient(new Uri(endpoint), new AzureKeyCredential(key));
var blocklistName = "<your_list_name>";
string blockItemText1 = "<block_item_text_1>";
string blockItemText2 = "<block_item_text_2>";
var blockItems = new TextBlockItemInfo[] { new TextBlockItemInfo(blockItemText1), new TextBlockItemInfo(blockItemText2) };
var addedBlockItems = client.AddBlockItems(blocklistName, new AddBlockItemsOptions(blockItems));
if (addedBlockItems != null && addedBlockItems.Value != null)
{
Console.WriteLine("\nBlockItems added:");
foreach (var addedBlockItem in addedBlockItems.Value.Value)
{
Console.WriteLine("BlockItemId: {0}, Text: {1}, Description: {2}", addedBlockItem.BlockItemId, addedBlockItem.Text, addedBlockItem.Description);
}
}
Replace <your_list_name> with the name you used in the list creation step.
Replace the values of the blockItemText1 and blockItemText2 fields with the items you'd like to add to your blocklist. The maximum length of a blockItem is 128 characters.
Optionally add more blockItem strings to the blockItems parameter.
Run the code.
Create a new Python script and open it in your preferred editor or IDE. Paste in the following code.
import os
from azure.ai.contentsafety import ContentSafetyClient
from azure.core.credentials import AzureKeyCredential
from azure.ai.contentsafety.models import (
TextBlockItemInfo,
AddBlockItemsOptions
)
from azure.core.exceptions import HttpResponseError
key = os.environ["CONTENT_SAFETY_KEY"]
endpoint = os.environ["CONTENT_SAFETY_ENDPOINT"]
# Create an Content Safety client
client = ContentSafetyClient(endpoint, AzureKeyCredential(key))
blocklist_name = "<your_list_name>"
block_item_text_1 = "<block_item_text_1>"
block_item_text_2 = "<block_item_text_2>"
block_items = [TextBlockItemInfo(text=block_item_text_1), TextBlockItemInfo(text=block_item_text_2)]
try:
result = client.add_block_items(
blocklist_name=blocklist_name,
body=AddBlockItemsOptions(block_items=block_items),
)
if result and result.value:
print("\nBlock items added: ")
for block_item in result.value:
print(f"BlockItemId: {block_item.block_item_id}, Text: {block_item.text}, Description: {block_item.description}")
except HttpResponseError as e:
print("\nAdd block items failed: ")
if e.error:
print(f"Error code: {e.error.code}")
print(f"Error message: {e.error.message}")
raise
print(e)
raise
Replace <your_list_name> with the name you used in the list creation step.
Replace the values of the block_item_text_1 and block_item_text_2 fields with the items you'd like to add to your blocklist. The maximum length of a blockItem is 128 characters.
Optionally add more blockItem strings to the block_items parameter.
Run the script.
Note
There will be some delay after you add or edit a blockItem before it takes effect on text analysis, usually not more than five minutes.
Copy the cURL command below to a text editor and make the following changes:
Replace <endpoint> with your endpoint URL.
Replace <enter_your_key_here> with your key.
Replace <your_list_name> with the name you used in the list creation step. The "blocklistNames" field can contain an array of multiple list IDs.
Optionally change the value of "breakByBlocklists". true indicates that once a blocklist is matched, the analysis will return immediately without model output. false will cause the model to continue to do analysis in the default categories.
Optionally change the value of the "text" field to whatever text you want to analyze.
curl --location --request POST '<endpoint>/contentsafety/text:analyze?api-version=2023-10-01&' \
--header 'Ocp-Apim-Subscription-Key: <enter_your_key_here>' \
--header 'Content-Type: application/json' \
--data-raw '{
"text": "I want to beat you till you bleed",
"categories": [
"Hate",
"Sexual",
"SelfHarm",
"Violence"
],
"blocklistNames":["<your_list_name>"],
"haltOnBlocklistHit": false,
"outputType": "FourSeverityLevels"
}'
The JSON response will contain a "blocklistMatchResults" that indicates any matches with your blocklist. It reports the location in the text string where the match was found.
Create a new C# console app and open it in your preferred editor or IDE. Paste in the following code.
string endpoint = Environment.GetEnvironmentVariable("CONTENT_SAFETY_ENDPOINT");
string key = Environment.GetEnvironmentVariable("CONTENT_SAFETY_KEY");
ContentSafetyClient client = new ContentSafetyClient(new Uri(endpoint), new AzureKeyCredential(key));
var blocklistName = "<your_list_name>";
// After you edit your blocklist, it usually takes effect in 5 minutes, please wait some time before analyzing with blocklist after editing.
var request = new AnalyzeTextOptions("<your_input_text>");
request.BlocklistNames.Add(blocklistName);
request.BreakByBlocklists = true;
Response<AnalyzeTextResult> response;
try
{
response = client.AnalyzeText(request);
}
catch (RequestFailedException ex)
{
Console.WriteLine("Analyze text failed.\nStatus code: {0}, Error code: {1}, Error message: {2}", ex.Status, ex.ErrorCode, ex.Message);
throw;
}
if (response.Value.BlocklistsMatchResults != null)
{
Console.WriteLine("\nBlocklist match result:");
foreach (var matchResult in response.Value.BlocklistsMatchResults)
{
Console.WriteLine("Blockitem was hit in text: Offset: {0}, Length: {1}", matchResult.Offset, matchResult.Length);
Console.WriteLine("BlocklistName: {0}, BlockItemId: {1}, BlockItemText: {2}, ", matchResult.BlocklistName, matchResult.BlockItemId, matchResult.BlockItemText);
}
}
Replace <your_list_name> with the name you used in the list creation step.
Replace the request input text with whatever text you want to analyze.
Run the script.
Create a new Python script and open it in your preferred editor or IDE. Paste in the following code.
import os
from azure.ai.contentsafety import ContentSafetyClient
from azure.core.credentials import AzureKeyCredential
from azure.ai.contentsafety.models import AnalyzeTextOptions
from azure.core.exceptions import HttpResponseError
key = os.environ["CONTENT_SAFETY_KEY"]
endpoint = os.environ["CONTENT_SAFETY_ENDPOINT"]
# Create an Content Safety client
client = ContentSafetyClient(endpoint, AzureKeyCredential(key))
blocklist_name = "<your_list_name>"
input_text = "<your_input_text>"
try:
# After you edit your blocklist, it usually takes effect in 5 minutes, please wait some time before analyzing with blocklist after editing.
analysis_result = client.analyze_text(AnalyzeTextOptions(text=input_text, blocklist_names=[blocklist_name], break_by_blocklists=False))
if analysis_result and analysis_result.blocklists_match_results:
print("\nBlocklist match results: ")
for match_result in analysis_result.blocklists_match_results:
print(f"Block item was hit in text, Offset={match_result.offset}, Length={match_result.length}.")
print(f"BlocklistName: {match_result.blocklist_name}, BlockItemId: {match_result.block_item_id}, BlockItemText: {match_result.block_item_text}")
except HttpResponseError as e:
print("\nAnalyze text failed: ")
if e.error:
print(f"Error code: {e.error.code}")
print(f"Error message: {e.error.message}")
raise
print(e)
raise
Replace <your_list_name> with the name you used in the list creation step.
Replace the input_text variable with whatever text you want to analyze.
Run the script.
Other blocklist operations
This section contains more operations to help you manage and use the blocklist feature.
Copy the cURL command below to a text editor and make the following changes:
Replace <endpoint> with your endpoint URL.
Replace <enter_your_key_here> with your key.
Replace <your_list_name> (in the request URL) with the name you used in the list creation step.
Replace <your_item_id> with the ID value for the blocklistItem. This is the value of the "blocklistItemId" field from the Add blocklistItem or Get all blocklistItems API calls.
Copy the cURL command below to a text editor and make the following changes:
Replace <endpoint> with your endpoint URL.
Replace <enter_your_key_here> with your key.
Replace <your_list_name> (in the request URL) with the name you used in the list creation step.
Replace <item_id> with the ID value for the blocklistItem. This is the value of the "blocklistItemId" field from the Add blocklistItem or Get all blocklistItems API calls.