How does Azure AI Language service detect the PII entities ?

Question

How does Azure AI Language service detect the PII entities ?

Mohsin Khan 60

Hey,

I have question related to PII detection, some of them detected based on the format and some of them detected based on the real data only.

As this leads to confusion to test the application with valid/invalid data.

Please suggest.

Regards,

Mohsin

Ravada Shivaprasad 535 Reputation points Microsoft External Staff Moderator

2025-05-14T22:48:05.56+00:00

Hi Mohsin Khan

Azure AI Language service employs a combination of techniques to detect Personally Identifiable Information (PII) entities within text, allowing for identification, categorization, and redaction of sensitive data in unstructured content. Detection follows two primary methods: format-based detection, where structured identifiers such as phone numbers, email addresses, and social security numbers are recognized based on predefined patterns, and data-based detection, which analyzes contextual information to identify PII that does not necessarily adhere to a fixed format.
To ensure accurate detection and assess model performance, testing should incorporate both valid data inputs and invalid data cases. Valid data testing involves providing real-world examples of PII—such as authentic phone numbers, email addresses, and identification numbers—to evaluate precision and reliability. Meanwhile, invalid data testing focuses on inputting fabricated or incorrectly structured information that mimics PII, helping identify how the system responds to ambiguous or misleading patterns. By implementing structured testing across both categories, organizations can effectively gauge the model’s accuracy and robustness in detecting and handling sensitive information.
Reference : Azure AI Language PII Detection Overview, How to Redact Text PII, and Native Document Support for PII Detection.
Please Let me know if you need further refinements!

Thanks
Mohsin Khan 60 Reputation points

2025-05-15T08:42:52.5866667+00:00

Hey @Ravada Shivaprasad,

Thank you for sharing.

This helps in further testing, Just quick do Microsoft have any link for the sample valid data with respective all categories mentioned ?

It is difficult to get all valid data for the all the categories mentioned.

https://learn.microsoft.com/en-us/azure/ai-services/language-service/personally-identifiable-information/concepts/entity-categories
Ravada Shivaprasad 535 Reputation points Microsoft External Staff Moderator

2025-05-21T19:43:59.58+00:00

Hi Mohsin Khan
Sorry for Delayed Response

Microsoft does not provide a direct repository of sample valid data for all PII detection categories. However, you can refer to multiple resources that offer insights into structuring test cases for PII detection. The Azure AI Language PII Detection Overview explains the predefined categories, detection methodologies, and redaction policies available within Azure AI Language.

You can use the Azure AI Language Samples GitHub Repository contains sample code and examples that can be useful for building structured test cases. For further guidance on integrating PII detection with external tools, Microsoft Presidio’s implementation with Azure AI Language, detailed in this guide, demonstrates how structured detection methodologies can be applied in real-world scenarios. These references should help streamline testing efforts and improve PII detection reliability.

Thanks
Ravada Shivaprasad 535 Reputation points Microsoft External Staff Moderator

2025-05-23T19:17:28.05+00:00

Hi Mohsin Khan

Just checking in to see if the below answer helped. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Thanks
Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Ravada Shivaprasad 535 Reputation points Microsoft External Staff Moderator

2025-05-26T22:43:08.16+00:00

Hi Mohsin Khan
Following up to see if the above answer was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.
Thanks

1 answer

Your answer

Ravada Shivaprasad 535 Reputation points Microsoft External Staff Moderator

2025-05-14T22:48:05.56+00:00

Hi Mohsin Khan

Azure AI Language service employs a combination of techniques to detect Personally Identifiable Information (PII) entities within text, allowing for identification, categorization, and redaction of sensitive data in unstructured content. Detection follows two primary methods: format-based detection, where structured identifiers such as phone numbers, email addresses, and social security numbers are recognized based on predefined patterns, and data-based detection, which analyzes contextual information to identify PII that does not necessarily adhere to a fixed format.
To ensure accurate detection and assess model performance, testing should incorporate both valid data inputs and invalid data cases. Valid data testing involves providing real-world examples of PII—such as authentic phone numbers, email addresses, and identification numbers—to evaluate precision and reliability. Meanwhile, invalid data testing focuses on inputting fabricated or incorrectly structured information that mimics PII, helping identify how the system responds to ambiguous or misleading patterns. By implementing structured testing across both categories, organizations can effectively gauge the model’s accuracy and robustness in detecting and handling sensitive information.
Reference : Azure AI Language PII Detection Overview, How to Redact Text PII, and Native Document Support for PII Detection.
Please Let me know if you need further refinements!

Thanks
Mohsin Khan 60 Reputation points

2025-05-15T08:42:52.5866667+00:00

Hey @Ravada Shivaprasad,

Thank you for sharing.

This helps in further testing, Just quick do Microsoft have any link for the sample valid data with respective all categories mentioned ?

It is difficult to get all valid data for the all the categories mentioned.

https://learn.microsoft.com/en-us/azure/ai-services/language-service/personally-identifiable-information/concepts/entity-categories
Ravada Shivaprasad 535 Reputation points Microsoft External Staff Moderator

2025-05-21T19:43:59.58+00:00

Hi Mohsin Khan
Sorry for Delayed Response

Microsoft does not provide a direct repository of sample valid data for all PII detection categories. However, you can refer to multiple resources that offer insights into structuring test cases for PII detection. The Azure AI Language PII Detection Overview explains the predefined categories, detection methodologies, and redaction policies available within Azure AI Language.

You can use the Azure AI Language Samples GitHub Repository contains sample code and examples that can be useful for building structured test cases. For further guidance on integrating PII detection with external tools, Microsoft Presidio’s implementation with Azure AI Language, detailed in this guide, demonstrates how structured detection methodologies can be applied in real-world scenarios. These references should help streamline testing efforts and improve PII detection reliability.

Thanks
Ravada Shivaprasad 535 Reputation points Microsoft External Staff Moderator

2025-05-23T19:17:28.05+00:00

Hi Mohsin Khan

Just checking in to see if the below answer helped. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Thanks
Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Ravada Shivaprasad 535 Reputation points Microsoft External Staff Moderator

2025-05-26T22:43:08.16+00:00

Hi Mohsin Khan
Following up to see if the above answer was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.
Thanks

Answer 1

Shikha Ghildiyal 6,615 Microsoft Employee Moderator

Hi Mohsin Khan,

Thanks for reaching out to Microsoft Q&A.

. Azure AI Language PII detection uses Named Entity Recognition (NER) to identify and redact sensitive information from input data. The service classifies sensitive personal data into predefined categories. These categories include phone numbers, email addresses, and identification documents. This classification helps to efficiently detect and eliminate such information.

Kindly refer more details here- https://learn.microsoft.com/en-us/azure/ai-services/language-service/personally-identifiable-information/overview?tabs=text-pii

Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.

Mohsin Khan 60 Reputation points

2025-05-14T09:17:12.25+00:00

Hey

Thank you for the response, currently for this categoryKRResidentRegistrationNumber data is detected as format based there is no check sum logic is validating, but for PTCitizenCardNumber category it is validating data based on the check sum validation.

Above is just reference example, there are so many categories which have different validation logic.

Also is there any link for sample data which help to test all model categories ?
Ravada Shivaprasad 535 Reputation points Microsoft External Staff Moderator

2025-05-26T22:42:57.2933333+00:00

Hi Mohsin Khan

Azure AI Language PII detection indeed applies different validation logic across categories. Some, like KRResidentRegistrationNumber, rely on format-based detection, while others, such as PTCitizenCardNumber, incorporate checksum validation.

For a comprehensive list of supported PII entity categories, you can refer to this documentation. Additionally, if you're looking for sample data to test various model categories, you might find useful examples in the Microsoft Presidio integration guide, which demonstrates Azure AI Language's Named Entity Recognition (NER) capabilities.

Thanks

Share via

How does Azure AI Language service detect the PII entities ?

1 answer

Your answer