Azure AI Language PII/PHI detection would not detect PII properly

Question

Azure AI Language PII/PHI detection would not detect PII properly

Mohsin Khan 60

As refer from the link below, most of the PII/PHI are not detected even following the correct format and applying the country-specific language.

What is the correct way or sample data that can be used to detect below categories during PII detection (applied text format did not work).

https://learn.microsoft.com/en-us/azure/ai-services/language-service/personally-identifiable-information/concepts/entity-categories

Prashanth Veeragoni 4,440 Reputation points Microsoft External Staff Moderator

2025-05-08T02:13:54.2133333+00:00
Hi Mohsin Khan,

Azure AI Language's PII/PHI detection sometimes doesn't identify expected entities, even when text formatting seems correct and the country-specific model is applied.

Azure AI Language uses pre-trained models for PII detection, but its accuracy depends on:

1.      Input formatting (e.g., punctuation, spacing, context).

2.      Country-specific configuration (US, UK, etc.).

3.      Language set (English is best supported).

4.      Confidence score thresholds (default is often 0.5 or 0.65).

5.      Model limits — it may miss data types not strongly represented in its training data.

Steps to Resolve:

1.Ensure Country is Properly Set

Specify the country code in the API call (e.g., "US" for U.S.). Some PII types are only supported in specific regions.

"parameters": { "domain": "phi", "piiCategories": ["All"], "countryHint": "US" }

2.Raise the Confidence Threshold

Lowering the threshold can help you see borderline detections:

"piiCategoriesFilter": ["All"], "confidenceScoreThreshold": 0.3

3.Use Azure's "PHI" Domain Instead of Generic

If you're detecting PHI (HIPAA-related), make sure you select the "phi" domain model:

POST https://<region>.api.cognitive.microsoft.com/text/analytics/v3.2-preview.2/entities/recognition/pii

Body:

{ "documents": [ { "id": "1", "language": "en", "text": "The patient’s MRN is 12345678. His SSN is 123-45-6789." } ], "parameters": { "domain": "phi", "piiCategories": ["All"] } }

Hope this helps, do let me know if you have any further queries.

Thank you!
Mohsin Khan 60 Reputation points

2025-05-08T03:52:42.2133333+00:00

Thank you @Prashanth Veeragoni for the response.

Yes I have followed all the recommended approach as mentioned even normalized the text before sending to PII detection process. Is there any link which I can refer for test data to be input include all the supported category ?

Also there is country Luxembourg (few more others) it is mentioned to be supported in category list (https://learn.microsoft.com/en-us/azure/ai-services/language-service/personally-identifiable-information/concepts/entity-categories) but the list mentioned below does not have country code for PII.
https://learn.microsoft.com/en-us/azure/ai-services/language-service/concepts/language-support

https://learn.microsoft.com/en-us/azure/ai-services/language-service/personally-identifiable-information/language-support?tabs=text

Note : Enabling PHI does not detect Passport numbers, Tax IDs.
Prashanth Veeragoni 4,440 Reputation points Microsoft External Staff Moderator

2025-05-08T04:39:37.5833333+00:00

Mohsin Khan,

Thanks for the added detail—this is known limitation and a documentation gap in Azure AI Language's PII detection capabilities.

Microsoft does not currently publish a comprehensive sample input dataset that:

·       Contains examples for each supported PII/PHI category.

·       Shows valid formats per country (e.g., Tax IDs for Luxembourg or Passport numbers for the UK).

You’re right—Luxembourg (LU) is mentioned in the PII category support table, but:

·       It’s missing from the PII Language Support.

·       It’s also not present in the broader Language Support.

This inconsistency likely means:

·       Partial support (e.g., LU may be supported in the model architecture, but not actively optimized/tested).

·       Categories like passport numbers, tax IDs, or driver's licenses for LU may not be triggered due to lack of training data or rules.

Hope this helps.

Thank you!
Mohsin Khan 60 Reputation points

2025-05-08T05:54:41.3533333+00:00

Hey Prashanth Veeragoni,

Thank you for the sharing and clarification the doubts,

With similar to LU does there are other categories which are partial supported, will be great help to craft the solution we are implementing.

Please share any list if available.
Prashanth Veeragoni 4,440 Reputation points Microsoft External Staff Moderator

2025-05-08T06:25:43.9133333+00:00

Hi Mohsin Khan,

Thank you for your follow-up. Your observation regarding Luxembourg (LU) and similar countries is valid. While the PII entity category documentation lists LU among supported regions, the PII language support page does not include LU in its country code list. This inconsistency suggests that certain countries may have partial or limited support for PII detection.

Recommendations for Handling Partial Support:

· Explicitly Specify PII Categories: When making API calls, use the piiCategories parameter to define the specific PII categories you want to detect. This approach can help ensure that the model attempts to identify the desired entities, even in regions with partial support.

· Utilize Supported Language Codes: If you're working with data from countries with limited support, consider using a supported language code (e.g., en for English) in your API requests. This can sometimes enhance detection accuracy, especially if the content is in English.

· Implement Custom Detection Mechanisms: For critical PII entities not reliably detected by Azure's built-in capabilities, consider implementing custom detection methods, such as regular expressions or third-party libraries, to supplement Azure's PII detection.

· Stay Informed on Updates: Regularly check Azure's documentation and release notes for updates on PII detection capabilities and supported regions. Microsoft may expand support over time, which could benefit your use case.

Hope this helps.

Thank you!
Mohsin Khan 60 Reputation points

2025-05-08T07:26:21.4166667+00:00

Hey @Prashanth Veeragoni,

Thank you for sharing.

Yes, I have already followed mentioned dos and dont's for PII detection with Azure AI.

Only requesting to provide list of categories those are partially supported or any note which state the clarification on this.

Thanks
Mohsin Khan 60 Reputation points

2025-05-09T03:31:15.66+00:00

Thank you for the clarification, This will help further.
Prashanth Veeragoni 4,440 Reputation points Microsoft External Staff Moderator

2025-05-09T03:36:11.1466667+00:00

Hi Mohsin Khan,

Glad to hear that and as it is helpful can you please take some time to up-vote and accept my previous response which I converted to actual answer. your feedback is really appriciated.

Thank you!

Accepted answer

0 additional answers

Your answer

Prashanth Veeragoni 4,440 Reputation points Microsoft External Staff Moderator

2025-05-08T02:13:54.2133333+00:00

Hi Mohsin Khan,

Azure AI Language's PII/PHI detection sometimes doesn't identify expected entities, even when text formatting seems correct and the country-specific model is applied.

Azure AI Language uses pre-trained models for PII detection, but its accuracy depends on:

1. Input formatting (e.g., punctuation, spacing, context).

2. Country-specific configuration (US, UK, etc.).

3. Language set (English is best supported).

4. Confidence score thresholds (default is often 0.5 or 0.65).

5. Model limits — it may miss data types not strongly represented in its training data.

Steps to Resolve:

1.Ensure Country is Properly Set

Specify the country code in the API call (e.g., "US" for U.S.). Some PII types are only supported in specific regions.

"parameters": { "domain": "phi", "piiCategories": ["All"], "countryHint": "US" }

2.Raise the Confidence Threshold

Lowering the threshold can help you see borderline detections:

"piiCategoriesFilter": ["All"], "confidenceScoreThreshold": 0.3

3.Use Azure's "PHI" Domain Instead of Generic

If you're detecting PHI (HIPAA-related), make sure you select the "phi" domain model:

POST https://<region>.api.cognitive.microsoft.com/text/analytics/v3.2-preview.2/entities/recognition/pii

Body:

{ "documents": [ { "id": "1", "language": "en", "text": "The patient’s MRN is 12345678. His SSN is 123-45-6789." } ], "parameters": { "domain": "phi", "piiCategories": ["All"] } }

Hope this helps, do let me know if you have any further queries.

Thank you!
Mohsin Khan 60 Reputation points

2025-05-08T03:52:42.2133333+00:00

Thank you @Prashanth Veeragoni for the response.

Yes I have followed all the recommended approach as mentioned even normalized the text before sending to PII detection process. Is there any link which I can refer for test data to be input include all the supported category ?

Also there is country Luxembourg (few more others) it is mentioned to be supported in category list (https://learn.microsoft.com/en-us/azure/ai-services/language-service/personally-identifiable-information/concepts/entity-categories) but the list mentioned below does not have country code for PII.
https://learn.microsoft.com/en-us/azure/ai-services/language-service/concepts/language-support

https://learn.microsoft.com/en-us/azure/ai-services/language-service/personally-identifiable-information/language-support?tabs=text

Note : Enabling PHI does not detect Passport numbers, Tax IDs.
Prashanth Veeragoni 4,440 Reputation points Microsoft External Staff Moderator

2025-05-08T04:39:37.5833333+00:00

Mohsin Khan,

Thanks for the added detail—this is known limitation and a documentation gap in Azure AI Language's PII detection capabilities.

Microsoft does not currently publish a comprehensive sample input dataset that:

· Contains examples for each supported PII/PHI category.

· Shows valid formats per country (e.g., Tax IDs for Luxembourg or Passport numbers for the UK).

You’re right—Luxembourg (LU) is mentioned in the PII category support table, but:

· It’s missing from the PII Language Support.

· It’s also not present in the broader Language Support.

This inconsistency likely means:

· Partial support (e.g., LU may be supported in the model architecture, but not actively optimized/tested).

· Categories like passport numbers, tax IDs, or driver's licenses for LU may not be triggered due to lack of training data or rules.

Hope this helps.

Thank you!
Mohsin Khan 60 Reputation points

2025-05-08T05:54:41.3533333+00:00

Hey Prashanth Veeragoni,

Thank you for the sharing and clarification the doubts,

With similar to LU does there are other categories which are partial supported, will be great help to craft the solution we are implementing.

Please share any list if available.
Prashanth Veeragoni 4,440 Reputation points Microsoft External Staff Moderator

2025-05-08T06:25:43.9133333+00:00

Hi Mohsin Khan,

Thank you for your follow-up. Your observation regarding Luxembourg (LU) and similar countries is valid. While the PII entity category documentation lists LU among supported regions, the PII language support page does not include LU in its country code list. This inconsistency suggests that certain countries may have partial or limited support for PII detection.

Recommendations for Handling Partial Support:

· Explicitly Specify PII Categories: When making API calls, use the piiCategories parameter to define the specific PII categories you want to detect. This approach can help ensure that the model attempts to identify the desired entities, even in regions with partial support.

· Utilize Supported Language Codes: If you're working with data from countries with limited support, consider using a supported language code (e.g., en for English) in your API requests. This can sometimes enhance detection accuracy, especially if the content is in English.

· Implement Custom Detection Mechanisms: For critical PII entities not reliably detected by Azure's built-in capabilities, consider implementing custom detection methods, such as regular expressions or third-party libraries, to supplement Azure's PII detection.

· Stay Informed on Updates: Regularly check Azure's documentation and release notes for updates on PII detection capabilities and supported regions. Microsoft may expand support over time, which could benefit your use case.

Hope this helps.

Thank you!
Mohsin Khan 60 Reputation points

2025-05-08T07:26:21.4166667+00:00

Hey @Prashanth Veeragoni,

Thank you for sharing.

Yes, I have already followed mentioned dos and dont's for PII detection with Azure AI.

Only requesting to provide list of categories those are partially supported or any note which state the clarification on this.

Thanks
Mohsin Khan 60 Reputation points

2025-05-09T03:31:15.66+00:00

Thank you for the clarification, This will help further.
Prashanth Veeragoni 4,440 Reputation points Microsoft External Staff Moderator

2025-05-09T03:36:11.1466667+00:00

Hi Mohsin Khan,

Glad to hear that and as it is helpful can you please take some time to up-vote and accept my previous response which I converted to actual answer. your feedback is really appriciated.

Thank you!

Answer 1

Hi Mohsin Khan,

Thank you for your continued inquiries regarding Azure AI Language's PII detection capabilities. Based on the available documentation and transparency notes, here's a consolidated overview to address your concerns:

Azure AI Language's PII detection feature supports a variety of entity categories, such as Social Security Numbers, Passport Numbers, Tax IDs, and more. However, the detection accuracy and support for these categories can vary based on the language and locale specified in the API request.

While the Supported PII Entity Categories documentation lists the types of entities that can be detected, it does not provide a detailed mapping of which categories are fully or partially supported per country or language.

Furthermore, the PII Language Support page outlines the languages supported for PII detection. However, it's important to note that:

· Not all entity categories are supported in all languages.

· Detection capabilities may be limited or partial for certain locales.

For instance, while Luxembourg (LU) is mentioned in the entity categories documentation, it is not listed in the language support page, indicating potential partial support.

I found one more supported document: Transparency note for Personally Identifiable Information (PII)

Hope this helps.

Thank you!

Share via

Azure AI Language PII/PHI detection would not detect PII properly

0 additional answers

Your answer