DLP for South African ID Number

Lethukuthula Shabalala 21

Hi All,

I need assistance in creating a functional Sensitive Info Type (SIT) to prevent the unintended or unauthorized sharing of South African ID numbers. I’ve tried using the existing SIT, but it doesn't detect the ID numbers during testing. Additionally, I attempted to create a custom SIT using a regular expression, and although I've found some expressions that are typically used to validate South African ID numbers, they aren't working as expected. The regular expressions work fine on https://regex101.com, but when I apply them in the SIT configuration, they fail.

Any guidance would be greatly appreciated!

vaibhav kamra 0 Reputation points

2024-11-20T12:40:07.4066667+00:00
When using custom SIT, some regular expressions might:

Fail due to unsupported constructs in Microsoft's regex engine.

Require additional context to reduce false positives or improve accuracy.

Create a Custom SIT

Follow these steps to create a custom SIT to detect South African ID numbers:

Step 1: Build a Simplified Regex

Start with a regex tailored for Microsoft's SIT capabilities:

Basic Regex: \b\d{2}(0[1-9]|1[0-2])(0[1-9]|[12][0-9]|3[01])\d{4}[01]8\d\b

This regex checks:

13 digits total.

The date portion (YYMMDD) matches valid months and days.

Citizenship (C) is either 0 or 1.

Reserved field (A) is 8.

Add a Supporting Element

Microsoft SIT often requires confidence boosters (keywords, patterns) to enhance detection accuracy. Examples:

Keywords: "South African ID", "RSA ID", "ZA ID number".

Proximity: Ensure these keywords are near the potential ID numbers (e.g., within 50 characters).

Configure the Custom SIT

Navigate to Microsoft Purview Compliance Portal:

Go to Compliance > Data Classification > Sensitive Info Types.

Click Create a sensitive info type.

Define the SIT:

Name: South African ID

Description: Custom SIT to detect South African ID numbers.

Add the Regular Expression:

In the Pattern section, add the regex: \b\d{2}(0[1-9]|1[0-2])(0[1-9]|[12][0-9]|3[01])\d{4}[01]8\d\b

Set confidence level to medium or high based on testing.

Add Keywords as Supporting Elements:

Include keywords like "South African ID" or "ZA ID" as context.

Set Proximity:

Configure proximity for keywords (e.g., 50 characters).

Save and Publish.When using custom SIT, some regular expressions might:

Fail due to unsupported constructs in Microsoft's regex engine.

Require additional context to reduce false positives or improve accuracy.

3. Create a Custom SIT Follow these steps to create a custom SIT to detect South African ID numbers: Step 1: Build a Simplified Regex Start with a regex tailored for Microsoft's SIT capabilities:

Basic Regex: \b\d{2}(0[1-9]|1[0-2])(0[1-9]|[12][0-9]|3[01])\d{4}[01]8\d\b

This regex checks:

13 digits total.

The date portion (YYMMDD) matches valid months and days.

Citizenship (C) is either 0 or 1.

Reserved field (A) is 8.

Step 2: Add a Supporting Element Microsoft SIT often requires confidence boosters (keywords, patterns) to enhance detection accuracy. Examples:

Keywords: "South African ID", "RSA ID", "ZA ID number".

Proximity: Ensure these keywords are near the potential ID numbers (e.g., within 50 characters).

Step 3: Configure the Custom SIT

Navigate to Microsoft Purview Compliance Portal:

Go to Compliance > Data Classification > Sensitive Info Types.

Click Create a sensitive info type.

Define the SIT:

Name: South African ID

Description: Custom SIT to detect South African ID numbers.

Add the Regular Expression:

In the Pattern section, add the regex:
\b\d{2}(0[1-9]|1[0-2])(0[1-9]|[12][0-9]|3[01])\d{4}[01]8\d\b

Set confidence level to medium or high based on testing.

Add Keywords as Supporting Elements:

Include keywords like "South African ID" or "ZA ID" as context.

Set Proximity:

Configure proximity for keywords (e.g., 50 characters).

Save and Publish.

Let me know if any extra help is needed
phemanth 11,975 Reputation points Microsoft Vendor

2024-11-22T19:03:18.8+00:00

@Lethukuthula Shabalala We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Lethukuthula Shabalala 21 Reputation points

2024-11-23T14:31:09.04+00:00

Thank you for your response. But, the provided solution appears to be repeated. No offense intended, but it seems like a suggested solution from ChatGPT or a similar AI tool. I did attempt the mentioned regex before reaching out for further assistance.

After further investigation, I discovered that Microsoft Purview utilizes a subset of .NET regex, which can cause certain features to behave differently. For instance, I had to avoid using \b (word boundary) and instead opted for (?<!\d) as an opener and (?!\d) as a closer for the regex.

Here’s an example comparing the original regex we use on Mimecast and the modified version for Purview:

Original Regex (Mimecast):

\b(([0-57-9]\d(0[1-9]|1[012]))|(61-9)|(601[012]))(0[1-9]|[12][0-9]|3[01])[ -]?\d\d\d\d[ -]?\d\d\d\b

Modified Regex (Purview):

(?<!\d)(([0-57-9]\d(0[1-9]|1[0-2]))|(61-9)|(601[0-2]))(0[1-9]|[12][0-9]|3[01])[ -]?\d{4}[ -]?\d{3}(?!\d)

The modified regex successfully worked in my SIT environment and can distinguish between valid and invalid ID numbers.

Thank you once again, and I appreciate the community's continued support.
phemanth 11,975 Reputation points Microsoft Vendor

2024-11-27T15:02:52.75+00:00

@Lethukuthula Shabalala Glad to know your issue has been resolved. Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others "I'll repost your solution in case you'd like to accept the answer.

Accepted answer

phemanth 11,975 Reputation points Microsoft Vendor

2024-11-27T15:01:07.52+00:00

@Lethukuthula Shabalala

I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to accept the answer .

Ask: I need assistance in creating a functional Sensitive Info Type (SIT) to prevent the unintended or unauthorized sharing of South African ID numbers. I’ve tried using the existing SIT, but it doesn't detect the ID numbers during testing. Additionally, I attempted to create a custom SIT using a regular expression, and although I've found some expressions that are typically used to validate South African ID numbers, they aren't working as expected. The regular expressions work fine on https://regex101.com, but when I apply them in the SIT configuration, they fail.

**Solution:**After further investigation, I discovered that Microsoft Purview utilizes a subset of .NET regex, which can cause certain features to behave differently. For instance, I had to avoid using \b (word boundary) and instead opted for (?<!\d) as an opener and (?!\d) as a closer for the regex.

Here’s an example comparing the original regex we use on Mimecast and the modified version for Purview:

Original Regex (Mimecast):

\b(([0-57-9]\d(0[1-9]|1[012]))|(61-9)|(601[012]))(0[1-9]|[12][0-9]|3[01])[ -]?\d\d\d\d[ -]?\d\d\d\b

Modified Regex (Purview):

(?<!\d)(([0-57-9]\d(0[1-9]|1[0-2]))|(61-9)|(601[0-2]))(0[1-9]|[12][0-9]|3[01])[ -]?\d{4}[ -]?\d{3}(?!\d)

The modified regex successfully worked in my SIT environment and can distinguish between valid and invalid ID numbers.

Thank you once again, and I appreciate the community's continued support.

If I missed anything please let me know and I'd be happy to add it to my answer, or feel free to comment below with any additional information.

If you have any other questions, please let me know. Thank you again for your time and patience throughout this issue.

Please don’t forget to Accept Answer and Yes for "was this answer helpful" wherever the information provided helps you, this can be beneficial to other community members.
Please sign in to rate this answer.

1 person found this answer helpful.

0 comments No comments
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

DLP for South African ID Number

0 additional answers

Your answer