Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Important
This feature is in Beta. Account admins can control access to this feature from the account console Previews page. See Manage Azure Databricks previews.
Sensitive Data Detection is a built-in service policy that finds sensitive data, such as personally identifiable information (PII), in requests and responses and either blocks the interaction or redacts the matched value in place. Unlike the LLM-as-a-judge built-in policies, it's deterministic and adds little latency.
You attach Sensitive Data Detection to a Model Service or Model Provider Service. You choose which categories to look for and whether a match blocks the interaction or redacts the value.
Attach the policy
Note
During the Beta, the policy UI might change. If a label differs from these steps, follow the in-product labels. The Principals and scope fields aren't configurable yet: a policy applies to all account users on the service.
You attach Sensitive Data Detection through the Unity AI Gateway UI, the same way you attach any built-in policy. You need MANAGE on the target service. During the Beta, built-in policies are Azure Databricks-managed and don't appear as functions you can browse or grant on in the system.ai schema, so there's no EXECUTE privilege to set; you select the policy by name in the UI.
- In the workspace sidebar, click AI Gateway.
- Select the service to govern. Either a model service on the Models tab, or a model provider service on the Providers tab.
- Open the Policies tab, then click New policy.
- Enter a Name for the policy.
- In Guardrail type, select Sensitive Data Detection.
- Select the Classification tags to detect, and choose whether a match blocks the interaction or redacts the matched values.
- Under Phase, select Input, Output, or both.
- Set the Rank to control evaluation order relative to other policies on the service. See Order of evaluation.
- Click Create policy.
The policy appears on the service's Policies tab. During the Beta, allow a short time for it to propagate before you test.
How the policy works
When you attach the policy, you set three things:
- Categories (required): the types of sensitive data to detect. See Supported categories.
- Action: what happens on a match.
- Block: Azure Databricks denies the interaction. Instead of an error, the caller receives a successful (HTTP 200) response whose assistant turn names the service policy that blocked, and a top-level
databricks_service_policyobject carries the blockreason, which names the matched categories, for exampleDetected sensitive data (US_SSN, CREDIT_CARD). - Redact: Azure Databricks replaces each matched value with a placeholder token, such as
[US_SSN], and forwards the rewritten content. On a request, the model receives the redacted text. On a response, the caller receives it. The built-in policy transforms content rather than only returning a decision.
- Block: Azure Databricks denies the interaction. Instead of an error, the caller receives a successful (HTTP 200) response whose assistant turn names the service policy that blocked, and a top-level
- Phase: whether the policy runs on the input, the output, or both.
Because detection is deterministic, Azure Databricks tunes blocking and redaction separately. Redaction fires on the category's format, plus a checksum where one applies, which favors catching more. Blocking is stricter for categories that are just a run of digits, since it requires a checksum or a nearby context word, so a random number doesn't deny a request. Distinctive formats such as email and IP addresses block on the match alone.
Supported categories
Sensitive Data Detection detects the following 15 categories, grouped by jurisdiction. In the Classification tags field, select the tags you want to detect. Each is listed in the Tag column below, for example, class.us_ssn. See Data Classification system tags for accepted tag values.
The Detection method column shows how Azure Databricks identifies each category:
- Regex: a pattern match on the value's format.
- Checksum: a validation check on the matched value, such as the Luhn check on a card number, that rejects values that fit the format but can't be valid.
- Context keywords: a related word must appear near the match, for example "SSN" near a nine-digit number. Because a bare number is ambiguous, these categories require a nearby keyword to block, though redaction still fires on the format alone.
Global identifiers
| Tag | Detects | Detection method |
|---|---|---|
class.email_address |
Email address | Regex |
class.ip_address |
IP address | Regex |
class.mac_address |
MAC address | Regex |
class.vin |
Vehicle Identification Number | Regex + check digit + context keywords |
class.credit_card |
Credit card number | Regex + Luhn checksum |
class.iban_code |
International Bank Account Number | Regex + ISO 7064 checksum |
class.phone_number |
Phone number | Regex + context keywords |
United States
| Tag | Detects | Detection method |
|---|---|---|
class.us_ssn |
US Social Security Number | Regex + context keywords |
class.us_itin |
US Individual Taxpayer Identification Number | Regex + context keywords |
class.us_passport |
US passport | Regex + context keywords |
class.us_bank_number |
US bank account number | Regex + context keywords |
United Kingdom
| Tag | Detects | Detection method |
|---|---|---|
class.uk_nhs |
UK NHS number | Regex + mod-11 checksum + context keywords |
class.uk_nino |
UK National Insurance Number | Regex |
India
| Tag | Detects | Detection method |
|---|---|---|
class.in_pan |
Indian Permanent Account Number | Regex |
class.in_aadhaar |
Indian Aadhaar number | Regex + Verhoeff checksum + context keywords |
What isn't detected
Sensitive Data Detection covers structured data that a pattern can match reliably. It doesn't detect:
- Names, locations, and organizations, and other free-text entities that need language understanding to identify. Detecting these reliably requires a model, not a pattern. To cover this content, see Block other sensitive data with an LLM judge.
- Identifiers that are only a run of digits with no checksum or standard format, such as some national IDs and device identifiers. Matching these on shape alone would flag too much ordinary text.
Detection accuracy
Sensitive Data Detection is tuned differently for its two actions. Redaction should catch as much sensitive data as possible, since a missed value is worse than an over-redaction, so it is optimized for recall. A block denies the request outright, so it should fire only when confident, and is optimized for precision.
On public benchmarks, block precision is 0.99 across the 15 categories, so blocks rarely fire on non-sensitive content, and redaction recall is 0.96. Recall is at or near 1.0 for most categories; it is lower for a few types, such as US passport numbers, where the benchmark data carries heavy format variety rather than the detector missing well-formed values.
Detection is deterministic and fast. It adds well under 50 ms to a request, even on a large (100-turn) conversation.
These numbers come from public sources: the AI4Privacy pii-masking-200k dataset, the Faker generator, and values built from each identifier's published check-digit and format specifications (for example, the Aadhaar Verhoeff and UK NHS mod-11 checks).
Block other sensitive data with an LLM judge
To also catch the names and other free-text data that Sensitive Data Detection can't identify, add a custom LLM-as-a-judge guardrail to the same service:
- Keep Sensitive Data Detection at a lower rank to redact structured data.
- Add the LLM judge at a higher rank so it evaluates the already-redacted content.
- In the judge's instruction, describe the sensitive data to flag, and tell it to ignore the redaction placeholders (such as
[US_SSN]) left behind by the first guardrail.
Example judge instruction
You are a data-privacy classifier for an AI gateway guardrail. Decide whether the content contains personal information about a private individual, such as a person's name (including a first name used to address or refer to someone), home or personal contact details, or other identifiers tied to that individual.
Set flagged to true whenever the content names or refers to a private individual, even if only a first name is given and no other identifying details are present. For example, a message addressed to "Kattie" or one that mentions a colleague by name contains personal information.
Do not flag:
- Public figures, executives, politicians, celebrities, or historical people named in a public, business, or newsworthy context (for example, Warren Buffett, Elon Musk).
- Companies, brands, organizations, funds, or institutions (for example, Morgan Stanley, Apple, Goldman Sachs).
- Redaction placeholders that already mask a value, such as [EMAIL_ADDRESS], [US_SSN], [PHONE_NUMBER], or any similar bracketed category token in square brackets; these have already been handled and are not personal information.
- Obvious placeholders that do not refer to a real person (for example, "John Doe", "test user").
- Content with no reference to an individual.
Limitations
The following limitations apply during the Beta:
- Sensitive Data Detection applies to model services and model provider services.
- Only the categories listed in Supported categories are supported. Free-text entities such as names, locations, or organizations are not supported. See What isn't detected.