Metastore-level ABAC policies (Beta)

Important

Metastore-level attribute-based access control (ABAC) policies are in Beta. You can attach ABAC policies at the metastore, the root of the Unity Catalog object hierarchy, so a single policy applies across every catalog in the metastore.

The metastore is a valid scope for ABAC policies. A policy attached to the metastore applies to every catalog, schema, and object supported by each policy type within the metastore, including objects created after you define the policy. This gives governance teams one place to define organization-wide data protection rules, without configuring or replicating a policy for each catalog. A metastore can't be tagged directly, so tag-based policies match objects by the governed tags applied to catalogs and lower-level objects.

All four ABAC policy types are supported at the metastore level: row filter, column mask, GRANT, and DENY (Beta). A metastore-level policy uses the same fields, conditions, and evaluation behavior as any other ABAC policy. Only its scope differs.

You can create and manage metastore-level policies with Catalog Explorer, SQL, or the REST API. For an overview of ABAC and core concepts, including governed tags and built-in functions such as has_tag and has_tag_value, see Core concepts for attribute-based access control (ABAC).

Requirements

Compute requirements

Using SQL to create, modify, or drop metastore-level policies requires compute running Databricks Runtime 19 or above.

Permission requirements

  • Creating, updating, and deleting metastore-level policies requires the metastore admin role.
  • Listing and viewing metastore-level policies requires READ METADATA or the metastore admin role.

How metastore-level policies work

A metastore-level policy inherits down the object hierarchy using the same rules as other ABAC policies:

  • A single policy covers the whole metastore. A policy attached to the metastore evaluates against every object of the type named in the FOR clause across all catalogs and schemas in the metastore.
  • New catalogs are covered automatically. A catalog created after the policy is defined is covered when it matches the policy's conditions, with no action from the catalog owner or admin.
  • Behavior matches lower-level policies. Metastore-level policies preserve the standard capabilities and evaluation behavior of each policy type, such as DENY precedence, workspace bindings, and row filter composition. There is no evaluation logic unique to the metastore scope.

A metastore can't be tagged directly, so if a metastore-level policy uses a tag condition, apply the governed tag to each catalog or lower-level object you want the policy to match.

Supported policy targets

A metastore-level policy can apply to (FOR) every securable type that its policy type supports at the catalog level:

GRANT and DENY policies can't target non-catalog objects in the metastore, such as external locations, storage credentials, shares, recipients, providers, and connections. See Limitations.

Create a metastore-level policy

You can create a metastore-level policy using the Catalog Explorer UI, the CREATE POLICY SQL statement, or the REST API.

Catalog Explorer

  1. In your Azure Databricks workspace, click Data icon. Catalog.
  2. At the top of the Catalog pane, click the Gear icon. gear icon and select Metastore.
  3. Click the Policies tab.
  4. Click New policy.
  5. Fill in the policy fields required for your selected Policy type. The Scope field is set to the current metastore.
  6. Click Create policy.

SQL

Attach the policy with ON METASTORE. The policy targets the metastore you are currently operating in. Do not specify a metastore name. See Limitations.

The following column mask policy masks every column tagged ssn across all tables in the metastore, for all account users except the HR admins group:

CREATE POLICY ssn_mask
ON METASTORE
COLUMN MASK ssn_to_last_nr
TO `account users` EXCEPT `HR admins`
FOR TABLES
MATCH COLUMNS has_tag('ssn') AS ssn
ON COLUMN ssn
USING COLUMNS (4);

The following DENY policy prevents non-admins from managing access control on every table tagged sensitive across the metastore. It applies to all account users except the data_admins group:

CREATE POLICY managed_access
ON METASTORE
COMMENT 'Prevent non-admins from managing permissions on sensitive tables'
TO `account users` EXCEPT `data_admins`
DENY MANAGE ACCESS CONTROL
FOR TABLES
WHEN has_tag('sensitive');

For the full SQL syntax, see CREATE POLICY.

REST API

Set on_securable_type to METASTORE and on_securable_fullname to your current metastore's name. The rest of the payload is identical to a catalog-scoped policy.

The following example creates the DENY policy shown in the SQL tab:

curl -X POST "https://${DATABRICKS_HOST}/api/2.1/unity-catalog/policies" \
  -H "Authorization: Bearer ${DATABRICKS_TOKEN}" \
  -H "Content-Type: application/json" \
  --data-binary @- << 'EOF'
{
  "name": "managed_access",
  "comment": "Prevent non-admins from managing permissions on sensitive tables",
  "on_securable_type": "METASTORE",
  "on_securable_fullname": "my_metastore",
  "for_securable_type": "TABLE",
  "policy_type": "POLICY_TYPE_DENY",
  "to_principals": ["account users"],
  "except_principals": ["data_admins"],
  "deny": {
    "privileges": ["MANAGE_ACCESS_CONTROL"]
  },
  "when_condition": "has_tag('sensitive')"
}
EOF

on_securable_fullname accepts only the name of the metastore you are currently operating in. A metastore ID, or the name of a different metastore, is rejected. To find your metastore's name, click Gear icon. the gear icon at the top of the Catalog pane in Catalog Explorer and select Metastore. For request and response details, see Create policy in the REST API reference.

Delete a metastore-level policy

Catalog Explorer

Manage metastore-level policies from the Policies tab of the metastore.

  1. In your Azure Databricks workspace, click Data icon. Catalog.
  2. At the top of the Catalog pane, click the Gear icon. gear icon and select Metastore.
  3. Click the Policies tab.
  4. Select the policy.
  5. Click Delete policy.

SQL

To delete a metastore-level policy with SQL, run DROP POLICY:

DROP POLICY ssn_mask ON METASTORE;

REST API

curl -X DELETE "https://${DATABRICKS_HOST}/api/2.1/unity-catalog/policies/METASTORE/my_metastore/ssn_mask" \
  -H "Authorization: Bearer ${DATABRICKS_TOKEN}"

Show policies

Use SHOW POLICIES to list the policies defined on a securable object. Use SHOW EFFECTIVE POLICIES to also include policies inherited from parent scopes, such as metastore-level policies that affect a catalog.

SHOW [EFFECTIVE] POLICIES ON METASTORE

The result includes the policy name, policy type, the catalog and schema of the securable each policy is defined on, and that securable's type and full name. Metastore-level policies are returned with on_securable_type set to METASTORE and on_securable_fullname set to the metastore name. The Table column is populated only when a policy is defined on a table.

Example:

SHOW POLICIES ON METASTORE;

Policy Name     Policy Type  Catalog  Schema  Table  Comment                                                           on_securable_type  on_securable_fullname
--------------  -----------  -------  ------  -----  ----------------------------------------------------------------  -----------------  ---------------------
ssn_mask        COLUMN_MASK  NULL     NULL    NULL   NULL                                                              METASTORE          my_metastore
managed_access  DENY         NULL     NULL    NULL   Prevent non-admins from managing permissions on sensitive tables  METASTORE          my_metastore

Describe a policy

Use DESCRIBE POLICY to view the details of a specific metastore-level policy. Requires READ METADATA on the metastore or the metastore admin role.

{ DESC | DESCRIBE } POLICY policy_name ON METASTORE

The result shows the policy's properties as key-value pairs, including name, securable object type, securable object name, principals, privileges, and the WHEN condition.

Query policy definitions with Information Schema

To list metastore-level policy definitions, query SYSTEM.INFORMATION_SCHEMA.ABAC_POLICY_DEFINITIONS and filter on on_securable_type:

SELECT *
FROM system.information_schema.abac_policy_definitions
WHERE on_securable_type = 'METASTORE';

Metastore-level policies return METASTORE in on_securable_type, with catalog_name, schema_name, and securable_name all NULL. For the available columns and additional examples, see ABAC_POLICY_DEFINITIONS.

Policy quotas

Resource Policy type Limit
Policies per metastore (all objects) Row filter / column mask 10,000
Policies per metastore (all objects) GRANT / DENY 10,000
Policies per metastore, attached directly Row filter / column mask 100
Policies per metastore, attached directly GRANT / DENY 100

Row filter and column mask policies share one quota, and GRANT and DENY policies share a separate quota. The two groups are counted independently.

For details about row filter and column mask policy quotas, see Policy quotas. For details about GRANT policy quotas, see Policy quotas. For details about DENY policy quotas, see Policy quotas.

Audit logging

Metastore-level policy create, alter, and drop operations are logged under the same createPolicy, deletePolicy, getPolicy, and listPolicies actions as other ABAC policies, with on_securable_type set to METASTORE. See Audit logging for example audit log queries.

Limitations

  • SQL targets the current metastore only. ON METASTORE always targets the metastore you are operating in. Specifying a metastore name after ON METASTORE returns a syntax error, and you can't manage policies on a different metastore.
  • GRANT and DENY targets. GRANT and DENY policies can't target non-catalog objects in the metastore, such as external locations, storage credentials, shares, recipients, providers, and connections. For the full object hierarchy, see The Unity Catalog object hierarchy.
  • Information schema. The ABAC_POLICY_DEFINITIONS view doesn't have a metastore_name column. A metastore-level policy is identified by on_securable_type = METASTORE.
  • Disaster recovery. Unity Catalog-managed disaster recovery does not replicate metastore-level policies when replicating your Azure Databricks deployment to a secondary region. In the case of an outage, objects previously protected by a metastore-level ABAC policy are replicated without protection.

More information

  • Last updated on