Discover and govern Azure SQL Database in Microsoft Purview
Článok
This article outlines the process to register an Azure SQL database source in Microsoft Purview. It includes instructions to authenticate and interact with the SQL database.
SP lineage extraction scan isn't supported if your logical server in Azure disables public access or doesn't allow Azure services to access it.
SP lineage extraction scan is currently not supported if your stored procedures contain an INSERT or a DROP query.
SP lineage extraction scan isn't supported if your Microsoft Purview account disables public access.
SP lineage extraction scan is scheduled to run every six hours by default. The frequency can't be changed.
Lineage is captured only when the stored procedure execution transfers data from one table to another. And it's not supported for temporary tables.
Lineage extraction isn't supported for functions or triggers.
Due to the following limitations, you could see duplicate assets in the catalog if you have scenarios such as:
The object names in assets and fully qualified names follow the case used in stored procedure statements, which might not align with the object case in original data source.
When SQL views are referenced in stored procedures, they're currently captured as SQL tables.
For scanned assets:
For a full scan, instead of INT data type, DECIMAL is displayed as a data type for SQL asset properties.
For NCHAR data type, column length is displayed in bytes and not as character count.
Create the collection hierarchy by going to Collections and then selecting Add a collection. Assign permissions to individual subcollections as required.
Go to the appropriate collection under Sources, and then select the Register icon to register a new SQL database.
Select the Azure SQL Database data source, and then select Continue.
For Name, provide a suitable name for the data source. Select relevant names for Azure subscription, Server name, and Select a collection, and then select Apply.
Confirm that the SQL database appears under the selected collection.
Update firewall settings
If your database server has a firewall enabled, you need to update the firewall to allow access in one of the following ways:
Use a managed virtual network. Setting up a managed virtual network with your Microsoft Purview account allows you to connect to Azure SQL by using the Azure integration runtime in a closed network.
Check your database server's networking configuration to confirm that a private endpoint is accessible to the machine that contains the self-hosted integration runtime. Add the IP address of the machine if it doesn't already have access.
If your logical server is behind a private endpoint or in a virtual network, you can use an ingestion private endpoint to ensure end-to-end network isolation.
Configure authentication for a scan
To scan your data source, you need to configure an authentication method in Azure SQL Database.
Dôležité
If you're using a self-hosted integration runtime to connect to your resource, system-assigned and user-assigned managed identities won't work. You need to use service principal authentication or SQL authentication.
Microsoft Purview supports the following options:
System-assigned managed identity (SAMI) (recommended). This is an identity that is associated directly with your Microsoft Purview account. It allows you to authenticate directly with other Azure resources without needing to manage a go-between user or credential set.
The SAMI is created when your Microsoft Purview resource is created. The SAMI is managed by Azure and uses your Microsoft Purview account's name. The SAMI can't currently be used with a self-hosted integration runtime for Azure SQL.
User-assigned managed identity (UAMI) (preview). Similar to a SAMI, a UAMI is a credential resource that allows Microsoft Purview to authenticate against Microsoft Entra ID.
The UAMI is managed by users in Azure, rather than by Azure itself, which gives you more control over security. The UAMI can't currently be used with a self-hosted integration runtime for Azure SQL.
Service principal. A service principal is an application that can be assigned permissions like any other group or user, without being associated directly with a person. Authentication for service principals has an expiration date, so it can be useful for temporary projects.
Only the server-level principal login (created by the provisioning process) or members of the loginmanager database role in the master database can create new logins. The Microsoft Purview account should be able to scan the resources about 15 minutes after it gets permissions.
You need a SQL login with at least db_datareader permissions to be able to access the information that Microsoft Purview needs to scan the database. You can follow the instructions in CREATE LOGIN to create a login for Azure SQL Database. Save the username and password for the next steps.
Go to your key vault in the Azure portal.
Select Settings > Secrets, and then select + Generate/Import.
For Name and Value, use the username and password (respectively) from your SQL database.
If you're using a self-hosted integration runtime to connect to your resource, system-assigned and user-assigned managed identities won't work. You need to use SQL authentication or service principal authentication.
Configure Microsoft Entra authentication in the database account
The managed identity needs permission to get metadata for the database, schemas, and tables. It must also be authorized to query the tables to sample for classification.
It's important to give your Microsoft Purview account's system-assigned managed identity or user-assigned managed identity the permission to scan the SQL database. You can add the SAMI or UAMI at the subscription, resource group, or resource level, depending on the breadth of the scan.
Poznámka
To add a managed identity on an Azure resource, you need to be an owner of the subscription.
From the Azure portal, find the subscription, resource group, or resource (for example, a SQL database) that the catalog should scan.
Select Access control (IAM) on the left menu, and then select + Add > Add role assignment.
Set Role to Reader. In the Select box, enter your Microsoft Purview account name or UAMI. Then, select Save to give this role assignment to your Microsoft Purview account.
Create a new service principal
If you don't have a service principal, you can follow the service principal guide to create one.
Poznámka
To create a service principal, you must register an application in your Microsoft Entra tenant. If you don't have the required access, your Microsoft Entra Application Administrator can perform this operation.
Grant the service principal access to your SQL database
The service principal needs permission to get metadata for the database, schemas, and tables. It must also be authorized to query the tables to sample for classification.
Select Settings > Secrets, and then select + Generate/Import.
For Name, give the secret a name of your choice.
For Value, use the service principal's secret value. If you've already created a secret for your service principal, you can find its value in Client credentials on your secret's overview page.
For Database selection method, select Enter manually.
For Database name and Credential, enter the values that you created earlier.
For Select a connection, choose the appropriate collection for the scan.
Select Test connection to validate the connection. After the connection is successful, select Continue.
For Name, provide a name for the scan.
Select the SAMI or UAMI under Credential, and choose the appropriate collection for the scan.
Select Test connection. After the connection is successful, select Continue.
For Name, provide a name for the scan.
Choose the appropriate collection for the scan, and select the credential that you created earlier under Credential.
Select Test connection. After the connection is successful, select Continue.
Scope and run the scan
You can scope your scan to specific database objects by choosing the appropriate items in the list.
Select a scan rule set. You can use the system default, choose from existing custom rule sets, or create a new rule set inline. Select Continue when you're finished.
If you select New scan rule set, a pane opens so that you can enter the source type, the name of the rule set, and a description. Select Continue when you're finished.
For Select classification rules, choose the classification rules that you want to include in the scan rule set, and then select Create.
The new scan rule set then appears in the list of available rule sets.
Choose your scan trigger. You can set up a schedule or run the scan once.
Review your scan, and then select Save and run.
View a scan
To check the status of a scan, go to the data source in the collection, and then select View details.
The scan details indicate the progress of the scan in Last run status, along with the number of assets scanned and classified.
Last run status is updated to In progress and then Completed after the entire scan has run successfully.
Manage a scan
After you run a scan, you can use the run history to manage it:
Under Recent scans, select a scan.
In the run history, you have options for running the scan again, editing it, or deleting it.
If you select Run scan now to rerun the scan, you can then choose either Incremental scan or Full scan.
Troubleshoot scanning
If you have problems with scanning, try these tips:
The following types of Microsoft Purview policies are supported on this data resource:
Data owner policies - a set of policy statements that allow you to grant users and groups access to data sources.
Self-service policies - policy that allows users to request access to data sources registered to Microsoft Purview.
Protection policies - denies access to data tagged with sensitivity labels to all users except those specified by the policy.
DevOps policies - grants access to database system metadata across multiple sources. They simplify access provisioning for IT operations and security auditing personnel. They only grant access, and don't deny access.
Enforcement of Microsoft Purview policies is available only in the following regions for Azure SQL Database:
Public cloud:
East US
East US2
South Central US
West Central US
West US3
Canada Central
Brazil South
West Europe
North Europe
France Central
UK South
South Africa North
Central India
South East Asia
East Asia
Australia East
Sovereign clouds:
USGov Virginia
China North 3
Configure the Azure SQL Database instance for policies from Microsoft Purview
For the logical server associated with Azure SQL Database to honor policies from Microsoft Purview, you need to configure a Microsoft Entra admin. In the Azure portal, go to the logical server that hosts the Azure SQL Database instance. On the side menu, select Microsoft Entra ID. Set an admin name to any Microsoft Entra user or group that you prefer, and then select Save.
Then, on the side menu, select Identity. Under System assigned managed identity, turn the status to On, and then select Save.
Configure the Microsoft Purview account for policies
Register the data source in Microsoft Purview
Before a policy can be created in Microsoft Purview for a data resource, you must register that data resource in Microsoft Purview Studio. You will find the instructions related to registering the data resource later in this guide.
Poznámka
Microsoft Purview policies rely on the data resource ARM path. If a data resource is moved to a new resource group or subscription it will need to be de-registered and then registered again in Microsoft Purview.
Configure permissions to enable Data policy enforcement on the data source
Once a resource is registered, but before a policy can be created in Microsoft Purview for that resource, you must configure permissions. A set of permissions are needed to enable the Data policy enforcement. This applies to data sources, resource groups, or subscriptions. To enable Data policy enforcement, you must have both specific Identity and Access Management (IAM) privileges on the resource as well as specific Microsoft Purview privileges:
You must have either one of the following IAM role combinations on the resource's Azure Resource Manager path or any parent of it (that is, using IAM permission inheritance):
IAM Owner
Both IAM Contributor and IAM User Access Administrator
To configure Azure role-based access control (RBAC) permissions, follow this guide. The following screenshot shows how to access the Access Control section in the Azure portal for the data resource to add a role assignment.
Poznámka
The IAM Owner role for a data resource can be inherited from a parent resource group, a subscription, or a subscription management group. Check which Microsoft Entra users, groups, and service principals hold or are inheriting the IAM Owner role for the resource.
You also need to have the Microsoft Purview Data source admin role for the collection or a parent collection (if inheritance is enabled). For more information, see the guide on managing Microsoft Purview role assignments.
The following screenshot shows how to assign the Data source admin role at the root collection level.
Configure Microsoft Purview permissions to create, update, or delete access policies
To create, update or delete policies, you need to get the Policy author role in Microsoft Purview at root collection level:
The Policy author role can create, update, and delete DevOps and Data Owner policies.
The Policy author role can delete self-service access policies.
Policy author role must be configured at the root collection level.
In addition, to easily search Microsoft Entra users or groups when creating or updating the subject of a policy, you can greatly benefit from getting the Directory Readers permission in Microsoft Entra ID. This is a common permission for users in an Azure tenant. Without the Directory Reader permission, the Policy Author will have to type the complete username or email for all the principals included in the subject of a data policy.
Configure Microsoft Purview permissions for publishing Data Owner policies
Data Owner policies allow for checks and balances if you assign the Microsoft Purview Policy author and Data source admin roles to different people in the organization. Before a Data owner policy takes effect, a second person (Data source admin) must review it and explicitly approve it by publishing it. This does not apply to DevOps or Self-service access policies as publishing is automatic for them when those policies are created or updated.
To publish a Data owner policy you need to get the Data source admin role in Microsoft Purview at root collection level.
To publish Data owner policies, the Data source admin role must be configured at the root collection level.
Delegate access provisioning responsibility to roles in Microsoft Purview
After a resource has been enabled for Data policy enforcement, any Microsoft Purview user with the Policy author role at the root collection level can provision access to that data source from Microsoft Purview.
Poznámka
Any Microsoft Purview root Collection admin can assign new users to root Policy author roles. Any Collection admin can assign new users to a Data source admin role under the collection. Minimize and carefully vet the users who hold Microsoft Purview Collection admin, Data source admin, or Policy author roles.
If a Microsoft Purview account with published policies is deleted, such policies will stop being enforced within an amount of time that depends on the specific data source. This change can have implications on both security and data access availability. The Contributor and Owner roles in IAM can delete Microsoft Purview accounts. You can check these permissions by going to the Access control (IAM) section for your Microsoft Purview account and selecting Role Assignments. You can also use a lock to prevent the Microsoft Purview account from being deleted through Resource Manager locks.
Register the data source and enable Data policy enforcement
The Azure SQL Database resource needs to be registered with Microsoft Purview before you can create access policies. To register your resources, follow the "Prerequisites" and "Register the data source" sections in Enable Data policy enforcement on your Microsoft Purview sources.
After you register the data source, you need to enable Data policy enforcement. This is a prerequisite before you can create policies on the data source. Data policy enforcement can affect the security of your data, because it delegates to certain Microsoft Purview roles that manage access to the data sources. Go through the security practices in Enable Data policy enforcement on your Microsoft Purview sources.
After your data source has the Data policy enforcement option set to Enabled, it will look like this screenshot:
Return to the Azure portal for Azure SQL Database to verify it's now governed by Microsoft Purview:
Select the Azure SQL Server that you want to configure.
Go to Microsoft Entra ID on the left pane.
Scroll down to Microsoft Purview access policies.
Select the button to Check for Microsoft Purview Governance. Wait while the request is processed. It could take a few minutes.
Confirm that the Microsoft Purview Governance Status shows Governed. Note that it could take a few minutes after you enable Data policy enforcement in Microsoft Purview for the correct status to be reflected.
Poznámka
If you disable Data policy enforcement for this Azure SQL Database data source, it might take up to 24 hours for the Microsoft Purview Governance Status to be updated automatically to Not Governed. This can be accelerated by selecting Check for Microsoft Purview Governance. Before you enable Data policy enforcement for the data source in another Microsoft Purview account, ensure that the Purview Governance Status shows as Not Governed. Then repeat the steps above with the new Microsoft Purview account.
Create an access policy
To create an access policy for Azure SQL Database, follow these guides:
Protection access control policies (protection policies) enable organizations to automatically protect sensitive data across data sources. Microsoft Purview already scans data assets and identifies sensitive data elements, and this feature allows you to automatically restrict access to that data using sensitivity labels from Microsoft Purview Information Protection. Learn how to create a protection policy.
Extract lineage (preview)
Microsoft Purview supports lineage for views and stored procedures from Azure SQL Database. While lineage for views is supported as part of scanning, you'll need to turn on the Lineage extraction toggle to extract stored procedure lineage when you're setting up a scan.
Poznámka
Lineage isn't supported using a self-hosted integration runtime or managed VNET runtime and an Azure SQL private endpoint. You need to enable Azure services to access the server under network settings for your Azure SQL Database, and your Microsoft Purview account needs to allow public access. Learn more about the known limitations in lineage extraction scan.
Lineage for SQL DB views
SQL DB metadata scan includes lineage extraction for views. Only new scans include the view lineage extraction. Lineage is extracted at all scan levels (L1/L2/L3). In case of an incremental scan, whatever metadata is scanned as part of incremental scan, the corresponding static lineage for tables/views will be extracted.
Prerequisites for setting up a scan with SP lineage extraction
Follow the steps in the Configure authentication for a scan section of this article to authorize Microsoft Purview to scan your SQL database.
Sign in to Azure SQL Database with your Microsoft Entra account, and assign db_owner permissions to the Microsoft Purview managed identity.
Poznámka
The 'db_owner' permissions is needed because lineage is based on XEvent sessions. So Microsoft Purview needs the permission to manage the XEvent sessions in SQL.
Use the following example SQL syntax to create a user and grant permission. Replace <purview-account> with your account name.
SQL
Createuser <purview-account> FROMEXTERNAL PROVIDER
GO
EXEC sp_addrolemember 'db_owner', <purview-account>
GO
Run the following command on your SQL database to create a master key:
SQL
CreatemasterkeyGo
Ensure that Allow Azure services and resources to access this server is enabled under networking/firewall for your Azure SQL resource.
Create a scan with lineage extraction turned on
On the pane for setting up a scan, turn on the Enable lineage extraction toggle.
Select your method of authentication by following the steps in the Create the scan section of this article.
After you successfully set up the scan, a new scan type called Lineage extraction will run incremental scans every six hours to extract lineage from Azure SQL Database. Lineage is extracted based on the stored procedure runs in the SQL database.
Search Azure SQL Database assets and view runtime lineage
Go to the Lineage tab for the asset. When applicable, the asset lineage appears here.
When applicable, you can further drill down to see the lineage at SQL statement level within a stored procedure, along with column level lineage. When using Self-hosted Integration Runtime for scan, retrieving the lineage drilldown information during scan is supported since version 5.25.8374.1.
Go to the stored procedure asset. On the Properties tab, go to Related assets to get the latest run details of stored procedures.
Select the stored procedure hyperlink next to Runs to see the Azure SQL Stored Procedure Run overview. Go to the Properties tab to see enhanced runtime information from the stored procedure, such as executedTime, rowCount, and Client Connection.
Troubleshoot lineage extraction for Stored Procedures
The following tips can help you solve problems related to lineage:
If no lineage is captured after a successful Lineage extraction run, it's possible that no stored procedures have run at least once since you set up the scan.
Lineage is captured for stored procedure runs that happen after a successful scan is set up. Lineage from past stored procedure runs isn't captured.
If your database is processing heavy workloads with lots of stored procedure runs, lineage extraction filters only the most recent runs. Stored procedure runs early in the six-hour window, or the run instances that create heavy query load, won't be extracted. Contact support if you're missing lineage from any stored procedure runs.
If a stored procedure contains drop or create statements, they aren't currently captured in lineage
Next steps
To learn more about Microsoft Purview and your data, use these guides:
Administer an SQL Server database infrastructure for cloud, on-premises and hybrid relational databases using the Microsoft PaaS relational database offerings.