Connect to and manage Azure Arc-enabled SQL Server in Microsoft Purview (public preview)

Important

This feature is currently in preview. The Supplemental Terms of Use for Microsoft Azure Previews include additional legal terms that apply to Azure features that are in beta, in preview, or otherwise not yet released into general availability.

This article shows how to register an Azure Arc-enabled SQL Server instance. It also shows how to authenticate and interact with Azure Arc-enabled SQL Server in Microsoft Purview. For more information about Microsoft Purview, read the introductory article.

Supported capabilities

Metadata extraction Full scan Incremental scan Scoped scan Classification Access policy Lineage Data sharing
Yes Yes Yes Yes Yes Yes - GA Limited** No

** Lineage is supported if the dataset is used as a source/sink in the Azure Data Factory copy activity.

The supported SQL Server versions are 2012 and later. SQL Server Express LocalDB is not supported.

When you're scanning Azure Arc-enabled SQL Server, Microsoft Purview supports extracting the following technical metadata:

  • Instances
  • Databases
  • Schemas
  • Tables, including the columns
  • Views, including the columns

When you're setting up a scan, you can choose to specify the database name to scan one database. You can further scope the scan by selecting tables and views as needed. The whole Azure Arc-enabled SQL Server instance will be scanned if you don't provide a database name.

Prerequisites

Register

This section describes how to register an Azure Arc-enabled SQL Server instance in Microsoft Purview by using the Microsoft Purview governance portal.

Authentication for registration

There are two ways to set up authentication for scanning Azure Arc-enabled SQL Server with a self-hosted integration runtime:

  • Windows authentication
  • SQL Server authentication

To configure authentication for the SQL Server deployment:

  1. In SQL Server Management Studio (SSMS), go to Server Properties, and then select Security on the left pane.

  2. Under Server authentication:

    • For Windows authentication, select either Windows Authentication mode or SQL Server and Windows Authentication mode.
    • For SQL Server authentication, select SQL Server and Windows Authentication mode.

    Screenshot that shows the Security page of the Server Properties window, with options for selecting authentication mode.

A change to the server authentication requires you to restart the SQL Server instance and SQL Server Agent. In SSMS, go to the SQL Server instance and select Restart on the right-click options pane.

Create a new login and user

If you want to create a new login and user to scan your SQL Server instance, use the following steps.

The account must have access to the master database, because sys.databases is in the master database. The Microsoft Purview scanner needs to enumerate sys.databases in order to find all the SQL databases on the server.

Note

You can run all the following steps by using this code.

  1. Go to SSMS, connect to the server, and then select Security on the left pane.

  2. Select and hold (or right-click) Login, and then select New login. If Windows authentication is applied, select Windows authentication. If SQL Server authentication is applied, select SQL Server authentication.

    Screenshot that shows selections for creating a new login and user.

  3. Select Server Roles on the left pane, and ensure that a public role is assigned.

  4. Select User Mapping on the left pane, select all the databases in the map, and then select the db_datareader database role.

    Screenshot that shows user mapping.

  5. Select OK to save.

  6. If SQL Server authentication is applied, you must change your password as soon as you create a new login:

    1. Select and hold (or right-click) the user that you created, and then select Properties.
    2. Enter a new password and confirm it.
    3. Select the Specify old password checkbox and enter the old password.
    4. Select OK.

    Screenshot that shows selections for changing a password.

Store your SQL Server login password in a key vault and create a credential in Microsoft Purview

  1. Go to your key vault in the Azure portal. Select Settings > Secrets.

  2. Select + Generate/Import. For Name and Value, enter the password from your SQL Server login.

  3. Select Create.

  4. If your key vault is not connected to Microsoft Purview yet, create a new key vault connection.

  5. Create a new credential by using the username and password to set up your scan.

    Be sure to select the right authentication method when you're creating a new credential. If Windows authentication is applied, select Windows authentication. If SQL Server authentication is applied, select SQL Server authentication.

Steps to register

  1. Go to your Microsoft Purview account.

  2. Under Sources and scanning on the left pane, select Integration runtimes. Make sure that a self-hosted integration runtime is set up. If it isn't set up, follow the steps to create a self-hosted integration runtime for scanning on an on-premises or Azure virtual machine that has access to your on-premises network.

  3. Select Data Map on the left pane.

  4. Select Register.

  5. Select Azure Arc-enabled SQL Server, and then select Continue.

    Screenshot that shows selecting a SQL data source.

  6. Provide a friendly name, which is a short name that you can use to identify your server. Also provide the server endpoint.

  7. Select Finish to register the data source.

Scan

Use the following steps to scan Azure Arc-enabled SQL Server instances to automatically identify assets and classify your data. For more information about scanning in general, see Scans and ingestion in Microsoft Purview.

To create and run a new scan:

  1. In the Microsoft Purview governance portal, select the Data Map tab on the left pane.

  2. Select the Azure Arc-enabled SQL Server source that you registered.

  3. Select New scan.

  4. Select the credential to connect to your data source. Credentials are grouped and listed under the authentication methods.

    Screenshot that shows selecting a credential for a scan.

  5. You can scope your scan to specific tables by choosing the appropriate items in the list.

    Screenshot that shows selected assets for scoping a scan.

  6. Select a scan rule set. You can choose between the system default, existing custom rule sets, or creation of a new rule set inline.

    Screenshot that shows selecting a scan rule set.

  7. Choose your scan trigger. You can set up a schedule or run the scan once.

    Screenshot that shows setting up a recurring scan trigger.

  8. Review your scan, and then select Save and run.

View your scans and scan runs

To view existing scans:

  1. Go to the Microsoft Purview governance portal. Select the Data map tab on the left pane.
  2. Select the desired data source. You can view a list of existing scans on that data source under Recent scans, or you can view all scans on the Scans tab.
  3. Select the scan that has results you want to view. The page shows you all of the previous scan runs, along with the status and metrics for each scan run.
  4. Click the run ID to check more about the scan run details.

Manage your scans - edit, delete, or cancel

To manage or delete a scan:

  1. Go to the Microsoft Purview governance portal. Select the Data Map tab on the left pane.

  2. Select the desired data source. You can view a list of existing scans on that data source under Recent scans, or you can view all scans on the Scans tab.

  3. Select the scan that you want to manage. You can then:

    • Edit the scan by selecting Edit scan.
    • Cancel an in-progress scan by selecting Cancel scan run.
    • Delete your scan by selecting Delete scan.

Note

  • Deleting your scan does not delete catalog assets created from previous scans.
  • The asset will no longer be updated with schema changes if your source table has changed and you re-scan the source table after editing the description on the Schema tab of Microsoft Purview.

Access policy

Supported policies

The following types of policies are supported on this data resource from Microsoft Purview:

Access policy prerequisites on Azure Arc-enabled SQL Server

Region support

Policy enforcement is available in only the following regions for Microsoft Purview:

  • East US
  • East US 2
  • South Central US
  • West Central US
  • West US
  • West US2
  • West US3
  • Canada Central
  • Brazil South
  • North Europe
  • West Europe
  • France Central
  • Switzerland North
  • UK South
  • UAE North
  • South Africa North
  • Central India
  • Korea Central
  • Japan East
  • Australia East

Security considerations for Azure Arc-enabled SQL Server

  • The server admin can turn off the Microsoft Purview policy enforcement.
  • Azure Arc admin and server admin permissions provide the ability to change the Azure Resource Manager path of the server. Because mappings in Microsoft Purview use Resource Manager paths, this can lead to wrong policy enforcements.
  • A SQL Server admin (database admin) can gain the power of a server admin and can tamper with the cached policies from Microsoft Purview.
  • The recommended configuration is to create a separate app registration for each SQL server instance. This configuration prevents the second SQL Server instance from reading the policies meant for the first SQL Server instance, in case a rogue admin in the second SQL Server instance tampers with the Resource Manager path.

Verify the pre-requisites

  1. Sign in to the Azure portal through this link

  2. Navigate to SQL servers on the left pane. You will see a list of SQL Server instances on Azure Arc.

  3. Select the SQL Server instance that you want to configure.

  4. Go to Azure Active Directory on the left pane.

  5. Ensure that Azure Active Directory authentication is configured with an admin login. If not, refer to the access policy prerequisites section in this guide.

  6. Ensure that a certificate has been provided to for SQL Server to authenticate to Azure. If not, refer to the access policy prerequisites section in this guide.

  7. Ensure that an app registration has been entered to create a trust relationship between SQL Server and Azure AD. If not, refer to the access policy prerequisites section in this guide.

  8. If you made any changes, select the Save button to save the configuration and wait until it shows it completed successfully.

    Screenshot that shows pre-requisites to configure a Microsoft Purview endpoint in the Azure Active Directory section.

Configure the Microsoft Purview account for policies

Configure permissions to enable Data use management on the data source

Before a policy can be created in Microsoft Purview for a resource, you must configure permissions. To enable the Data use management toggle for a data source, resource group, or subscription, the same user must have both specific identity and access management (IAM) privileges on the resource and specific Microsoft Purview privileges:

  • The user must have either one of the following IAM role combinations on the resource's Azure Resource Manager path or any parent of it (that is, using IAM permission inheritance):

    • IAM Owner
    • Both IAM Contributor and IAM User Access Administrator

    To configure Azure role-based access control (RBAC) permissions, follow this guide. The following screenshot shows how to access the Access Control section in the Azure portal for the data resource to add a role assignment.

    Screenshot that shows the section in the Azure portal for adding a role assignment.

  • The same user needs to have the Microsoft Purview Data source admin role for the collection or a parent collection (if inheritance is enabled). For more information, see the guide on managing Microsoft Purview role assignments.

    The following screenshot shows how to assign the Data source admin role at the root collection level.

    Screenshot that shows selections for assigning the Data source admin role at the root collection level.

Configure Microsoft Purview permissions to create, update, or delete access policies

The following permissions are needed in Microsoft Purview at the root collection level:

  • The Policy author role can create, update, and delete DevOps and Data Owner policies.
  • The Policy author role can delete self-service access policies.

For more information about managing Microsoft Purview role assignments, see Create and manage collections in the Microsoft Purview Data Map.

Note

Currently, Microsoft Purview roles related to creating, updating, and deleting policies must be configured at the root collection level.

In addition to the Microsoft Purview Policy author role, users might need Directory Readers permission in Azure Active Directory to create a policy. This is a common permission for users in an Azure tenant.

Configure Microsoft Purview permissions for publishing Data Owner policies

Data Owner policies allow for checks and balances if you assign the Microsoft Purview Policy author and Data source admin roles to different people in the organization. Before a data policy takes effect, a second person (Data source admin) must review it and explicitly approve it by publishing it. Publishing is automatic after DevOps or self-service access policies are created or updated, so it doesn't apply to these types of policies.

The following permissions are needed in Microsoft Purview at the root collection level:

  • The Data source admin role can publish a policy.

For more information about managing Microsoft Purview role assignments, see Create and manage collections in the Microsoft Purview Data Map.

Note

Currently, Microsoft Purview roles related to publishing Data Owner policies must be configured at the root collection level.

Delegate access provisioning responsibility to roles in Microsoft Purview

After a resource has been enabled for Data use management, any Microsoft Purview user with the Policy author role at the root collection level can provision access to that data source from Microsoft Purview.

The IAM Owner role for a data resource can be inherited from a parent resource group, a subscription, or a subscription management group. Check which Azure AD users, groups, and service principals hold or are inheriting the IAM Owner role for the resource.

Note

Any Microsoft Purview root Collection admin can assign new users to root Policy author roles. Any Collection admin can assign new users to a Data source admin role under the collection. Minimize and carefully vet the users who hold Microsoft Purview Collection admin, Data source admin, or Policy author roles.

If a Microsoft Purview account with published policies is deleted, such policies will stop being enforced within an amount of time that depends on the specific data source. This change can have implications on both security and data access availability. The Contributor and Owner roles in IAM can delete Microsoft Purview accounts.

You can check these permissions by going to the Access control (IAM) section for your Microsoft Purview account and selecting Role Assignments. You can also use a lock to prevent the Microsoft Purview account from being deleted through Resource Manager locks.

Register the data source and enable Data use management

Before you can create policies, you must register the Azure Arc-enabled SQL Server data source with Microsoft Purview:

  1. Sign in to Microsoft Purview Studio.

  2. Go to Data map on the left pane, select Sources, and then select Register. Enter Azure Arc in the search box and select SQL Server on Azure Arc. Then select Continue.

    Screenshot that shows selecting a source for registration.

  3. For Name, enter a name for this registration. It's best practice to make the name of the registration the same as the server name in the next step.

  4. Select values for Azure subscription, Server name, and Server endpoint.

  5. For Select a collection, choose a collection to put this registration in.

  6. Enable Data use management. Data use management needs certain permissions and can affect the security of your data, because it delegates to certain Microsoft Purview roles to manage access to the data sources. Go through the secure practices related to Data use management in this guide: Enable Data use management on your Microsoft Purview sources.

  7. Upon enabling Data Use Management, Microsoft Purview will automatically capture the Application ID of the App Registration related to this Azure Arc-enabled SQL Server if one has been configured. Come back to this screen and hit the refresh button on the side of it to refresh, in case the association between the Azure Arc-enabled SQL Server and the App Registration changes in the future.

  8. Select Register or Apply.

Screenshot that shows selections for registering a data source for a policy.

Enable policies in Azure Arc-enabled SQL Server

This section describes the steps to configure SQL Server on Azure Arc to use Microsoft Purview. Execute these steps after you enable the Data use management option for this data source in the Microsoft Purview account.

  1. Sign in to the Azure portal through this link

  2. Navigate to SQL servers on the left pane. You will see a list of SQL Server instances on Azure Arc.

  3. Select the SQL Server instance that you want to configure.

  4. Go to Azure Active Directory on the left pane.

  5. Scroll down to Microsoft Purview access policies.

  6. Select the button to Check for Microsoft Purview Governance. Wait while the request is processed. While that happens, this message will be displayed at the top of the page. You may need to scroll up to see it. Screenshot that shows Arc-SQL agent is processing a request

  7. Do a browser refresh. At the bottom of the page, confirm that the Microsoft Purview Governance Status shows Governed. Note that it may take up to 15 minutes for the correct status to be reflected.

  8. Confirm that the Microsoft Purview Endpoint points to the Microsoft Purview account where you registered this data source and enabled the Data use management

  9. Compare the App registration ID with the one shown in the Microsoft Purview account registration for this data source

    Screenshot that shows Microsoft Purview endpoint status in the Azure Active Directory section.

Create a policy

To create an access policy for Azure Arc-enabled SQL Server, follow these guides:

To create policies that cover all data sources inside a resource group or Azure subscription, see Discover and govern multiple Azure sources in Microsoft Purview.

Next steps

Now that you've registered your source, use the following guides to learn more about Microsoft Purview and your data: