Discover and govern multiple Azure sources in Microsoft Purview

This article outlines how to register multiple Azure sources and how to authenticate and interact with them in Microsoft Purview. For more information about Microsoft Purview, read the introductory article.

Supported capabilities

Metadata Extraction Full Scan Incremental Scan Scoped Scan Classification Access Policy Lineage Data Sharing
Yes Yes Yes Yes Yes Yes Source Dependant No

Prerequisites

Register

This section describes how to register multiple Azure sources in Microsoft Purview using the Microsoft Purview governance portal.

Prerequisites for registration

Microsoft Purview needs permissions to be able to list resources under a subscription or resource group.

  1. Go to the subscription or the resource group in the Azure portal.
  2. Select Access Control (IAM) from the left menu.
  3. Select +Add.
  4. In the Select input box, select the Reader role and enter your Microsoft Purview account name (which represents its MSI file name).
  5. Select Save to finish the role assignment. This will allow Microsoft Purview to list resources under a subscription or resource group.

Authentication for registration

There are two ways to set up authentication for multiple sources in Azure:

  • Managed identity
  • Service principal

You must set up authentication on each resource within your subscription or resource group that you want to register and scan. Azure Storage resource types (Azure Blob Storage and Azure Data Lake Storage Gen2) make it easy by allowing you to add the MSI file or service principal at the subscription or resource group level as a storage blob data reader. The permissions then trickle down to each storage account within that subscription or resource group. For all other resource types, you must apply the MSI file or service principal on each resource, or create a script to do so.

To learn how to add permissions on each resource type within a subscription or resource group, see the following resources:

Steps to register

  1. Go to your Microsoft Purview account.

  2. Select Data Map on the left menu.

  3. Select Register.

  4. On Register sources, select Azure (multiple).

    Screenshot that shows the tile for Azure Multiple on the screen for registering multiple sources.

  5. Select Continue.

  6. On the Register sources (Azure) screen, do the following:

    1. In the Name box, enter a name that the data source will be listed with in the catalog.

    2. In the Management group box, optionally choose a management group to filter down to.

    3. In the Subscription and Resource group dropdown list boxes, select a subscription or a specific resource group, respectively. The registration scope will be set to the selected subscription or resource group.

      Screenshot that shows the boxes for selecting a subscription and resource group.

    4. In the Select a collection box, select a collection or create a new one (optional).

    5. Select Register to register the data sources.

Scan

Follow the steps below to scan multiple Azure sources to automatically identify assets and classify your data. For more information about scanning in general, see our introduction to scans and ingestion.

Create and run scan

To create and run a new scan, do the following:

  1. Select the Data Map tab on the left pane in the Microsoft Purview governance portal.

  2. Select the data source that you registered.

  3. Select View details > + New scan, or use the Scan quick-action icon on the source tile.

  4. For Name, fill in the name.

  5. For Type, select the types of resources that you want to scan within this source. Choose one of these options:

    • Leave it as All. This selection includes future resource types that might not currently exist within that subscription or resource group.
    • Use the boxes to specifically select resource types that you want to scan. If you choose this option, future resource types that might be created within this subscription or resource group won't be included for scans, unless the scan is explicitly edited in the future.

    Screenshot that shows options for scanning multiple sources.

  6. Select the credential to connect to the resources within your data source:

    • You can select a credential at the parent level as an MSI file, or you can select a credential for a particular service principal type. You can then use that credential for all the resource types under the subscription or resource group.
    • You can specifically select the resource type and apply a different credential for that resource type.

    Each credential will be considered as the method of authentication for all the resources under a particular type. You must set the chosen credential on the resources in order to successfully scan them, as described earlier in this article.

  7. Within each type, you can select to either scan all the resources or scan a subset of them by name:

    • If you leave the option as All, then future resources of that type will also be scanned in future scan runs.
    • If you select specific storage accounts or SQL databases, then future resources of that type created within this subscription or resource group won't be included for scans, unless the scan is explicitly edited in the future.
  8. Select Test connection. This will first test access to check if you've applied the Microsoft Purview MSI file as a reader on the subscription or resource group. If you get an error message, follow these instructions to resolve it. Then it will test your authentication and connection to each of your selected sources and generate a report. The number of sources selected will impact the time it takes to generate this report. If failed on some resources, hovering over the X icon will display the detailed error message.

    Screenshot showing the scan setup slider, with the Test Connection button highlighted. Screenshot showing an example test connection report, with some connections passing and some failing. Hovering over one of the failed connections shows a detailed error report.

  9. After your test connection has passed, select Continue to proceed.

  10. Select scan rule sets for each resource type that you chose in the previous step. You can also create scan rule sets inline.

    Screenshot that shows scan rules for each resource type.

  11. Choose your scan trigger. You can schedule it to run weekly, monthly, or once.

  12. Review your scan and select Save to complete setup.

View your scans and scan runs

  1. View source details by selecting View details on the tile under the Data Map section.

    Screenshot that shows source details.

  2. View scan run details by going to the Scan details page.

    The status bar is a brief summary of the running status of the child resources. It's displayed on the subscription level or resource group level. The colors have the following meanings:

    • Green: The scan was successful.
    • Red: The scan failed.
    • Gray: The scan is still in progress.

    You can select each scan to view finer details.

    Screenshot that shows scan details.

  3. View a summary of recent failed scan runs at the bottom of the source details. You can also view more granular details about these runs.

Manage your scans: edit, delete, or cancel

To manage a scan, do the following:

  1. Go to the management center.

  2. Select Data sources under the Sources and scanning section, and then select the desired data source.

  3. Select the scan that you want to manage. Then:

    • You can edit the scan by selecting Edit.
    • You can delete the scan by selecting Delete.
    • If the scan is running, you can cancel it by selecting Cancel.

Access Policy

Supported policies

The following types of policies are supported on this data resource from Microsoft Purview:

Access policy pre-requisites on Azure Storage accounts

To be able to enforce policies from Microsoft Purview, data sources under a resource group or subscription need to be configured first. Instructions vary based on the data source type. Please review whether they support Microsoft Purview policies, and if so, the specific instructions to enable them, under the Access Policy link in the Microsoft Purview connector document.

Configure the Microsoft Purview account for policies

Configure permissions to enable Data use management on the data source

Before a policy can be created in Microsoft Purview for a resource, you must configure permissions. To enable the Data use management toggle for a data source, resource group, or subscription, the same user must have both specific identity and access management (IAM) privileges on the resource and specific Microsoft Purview privileges:

  • The user must have either one of the following IAM role combinations on the resource's Azure Resource Manager path or any parent of it (that is, using IAM permission inheritance):

    • IAM Owner
    • Both IAM Contributor and IAM User Access Administrator

    To configure Azure role-based access control (RBAC) permissions, follow this guide. The following screenshot shows how to access the Access Control section in the Azure portal for the data resource to add a role assignment.

    Screenshot that shows the section in the Azure portal for adding a role assignment.

  • The same user needs to have the Microsoft Purview Data source admin role for the collection or a parent collection (if inheritance is enabled). For more information, see the guide on managing Microsoft Purview role assignments.

    The following screenshot shows how to assign the Data source admin role at the root collection level.

    Screenshot that shows selections for assigning the Data source admin role at the root collection level.

Configure Microsoft Purview permissions to create, update, or delete access policies

The following permissions are needed in Microsoft Purview at the root collection level:

  • The Policy author role can create, update, and delete DevOps and Data Owner policies.
  • The Policy author role can delete self-service access policies.

For more information about managing Microsoft Purview role assignments, see Create and manage collections in the Microsoft Purview Data Map.

Note

Currently, Microsoft Purview roles related to creating, updating, and deleting policies must be configured at the root collection level.

In addition to the Microsoft Purview Policy author role, users might need Directory Readers permission in Azure Active Directory to create a policy. This is a common permission for users in an Azure tenant.

Configure Microsoft Purview permissions for publishing Data Owner policies

Data Owner policies allow for checks and balances if you assign the Microsoft Purview Policy author and Data source admin roles to different people in the organization. Before a data policy takes effect, a second person (Data source admin) must review it and explicitly approve it by publishing it. Publishing is automatic after DevOps or self-service access policies are created or updated, so it doesn't apply to these types of policies.

The following permissions are needed in Microsoft Purview at the root collection level:

  • The Data source admin role can publish a policy.

For more information about managing Microsoft Purview role assignments, see Create and manage collections in the Microsoft Purview Data Map.

Note

Currently, Microsoft Purview roles related to publishing Data Owner policies must be configured at the root collection level.

Delegate access provisioning responsibility to roles in Microsoft Purview

After a resource has been enabled for Data use management, any Microsoft Purview user with the Policy author role at the root collection level can provision access to that data source from Microsoft Purview.

The IAM Owner role for a data resource can be inherited from a parent resource group, a subscription, or a subscription management group. Check which Azure AD users, groups, and service principals hold or are inheriting the IAM Owner role for the resource.

Note

Any Microsoft Purview root Collection admin can assign new users to root Policy author roles. Any Collection admin can assign new users to a Data source admin role under the collection. Minimize and carefully vet the users who hold Microsoft Purview Collection admin, Data source admin, or Policy author roles.

If a Microsoft Purview account with published policies is deleted, such policies will stop being enforced within an amount of time that depends on the specific data source. This change can have implications on both security and data access availability. The Contributor and Owner roles in IAM can delete Microsoft Purview accounts.

You can check these permissions by going to the Access control (IAM) section for your Microsoft Purview account and selecting Role Assignments. You can also use a lock to prevent the Microsoft Purview account from being deleted through Resource Manager locks.

Register the data source in Microsoft Purview for Data Use Management

The Azure subscription or resource group needs to be registered first with Microsoft Purview before you can create access policies. To register your resource, follow the Prerequisites and Register sections of this guide:

After you've registered the data resource, you'll need to enable Data Use Management. This is a pre-requisite before you can create policies on the data resource. Data Use Management can impact the security of your data, as it delegates to certain Microsoft Purview roles managing access to the data sources. Go through the secure practices related to Data Use Management in this guide: How to enable Data Use Management

Once your data source has the Data Use Management option set to Enabled, it will look like this screenshot: Screenshot shows how to register a data source for policy with the option Data use management set to enable.

Create a policy

To create an access policy on an entire Azure subscription or resource group, follow these guides:

Next steps

Now that you've registered your source, follow the below guides to learn more about Microsoft Purview and your data.