Duplicate detection

  • 5 minutes

Microsoft Dataverse includes basic duplicate detection functionality. The purpose of duplicate detection is to alert the user to potential duplicates when creating or updating records in model-driven apps ad keep data clean.

Enable duplicate detection

Duplicate detection must be enabled in the environment's settings in the Power Platform admin center under the Data management section.

Screenshot of duplicate detection settings.

Note

Duplicate detection is enabled on the environment by default.

To detect duplicates for a table's rows, you must first edit the table's properties in the Power Apps maker portal and enable the Apply duplicate detection rules option.

Screenshot of duplicate detection table properties.

Configure duplication detection rules

To detect duplicates, you need to define rules to determine potential duplicate data. You can only create duplicate detection rules for tables that have been enabled for duplicate detection.

Duplicate detection rules are managed in the environment's settings in the Power Platform admin center under the Data management section.

To create a duplicate detection rule:

  • Select the base and matching tables. The matching table defaults to the selected base table but can be changed.

  • Select column

  • Select matching criteria

The matching criteria you can select is simple.

Screenshot of duplicate detection rule.

Tip

The duplicate detection rule criteria lines within a rule performs an AND. To perform an OR, create separate rules.

Duplication detection rules must be published and might take some time to be Published as matchcodes are creating in the background for every row on the table.

You should select Exclude inactive matching records to prevent duplicates being found for deactivated rows and you should select Ignore blank values to prevent multiple blank to blank duplicates being detected.

Duplicate detection jobs

Duplicate detection jobs run as scheduled background jobs to find duplicates on a table. Duplicate detection jobs are managed in the environment's settings in the Power Platform admin center under the Data management section.

Duplicate detection jobs are created by using a wizard.

Screenshot of duplicate detection wizard.

Note

You can only create duplicate detection jobs for tables that has a published duplicate detection rule.

You select the table and then either use an existing view or create your own query to determine the rows to check for duplicates. You can then run the job or schedule it to run every 7, 30, 90, 180, or 365 days.

After the job has run, you can view the duplicates it has discovered and take the action required to keep your data clean.