Detect duplicate data

Article
07/22/2016

Applies To: Dynamics CRM 2015

Duplicate detection lets organizations set duplicate detection policies and create duplicate detection rules for business and custom entities. These rules can be applied across different record types in Microsoft Dynamics CRM. For example, an organization may define that a lead is a duplicate of a contact, if they have the same name and phone number. Based on the duplicate detection rules set by the administrator, the system alerts the user about potential duplicates when the user tries to create new records or update existing records. To maintain data quality, you can schedule a duplicate detection job to check for duplicates for all records that match a certain criteria. You can clean the data by deleting, deactivating, or merging the duplicates reported by a duplicate detection job.

To detect duplicates in the system, create a duplicate detection rule for a specific entity type. A duplicate detection rule is represented by the duplicate rule (DuplicateRule) entity. You can create multiple detection rules for the same entity type. However, you can publish a maximum of five duplicate detection rules per entity type at one time.

A rule can have one or more duplicate detection rule conditions that are represented by the duplicate rule condition (DuplicateRuleCondition) entity. The conditions are combined by the system as in logical AND operation. A duplicate detection rule specifies a base entity type and a matching entity type. A duplicate rule condition specifies the name of a base attribute and the name of a matching attribute. For example, specify an account as a base entity and a contact as a matching entity to compare last names and addresses. The matching criteria consist of operators such as exactly match, first n-number of characters, or last n-number of characters.

To create duplicate detection rules and duplicate detection rule conditions in the Microsoft Dynamics CRM database, use the IOrganizationService.Create method or the CreateRequest message.

Important

Take special precautions when you create duplicate detection rules for appointments. The recurring appointment master and the recurring appointment records share some of the same attribute values, such as subject, location, and regarding object. Using any of these attributes as a duplicate detection rule condition (DuplicateRuleCondition), may result in incorrectly marking recurring appointment records as duplicates. To avoid this, also include a start time attribute as a duplicate detection rule condition. For more information about working with recurring appointments, see Schedule and appointment entities.

The duplicate detection rules are system-wide. You must publish them before running a duplicate detection job to detect duplicates for bulk data or retrieve duplicates for a particular entity record. To publish a duplicate detection rule, use the PublishDuplicateRuleRequest message. Duplicate rule publishing is an asynchronous operation that runs in the background.