Share via

Support-Info: (Sync): Run Profile Optimization

  • Article

Products Involved Discussed

  • Forefront Identity Manager Synchronization Service
  • Microsoft Identity Manager Synchronization Service

Purpose:

The purpose of this document is to discuss from a high level how to optimally execute run profiles to effectively and efficiently reach the intended goal.

Disclaimer This document will discuss the execution of run profiles simultaneously.  However, running more than a single synchronization run profile execution at a time is not supported.  The supported method is to execute run profiles asynchronously, allowing each run profile to complete before executing another.

 

We need to understand how data flows through the Synchronization Service Engine before we can understand how to optimize run profile execution.

Data flows through the Synchronization Service Engine via the execution of what is known as a Run Profile.  A Run Profile executes an action which moves the data through the Synchronization Service Engine.  The Import and Synchronization run profiles contain the two key words Full and Delta.  These describe the type of Import/Synchronization. 

  • Full: This will process every single record
  • Delta (Incremental): This looks for records that have been modified and processes them.
    • Export Run Profile is always a Delta

 

NOTE

 This document will utilize terms such as Connected Data Source (CDS), Connector Space (CS) and Metaverse (MV).  These terms are very important in the understanding of how data flows through the Synchronization Service Engine and the optimization of executing run profiles.
  • Connected Data Source (External Data Source) : The location of the actual data.
  • Connector Space: A storage area in the backend SQL Server database where object additions, deletions, and modifications.  A portion of the Connector Space is dedicated to each individual management agent (connector).
  • Metaverse: This is a series of tables in the backend SQL Server that contains combined identity information from multiple Connector Spaces on person and/or resource objects. 
  • More Information:

 

 

 

Essentially there is three basic steps that moves the data through the engine.  Import, Synchronization and Export.  The following illustration displays how the data flows through the engine.

Run Profile What happens when executed
Import Reads objects and related attribute information from the External Data Source (EDS) and then either Adds, Modifies (Updates) and/or Deletes the object from the specified Connector Space (CS).
Synchronization The Synchronization process that ties the objects together between the different Connector Spaces (CS).  This is done through the use of several tables in the backend SQL Server database that links these objects together. A synchronization goes through several steps in merging the objects. 
  • Filter: Check to see if the object should be filtered from the metaverse
  • Join: If the object is not to be filtered, then the synchronization service engine checks to see if the object can join to an existing object in the metaverse.
  • Project: If the object cannot join to an existing metaverse object, then it will project (create a metaverse object).
  • Import Attribute Flow (IAF) – Flows the attribute values from the source connector space to the metaverse
  • Provision: If there is no join between the metaverse object and the target connector space object, then it will Provision (create a connector space) object.
  • Export Attribute Flow (EAF) – Flows the attribute values from the target connector space
Export Reads the objects from the specified Connector Space (CS) and writes (Adds, Modifies and/or Deletes) an object in the External Data Source.  Export is always a Delta, which means it will always on update the objects that need to be updated. 

*NOTE: If you never run an export, you will never update the Connected Data Source (CDS).

 

Is it supported to run multiple run profiles at one time?  

Disclaimer This document will discuss the execution of run profiles simultaneously.  However, running more than a single synchronization run profile execution at a time is not supported.  The supported method is to execute run profiles asynchronously, allowing each run profile to complete before executing another.

Executing more than a single synchronization run profile is not supported.  Executing more than a single Import or Export run profile is not recommended; however, we do understand that sometimes there is a business need to synchronously execute run profiles.  If the business need arises to synchronously execute run profiles, you should follow a few simple rules.

  1. Imports and/or Exports are the only run profiles you should consider to run simultaneously
  2. It’s forbidden to simultaneously run Import and Export profiles on the same connector
    1. Do not start an Import on a connector and then prior to the Import finishing, start an Export on the same connector.  This is forbidden!
  3. Never run Synchronizations simultaneously
  4. Never run Synchronizations while Imports or Exports are running.

Simultaneous Imports / Exports

The rule to follow should a business need arises to run Imports or Exports simultaneously is to only run Imports or only run Exports.  If you are running simultaneous imports/exports, then we recommend using a staggered approach to execute in a simultaneous fashion.  A staggered approach would resemble:

  1. Start your first import or export
  2. Wait about 10-15 seconds and then start the next import or export
  3. Repeat Step 2
NOTE Simultaneous Imports/Exports can cause performance issues and possibly other issues.

 

Simultaneous Synchronizations (Full or Delta) [Not Supported]

First, it is important to note, that simultaneous synchronizations is not supported.  Running Imports/Exports while a synchronization is running is not supported.

A synchronization is the process of linking objects together throughout the synchronization service engine.  Creating links between the Connector Spaces and the Metaverse.  If synchronizations are executing simultaneously it could generate the following issues:

  • Could possibly have these steps touch the same object at the same time which could cause data integrity issues.
  • Could possibly cause Data Corruption issues
  • Could possibly cause SQL Server Dead-Lock issues
  • Could possibly cause SQL Server blocking

What are some optimization techniques that could be utilized?

  1. Utilize WMI Scripting to stagger the launching of Imports / Exports and the synchronizations.
    1. Here you could utilize the Execute Method of the MIIS_ManagementAgent Class to check the status of the run on a management agent.
  2. Utilize the WMI provider to check the status of the current run to ensure that nothing is running before starting a synchronization.
    1. Here you could utilize the RunStatus Method of the MIIS_ManagementAgent Class to check the status of the run on a management agent.

Additional Resources

Comments

  • Anonymous
    August 27, 2015
    Hello.
    Run Profiles is a consistent conversation that we have in support. In an effort to help relay
  • Anonymous
    August 27, 2015
    this is exactly the kind of data I was looking for
    very very helpful
    Thanks