Understanding Deletions in ILM
Have you ever had problems getting ILM to delete objects in a connected data source?
If the answer is yes, you might want to read this article.
The deletion scenario is in my opinion the best scenario to learn how ILM works, because you are not dealing with an isolated operation.
It requires an understanding of the relationship between the synchronization rules and the “what triggers what” process.
From that perspective, you want to also read this article if you would like to get a better understanding of ILM and the synchronization process.
In the following sections, I will introduce you to the ingredients – the synchronization rules that are part of a typical deletion scenario and I will also explain to you, how to connect the dots together to make a deletion scenario work.
It is my hope that you know after reading this article more about ILM than you did before.
If you have questions or other feedback, please feel free to use the “reply” function on this forum.
When to delete objects in a connected data source
In many organizations, policies govern the schedule for deletion of an object that has had its actual identity state change.
For instance, if a full-time employee’s information in a human-resource system is changed by mistake, it would be a serious mistake to delete that employee’s accounts and mailbox on the next synchronization cycle.
Many organizations effect policies that control when a system should delete the accounts, mailboxes, and other resources that a user may have held.
This type of policy can be described as a “wait and see” policy.
The typical policy is to wait at least one day before doing anything. (An obvious exception to this would be an urgent termination).
After the wait, the accounts can be disabled and moved, so that access to mailboxes and other resources would thus be restricted.
After a suitable duration, the objects finally can be deleted.
This article describes the techniques for actually deleting the connectors; it doesn’t cover the “wait and see” policy implementation.
Note
In many cases, deletions are destructive operations.
Before deleting an object, you should make sure that the operation is really safe to perform.
As the “wait and see” policy is a recommended best practice, it might be advisable to use manual steps for the final deletion of an object.
Deleting objects in a connected data source
When managing identities with ILM, you may need to delete objects in a connected data source.
Deleting objects is necessary when the authoritative masters for an identity was deleted in a connected data source, and you need to make sure that ILM deletes all other objects that are also representations of the deleted identity from your environment.
For example, if you have a HR system that functions as authoritative source for objects in your environment and a record for an employee was deleted in the HR database, a common business requirement is to also delete all other objects in your system that are representations of the removed identity.
In the picture below, the authoritative representation of Britta Simon was deleted in the HR system, which also requires Britta’s account to be deleted from Active Directory.
In the context of ILM, you find two types of deletions: import deletions and export deletions.
As indicated by the name, an import deletion is a deletion ILM has imported from a connected data source.
In a managed scenario, you can only delete a connector space object by importing the deletion of the related object from the connected data source.
An export deletion is a deletion the synchronization engine stages on a connector space object with the intention to delete the related object in a connected data source.
ILM applies export deletions during a run of an export run profile.
An export deletion does not delete a connector space object.
When a management agent processes a connector space object with a staged export deletion, it forwards the request to the connected data source.
In our current example, if you want to delete Britta’s representation in AD, you need to get an export deletion staged on her connector space object in the AD connector space.
ILM stages an export deletion request on a connector space object during a synchronization run.
So, if you want to get Britta’s account deleted from Active Directory, you need to
- Get an export deletion staged on Britta’s object in the Active Directory connector space
- Export the staged deletion request to Active Directory
Staging export deletions with synchronization rules
ILM stages export deletions on a connector spaces object during a synchronization run.
Getting an export deletion staged involves a processing chain that consists of several synchronization rules.
In other words, you can’t just manually select objects in a connector space and then flag them for a staged export deletion.
Also, while there is one synchronization rule, that has the potential to stage an export deletion on a connector space object, you need to get this rule triggered.
To understand how and when the synchronization engine stages an export on a connector space object, you need to understand how the synchronization process works.
The synchronization process consists of two separate phases: inbound synchronization and outbound synchronization. The following illustration outlines the data flow during a synchronization run.
The objective of the inbound synchronization phase is to update the metaverse with authoritative data that is pending on connector space objects.
When the synchronization engine updates a metaverse object, it also determines whether there is a need to distribute data to the connectors of the metaverse object.
Updating metaverse objects with authoritative metaverse data is the objective of the outbound synchronization phase.
In ILM, a link relationship is an essential requirement to manage objects.
Without a link relationship, it is not possible for two objects to exchange data.
The ILM terminology defines connector space objects with a link relationship to a metaverse object as connectors.
A connector space object without a link relationship to a metaverse object is a disconnector.
Depending on the context, I also prefer using the terms managed object for connectors and unmanaged objects for disconnectors.
In the previous section, I have introduced a simple scenario that consists of a HR database and an Active Directory.
The following illustration shows the established architecture for Britta Simon in this scenario.
I have already mentioned that you need to get an export deletion staged on the managed object for Britta Simon in the AD connector space if you want to synchronize the deletion in the HR database .
In the following sections, I will explain, how you can get from a deletion in a source to a staged deletion in a target.
However, I will do this backwards, which means, I will start with the staged export deletion on the target.
Introduction to the deprovisioning synchronization rule
The name of the synchronization rule that has the potential to stage an export deletion on a managed object is deprovisioning synchronization rule.
This rule is triggered "when the synchronization engine removes link relationship between a metaverse and a connector space object during the outbound synchronization phase".
Removing a link relationship from a connector turns the affected object into an unmanaged object.
Turning a managed object into an unmanaged object is a major change in an identity management scenario.
The synchronization engine consults the deprovisioning synchronization rule to determine what the consequence of the link removal for the disconnected object is.
You typically have a good reason for the disconnection - is there anything else you want to do with the object besides disconnecting it?.
In essence, the objective of this rule is to find out whether you want to just keep the object as unmanaged object in the connector space or whether you also want to stage a deletion on it.
Answering this question is not always straightforward.
This is why you have several configuration options to define what it is you want the synchronization engine to do with the disconnected objects:
- Make them disconnectors.
- Make them explicit disconnectors.
- Stage a delete on the object for the next export run.
- Determine with a rules extension.
The following illustration shows the related configuration page.
If you always want to stage a deletion on a disconnected object, you configure this rule to “stage a delete on the object for the next export run”.
However, there are also cases where the final decision for the correct next step requires further investigation.
To support these cases, the deprovisioning synchronization rule has an option to determine the appropriate next step in a rules extension.
The following pseudo code snippet shows an example for a deprovisioning evaluation in a rules extension function.
In this simple example, the code only stages a deletion when the affected connector space object is of type "Person"; all other objects are just disconnected.
Public Function Deprovision(ByVal csentry AD CSEntry) As …
If csentry.ObjectType = "Person" Then
Deprovision = DeprovisionAction.Delete
Else
Deprovision = DeprovisionAction.Disconnect
End If
End Function
Handling your deprovisioning activity in a rules extension is also an option when you need to set attribute values on the connector space object prior to the disconnection.
However, you can only set values on a connector space object, when you actually keep it in the connector space.
If you would like to read more about this, see “Example: Deprovisioning a Connector Space Object”.
The process of staging export deletions
In summary, to get a deletion staged on a connector-space object, you need the following:
- A managed connector space object with a link relationship to a metaverse object
- The removal of the link relationship between the metaverse object and the connector space object during the outbound synchronization phase of a synchronization run
- The deprovisioning synchronization rule on the affected management agent configured with an option that stages an export deletion on the affected connector space object.
At this point, you know what the trigger for the deprovisioning synchronization rule is.
As a next step, we need to discuss, how you can initiate a link removal during the outbound synchronization phase.
ILM removes a link relationship between a metaverse object and a connector space object during the outbound synchronization phase, when:
- An object deletion rule has deleted a metaverse object.
- A deprovisioning API call in your provisioning code initiates the link removal.
Introduction to the object deletion synchronization rule
As you can see in the previous list, the deprovisioning synchronization rule requires a specific processing result as trigger.
One option is the object deletion rule.
Like the deprovisioning synchronization rule, the object deletion synchronization rule defines a reaction on a link removal.
The major difference between the object-deletion synchronization rule and deprovisioning synchronization rule is the phase in which this synchronization rule is triggered.
Whereas the deprovisioning synchronization rule is part of the outbound phase, ILM triggers the object deletion synchronization rule upon "a link removal during the inbound synchronization phase".
ILM removes a link relationship between a metaverse object and a connector space object during the outbound synchronization phase, when:
- A connector with a staged import deletion is processed.
- A connector filter synchronization rule has disconnected a connector.
When the synchronization engine has removed a link between a connector space and a metaverse object during the inbound synchronization phase, it evaluates the object deletion synchronization rule to determine the consequence of the link removal.
The disconnected connector space object might have been the authoritative master for the metaverse object.
In this case, the metaverse object can become obsolete and must be removed.
Depending on your scenario, one option to identify an authoritative master is to use the connector space membership of the disconnected object.
For example, in case of the HR scenario in this article, it could be safe to delete a metaverse object after the synchronization engine has removed the link to an object in the HR connector space.
However, this is not necessarily true for all scenarios that involve a HR system I have seen while working with ILM.
So, please don’t take this as a silver bullet solution.
The connector space membership is in any case not sufficient as deletion criterion, if you have multiple masters in your environment.
The synchronization of user objects as contacts between two Active Directory forests is a good example for such a case.
The following picture shows an example for this.
In this case, just looking at the connector space membership of Britta Simon would not help you when an imported deletion is processed.
If the determination for whether it is safe to delete a metaverse object requires a more complex evaluation, you can accomplish this with the code of a rules extension.
In the current example, determining the object type of the disconnected object could be sufficient to make a deletion decision.
The following pseudo code snippet shows an example for this:
Public Function ShouldDeleteFromMV(…) As Boolean Implements …
ShouldDeleteFromMV = False
If csentry.ObjectType.Equals("Person") Then
ShouldDeleteFromMV = True
End If
End Function
Finally, there is one case, where a determination of a proper reaction is not necessary – when the removed link was also the last link to a metaverse objects.
The objective of ILM is to manage distributed identities from a central point.
If a metaverse object has no link relationship to any connector space object, the metaverse object becomes obsolete and must be deleted.
The deletion based on connector space membership, the deletion based on a determination in a rules extension and the deletion of a metaverse object when the last connector was removed are the three options you have to configure the object deletion rule.
The following illustration shows the related configuration dialog of the object-deletion synchronization rule..
In conjunction with the object deletion rule, I have seen two reoccurring issues in posts on this forum, which is why I want to discuss them briefly.
The first one is related to the default configuration of the object deletion rule:
| “I have imported a deletion from my HR system; however, after running a synchronization run, the metaverse object was not deleted. What is wrong with ILM?” |
The default configuration of the object deletion rule is to “Delete metaverse object when last connector is disconnected”. When you look at the picture below, you will notice that after processing the staged import deletion for Britta Simon from the HR system, you still have one connector left at the end of the synchronization run– the connector in the AD connector space.
This means, nothing is wrong with ILM; the object deletion rule didn’t delete the metaverse object because the deletion criterion was not satisfied.
The second issue is related to the “Rules Extension” setting:
| “I have configured the object deletion rule to make a determination in a rules extension because I want to log processed deletions in a file; however, after running a synchronization, my code was not processed but the metaverse object was deleted. What is wrong with ILM?” |
In this case, you need to know that “Delete metaverse object when last connector is disconnected” is a master rule that supersedes the other configurations.
In other words, regardless of your configuration, a metaverse object is always deleted when the last connector was removed.
If the deletion type of your object deletion rule is rules extension and the removed connector was as outlined in the following picture the last connector, your extension code is not processed.
Why should it be?
Since the synchronization engine has removed the last connector, there is no decision in conjunction with the metaverse object deletion your code can make.
Having the deletion of a metaverse object when the last connector was disconnected as sort of a “master” rule is necessary because ILM provides a central point of administration for distributed identities.
If a metaverse object has no linked relationship, there is nothing the object can manage anymore, which makes the metaverse object obsolete.
When the object-deletion synchronization rule deletes a metaverse object, all remaining link relationships are removed and the deprovisioning synchronization rule is triggered on the affected objects.
The second option to get the deprovisioning synchronization rule triggered is a related implementation in your provisioning code.
The synchronization engine evaluates your provision code:
- During a full synchronization run.
- When the synchronization engine applies an update to a metaverse object during a delta synchronization run.
Note
A link removal during the inbound synchronization phase that does not cause the deletion of the related metaverse object is a provisioning trigger.
Introduction to the deprovisioning APIs
The objective of your provisioning code is to create a new connector, to rename an existing connector and to disconnect existing connectors from a metaverse object.
In your disconnection scenario, you either might want to disconnect all connectors a metaverse object has or you might want to disconnect specific connector space objects from a metaverse object.
To support both cases, the rules extension framework provides two related API calls to disconnect connector space objects from metaverse objects:
- DeprovisonAll from the ConnectedMACollection class
- Deprovision from the CSEntry class
Each call triggers the deprovisioning synchronization rule on the affected connector space object.
In this context, it might be important to note that the synchronization engine deletes a metaverse object if the deprovisioning API has disconnected the last connector of a metaverse object.
If you would like to read more about how to disconnect objects with your provisioning code, see “Example: Deprovisioning an Object After a Specified Time”.
Deleting unmanaged objects
If you want to use ILM to initiate the deletion of objects in a connected data source, the affected objects must be managed objects (connectors).
As you have learned, you need a link removal during the outbound synchronization phase and a deprovisioning rule that would stage a deletion on the disconnected object.
Now, what can you do with unmanaged object objects (disconnectors)?
For example, you need to delete a set of unmanaged objects as part of a cleanup process.
As a rule of a thumb, you should clean up inaccurate data in your data source and import the deletions into ILM to remove the affected connector space objects.
However, for various reasons, this is not always possible.
There are also scenarios where you need a list of disconnectors to have a list of objects requiring your attention.
A command-line tool such as CSExport can help you in this scenario.
CSExport, which ships with ILM, is a tool that generates an XML-based drop file of your connector space content.
When running this tool, you can set a filter to include only disconnectors in the output file.
You can then use the content of the drop file to delete the objects in the connected data source directly.
If you want to use ILM to delete unmanaged objects, you need to bring them into the metaverse first, and then implement a processing chain as outlined in this article.
There are several possible implementation strategies for an ILM initiated cleanup; however, a detailed explanation of them is outside the scope of this article.
Please feel free to provide feedback if you would like to read about these implementation options.
Illustrated example for deleting objects with synchronization rules
At this point, it is probably time to look at how the concepts I have explained in this article apply to an end-to-end scenario.
The starting point of this example is a physical identity, Britta Simon, with a representation in two connected data sources CDS1 and CDS2 as outlined in the following picture.
The administrator of CDS1 deletes the representation of Britta in the connected data source CDS1.
During an import run on the management agent of CDS1, ILM stages the deletion on the connector for Britta in CS1.
As you can see in the previous picture, when ILM imports a deletion for a connector from a connected data source, the connector remains in the connector space with a modification type of delete.
The following picture shows and example for the properties of a connector with an imported deletion from a connected data source.
It is necessary to keep a connector with a modification type of delete in the connector space because, when processed during a synchronization run, the final deletion of the connector causes a link removal on the linked metaverse object, which is a trigger for a synchronization rule that must be processed.
During a following synchronization run, ILM deletes the connector, which also removes the link relationship between the connector space and the metaverse object..
As you have learned, a removal of the link relationship between the connector space object and the metaverse object during the inbound synchronization phase triggers the object deletion synchronization rule on the affected metaverse object.
The next processing step, depends on the configuration of the object deletion synchronization rule.
In the following sections, I will explain to you what ILM does for the following settings of this rule:
- Delete metaverse object when connector from CS1 is disconnected
- Delete metaverse object when last connector is disconnected
Deprovisioning based on the object deletion rule
When you configure the object-deletion synchronization rule to delete a metaverse object when a connector from CS1 is disconnected, ILM deletes the metaverse object that is the aggregated representation of Britta.
Because of the metaverse object deletion, ILM also removes the link to Britta’s connector space object in CS2.
The removal of the link relationship during the outbound synchronization phase triggers the deprovisioning synchronization rule on Britta’s connector space object CS2.
ILM stages a deletion on Britta’s connector space object in CS, if this is the result of processing the deprovisioning synchronization rule.
During the next export run, ILM applies the staged deletion to Britta’s representation in the connected data source CDS2, which also deletes her connector space object in CS2.
Deprovisioning base on the metaverse rules extension
You already know, that the object deletion rule is not the only place in the synchronization rule architecture to address deletions.
If you have configured the object deletion rule to delete a metaverse object when last connector is disconnected, ILM does not delete Britta’s metaverse, as there is still one connector left in the connector space CS2.
However, the link removal that is caused by the deletion of Britta’s connector space object in CS1, triggers provisioning.
If your provisioning code calls the deprovisioning API, ILM removes the link relationship between Britta’s metaverse and the connector space object in CS2.
In our scenario, the removed link between the metaverse object and the connector space object was the last connector the metaverse object had, which is why ILM deletes the metaverse object.
Like in the previous example, the link removal between the metaverse object and the connector space object during the outbound synchronization phase triggers the deprovisioning synchronization rule.
ILM stages a deletion on Britta’s connector space object in CS, if this is the result of processing the deprovisioning synchronization rule.
During the next export run, the staged deletion is applied to Britta’s representation in the connected data source CDS2, which also deletes her connector space object in CS2.
Summary
What you can take from this little walkthrough scenario in the previous section is that there are several ways to process the same information (the deletion of an object in a connected data source) throughout the system and to get the same processing results (a deletion in a target system).
Both implementations are equally good or bad, which means without knowing more details about the business requirements of a scenario, it is very hard to say what the optimal solution for you is.
A good understanding of how the synchronization rules work, is based on my experience paramount for an efficient implementation of a solution in your environment.
In this context, I can highly recommend the article a colleague of mine, Ahmad Abdel-wahed, wrote on Learning ILM.
ILM is not a “learning by doing” technology.
As soon as you have accidentally deleted several thousands of accounts, you will know what that means.
However, you will hopefully not need this kind of experience to trust me on this.
What you need to know about deletions can be summarized as follows:
- To delete an object in a connected data source by using ILM, you need a staged export deletion on the related connector space object.
- To stage an export deletion on a connector-space object, you need:
- A link removal on the connector space object during the outbound synchronization phase of a synchronization run.
- The deprovisioning synchronization rule to stage a deletion on the disconnected object.
- To force the deprovisioning synchronization rule to stage a deletion on a connector space object, need to configure the rule to:
- “Stage a delete on the object for the next export run”.
- “Determine with a rules extension” and return the related value in the function that is associated with this setting.
- To get a link between a metaverse object and a connector space object removed during the outbound synchronization phase, you need:
- The object deletion synchronization rule to deleted a metaverse object.
- Your provisioning to call a deprovisioning related API.
- To trigger the object-deletion synchronization rule, you need a link removal between a connector space object and a metaverse object during the inbound synchronization phase.
- To get a link between a connector space object and a metaverse object removed during the inbound synchronization phase, you need:
- A processed import deletion on a connector space object.
- The connector filter synchronization rule to filter a connector
The following flowchart outlines the complete process flow of getting an export deletion staged on a connector space object:
Additional Information
For more information, please see
- The Microsoft Identity Integration Server 2003 Technical Reference.
- Planning Synchronization Rules for MIIS 2003
Related Forum Posts
- Deprovisioning Question
- How to disconnect objects based on the MV value?
- No exception is Fired by MIIS even if connector does not exist
- Question regarding deletion
- Remove enties from pending export
- Deprovisioning from multiple sources
- Deleting users from AD
- disconnectors and re-joining
- Force MIIS to delete previously disconnected objects
- Deprovision rules extension - how to update data
- Deprovisioning in Active Directory
- Deprovisioning
- Object Deletion Rule
- Question regarding deletion
- Help configuring deprovisioning
- How to retrieve unmatched records (Disconnectors)