Office 365: Exchange Migrations and the data migrated information…

Article
07/03/2018

In Office 365 migration batches are the preferred method of establishing mailbox migrations to Office 365. When a migration batch is established the users included in the batch are represented as migration users within the service.

Within the context of each migration user is where we track the migration process. One of the data points contained within the migration process is the DATA MIGRATED tab. When enumerating the users within a migration batch, and selecting a migration user, this information is visible within the portal.

USER@domain.com

Status: Synced

USER@domain.com Skipped item details

Data migrated: 1.4 GB ‎(1,503,707,494 bytes)‎
Migration rate: 0 B ‎(0 bytes)‎
Error:
Report: USER@domain.com Download the report for this user

Last successful sync date: 7/2/2018 6:12:47 PM

Status:

Queued duration: 00:00:09.5015268
In-progress duration: 6.04:36:17.0329441
Synced duration: 36.10:02:28.6166962
Stalled duration: 02:12:43.4040899

One of the common questions and comments that I receive is that the DATA MIGRATED field may often reflect sizes that are slightly more to drastically more than the mailbox being migrated. This on occasion leads to confusion about the migration process and the data that was migrated. Here’s an example.

I recently migrated a mailbox from an on premises Exchange 2010 installation. With Exchange 2010 being the source we can already expect that the data migrated to the service to be more than the reported mailbox size due to the inconsistencies in the methods Exchange 2010 reports mailbox sizes. At the time that the mailbox migration completed the following was reported within the migration portal.

user@domain.com

Status: Completed

user@domain.com Skipped item details

Data migrated: 2.485 GB ‎(2,668,722,884 bytes)‎
Migration rate: 0 B ‎(0 bytes)‎
Error:
Report: user@domain.com Download the report for this user

Last successful sync date: 7/3/2018 3:14:56 PM

Status:

Queued duration: 00:00:08.5213717
In-progress duration: 01:01:30.1987283
Synced duration: 00:00:00
Stalled duration: 01:21:09.3851867

In this example the data migrated field shows 2.485 GB migrated. If we review the migration statistics associated with the mailbox we can get an idea of the reported mailbox size to be migrated.

PS C:\> Get-MigrationUserStatistics user@domain.com -IncludeReport | Export-Clixml -Path c:\temp\migration.xml

PS C:\> $stats=Import-Clixml -Path C:\temp\migration.xml

PS C:\> $stats.EstimatedTotalTransferSize

1.501 GB (1,612,092,317 bytes)

In this particular instance the estimated mailbox transfer size was 1.5 GB. If we look at the data migrated verses the estimated data transfer size the difference is almost 1 GB difference. Even accounting for reporting inconsistencies from Exchange 2010 mailbox size estimates 1 GB is a significant difference. Why has this happened?

The data migrated field is a summary counter of all data transmissions between Office 365 and the on premises Exchange environment. It is NOT a reflection of the data migrated verses the estimated mailbox size. In this particular instance the mailbox move process was interrupted several times. The interruptions may be the result of throttling, connection problems, or other migration interruptions.

7/3/2018 2:22:35 PM [CO2PR07MB2711] The job has been paused temporarily due to unfavorable server health, with request throttling state: 'StalledDueToTarget_DiskLatency'. It will automatically resume after '7/3/2018 2:23:35 PM'.

7/3/2018 2:28:38 PM [CO2PR07MB2711] The Microsoft Exchange Mailbox Replication service 'CO2PR07MB2711.namprd07.prod.outlook.com' (15.20.930.16 ServerCaps:FFFFFF, ProxyCaps:0FFFC7FD6DFDBF5FFFFFCB07FFFF, MailboxCaps:, legacyCaps:FFFFFF) is examining the request.

7/3/2018 2:28:38 PM [CO2PR07MB2711] Content from the Shard mailbox (Mailbox Guid: 6021ac05-d3f5-4bf4-ad03-f749aa02a1e4, Database: 545e461d-55da-4037-9c3e-d57a539e5e43) will be merged into the target mailbox.

7/3/2018 2:28:38 PM [CO2PR07MB2711] Connected to target mailbox 'live.domain.com\f28fad8a-546d-4965-8bcd-94806d53967d (Primary)', database 'NAMPR07DG170-db053', Mailbox server 'CO2PR07MB2711.namprd07.prod.outlook.com' Version 15.20 (Build 930.0).

7/3/2018 2:28:45 PM [CO2PR07MB2711] Connected to source mailbox 'live.domain.com\f28fad8a-546d-4965-8bcd-94806d53967d (Primary)', database 'DAG-Administration', Mailbox server 'MICHAEL.domain.com' Version 14.3 (Build 382.0), proxy server 'PROXY.domain.com' 15.1.1531.3 ServerCaps:, ProxyCaps:, MailboxCaps:, legacyCaps:0FFD6FFFBF5FFFFFCB07FFFF.

7/3/2018 2:28:47 PM [CO2PR07MB2711] Request processing continued, stage LoadingMessages.

7/3/2018 2:28:47 PM [CO2PR07MB2711] Stage: LoadingMessages. Percent complete: 20.

7/3/2018 2:41:04 PM [CO2PR07MB2711] Stage: CopyingMessages. Percent complete: 85.

7/3/2018 2:41:04 PM [CO2PR07MB2711] Copy progress: 17097/21426 messages, 1.299 GB (1,394,743,256 bytes)/1.501 GB (1,611,958,891 bytes), 30/67 folders completed.

7/3/2018 2:41:04 PM [CO2PR07MB2711] The job has been paused temporarily due to unfavorable server health, with request throttling state: 'StalledDueToTarget_DiskLatency'. It will automatically resume after '7/3/2018 2:42:04 PM'.

7/3/2018 2:49:49 PM [CO2PR07MB2711] The Microsoft Exchange Mailbox Replication service 'CO2PR07MB2711.namprd07.prod.outlook.com' (15.20.930.16 ServerCaps:FFFFFF, ProxyCaps:0FFFC7FD6DFDBF5FFFFFCB07FFFF, MailboxCaps:, legacyCaps:FFFFFF) is examining the request.

7/3/2018 2:49:55 PM [CO2PR07MB2711] Content from the Shard mailbox (Mailbox Guid: 6021ac05-d3f5-4bf4-ad03-f749aa02a1e4, Database: 545e461d-55da-4037-9c3e-d57a539e5e43) will be merged into the target mailbox.

7/3/2018 2:49:55 PM [CO2PR07MB2711] Connected to target mailbox 'live.domain.com\f28fad8a-546d-4965-8bcd-94806d53967d (Primary)', database 'NAMPR07DG170-db053', Mailbox server 'CO2PR07MB2711.namprd07.prod.outlook.com' Version 15.20 (Build 930.0).

7/3/2018 2:50:02 PM [CO2PR07MB2711] Connected to source mailbox 'live.domain.com\f28fad8a-546d-4965-8bcd-94806d53967d (Primary)', database 'DAG-Administration', Mailbox server 'MICHAEL.domain.com' Version 14.3 (Build 382.0), proxy server 'PROXY.domain.com' 15.1.1531.3 ServerCaps:, ProxyCaps:, MailboxCaps:, legacyCaps:0FFD6FFFBF5FFFFFCB07FFFF.

7/3/2018 2:50:04 PM [CO2PR07MB2711] Request processing continued, stage LoadingMessages.

7/3/2018 2:50:04 PM [CO2PR07MB2711] Stage: LoadingMessages. Percent complete: 20.

7/3/2018 3:00:05 PM [CO2PR07MB2711] Stage: CopyingMessages. Percent complete: 89.

In this example the move request was stalled several times. Each time after the stall the migration request is picked back up – and message loading occurs. This would be an example of additional data added to the transfer set that was not mailbox level items that would have been accounted for in the mailbox size.

If I was curious about the status of the migration and understanding the full migration status – how can I ensure the data was moved. When a move enters the finalization phase the source mailbox and target mailbox are locked. We have a mailbox folder verification process that is executed. The mailbox folder verification process iterates through each folder in the source mailbox and compares it to the target mailbox. Item counts and sizes are compared in real time. If there are any discrepancies identified the move will be placed into a failed state. The mailbox verification is logged into the migration log file and can be viewed by the administrator.

PS C:\> Get-MigrationUserStatistics user@domain.com -IncludeReport | Export-Clixml -Path c:\temp\migration.xml

PS C:\> $stats=Import-Clixml -Path C:\temp\migration.xml

PS C:\> $stats.Report.MailboxVerification

Source : 10188 [1.152 GB (1,237,342,091 bytes)]

SourceFAI : 27 [1.203 MB (1,261,537 bytes)]

Target : 10188 [1.464 GB (1,571,440,111 bytes)]

TargetFAI : 27 [1.227 MB (1,286,579 bytes)]

Corrupt : 0 [0 B (0 bytes)]

Large : 0 [0 B (0 bytes)]

Skipped : 0 [0 B (0 bytes)]

FolderSourcePath : /Top of Information Store/Inbox

FolderTargetPath : /Top of Information Store/Inbox

FolderSourceID : {0, 0, 0, 0...}

FolderTargetID : {0, 0, 0, 0...}

ParentSourceID : {0, 0, 0, 0...}

ParentTargetID : {0, 0, 0, 0...}

WKFType : Inbox

WKFTypeInt : 10

FolderIsMissing : False

FolderIsMisplaced : False

MismatchedFlagsCount : 0

MissingItemsInTargetBucket : 0 [0 B (0 bytes)]

MismatchedFlagsSample : {}

DuplicatedItemsInTargetBucket : 0 [0 B (0 bytes)]

MissingInSourceExtraItemsInTargetBucket : 0 [0 B (0 bytes)]

MismatchedSyncFolderIdExtraItemsInTargetBucket : 0 [0 B (0 bytes)]

NonMRSSyncedExtraItemsInTargetBucket : 0 [0 B (0 bytes)]

FolderExclusionExtraItemsInTargetBucket : 0 [0 B (0 bytes)]

MailboxGuid : f28fad8a-546d-4965-8bcd-94806d53967d

UnknownElements :

UnknownAttributes :

XmlSchemaType :

Source : 5129 [247.9 MB (259,986,587 bytes)]

SourceFAI : 3 [5.91 KB (6,052 bytes)]

Target : 5129 [500 MB (524,288,275 bytes)]

TargetFAI : 3 [8.085 KB (8,279 bytes)]