Share via

Mailbox migration from on-premise to Exchange online is failing with the following error: Transient error ObjectNotFoundException

Anonymous
2024-02-27T20:20:03+00:00

We have successfully migrated many mailboxes but one mailbox migration from on-premise to Exchange online is failing with the following error:

Transient error ObjectNotFoundException has occurred.

The inital sync is fine but the errors creep in after the migration is 95%, during the IncrementalSync phase.

An excerpt from the log from the user report on Exchange Admin Center (confidential info hidden):

2/27/2024 7:58:45 PM [CP7P275MB2536] The Microsoft Exchange Mailbox Replication service 'CP7P275MB2536.ZAFP275.PROD.OUTLOOK.COM' (15.20.7316.37 ServerCaps:FFFFFFFF, ProxyCaps:07FFFFFFFFFFFFFFC7DD2DFDBF5FFFFFCB07EFFF, MailboxCaps:, legacyCaps:FFFFFFFF) is examining the request.

2/27/2024 7:58:45 PM [CP7P275MB2536] Resource information was found with reservation.

2/27/2024 7:58:45 PM [CP7P275MB2536] Content from the Shard mailbox (Mailbox Guid: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx, Database: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx) will be merged into the target mailbox.

2/27/2024 7:58:45 PM [CP7P275MB2536] Connected to target mailbox 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.onmicrosoft.com\xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx (Primary)', database 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx', Mailbox server 'CP7P275MB2536.ZAFP275.PROD.OUTLOOK.COM' Version 15.20 (Build 7316.0).

2/27/2024 7:58:48 PM [CP7P275MB2536] Connected to source mailbox 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.onmicrosoft.com\xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx (Primary)', database 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx', Mailbox server 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.contoso.com' Version 14.3 (Build 513.0), proxy server 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.contoso.com' 14.3.498.0 ServerCaps:, ProxyCaps:, MailboxCaps:, legacyCaps:05FFFF.

2/27/2024 7:58:49 PM [CP7P275MB2536] Request processing continued, stage IncrementalSync.

2/27/2024 7:58:50 PM [CP7P275MB2536] Folder hierarchy changes reported in source 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.onmicrosoft.com\xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx (Primary)': 49 changed folders, 0 deleted folders.

2/27/2024 7:58:50 PM [CP7P275MB2536] Job starts to validate migration user.

2/27/2024 7:59:04 PM [CP7P275MB2536] Stage: IncrementalSync. Percent complete: 95.

2/27/2024 7:59:04 PM [CP7P275MB2536] Transient error ObjectNotFoundException has occurred. The system will retry (2/60, 2/611).

The transient error repeats 60 times before becoming a permanent error.

The underlying error from Get-MoveRequestStatistics powershell command run on Exchange online is:

PS C:\Windows\system32> $r.Report.Failures[24]

Timestamp : 2024/02/27 10:08:58

FailureType : ObjectNotFoundException

FailureHash : 9b0a

FailureCode : -2146233088

MapiLowLevelError : -2147221233

FailureSide : Target

FailureSideInt : 2

ExceptionTypes : {ObjectNotFound, Storage, DataProviderPermanent, Exchange...}

ExceptionTypesInt : {106, 70, 102, 1...}

WorkItem : IncrementalSync

Message : Cannot set search criteria in SearchFolder. Try using fewer keywords at the same time, reducing the number of users in the From, To, Cc, and Bcc 

                fields, and reducing the number of mailboxes that are searched at the same time. --> **MapiExceptionNotFound: Unable to SetSearchCriteria.** 

                (hr=0x8004010f, ec=-2147221233) 

                0.35250:0201027D, 1.36674:30000000, 1.61250:00000000, 1.45378:02000000, 1.44866:58000000, 16.55847:EA000000, 

                17.43559:0000000050030000000000000000000000000000, 20.52176:140F949C2200103100000000, 20.50032:140F949C9217000000000000, 0.53414:140F949C, 

                0.35180:00000000, 255.23226:140F949C, 255.27962:30000000, 255.17082:0F010480, 0.27745:00000000, 4.21921:0F010480, 255.27962:FA000000, 

                255.1494:07000000, 0.38698:47000000, 1.41134:47000000, 4.65304:0F010480, 7.62138:010000018169F2E387000000, 7.45754:08000030751A2C2E87000000, 

                4.52728:0F010480, 7.61228:010000018169F2E387000000, 4.39640:0F010480, 1.63016:30000000, 8.45434:B2AB09ADA0CC4B4C997538C48A88B65487000000, 

                1.46798:14000000, 5.10786:0000000031352E32302E373331362E3033343A435037503237354D42323533363A32616430363434362D383265332D343733642D623833382D306531646364 

                6266623338333A3135343735363A2E4E455420362E302E323600000000, 7.51330:D7C3391D7C37DC0802000000, 0.39570:02000000, 1.64146:30000000, 1.33010:30000000, 

                2.54258:0F010480, 1.33010:30000000, 2.54258:0F010480, 255.1750:1F000430, 255.31418:C3000000, 0.22753:D2000000, 255.21817:0F010480, 0.40514:C8000000, 

                4.44610:0F010480, 0.30353:0B000A36, 4.19089:0F010480, 0.18065:DA000000, 4.26257:0F010480

DataContext : Mrs#ProxyService, ADInconsistencyCheck, NonInteractiveOther 

                -------- 

                Operation: IDestinationFolder.SetSearchCriteria 

                xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.onmicrosoft.com\xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx (Primary) 

                OperationSide: Target 

                Restriction: Restriction: COUNT[100, AND[count:1, CONTENT[ptag:0xeaf001f, PrefixOnWords, Loose, val:[Tag:0xeaf001f, Value:"BERNARD"(string)]]]] 

                EntryIDs: [count:1, [len=46, data=000000002B2B30723EEEF64A807DE470EA2D88B501008AEB70073A4A5D49ABD7B01E89D9AC960030751A2C2E0000]] 

                Flags: Recursive, FailOnForeignEID 

                FolderId: entryId [len=46, data=000000002B2B30723EEEF64A807DE470EA2D88B5010031E86643737E814FB7D717592AC5BA9500018169F2E30000] 

                Folder: type Search, wkf: None, RegularStorageSourceMailbox, entryId [len=46, 

                data=000000002B2B30723EEEF64A807DE470EA2D88B5010031E86643737E814FB7D717592AC5BA9500018169F2E30000], parentId [len=46, 

                data=000000002B2B30723EEEF64A807DE470EA2D88B5010031E86643737E814FB7D717592AC5BA950000000000B70000] 

                >>>> Current WorkItem: IncrementalSync, Scheduled WorkItems: WriteFolderMessages(P:31,R:-63844625243557,S:63844625243557,C:6297,D:0); 

                WriteFolderMessages(P:0,R:-63844625249854,S:63844625249854,C:171,D:0); EnumerateFolderMessages(P:3202141,R:-63844625250026,S:63844625250026,C:62,D:0); 

                EnumerateFolderMessages(P:3202204,R:-63844625250089,S:63844625250089,C:46,D:0); 

                EnumerateFolderMessages(P:3202251,R:-63844625250136,S:63844625250136,C:62,D:0); 

                EnumerateFolderMessages(P:3202313,R:-63844625250198,S:63844625250198,C:62,D:0); 

                EnumerateFolderMessages(P:3202376,R:-63844625250261,S:63844625250261,C:62,D:0); CopierFinished(P:3202438,R:-63844625250323,S:63844625250323,C:62,D:0); 

                InitialSeedingComplete(P:3202501,R:-63844625250386,S:63844625250386,C:76567,D:0); DoPeriodicCheck(P:0,R:0,S:0,C:312,D:0)

StackTrace : at Microsoft.Exchange.Data.StoreObjects.MapiAccessorExtensionMethods.SetSearchCriteria(IStoreSession storeSession, IMapiFolder mapiFolder, 

                Restriction restriction, Byte[][] folderScopeEntryIds, SearchCriteriaFlags searchCriteriaFlags) 

                   at Microsoft.Exchange.Data.Storage.CoreFolder.SetSearchCriteria(SearchFolderCriteria searchFolderCriteria, SetSearchCriteriaFlags 

                setSearchCriteriaFlags) 

                   at Microsoft.Exchange.Data.Storage.BigFunnelCoreFolderDecorator.SetSearchCriteria(SearchFolderCriteria searchFolderCriteria, SetSearchCriteriaFlags 

                setSearchCriteriaFlags) 

                   at Microsoft.Exchange.MailboxReplicationService.StorageDestinationFolder.Microsoft.Exchange.MailboxReplicationService.IDestinationFolder.SetSearchCri 

                teria(RestrictionData restriction, EntryId[] entryIds, SearchCriteriaFlags flags) 

                   at Microsoft.Exchange.MailboxReplicationService.DestinationFolderWrapper.Microsoft.Exchange.MailboxReplicationService.IDestinationFolder.SetSearchCri 

                teria(RestrictionData restriction, EntryId[] entryIds, SearchCriteriaFlags flags) 

                   at Microsoft.Exchange.MailboxReplicationService.MailboxCopierBase.CopySearchFolderCriteria(FolderRecWrapper folderRec, IDestinationFolder destFolder) 

                   at Microsoft.Exchange.MailboxReplicationService.MailboxCopierBase.CopyFolderProperties(FolderRecWrapper folderRec, ISourceFolder sourceFolder, 

                IDestinationFolder destFolder, FolderRecDataFlags dataToCopy, Boolean& wasPropertyCopyingSkipped) 

                   at Microsoft.Exchange.MailboxReplicationService.MailboxCopierBase.UpdateFolderAfterHierarchyChange(SyncContext ctx, FolderRecWrapper srcFolderRec) 

                   at Microsoft.Exchange.MailboxReplicationService.MailboxCopierBase.ApplyHierarchyChanges(SyncContext ctx, MailboxChangesManifest changes) 

                   at Microsoft.Exchange.MailboxReplicationService.BaseJob.EnumerateAndApplyIncrementalChanges(MailboxCopierBase mbxContext, SyncContext syncContext, 

                MailboxChangesManifest hierarchyChanges, EnumerateContentChangesFlags contentChangesFlags) 

                   at Microsoft.Exchange.MailboxReplicationService.MoveBaseJob.ApplyIncrementalChanges(Int32 iterationsCount, UInt64 totalChurn, Boolean finalSync) 

                   at Microsoft.Exchange.MailboxReplicationService.MoveBaseJob.IncrementalSync(Int32 iterationsCount, UInt64 totalChurn) 

                   at Microsoft.Exchange.MailboxReplicationService.Job.ExecuteWorkItemCallback(WorkItem workItem)

InnerException : MapiExceptionNotFound: MapiExceptionNotFound: Unable to SetSearchCriteria. (hr=0x8004010f, ec=-2147221233) 

                0.35250:0201027D, 1.36674:30000000, 1.61250:00000000, 1.45378:02000000, 1.44866:58000000, 16.55847:EA000000, 

                17.43559:0000000050030000000000000000000000000000, 20.52176:140F949C2200103100000000, 20.50032:140F949C9217000000000000, 0.53414:140F949C, 

                0.35180:00000000, 255.23226:140F949C, 255.27962:30000000, 255.17082:0F010480, 0.27745:00000000, 4.21921:0F010480, 255.27962:FA000000, 

                255.1494:07000000, 0.38698:47000000, 1.41134:47000000, 4.65304:0F010480, 7.62138:010000018169F2E387000000, 7.45754:08000030751A2C2E87000000, 

                4.52728:0F010480, 7.61228:010000018169F2E387000000, 4.39640:0F010480, 1.63016:30000000, 8.45434:B2AB09ADA0CC4B4C997538C48A88B65487000000, 

                1.46798:14000000, 5.10786:0000000031352E32302E373331362E3033343A435037503237354D42323533363A32616430363434362D383265332D343733642D623833382D306531646364 

                6266623338333A3135343735363A2E4E455420362E302E323600000000, 7.51330:D7C3391D7C37DC0802000000, 0.39570:02000000, 1.64146:30000000, 1.33010:30000000, 

                2.54258:0F010480, 1.33010:30000000, 2.54258:0F010480, 255.1750:1F000430, 255.31418:C3000000, 0.22753:D2000000, 255.21817:0F010480, 0.40514:C8000000, 

                4.44610:0F010480, 0.30353:0B000A36, 4.19089:0F010480, 0.18065:DA000000, 4.26257:0F010480

UnknownElements :

UnknownAttributes :

XmlSchemaType :

The underlying error from the log above is therefore:

Cannot set search criteria in SearchFolder. Try using fewer keywords at the same time, reducing the number of users in the From, To, Cc, and Bcc fields, and reducing the number of mailboxes that are searched at the same time. --> MapiExceptionNotFound: Unable to SetSearchCriteria.

This mailbox soes not have any search folders in addition to the default search folders in Outlook. I did run outlook.exe /cleanfinders before migrating to ensure there would be no issues with searchfolders.

Microsoft 365 and Office | Subscription, account, billing | For business | Other

Locked Question. This question was migrated from the Microsoft Support Community. You can vote on whether it's helpful, but you can't add comments or replies or follow the question.

0 comments

6 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Anonymous
    2024-03-01T12:35:19+00:00

    Hi OS92,

    I finally managed to migrate this mailbox. In the end, it turns out that there were old search folders in the mailbox, despite running the outlook.exe /cleanfinders command. Seems this command does not cleanout all search folders. The particular search folder that was causing the migration failure was titled: EAS Search Device:XXXXXXXXXXXXXX. It appears to have been created in the past by a smartphone that was linked to the mailbox. These folders remain even though this Exchange ActiveSync smartphone hasn't been used for years.

    Here are the steps I followed:

    1. Download MFCMAPI from github: https://github.com/microsoft/mfcmapi?tab=readme-ov-file
    2. Extract and double-click MFCMapi.exe from the computer with the mailbox that will not migrate
    3. Select Session -> Logon
    4. Choose the Outlook profile and click Ok
    5. Double-click on the mailbox (Default Store column will show as "True")
    6. Select Tools -> Entry ID -> Open given entry ID
    7. Paste the entry ID mentioned in the error shown in PowerShell into the "Entry ID:" field and click OK. In my case, from the log file above it was "000000002B2B30723EEEF64A807DE470EA2D88B5010031E86643737E814FB7D717592AC5BA9500018169F2E30000"
    8. A new window opens showing the Entry ID that was located. In my case I could confirm it was correct because one of the fields in the right pane showed the value from the log file above "Value:"BERNARD"(string)"
    9. In the left pane, right-click the name of the folder and say Delete Folder. In my case, the folder was titled: "EAS Search Device:XXXXXXXXXXXXXX"
    10. A pop-up will ask you to confirm deletion. Make sure to choose the "Hard Deletion" checkbox and click OK
    11. Close MFCMAPI. Open Outlook and ensure that it doesn't complain about mailbox corruption and give it time to sync the change from the local copy to the server
    12. Run the migration from scratch, from the EAC web page. Now it doesn't get stuck at the IncrementalSync step anymore
    13. Approve the migration (this was needed in my case due to corrupt items that I had previously deleted that were in the Recoverable Items folder)
    14. Migration completes successfully

    Some notes:

    • Be careful of deleting items using MFCMAPI, you could cause issues with the mailbox
    • Within MFCMAPI, search folders are stored in a folder in the "Root Container" called "Finder". However, it seems that the search folder that had issues was not listed in "Finder". When I search for the Entry ID as per the steps above, it found it under a different "Finder" folder. I could not find this folder manually, seems to be hidden. There are multiple "Finder" folders?
    • In the log file above, the error makes mention of three Entry IDs. Note that the first entry ID could not be found - it was missing and hence the migration tool failed. The second entry ID was the one that referenced the missing entry ID and therefore I deleted this one. The third Entry ID refers to the "Finder" folder. I would not delete this folder. To clarify from the log above, here are the three entry IDs:
      • 000000002B2B30723EEEF64A807DE470EA2D88B501008AEB70073A4A5D49ABD7B01E89D9AC960030751A2C2E0000 - missing
      • 000000002B2B30723EEEF64A807DE470EA2D88B5010031E86643737E814FB7D717592AC5BA9500018169F2E30000 - EAS Search Device:XXX -> delete this
      • 000000002B2B30723EEEF64A807DE470EA2D88B5010031E86643737E814FB7D717592AC5BA950000000000B70000 - Finder -> don't delete

    I hope this helps somebody in future. Here is the reference that helped me find the corruption: https://support.quest.com/archive-shuttle/kb/4289024/howto-check-item-properties-using-outlookspy-or-mfcmapi

    1 person found this answer helpful.
    0 comments
  2. Anonymous
    2024-03-01T07:37:16+00:00

    Hi OS92,

    I'm glad to know I'm not alone!

    I have not yet got a solution. Eleni has not yet replied further to my comment above. Also, Microsoft support are not helping - I did have a conference call to explain the issue in detail. Despite this being a clear error with the migration tool that fails during the IncrementalSync process, they aren't willing to fix the tool or even understand the problem further. Microsoft support also claim they are unable to use the references shown in the log file to help me pin point the object in question in the mailbox that is causing the migration tool the problems. I have offered to use MFCMAPI tool to delete the problem in the mailbox so that the migration could continue but they are not able to understand the log file error in that level of detail.

    I also tried running the migration from PowerShell in order to set a BadItemLimit. This was to test whether the issue lay with the Data Consistency Score (DCS) mechanism. Turns out that the problem was experienced even without DCS. For your information, here is the command:

    $onpremCred = Get-Credential

New-MoveRequest -Identity *** Email address is removed for privacy *** -Remote -RemoteHostName 'XXXX.contoso.com' -RemoteCredential $onpremCred -TargetDeliveryDomain 'XXXXXX.mail.onmicrosoft.com' -BadItemLimit 40

    Microsoft Support are giving me problems about the fact that we are migrating from Exchange 2010. Perhaps if you log the call you might get better assistance as you are migrating from Exchange 2019?

    Kind regards,

    0 comments
  3. Anonymous
    2024-02-29T14:35:23+00:00

    Hi

    I have exactly the same problem in my environment with EX2019 hybrid (CU13 with last SU).

    The same error on some mailboxes during migration to exchange on-line.

    Do you have any solution ?

    Maciek

    0 comments
  4. Anonymous
    2024-02-28T11:55:06+00:00

    Hi Eleni,

    Thank you for your reply. I have run through the sections troubleshooting migration issues page. Most sections don't apply because we have migrated many mailboxes successfully so our overall migration setup is correct. This issue is related directly with the mailbox in question. Here are answers to your questions:

    1. I am using Exchange Admin Center (EAC).
    2. I have run the migration on this mailbox 5 or 6 times. I have recreated the migration batch 5 times. It failed initially in a migration batch with other successfully migrated mailboxes. Since then I have run the mailbox in its own migration batch. The mailbox is about 100 or 200 Mb and the overall data transfer once the migration attempt fails is about 3.5 GB according the the EAC.
    3. Internet speed is 100 mbps. Other mailboxes transfer in about 30 to 50 minutes (about 8 GB). No speed issues with other mailbox migrations.
    4. Windows Server 2008 R2 and Exchange 2010 SP3 RU 32 - Version 14.3 (Build 513.0).
    5. By hybrid server I assume you refer to Exchange Online. According to the MigrationUserReport on the EAC it is Version 15.20 (Build 7316.0)
    6. There is one mailbox/CAS/Hub server, and it is located in the same location as the AD site
    7. Hybrid was setup on the mailbox server referred to in point 6
    8. There is a Forefront TMG 2010 in place but MRS was published as per https://learn.microsoft.com/en-us/previous-versions/exchange-server/exchange-141/mt805114(v=exchg.141)?redirectedfrom=MSDN and we have no issues with other migrations

    I have logged a support call but so far have had poor response. Level 1 passed it to level 2 and they are struggling to understand the problem despite the clear error message in the log files. Surely I can't be the first to ever experience the attached errors?

    Thank you for your assistance.

    Kind regards,

    0 comments
  5. Anonymous
    2024-02-28T11:09:19+00:00

    Hello BRBM,

    Greetings!

    Thank you for posting on Microsoft Community!

    If you receive an error after migrating the mailbox from on-premise to Exchange online, please run the guided walkthrough hybrid migration troubleshooter located at Troubleshoot migration issues in Exchange Server hybrid environment

    In order to resolve your issue in a timely manner, we need to collect further information on your environment:

    1. Please describe the method you are using to migrate mailboxes to Office 365. Are you using the Exchange Admin Center (EAC) or Remote PowerShell?
    2. Can you provide me with information on the average size of the mailboxes which are part of this migration batch?
    3. What is the speed of your link to the internet and do you recall if you had been experiencing any disruption to your internet connectivity during the moves?
    4. What versions of Windows & Exchange are running on the mailbox server?
    5. Can you confirm the version of OS and Exchange on the hybrid server?
    6. Are the Hybrid servers and Mailbox servers in the same AD site/location?
    7. How many hybrid servers do you have and are they load balanced?
    8. What firewalls or proxy servers do you use and how is the MRS published?

    I'd also suggest you to submit a support ticket to Microsoft support team, they have more resources and permissions to help you remotely in a timely manner:

    1. Go to the admin center at https://admin.microsoft.com. If you get a message that says you don't have permission to access this page or perform this action, you aren't an admin. For more information, see Who has admin permissions in my business?.
    2. On the bottom right side of the page, select Help & support.
    3. Type a question or keyword into the text box. If you get a drop-down list, select the one closest to your question, or continue typing your question, then press Enter.
    4. If the results don't help, at the bottom, select Contact Support.
    5. Enter a description of your issue, confirm your contact number and email address, select your preferred contact method, and then select Contact me. The expected wait time is indicated in the Contact support pane.

    Thank you for your time and cooperation. Wish you have a nice day!

    If you need further assistance, feel free to reply us! We will continue to support you!

    0 comments