Partager via

Migrate to SharePoint Online (O365–SPO) from On-Premise SharePoint or fileshare using new PowerShell migration APIs

  • Article

  •  

  • **Updated to include SharePoint On-Prem to Office 365 SharePoint Online Scenario

  • Important

    • This documentation covers features that are currently in preview and not yet finalized for use in production code.
    • Replace <TENANT> with actual tenant prefix, <SPOSite> with SPO Site prefix and <On-PremSPSite> with SP On-Prem Site Prefix in all PowerShell cmdlets.
    • Triple-click in code box to copy Code / PowerShell cmdlets if they don’t fit on screen!

  • Prerequisites

    • Valid Office 365 subscription

    • Valid Microsoft Azure subscription

    • Install SharePoint Online management shell version 16.0.4017.1200 or above from here. More details at https://technet.microsoft.com/en-us/library/fp161372.aspx

    • Run SharePoint Online management shell as Administrator and connect SPO service using the below cmdlet

      Connect-SPOService -Url https://<TENANT>-admin.sharepoint.com

  •  To keep things simple, lets divide the entire process in few steps:   

  • Step 1 - Create SPO Migration Package

  • Step 1a – Create SPO Migration Package from SharePoint On-Prem Content

    • RDP to a WFE for On Premise SharePoint Server

    • Open SharePoint Management Shell as administrator

    • Export the Web / Site Collection using the below cmdlet

      Export-SPWeb -Identity "https://<On-PremSPSite>" -ItemUrl "/OnPremDocLib" -Path "C:\SPOnPremExport" -NoFileCompression -IncludeVersions 4

    • Note:
    • Copy Data: Copy the Contents of this folder to the location on machine which has SharePoint Online Management Shell installed. i.e. I copied contents from C:\SPOnPremExport from my SharePoint On-Prem Server to C:\MigrationPackages\SPOTemp on my laptop.

image

  • OR

  • Step 1b – Create SPO Migration Package from a file share

  • Note:

    • OutputPackagePath folder hierarchy will be created if it doesn’t exist

  •  

  • Step 2 - Prepare Final SPO Migration Package

    • Prepare data / files to be migrated using below cmdlet

    • For Step 1a

      ConvertTo-SPOMigrationTargetedPackage -SourceFilesPath "C:\MigrationPackages\SPOTEMP" -SourcePackagePath "C:\MigrationPackages\SPOTEMP" -OutputPackagePath "C:\MigrationPackages\SPOMigrationPackage" -TargetWebUrl "https://<SPOSite>.sharepoint.com/" -TargetDocumentLibraryPath "SPOMigration"

SNAGHTMLbecfed5

    • For Step1b

      ConvertTo-SPOMigrationTargetedPackage -SourceFilesPath "\\nishant-lptp\SPOTEMP" -SourcePackagePath "C:\MigrationPackages\SPOTEMP" -OutputPackagePath "C:\MigrationPackages\SPOMigrationPackage" -TargetWebUrl "https://<SPOSite>.sharepoint.com/" -TargetDocumentLibraryPath "SPOMigration"

    • There are six required parameters to enter (others are optional)

      1. TargetwebURL: points to the destination Web
      2. SourceFilesPath: points to the you want to migrate
      3. SourcePackagePath: points to your Temporary package folder
      4. OutputPackagePath: points to your final package folder
      5. TargetDocumentLibraryPath: the path to your destination library
      6. Credentials: SPO credential that has admin rights to the destination site

    • More Info: ConvertTo-SPOMigrationTargetedPackage - https://technet.microsoft.com/en-us/library/mt143609.aspx

  •  

  • Step 3 - Upload SPO Migration Package to Azure Blob Containers

    • Follow the below steps to get Storage Account Name and two container names inside it. If you already have a storage account, you could use the same

      • Login to Azure Portal via https://manage.windowsazure.com
      • Browse to Storage > Create a new storage account e.g. nO365mig
      • In newly created storage account create two containers e.g. migration-files, migration-package

    • Prepare to upload the prepared data / files to Azure Blob Containers using below cmdlet

    • For Step 1a

      $azurelocations = Set-SPOMigrationPackageAzureSource -SourceFilesPath "C:\MigrationPackages\SPOTEMP" -SourcePackagePath "C:\MigrationPackages\SPOMigrationPackage" -FileContainerName migration-files -PackageContainerName migration-package -AccountName nO365mig -AccountKey "REAPLCE_WITH_YOUR_AZURE_STORAGE_ACCOUNT_ACCESS_KEY"

    • For Step1b

      $azurelocations = Set-SPOMigrationPackageAzureSource -SourceFilesPath "\\nishant-lptp\SPOTEMP" -SourcePackagePath "C:\MigrationPackages\SPOMigrationPackage" -FileContainerName migration-files -PackageContainerName migration-package -AccountName nO365mig -AccountKey "REAPLCE_WITH_YOUR_AZURE_STORAGE_ACCOUNT_ACCESS_KEY"

    • More Info: Set-SPOMigrationPackageAzureSource - https://technet.microsoft.com/en-us/library/mt143606.aspx

  •  

  • Step 4 - Submit Migration Job

    • Submit Migration Job to transfer data from Source <to> Azure Blob Containers <to> SPO Site Collection using below cmdlet

      Submit-SPOMigrationJob -TargetWebUrl "https://<SPOSite>.sharepoint.com/" -MigrationPackageAzureLocations $azurelocations

    • Above cmdlet is generic and applies to both Step1a and Step1b

    • More Info: Submit-SPOMigrationJob - https://technet.microsoft.com/en-us/library/mt143605.aspx

  •  

  • Step 5 (Optional) - Check Migration Job Status

    • Check migration job (to transfer data from Azure Blob Containers to SPO Site Collection) status using below cmdlet

      Get-SPOMigrationJobStatus -TargetWebUrl https://<SPOSite>.sharepoint.com/

       

  • Step 1a

  • Step 1b

  •  

  • Step 6 (Optional) - Remove Migration Job

    • In case if the migration job is stuck or if it needs to be removed for any reason you could use the below cmdlet (Note: This will work only if the job is not already completed)

      Remove-SPOMigrationJob -JobId e79c5022-3aa1-446c-a8e6-2431063706fa -TargetWebUrl https://<SPOSite>.sharepoint.com/

    • More Info: Remove-SPOMigrationJob - https://technet.microsoft.com/en-us/library/mt143607.aspx

Comments

  • Anonymous
    July 19, 2015
    Great article… Thanks for your great information, the contents are quiet interesting. I will be waiting for your next post.

  • Anonymous
    August 21, 2015
    if you have office 365 do you need to have a separate azure account? we have plenty of storage under available under office 365.

  • Anonymous
    September 03, 2015
    Thanks a lot Nishant for the great article.

  • Anonymous
    September 03, 2015
    @Charles: Yes, as I mentioned in requirements, valid Microsoft Azure subscription is required.

  • Anonymous
    September 03, 2015
    Nice one Nishant

  • Anonymous
    September 04, 2015
    I can confirm that the method works beautifully. However, I seem to have a bit of trouble. I am trying to figure why it takes so long for the job to finish on azure before it shows up on the sharepoint site. Is this common? Should there be a significant delay? Maybe I'm missing something here?

  • Anonymous
    September 04, 2015
    @Josh: Thanks, what is the size of Content being migrated and what is the approx. time it took? Is the status of job (Step 5) "Processing" when it takes time?

  • Anonymous
    September 04, 2015
    Since im just testing its about 8-12 mb. Processing takes about 10 seconds at most.

  • Anonymous
    September 14, 2015
    @Josh, there could be a geo difference in your azure instance and O365 instance. For me, both in same geo location and it takes minutes to move MBs data to SPO lib.

  • Anonymous
    September 17, 2015
    Thank you very much Nishant for this step by step explanation.
    Do you know how to migrate DocLib and each documents permissions?[CSV file logic]

  • Anonymous
    September 20, 2015
    Wonderful article. Nishant I was working on this PowerShell and found document created and modified by are not getting carried over. Everything are under System Account. Do you aware of this issue and any workaround?

  • Anonymous
    September 20, 2015
    @Jeevan @Senthi: Do you have User Mapping file in place in prescribed format as mentioned athttps://technet.microsoft.com/en-us/library/mt143609.aspx?

  • Anonymous
    September 22, 2015
    Hi,When i am migrating a site at ConvertTo-SPOMigrationTargetedPackage i got unrecognized GUID Format error.Can you help me in this case please

  • Anonymous
    September 22, 2015
    @Ratnakar: Please explain how did you perform Step 1. Was it SharePoint On-Prem scenario? Did you specify -ItemUrl with Export-SPWeb?

  • Anonymous
    September 22, 2015
    I tried Sharepoint OnPremise Scenario without itemurl parameter.Is it possible to migrate entire subsite directly into Sharepoint online ? I tried to take entire Subsite to Sharepoint Online.

  • Anonymous
    September 22, 2015
    The comment has been removed

  • Anonymous
    September 24, 2015
    Hey When i am trying to migrate my sharepoint list to sharepoint online i am getting file not found error in step 2.What might be the reason do you have any idea ??

  • Anonymous
    September 24, 2015
    The comment has been removed

  • Anonymous
    September 25, 2015
    @Binoj: Please ensure you are doing step 1 correctly as mentioned in blog

  • Anonymous
    September 25, 2015
    @Jeevan: I haven't yet tested this approach, will test and update the blog when I get a chance.

  • Anonymous
    September 29, 2015
    @Nishant: I am able to use CSV file user mapping approach. I was using wrong SSID values for AD users. With proper SSID values, I am able to migrate Author and Editor field values properly.
    Thanks.

    • Anonymous
      August 10, 2016
      Can anyone light some insight on the user mapping file internals. What is the correct SSID that I must use to put together the CSV and where do I get it?

  • Anonymous
    September 29, 2015
    @Jeevan: Thank you for the update.

  • Anonymous
    October 09, 2015
    Hey, 'Parameters specify a Document Library but manifest does not contain a matching Document Library node' during step 2.. what error is this about. Do you have any idea.. Please help.

  • Anonymous
    October 16, 2015
    @Fresher: Please detail out your step 1.

  • Anonymous
    October 22, 2015
    Hey.. Sorry.. I gave a List in '-ItemUrl' in step-1 and was trying with '-TargetDocumentLibraryPath' in step-2. Just saw the '-TargetListPath' parameter and tried with that and it worked.

  • Anonymous
    November 02, 2015
    Hi Nishant,

    To getting packages (xml files)
    is there any other alternative for "Export-SPWeb" command or can we generate these (8 xml packaged files) from the content database. because we don't have a permission on SP server 2010.
    By Server object model (using Microsoft.sharepoint.deployment.dll) i am able to generate these packages.

    my requirement is we need to migrate data from on-prem to O365 site using your concept (Azure pipeline).

    Thanks in advance.

  • Anonymous
    November 02, 2015
    Hey.. I used my office 365 import storage account for both -FileContainerName and -PackageContainerName and also as -AccountName in step-3. But while submitting spo migration job in step-4, i received an error as ' The specified Azure source file and manifest containers cannot be identical'.
    Can I use my office 365 import storage account instead of azure storage account..? If so what error is this about.. Please help.

  • Anonymous
    November 03, 2015
    @Gopi: You will need to use Export-SPWeb. There is no other alternative that I know of...

  • Anonymous
    November 03, 2015
    @Fresher: 1) The error is self-explanatory; you need to use two separate containers. 2) As I mentioned under prerequisites section of blog, you will need Azure Account.

  • Anonymous
    November 03, 2015
    Ok dude, I have an azure account and created a storage with two containers. I mentioned all of them in the $azurelocations. But I was wondering how will I provide authentication for the account. Coz, when i submitted the migration job I got error as 'Access denied. You do not have permission to perform this action or access this resource'.
    There was one parameter -Credential in Submit-SPOMigrationJob. So I thought I could provide authentication for azure account through that. But that gave me an error ' The Login server cannot issue the requested compact encrypted ticket because a Data Encryption Key (DEK) has not been uploaded to the site'. I could not recognize this error.. Please help.

  • Anonymous
    November 11, 2015
    Hi Nishant,
    Great post! but if I understand correctly this method is only available for SharePoint 2010 or SharePoint 2013 On Premises, what about SharePoint 2007?, the stsadm operation Export does not have an option for library or list.

    Thanks in advance.

  • Anonymous
    November 16, 2015
    @Fresher: Please open a support case.

    • Anonymous
      January 07, 2016
      Hi Nishant,Now I understood the commands and working fine.. Thanks..I'm eagerly waiting to know if you guys are working on the api that helps to migrate the whole site and when it would be out for usage..

  • Anonymous
    November 16, 2015
    @Ana: Correct, this method is available only for SharePoint 2010 or SharePoint 2013 On Premises like you mentioned. SharePoint 2007 will need an upgrade to SharePoint 2010 / SharePoint 2013 prior to SPO migration.

  • Anonymous
    November 25, 2015
    Hi Nishant,

    Can this migration approach be used for migrating content from one SPO site collection in one tenant to another SPO site collection on another tenant?

  • Anonymous
    November 27, 2015
    I have tried migrating from SharePoint On-Premise 2010 and 2013 and also from File Folder data to SharePoint Online, all steps gets completed without any error, however we can't see any migrated data and migration job gets completed in few seconds. something is going but not sure what.
    Is it because I am using trial version of both Office 365 and Microsoft Azure?
    It would be great if you help me fix the issue.

  • Anonymous
    December 01, 2015
    @Pankaj: Do you see the data copied to Azure containers after completing the steps?

  • Anonymous
    December 01, 2015
    @EValliarampath: I don't think so. Once you have SPO Migration Package created, you can proceed further however I dont see a way to create SPO Migration Package for content which resides on SPO.

  • Anonymous
    December 09, 2015
    Hi Nishanth, trying to migrate about 30-40k items (approx 15 GB). Not all files have moved to SPO (though all moved to Azure storage successfully).What could we check in this scenario?

    • Anonymous
      December 13, 2015
      Check the Migration Job status...

  • Anonymous
    December 23, 2015
    The comment has been removed

  • Anonymous
    January 27, 2016
    Very nice article. This really helped me to do SharePoint migration. Thanks alot

  • Anonymous
    March 25, 2016
    Nishant this is a great article and I have created a script file and it just works every time fro small data migrations. However more recently I am coming across larger data migrations resulting in not all data been transferred. I noticed errors in the copymigrationpackage and createmigrationpackage texts files. Can you advise the importance of checking these and can I assume that if there is any errors I will get data transfer failures. Or can you reference me to any documentation regarding these files. Thanks Neil.

  • Anonymous
    May 13, 2016
    Nice work just please edit Step 2. -SourceFilesPath should be same as in Step1 -"\nishant-lptp\SPOTEMP" and not"C:\MigrationPackages\SPOTEMP"

  • Anonymous
    February 26, 2017
    Full Videohttps://www.youtube.com/watch?v=VnjdzqvvczU&t=648s