Migrate SharePoint online sites document library to Local drive

Question

Hello Experts,

I have around 2000 subsites in site collection and each subsite has document library along with folders and documents, i want to migrate those document library data to local drive.

Can anyone suggest me the best way to achieve this.

Thanks in advance!

Answer 1

Hi @Anonymous ,

I traversed all subsites in a site collection and downloaded all files(including folders) in the OOB document library of each subsite to * C:\Documents\ * using the below PowerShell Command.

Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.dll"  
Add-Type -Path "C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.Runtime.dll"  
  
#Variables for Processing  
$SiteCollUrl = "https://contoso.sharepoint.com/sites/Sup"  
   
#Get Credentials to connect  
$Cred= Get-Credential  
$Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($Cred.Username, $Cred.Password)  
  
Function Download-AllFilesFromLibrary()  
{   
    param  
    (  
        [Parameter(Mandatory=$true)] [string] $SiteURL,  
        [Parameter(Mandatory=$true)] [Microsoft.SharePoint.Client.Folder] $SourceFolder,  
        [Parameter(Mandatory=$true)] [string] $TargetFolder  
    )  
    Try {  
           
        #Create Local Folder, if it doesn't exist  
        $FolderName = ($SourceFolder.ServerRelativeURL) -replace "/","\"  
        $LocalFolder = $TargetFolder + $FolderName  
        If (!(Test-Path -Path $LocalFolder)) {  
                New-Item -ItemType Directory -Path $LocalFolder | Out-Null  
        }  
           
        #Get all Files from the folder  
        $FilesColl = $SourceFolder.Files  
        $Ctx.Load($FilesColl)  
        $Ctx.ExecuteQuery()  
   
        #Iterate through each file and download  
        Foreach($File in $FilesColl)  
        {  
            $TargetFile = $LocalFolder+"\"+$File.Name  
            #Download the file  
            $FileInfo = [Microsoft.SharePoint.Client.File]::OpenBinaryDirect($Ctx,$File.ServerRelativeURL)  
            $WriteStream = [System.IO.File]::Open($TargetFile,[System.IO.FileMode]::Create)  
            $FileInfo.Stream.CopyTo($WriteStream)  
            $WriteStream.Close()  
            write-host -f Green "Downloaded File:"$TargetFile  
        }  
           
        #Process Sub Folders  
        $SubFolders = $SourceFolder.Folders  
        $Ctx.Load($SubFolders)  
        $Ctx.ExecuteQuery()  
        Foreach($Folder in $SubFolders)  
        {  
            If($Folder.Name -ne "Forms")  
            {  
                #Call the function recursively  
                Download-AllFilesFromLibrary -SiteURL $SiteURL -SourceFolder $Folder -TargetFolder $TargetFolder  
            }  
        }  
  }  
    Catch {  
        write-host -f Red "Error Downloading Files from Library!" $_.Exception.Message  
    }  
}  
   
#Custom function to get all subsites in a given site collection  
Function Get-SPOWebs($Url, $Credentials)  
{  
    #Set up the context  
    $Context = New-Object Microsoft.SharePoint.Client.ClientContext($Url)   
    $Context.Credentials = $credentials  
  
    $web = $Context.Web  
    $Context.Load($web)  
    $Context.Load($web.Webs)  
    $Context.ExecuteQuery()  
  
    #Do Something Here  
    Write-host $Web.URL  
  
    #Process Each subsite in current site  
    ForEach($web in $web.Webs)  
    {  
        #call the function recursively  
        Get-SPOWebs $web.Url $Credentials  
 $SiteURL=$web.URL  
 $LibraryName="Documents"  
        $TargetFolder="C:\Documents\"  
 $Ctx = New-Object Microsoft.SharePoint.Client.ClientContext($SiteURL)  
        $Ctx.Credentials = $Credentials  
        $List = $Ctx.Web.Lists.GetByTitle($LibraryName)  
        $Ctx.Load($List)  
        $Ctx.Load($List.RootFolder)  
        $Ctx.ExecuteQuery()  
 Download-AllFilesFromLibrary -SiteURL $SiteURL -SourceFolder $List.RootFolder -TargetFolder $TargetFolder  
    }  
}   
         
    Get-SPOWebs $SiteCollUrl $Credentials  

Test result in my end:

Reference:
SharePoint Online: PowerShell to Download All Files from a Document Library
SharePoint Online: Loop Through All Subsites in a Site Collection using PowerShell

---

If an Answer is helpful, please click "Accept Answer" and upvote it.
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

Answer 2

@Allen Xu_MSFT ,

I have used above code for downloading files for local drive.. and its working fine.. but sometimes i am facing below error, can you please advise.

I have attached error screenshot.

Error: Error Downloading Files from library! - Exception calling "OpenBinaryDirect" with "2" argument(s): The remote server returned an error: (404) Not Found