How do I copy data from a public sharepoint URL to Microsoft Blob Storage?

Question

Hi everyone!

I am new to using Azure services. My co-author at Stanford has uploaded several files (in csv format) to a public link. This link starts as: office365stanford-my.sharepoint.com/personal

It has 40 files, ranging from a few 100 MB to 100 GB. I don't want to have to download the files to my local machine to transfer it to Blob. How can I do this directly?

I tried to set up Azure Data Factory but the pipleline is failing with the following error:

ErrorCode=HttpRequestFailedWithClientError,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Http request failed with client error, status code 403 Forbidden, please check your activity settings. If you configured a baseUrl that includes path, please make sure it ends with '/'.

Request URL: https://office365stanford-my.sharepoint.com/personal/gxzheng_stanford_edu/_layouts/15/onedrive.aspx?id=%2Fpersonal%2Fgxzheng_stanford_edu%2FDocuments%2FCNpatents%2Ffull_text_en&ga=1/.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=System.Net.WebException,Message=The remote server returned an error: (403) Forbidden.,Source=System,'

Note, I have already set up Microsoft Blob and I'm able to use AzCopy to transfer my local files into it.

Thanks for all your help!!

Answer 1

You can use Copy Blob (programmatically) - more at https://stackoverflow.com/questions/45003502/save-an-image-from-url-to-azure-storage-directly?noredirect=1&lq=1 or use tools such as this one https://docs.fineuploader.com/branch/master/features/azure.html

---

If the above response helps answer your question, remember to "Accept Answer" so that others in the community facing similar issues can easily find the solution. Your contribution is highly appreciated.

hth

Marcin

Answer 2

Here are some alternative methods to transfer your files directly:

• Azure Logic Apps: You can create an Azure Logic App that automates the file transfer from SharePoint to Azure Blob Storage.
• Power Automate: Microsoft Power Automate can also be used to copy files from SharePoint to Azure Blob Storage, though it may have limitations on the file size.
• Server-to-Server Transfer: Utilize AzCopy’s server-to-server transfer capabilities to move files directly between storage servers.

Additional Reading: https://learn.microsoft.com/en-us/answers/questions/1513111/how-to-copy-folders-and-files-from-a-sharepoint-si