Migrate Data from Sharepoint to ADLS using Logic Apps

Priya N. Bidkar 0

How to Migrate all the current engagement folders and sub-folders from SharePoint to ADLS. Ensuring that the existing security roles and privileges associated with these folders in SharePoint are preserved and applied to the newly created ADLS folders.Using Logic Apps.

1 answer

Mike Urnun 9,831 Reputation points Microsoft Employee

2023-11-14T18:47:43.7433333+00:00
Hello @Priya N. Bidkar - Thanks for reaching out, and posting on the MS Q&A.

The step for transferring the files will be pretty easily accomplished using the following two connectors:

Azure Data Lake

SharePoint

As for the step for preserving existing security roles & privileges, could you provide more details? Are you looking to preserve the existing "Sharing & permissions" constructs (associated with folder/files on Sharepoint) to be converted to a combination of RBAC/ABAC/ACL rules on the ADLS side?
Please sign in to rate this answer.
Priya N. Bidkar 0 Reputation points

2023-11-15T06:57:01.68+00:00

Thanks For the Answer,
The information that is provided with the help of those links is limited. I am Interested to know the end-to-end solution for the answer.
For reference:
https://learn.microsoft.com/en-us/answers/questions/898508/copy-files-from-sharepoint-to-azure-blob-storage-u
here in this @Ravi Kanth Koppala has provided the steps, but still, my solution is not accomplished, It is showing me errors. So, I need extended help with the project scope mentioned above.

Mike Urnun 9,831 Reputation points Microsoft Employee

2023-11-21T03:12:57.2166667+00:00

@Priya N. Bidkar Sorry for the late reply. I'll try to implement and share the steps with you.

Priya N. Bidkar 0 Reputation points

2023-11-21T04:07:23.6233333+00:00

Thanks, @MikeUrnun, That Would Really Help A Lot!!!

Mike Urnun 9,831 Reputation points Microsoft Employee

2023-12-11T06:34:25.7133333+00:00

Hello @Priya N. Bidkar - Sorry for the long delay. I implemented below for migrating all subfolders and any files on SharePoint to the ADSL gen2 storage account:

Some considerations:

Sharepoint connector -> Get Files (properties only) action was chosen for returning the list of folders and files instead of other actions such as List Folders because it returns all of the folders & files in a one-dimensional array of objects that each contain useful details like Full Path including filename and extension, and the folders are flagged with {isFolder}: true property. As such, no recursive traversal operation is needed.

Using the details, fetch the content of each file and recreate them in ADSL gen2 using the Create blob (V2) action. Because "Full Path" is used for the "Blob name", folder structure will be automatically created, and can be navigated using Data Explorer.

Be sure to review Concurrency control and Sequential run settings to fine-tune performance & scale.

I tested with simple Request/Response and migrated 86 small files successfully without a timeout occurring on the Response. However, for a larger number of files that entail longer running work, HTTP may not be great. You can add any trigger of your choice to the above, and perhaps, add an action at the end to send an email notifying that the job is completed.

Priya N. Bidkar 0 Reputation points

2023-12-12T15:10:49.4733333+00:00

Thanks, @MikeUrnun, That Would Help A Lot!!!
Thank you for the effort that you have put in.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

Migrate Data from Sharepoint to ADLS using Logic Apps

1 answer

Your answer