How to copy one drive files to share point automatically with out logic apps and power automata

Podili Vijay 20

i have one logic app flow it will copy the files from SharePoint to ADLS. now i want to add one parallel branch to copy files from one drive to adls. but while add one drive action name like get file content it will ask sig in . but actually those one drive credential we don't have in hand and it is not possible to take it our hand. so i thought to move those files to sharepoint after that we place those file to adls. but i'm struct how to move one drive files to adls automatically at daily.

1 answer

Nehruji R 2,051 Reputation points Microsoft Vendor

2024-02-26T06:10:05.2+00:00
Hello Podili Vijay,

Greetings! Welcome to Microsoft Q&A forum.

I understand that you would like to copy files from one drive to ADLS automatically without using PowerAutomate flows or logic apps. Below are the certain ways to perform the same and you can choose which is appropriate for your use case,

1.Consider using Azure Data Factory, Create a pipeline in ADF that:

Uses the “Copy Data” tool.

Connects to your OneDrive as the source.

Connects to your ADLS Gen2 as the destination.

Schedule the pipeline to run daily.

This way, it will automatically move files from OneDrive to ADLS Gen2 on a regular basis.

refer -https://learn.microsoft.com/en-us/azure/data-factory/load-azure-data-lake-storage-gen2

2.You can also achieve your goal by using Storage Events Trigger in ADF. This will trigger the linked/associated pipeline when a new file is added in the onedrive.

refer - https://learn.microsoft.com/en-us/azure/data-factory/how-to-create-event-trigger?tabs=data-factory

Demonstration Video : Here is a demonstration video on event triggers in ADF - Event based Triggers in Azure Data Factory

3.Alternatively, you can try using PowerShell Script,

Write a custom PowerShell script that periodically checks for new files in OneDrive. Use the Azure Data Lake Store PowerShell module to move the files from OneDrive to ADLS Gen2.

Schedule the script to run daily using Task Scheduler or any other scheduling mechanism.

refer SO thread- https://stackoverflow.com/questions/58622619/how-to-move-files-in-azure-data-lake-store-with-powershell-recursive for more details.

Hope this answer helps! Please let us know if you have any further queries. I’m happy to assist you further.

Please "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.
Please sign in to rate this answer.
Vijay-2791 151 Reputation points

2024-02-26T10:31:20.71+00:00

if we connect amazon connection as a source is it connect to onedrive?
https://learn.microsoft.com/en-us/azure/data-factory/load-azure-data-lake-storage-gen2 using these as reference

Vijay-2791 151 Reputation points

2024-02-26T10:34:50.82+00:00

if we consider these as refrence https://learn.microsoft.com/en-us/azure/data-factory/load-azure-data-lake-storage-gen2
in source connection if we add amzon s3 , is it connect to onedrive location .
i serach in source data set conncetion but did not found onedrive conncetion

Nehruji R 2,051 Reputation points Microsoft Vendor

2024-02-27T06:39:44.27+00:00

Azure Data Factory (ADF) provides a connector for Amazon Simple Storage Service (S3). You can use this connector to copy data from Amazon S3 to other destinations or perform transformations within ADF pipelines.

To create an Amazon S3 linked service in ADF, follow these steps:

Browse to the Manage tab in your Azure Data Factory or Synapse workspace.

Select Linked Services, then click New.

Search for Amazon and select the Amazon S3 connector.

Configure the service details, test the connection, and create the new linked service.

refer - https://learn.microsoft.com/en-us/azure/data-factory/connector-amazon-simple-storage-service?tabs=data-factory

Unfortunately, as of now, there is no direct built-in connector for OneDrive in Azure Data Factory.

However, you can still achieve integration between OneDrive and Azure Data Factory using an intermediate step: Initially from OneDrive to Amazon S3: Set up a process to sync or replicate data from your OneDrive location to an Amazon S3 bucket. You can use third-party tools or custom scripts to automate this synchronization.

then Amazon S3 to Azure Data Factory:

Once the data is in Amazon S3, use the ADF Amazon S3 connector to copy it to your desired destination (e.g., Azure Blob Storage, Azure Data Lake Storage Gen2).

Configure your ADF pipeline to read from Amazon S3 and write to the target storage.

refer - https://learn.microsoft.com/en-us/azure/data-factory/data-migration-guidance-s3-azure-storage

As this approach involves an additional step (OneDrive to Amazon S3) and it may introduce some latency.

If you have specific requirements for real-time synchronization or frequent updates, consider using Azure Logic Apps or Azure Functions for more complex workflows.

Vijay-2791 151 Reputation points

2024-02-27T06:53:55.1+00:00

can you suggest me how we can move onedrive data to amazon s3 automatically Hi @Nehruji R

Nehruji R 2,051 Reputation points Microsoft Vendor

2024-02-27T07:47:14.76+00:00

To automatically move OneDrive data to Amazon S3, you have a few options,

Zapier Integration:

Zapier is an automation platform that allows you to create workflows (called “Zaps”) between different apps and services.

You can set up a Zap that triggers when a new file is added to OneDrive and automatically uploads a copy of that file to Amazon S3.

Here’s how it works:

Trigger: When a new file is added in a OneDrive folder.

Action: Upload the file to Amazon S3.

This provides a seamless and convenient file backup solution.

CData Sync:

CData Sync is a data synchronization tool that can replicate data from Microsoft OneDrive to Amazon S3.

Configure the necessary connection properties for both OneDrive and Amazon S3 in CData Sync.

Set up a synchronization task to replicate files from OneDrive to Amazon S3.

MultCloud:

MultCloud is a cloud management platform that allows you to transfer files between different cloud storage services.

You can use MultCloud to transfer files from OneDrive to Amazon S3:

Add both OneDrive and Amazon S3 to MultCloud.

Use the “Cloud Transfer” feature to select files and folders from OneDrive as the source and Amazon S3 as the destination.

Configure the necessary credentials and permissions for both OneDrive and Amazon S3 accordingly per your use case.

Note: The third-party articles are recommended to help you find alternatives, Microsoft does not verify if the third-party workarounds work completely.
Sign in to comment