Copy File from P drive into Azure Data Lake Gen 2 with Logic Apps

Question

I have spent the week trying to figure out this one and have got nowhere so far

Basically I have files on a P Drive On Premise and I want to copy them Across to an Azure Data Lake Gen 2 Container on a weekly basis

I cant use the Azure Data Lake Upload Data activity because its Gen 1 and Im on gen 2 so I thought it might be blob storage but I cant find any blob storageactivity for this?

So basically is this possible? and are there any good how to guides for it because I havent found anything anywhere

Thanks in advance

Answer 1

Hello @Debbie Edwards

Sorry for the late reply. I have reproduced this scenario and didn't run into any issues. Not sure if you've already got it to work but in case it's still helpful to you or anyone else visiting this thread, the below are my findings:

In short, as long as you've enabled Data Lake Gen2 on your Storage Account during its creation by going to the "Advanced" tab (as described here) and turning on the "Hierarchical namespace" option, you can use the Azure Blob Storage connector as-is without anything special. Here are the specific steps that I've taken to get this scenario working:

1. Installed On-Premise Data Gateway on my on-prem machine
2. Ensured that the Windows user has access to the directory where the files are expected to be picked up
3. Created the corresponding On-Premise Data Gateway on Azure
4. On Logic Apps, added the File System connector (which is capable of supporting On-Prem connections) and chose its "When a file is created (properties only)" trigger (note that the File System connector only supports Windows Authentication at this time. Also, when entering the username, be sure to enter it in the following format: Computername\Username)
5. Using the "Get File Content" action from the File System connector (consider this as a "copy" action)
6. Using the "Create Blob" action from the Blob Storage connector (consider this as a "paste" action)
7. Upon successful creation of the file on Gen2 Data Lake, I'm also using the Outlook connector to send an email notification to myself.

Below is the screenshot of the workflow:

Below is the screenshot of the result where a file was copied in a container named "copied" in my Storage Account (I'm using Storage Explorer):

I hope this is helpful. If any further questions, just let me know and we'd be happy to help and continue our discussion.