Azure Logic apps Large file handling

Question

Hi All,

I have 500 MB large file in SFTP. I need to load the file in logic apps parse Json, apply mapping and convert to xml and place the file SFTP.

Please suggest How to handle large file in logic apps without any performance issue.

Thanks
Prashanth

Answer 1

The SFTP-SSH Connector supports files up to 1GB with chunking enabled. Chunking needs to be set in the action settings.

But note that all actions don't support working with this chunked data. To work around this, you could employ Azure Functions to process the data with Logic Apps implementing the business workflow you require.

The Logic App would have to be something like this

1. Trigger based on your business requirement
2. Fetch file from SFTP
3. Create file in blob storage with the fetched content from above
4. Call Azure Function with path to created blob to process it and place the final XML file back in blob storage, returning the path to the new blob
5. Fetch file from blob storage using path returned from function
6. Create file in SFTP using the fetched content from above

As for the Azure Function, you could leverage the blob storage bindings to interact with blob storage with your function focused on processing the file.

Answer 2

We have time trigger, Standard logic Apps workflow move file from Data Lake gen2 with sftp , move file to one data lake gen 2 to other datalake gen2 with no sftp feature enable. does it support larger f file like 1GB?