Azure Logic apps Large file handling

prashanth suranani 6 Reputation points
2020-08-09T21:54:25.03+00:00

Hi All,

I have 500 MB large file in SFTP. I need to load the file in logic apps parse Json, apply mapping and convert to xml and place the file SFTP.

Please suggest How to handle large file in logic apps without any performance issue.

Thanks
Prashanth

Azure Logic Apps
Azure Logic Apps
An Azure service that automates the access and use of data across clouds without writing code.
2,843 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Pramod Valavala 20,516 Reputation points Microsoft Employee
    2020-08-10T05:14:19.04+00:00

    The SFTP-SSH Connector supports files up to 1GB with chunking enabled. Chunking needs to be set in the action settings.

    But note that all actions don't support working with this chunked data. To work around this, you could employ Azure Functions to process the data with Logic Apps implementing the business workflow you require.

    The Logic App would have to be something like this

    1. Trigger based on your business requirement
    2. Fetch file from SFTP
    3. Create file in blob storage with the fetched content from above
    4. Call Azure Function with path to created blob to process it and place the final XML file back in blob storage, returning the path to the new blob
    5. Fetch file from blob storage using path returned from function
    6. Create file in SFTP using the fetched content from above

    As for the Azure Function, you could leverage the blob storage bindings to interact with blob storage with your function focused on processing the file.