question

prashanthsuranani-4204 avatar image
0 Votes"
prashanthsuranani-4204 asked VinayNandargi-4069 published

Azure Logic apps Large file handling

Hi All,

I have 500 MB large file in SFTP. I need to load the file in logic apps parse Json, apply mapping and convert to xml and place the file SFTP.

Please suggest How to handle large file in logic apps without any performance issue.

Thanks
Prashanth

azure-logic-apps
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

@prashanthsuranani-4204 Just following up to see if my updated response helps

0 Votes 0 ·

1 Answer

PramodValavala-MSFT avatar image
0 Votes"
PramodValavala-MSFT answered VinayNandargi-4069 published

The SFTP-SSH Connector supports files up to 1GB with chunking enabled. Chunking needs to be set in the action settings.

But note that all actions don't support working with this chunked data. To work around this, you could employ Azure Functions to process the data with Logic Apps implementing the business workflow you require.

The Logic App would have to be something like this

  1. Trigger based on your business requirement

  2. Fetch file from SFTP

  3. Create file in blob storage with the fetched content from above

  4. Call Azure Function with path to created blob to process it and place the final XML file back in blob storage, returning the path to the new blob

  5. Fetch file from blob storage using path returned from function

  6. Create file in SFTP using the fetched content from above

As for the Azure Function, you could leverage the blob storage bindings to interact with blob storage with your function focused on processing the file.


· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Thanks Pramod for the reply.

I am able to retrieve the file from SFTP but it is failing while parsing due to large content.

Any idea How to parse large file in logic apps?

Thanks
Prashanth

0 Votes 0 ·

This is expected. Only connectors that support chunked data can process the large file. Missed to mention this earlier. I have update my answer to include the approach for such scenarios.


0 Votes 0 ·

Hi, Do you have more details on how to implement this?

0 Votes 0 ·