Hi Community, Assistance Needed Issue Description: I am encountering difficulties copying Excel, CSV, or ZIP files larger than 100MB (approximately 140MB) from an SFTP server to Azure Blob Storage using Logic Apps. Current Workflow: Trigger: Initiates when a file is added or modified on the SFTP server. Action: Attempts to create a blob storage in Azure. Specific Problem: The current trigger is not effectively identifying file changes or modifications on the SFTP server.

@Rama Thulasi Yemireddy Thanks for reaching out and apology for the inconvenience due to this issue. Can you please confirm what is the error/exception you are getting when the SFTP trigger is triggered. Please also confirm which trigger you are using? SFTP SHH trigger does support chunking as documented here and the supported action are listed here . In case if you are getting the size issue then you can try leveraging the When files are added or modified (properties only) trigger and use then use the Get file content using path or Get file content mentioned the chuck size and see if it helps you for consumption logic app as both the action supports chunking as per the document.

How to copy the large files from SFTP server to Azure blob storage with Azure logic app

Rama Thulasi Yemireddy 25

Hi Community, Assistance Needed
Issue Description:

I am encountering difficulties copying Excel, CSV, or ZIP files larger than 100MB (approximately 140MB) from an SFTP server to Azure Blob Storage using Logic Apps.
Current Workflow:

Trigger: Initiates when a file is added or modified on the SFTP server.
Action: Attempts to create a blob storage in Azure.

Specific Problem: The current trigger is not effectively identifying file changes or modifications on the SFTP server.

Deepanshukatara-6769 8,060 Reputation points

2024-02-05T09:45:05.67+00:00

HI Rama, Logic App design is within the limitations of Azure Logic Apps. There are certain restrictions on the maximum payload size and execution time. If your process exceeds these limits, consider breaking it into smaller steps or optimizing the workflow. The max message size could be 100 MB for all connectors and API under logic apps.

Please find below link for completed details https://learn.microsoft.com/en-us/azure/logic-apps/logic-apps-limits-and-config?tabs=consumption Kindly accept answer if it helps , Thanks!
Rama Thulasi Yemireddy 25 Reputation points

2024-02-05T09:50:24.18+00:00

Hi Deepanshu,
Thank you for answering.
The trigger is not functioning for 40 MB file also and thats why i put issue with max size which i faced.
Deepanshukatara-6769 8,060 Reputation points

2024-02-05T09:58:27.5133333+00:00
Hi Rama, Ok

Check SFTP Connection:

Verify that the connection to the SFTP server from your Logic App is successful. Check if the credentials, host, and port information are correct. If there are any connection issues, they might be preventing the trigger from functioning.

Test with a Different SFTP Client:

If possible, try connecting to the SFTP server using a different SFTP client to ensure that the server is functioning as expected. This can help rule out server-side issues.

Please try this as it will tell us the exact issue lie where accordingly we can check , Thanks
Rama Thulasi Yemireddy 25 Reputation points

2024-02-05T10:03:09.0133333+00:00

Hi Deepanshu,
Connection is working fine for small files which are < 10MB.
Deepanshukatara-6769 8,060 Reputation points

2024-02-05T10:55:36.1533333+00:00
I think it is because operation is getting timeout for larger files , so to fix this

Navigate to configuration of your standard logicapp and there would be one setting with name ServiceProviders.Sftp.OperationTimeout which has default value as 2 min and please make it around 10 mins , check below image for reference
Deepanshukatara-6769 8,060 Reputation points

2024-02-05T10:55:58.6066667+00:00

Please try this once and let us know
Deepanshukatara-6769 8,060 Reputation points

2024-02-05T11:25:46.9933333+00:00

For detailed information to the above comment, please check this https://learn.microsoft.com/en-us/azure/logic-apps/edit-app-settings-host-settings?tabs=azure-portal
Rama Thulasi Yemireddy 25 Reputation points

2024-02-06T07:56:33.66+00:00

Thank you for your quick assistance. Unfortunately, we do not have the configuration to update the timeout as we are using consumption whereas you are using standard plan.
MayankBargali-MSFT 70,016 Reputation points

2024-02-08T04:12:16.99+00:00

Hi @Rama Thulasi Yemireddy

Following up to see if my answer helps. Feel free to get back to me if you need any assistance.

Please 'Accept Answer' if it helped so that it can help others in the community looking for help on similar topics.

Accepted answer

MayankBargali-MSFT 70,016 Reputation points

2024-02-07T04:59:37.5033333+00:00

@Rama Thulasi Yemireddy Thanks for reaching out and apology for the inconvenience due to this issue. Can you please confirm what is the error/exception you are getting when the SFTP trigger is triggered. Please also confirm which trigger you are using? SFTP SHH trigger does support chunking as documented here and the supported action are listed here. In case if you are getting the size issue then you can try leveraging the When files are added or modified (properties only) trigger and use then use the Get file content using path or Get file content mentioned the chuck size and see if it helps you for consumption logic app as both the action supports chunking as per the document.
Please sign in to rate this answer.
Rama Thulasi Yemireddy 25 Reputation points

2024-02-07T06:11:32.6733333+00:00

Hi Mayankbargali, We are using SFTP-SSH Trigger (When a file is added or modified). File Changes Not Captured (>25MB with content): The current trigger mechanism not captured the file changes or modifications when dealing with files larger than 25MB, especially if the trigger includes the file content. Attempting to manually trigger the process results in errors, likely due to the size of the file causing issues with the trigger mechanism. PFB SS.

MayankBargali-MSFT 70,016 Reputation points

2024-02-07T06:25:42.6433333+00:00

@Rama Thulasi Yemireddy Chunking is not supported for the trigger. Please modify your workflow as mentioned in my previous comment for the action that supports chunking. The trigger should support upto 50 MB files without chunking.

you can try leveraging the When files are added or modified (properties only) trigger and use then use the Get file content using path or Get file content mentioned the chuck size and see if it helps you for consumption logic app as both the action supports chunking as per the document.
Sign in to comment

Share via

How to copy the large files from SFTP server to Azure blob storage with Azure logic app

0 additional answers