Logic app to copy files from azure storage to onprem server fails

Question

Logic app to copy files from azure storage to onprem server fails

MarcVanderhaegen 241

Hello,

I have a problem with a Logic App that should copy csv files from an azure storage account to an onprem server through a data gateway.
When the logic app starts, it lists the files on the azure storage and then there is a loop to get each file content and create a file on the onprem server.
I see the file being created on the share on the onprem server but they are not complete.
Each time I launch the logic app, the size of the same files being transfered is different.
Example, I would like to transfer two 16MB files, one time I have 2MB and 8MB transfered, one time I have 4MB and 4MB etc....
The files are never completely transfered.

Each run of the logic app has a status 'failed' and on each files I have the error in the step 'create file' :
{
"status": 400,
"message": "The requested action could not be completed. Check your request parameters to make sure the path '//xxxxxxxxxxxxxxxxx_20231010-153328.csv' exists on your file system.\r\nclientRequestId: 5133d7b2-57f0-43cc-860e-532e385db6a9",
"error": {
"message": "The requested action could not be completed. Check your request parameters to make sure the path '//xxxxxxxxxxxxxxxxx_20231010-153328.csv' exists on your file system."
},
"source": "filesystem-we.azconn-we-002.p.azurewebsites.net"
}

Does anyone have an idea what could be wrong and how solve this ?

Thanks

Marc

1 answer

Your answer

Answer 1

MikeUrnun 9,777 Moderator

Hi @MarcVanderhaegen - Thanks for reaching out. The ForEach loop, by default, runs all of its iterations at the same time in parallel which I suspect might be creating more connections and workload than the OPDG & its downstream processing at your on-prem can handle at a time. In order to prevent that from happening, you can enable the Sequential run: For each: Run sequentially Could you try this out and let us know if this helps?

MarcVanderhaegen 241 Reputation points

2023-10-11T19:00:50.2233333+00:00

Hello,
Thank you for your answer, I just tried changing the Concurrency Control by setting it on 1 but unfortunately it doesn't change anything.
On the first attempt the first file was 8M and the second 12MB. (each file should be 16MB)
On the second attempt, 15MB and 4MB.
On the third attempt, 2MB, 6MB.
It's the same error 400 on the 'Create File' step, 'The requested action could not be completed. Check your request parameters to make sure the path '//xxxxxxxxxxx_20231010-153328.csv' exists on your file system'

Marc
MarcVanderhaegen 241 Reputation points

2023-10-11T19:32:47.67+00:00
Hello again,
I just found that in the documentation :
Logic Apps, Power Apps, and Power Automate support both read and write operations through the gateway:

The gateway has a 2-MB payload limit for write operations.

The gateway has a 2-MB request limit and an 8-MB compressed data response limit for read operations.

URL for the GET request has a 2048 character limit.

I think that could be the cause, don't you think ?
MikeUrnun 9,777 Reputation points Moderator

2023-10-11T20:34:14.6833333+00:00

@MarcVanderhaegen That seems to be the case but I'm looking into it to confirm. I thought we had raised that limit to 20MB (see this issue), and the error message, when the limit is hit, is usually pretty explicit: "The request content-length is larger than 'xyz' bytes. Current: 'xyz' bytes." whereas the error you're hitting is about the absence of a file that was supposed to have been created hence my suspicion for Networking/Connectivity related disruption.

Share via

Logic app to copy files from azure storage to onprem server fails

1 answer

Your answer