Logic App Get File Content

clouddataquest 91 Reputation points
2023-05-09T08:47:29.8966667+00:00

Hello,

i want to move big files up to 1GB with a Logic App (Consumption only), i saw there are now new Chunking methods for some Actions but not for the Get file Content, its stillt limited to about 50mb i guess.

Can you please help me? I could also use the File System connector instead von sharepoint but its limited too :( to you have any ideas? thank you very much

My Logic App:

MUser's image

Azure Logic Apps
Azure Logic Apps
An Azure service that automates the access and use of data across clouds without writing code.
2,991 questions
{count} votes

Accepted answer
  1. RevelinoB 2,780 Reputation points
    2023-05-09T09:03:49.17+00:00

    Hi Clouddataquest,

    If you're trying to move large files up to 1GB using Logic Apps, there are a few options available to you.

    Firstly, some actions in Logic Apps now support chunking, which allows you to split a large file into smaller pieces and process them individually. However, the "Get file content" action doesn't currently support chunking. As a workaround, you could use the "Get file metadata" action to retrieve the file size, and then use a combination of the "Get file content" and "Append to blob" actions to upload the file in chunks.

    Alternatively, you could use the "FTP" connector in Logic Apps to transfer files using the FTP protocol. This approach doesn't support chunking, so you may need to manually split the file into smaller pieces before uploading.

    Another option is to use Azure Blob Storage, which is a cloud-based storage service designed for storing large amounts of unstructured data. You can use the "Create blob" action in the "Azure Blob Storage" connector to upload your large files to Blob Storage, and then use the "Get blob content" action to download them again if needed. Blob Storage supports chunking, so you can upload and download files of any size.

    In summary, while the "Get file content" action doesn't currently support chunking, you can work around this limitation by using other actions or connectors in Logic Apps, such as "Get file metadata," "Append to blob," "FTP," or "Azure Blob Storage."

    I hope this helps! Let me know if you have any more questions.


3 additional answers

Sort by: Most helpful
  1. RevelinoB 2,780 Reputation points
    2023-05-22T14:00:57.08+00:00

    Hi Clouddataquest,

    To download the file in chunks using the 'Get blob content' action, you can define the range using the 'Range' parameter in the action. The 'Range' parameter allows you to specify the byte range of the blob content you want to retrieve.

    To implement chunked downloading, you'll need to calculate the appropriate range for each chunk based on the desired chunk size and the total size of the blob. For example, if you want to download a 2 GB file in 500 MB chunks, for the first chunk you would set the 'Range' parameter to 'bytes=0-524288000' (0 to 500 MB), for the second chunk 'bytes=524288001-1048576000' (500 MB to 1 GB), and so on.

    Please note that the exact syntax and format of the 'Range' parameter may vary depending on the specific tools or programming languages you are using. I would recommend referring to the documentation or examples provided by the connector or library you are using for more specific instructions on how to set the range parameter correctly.

    I hope this helps with your query: "where to define the range?"


  2. Hai, Yan 0 Reputation points
    2023-11-13T14:22:11.0533333+00:00

    If I would like to download large file from sharepoint to Azure blob storage. can I directly use the following steps:

    1. trigger if there is a file create/modified in sharepoint
    2. get folder list from sharepoint
    3. for each : azure blob storage create blob

    Is there any other action I need to add?

    User's image

    0 comments No comments

  3. Mark Brown 0 Reputation points
    2023-12-14T21:22:31.97+00:00

    At the moment, the SharePoint connector does not have a "chunking" option for downloads. The option exists in the "create file" action but that's the only one that I've seen it on.

    I had the same need and found limitations with both Power Apps and Power Automate. I ended up having to use PowerShell in a "hybrid" Azure Runbook to move the files from SharePoint to Azure Storage. As a hybrid runbook, the task actually runs on of our Azure VMs, where it has the drive space and memory that it needs. PowerShell downloads the file via MS Graph REST API and uploads it to Azure Storage, chunking any file over 1GB. The runbook is triggered by a custom form (Power App) for the SharePoint library that stores the file, which sends the item ID as a parameter. But you could also trigger it with Power Automate as well.

    0 comments No comments