How to expand large 7z file(greater than a 1TB) on Auzre?

Madhav Vipin 20 Reputation points
2023-02-09T11:37:35.59+00:00

I have a large 7z file(greater than 1TB) on a blob storage in Azure and would like to expand and store it in another blob storage. I tried creating a pipeline in ADF to utilise an automation account that runs a runbooks for 7zip4powershell module and a powershell script. The pipeline works by taking downloading a file from bolb, expanding and uploading it back, which works well for 7z files of less size( less than a 100mb), but fails for larger files and shows the error: "Get-AzureStorageBlobContent : There is not enough space on the disk."

Is there another way to expand large 7z files on azure platform or is there a way in which I could resolve the space on disk error?

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,563 questions
Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
3,203 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,661 questions
Azure Automation
Azure Automation
An Azure service that is used to automate, configure, and install updates across hybrid environments.
1,369 questions
0 comments No comments
{count} votes

Accepted answer
  1. ShaikMaheer-MSFT 38,551 Reputation points Microsoft Employee Moderator
    2023-02-10T10:41:05.9766667+00:00

    Hi Madhav Vipin,

    Thank you for posting query in Microsoft Q&A Platform.

    Your error indicates there is not enough space on local machine to complete the operation.

    In your case it seems you may be using custom activity. Kindly correct me if I am wrong. Check your Integration runtime where its running and try have enough space on that machine to complete this operation. Feel free to share your activity name which are using to run your runbooks.

    Hope this helps. Please let me know how it goes.


    Please consider hitting Accept Answer button. Accepted answers help community as well.


0 additional answers

Sort by: Most helpful

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.