zip all files from a sharepoint folder and upload to storage account

Question

we have a requirement to list all files in a sharepoint folder , make a single zip and upload them to storage account.
my approach is to get content of all files in a single string separated by key identifiers using logicApp and then pass it on to function app to try to create a zip stream from that string and finally upload it to storage.

i am able to address all pieces of this problem but not able to figure out how to create a zip stream from a string without physically creating a file.

other idea i was thinking of is uploading all files on storage as is and then try to zip them up on storage on another location

@all experts : please feedback.

Answer 1

@Abhishake Jaiswal Thanks for reaching out. You should use Azure function and in memory you can create the stream and zip it then uploads to your storage account.

For your reference I have tested the setup at my end and used the below code which works perfectly fine at my end.

The below is only for reference and you need to modify as per your need.

using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.Extensions.Logging;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Blob;
using Newtonsoft.Json;
using System;
using System.Collections.Generic;
using System.IO;
using System.IO.Compression;
using System.Threading.Tasks;

namespace ZipFunc
{
    public static class Function1
    {
        [FunctionName("Function1")]
        public static async Task<IActionResult> Run(
            [HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)] HttpRequest req,
            ILogger log)
        {

            // Retrieve the connection string for the storage account
            string storageConnectionString = "connectionstring";

            // Read the request body as a string and deserialize it to a list of file objects
            string requestBody = await new StreamReader(req.Body).ReadToEndAsync();

            Data inputRequest = JsonConvert.DeserializeObject<Data>(requestBody);

            // Retrieve the container name and blob name from the request body
            string containerName = inputRequest.ContainerName;
            string blobName = inputRequest.BlobName;

            // Retrieve the blob reference
            CloudStorageAccount storageAccount = CloudStorageAccount.Parse(storageConnectionString);
            CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
            CloudBlobContainer container = blobClient.GetContainerReference(containerName);
            CloudBlockBlob blob = container.GetBlockBlobReference(blobName);

            //List<FileObject> files = new List<FileObject>
            //{
            //    new FileObject { FileName = "file1.txt", StreamData = "VGhpcyBpcyB0ZXN0MQ==" },
            //    new FileObject { FileName = "file2.txt", StreamData = "VGhpcyBpcyB0ZXN0Mg==" }
            //};



            // Create a memory stream to hold the zip archive
            using (MemoryStream memoryStream = new MemoryStream())
            {
                // Create a new zip archive
                using (ZipArchive zipArchive = new ZipArchive(memoryStream, ZipArchiveMode.Create, true))
                {


                    // Loop through the files and add them to the zip archive
                    foreach (var file in inputRequest.files)
                    {
                        // Create a new zip archive entry
                        ZipArchiveEntry entry = zipArchive.CreateEntry(file.FileName);
                        Stream entryStream = entry.Open();

                        // Write the file content to the zip archive entry
                        Stream fileStream = new MemoryStream(Convert.FromBase64String(file.StreamData));
                        await fileStream.CopyToAsync(entryStream);
                        entryStream.Close();
                        
                    }
                }
                memoryStream.Seek(0, SeekOrigin.Begin);

                using (var stream = new MemoryStream())
                {
                    // Write the Memory Stream object to the stream.
                    memoryStream.WriteTo(stream);

                    // Set the stream position to the beginning.
                    stream.Position = 0;

                    // Upload the stream to the blob.
                    await blob.UploadFromStreamAsync(stream);
                }

            }
            return new OkObjectResult($"Zip file created and uploaded to {containerName}/{blobName}");

        }
    }
}

Data Class for my json body where you can pass the containername, blobname and fileObject

    internal class Data
    {
        public string ContainerName { get; set; }
        public string BlobName { get; set; }
        public List<FileObject> files { get; set; }

    }

    internal class FileObject
    {
        public string FileName { get; set; }
        public string StreamData { get; set; }
    }

Sample JSON Object that needs to be passed from your logic app to azure function

{
   "ContainerName":"upload",
   "BlobName":"function.zip",
   "files":[
      {
         "FileName":"file1.txt",
         "StreamData":"VGhpcyBpcyB0ZXN0MQ=="
      },
      {
         "FileName":"file2.txt",
         "StreamData":"VGhpcyBpcyB0ZXN0Mg=="
      }
   ]
}

Make sure in your logic app when you are appending the array variable of files you need to pass the $content

Feel free to get back to me if you have any queries or concerns.

Please click on "Yes" if the answer is helpful so that it can help others in the community.