ADF - Zip files in chunks

Question

The source contains multiple files. If we zip all of these files together, the size of the zip file would be too huge. So we would need to zip groups of files, and have multiple zip files. Is there a support for this in ADF -

a) where we can provide a limit on the maximum number of files that can be included in a zip
b) or provide a limit on the size of the zip file

so that when the limit is reached, automatically multiple zip files will be created ?

Any pointers in this regard will be helpful.

Thanks,
Kothai.

Answer 1

@Kothai Ramanathan Thanks for using Microsoft Q&A.

No, there is no way in ADF to provide a limit on files or limit on size of the zip files to create multiple files. You can however, try using a staging storage location and use Metadata activity to move new files to the staging area based on the size. Once the specified file size or number of files is reached then you can transfer the files from staging as a zip to the final destination.