az storage blob copy start fails when files are larger than 256MB and requires-sync=true

Question

We are running a copy from one blob storage to another blob storage. However we don't want to the files to display in the azure storage until the file has fully copied.

Setting the flag --requires-sync true forces the files to copy and return prior to moving to the next copy. However it won't allow files 256MB to be copied.

Here is the following error:
The source request body for synchronous copy is too large and exceeds the maximum permissible limit (256MB).
RequestId:c34ce18e-501e-005b-1973-ec1b5e000000
Time:2021-12-08T20:36:27.0718053Z
ErrorCode:CannotVerifyCopySource
Error:None

Looking to get the files to not show up in azure storage blob until file has fully been copied or increase the maximum permissible limit.

Answer 1

@Matt Walstad May I know which version of SDK are you using?

Based on the error message: Please refer to the suggestion mentioned here

How to Copy Blob : The Copy Blob operation copies a blob to a destination within the storage account.

For testing purpose can you try to upload large files using Azure Storage Data Movement Library. It provides high-performance for uploading, downloading larger files. Please consider using this library for larger files.

Note: We also recommended to use Azcopy tool to upload files from on-premises or cloud (Use this command-line tool to easily copy data to and blobs from Azure Blobs, Blob Files, and Table storage Storage with optimal performance. ) AzCopy supports concurrency and parallelism, and the ability to resume copy operations when interrupted. It provides high-performance for uploading, downloading larger files. Please consider using this library for larger files.

Storage limits: https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/azure-subscription-service-limits#storage-limits

Note: Over time, the maximum upload size has increased. The Maximum blob size (via Put Block List) is approximately 190.7 TiB (4000 MiB X 50,000 blocks) for Version 2019-12-12 and later. The referenced doc gives size limits for other versions as well. See here for more information.

If the issue still persist, please share the complete details of the issue with the screenshot

Please let us know if you have any further queries. I’m happy to assist you further.

---------
Please do not forget to and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.