How to fix - Unable to write file at location: asset/g10000/17809/5694.bin. Fail: Code: 400 Value: The block list may not contain more than 50,000 blocks. details (if any): BlockListTooLongThe block list may not contain more than 50,000 blocks. RequestId:

Question

I have integrated Blob with Pimcore for storing the assets, but when I upload bigger assets, I am getting the error.

Unable to write file at location: asset/g10000/17809/5694.bin. Fail: Code: 400 Value: The block list may not contain more than 50,000 blocks. details (if any): BlockListTooLongThe block list may not contain more than 50,000 blocks. RequestId:d0024871-501e-005c-1c3e-ac1a04000000 Time:2024-05-22T11:50:17.6972629Z.

Answer 1

Hi Aryan, Raj - Thanks for reaching out.

Based on the exception it is potentially hitting a limitation where in a block blob can include up to 50,000 blocks. it seem to be trying to commit the blocks using PutBlockList API but the total number of blocks might be exceeding the scalability target for maximum number of blocks for block or append blob.

https://learn.microsoft.com/en-us/rest/api/storageservices/understanding-block-blobs--append-blobs--and-page-blobs#about-block-blobs

https://learn.microsoft.com/en-us/azure/storage/blobs/scalability-targets

What is the size of the blob you are trying to upload to the blob storage?
Also, please verify what is the block size being used by the PimCore application underneath? Try enabling the diagnostic logging and check for Requestsize field which might provide hint to the same.

The above details shall in the isolation to some extent further.

Answer 2

Hello Aryan, Raj,

Greetings! Welcome to Microsoft Q&A Platform.Adding to above information. The error message indicates that the block list for the asset exceeds the allowed limit of 50,000 blocks.

Block blobs are optimized for uploading large amounts of data efficiently. Block blobs are composed of blocks, each of which is identified by a block ID. A block blob can include up to 50,000 blocks. Each block in a block blob can be a different size, up to the maximum size permitted for the service version in use. To create or modify a block blob, write a set of blocks via the Put Block operation and then commit the blocks to a blob with the Put Block List operation.

Block blobs store text and binary data. Block blobs are made up of blocks of data that can be managed individually. Block blobs can store up to about 190.7 TiB.

Blobs that are less than a certain size (determined by service version) can be uploaded in their entirety with a single write operation via Put Blob.

refer - https://learn.microsoft.com/en-us/rest/api/storageservices/understanding-block-blobs--append-blobs--and-page-blobs#about-block-blobs for more detailed guidance.

• Azure Blob Storage divides large files into smaller blocks for efficient storage and retrieval. The error message suggests that the block list (collection of these smaller blocks) has exceeded the maximum allowed limit of 50,000 blocks. To resolve this, we need to find a way to reduce the number of blocks or optimize the storage process. There are some possible ways to overcome this issue are following,
• Check how Pimcore handles chunking or splitting large assets into smaller blocks. If possible, adjust the chunk size or consider using larger chunks to reduce the total number of blocks.
• Optimize the asset upload process to minimize the number of blocks generated and consider using larger chunks during the upload process. Ensure that the asset is uploaded in a single operation (if feasible) to avoid excessive block creation. Ensure that there are no custom settings or plugins affecting the block creation process.
• Review your Azure Blob Storage configuration and confirm that there are no specific limitations or restrictions set on the storage account.
• Consider upgrading to the latest version of Pimcore, as newer releases often include bug fixes and improvements and check if the issue has been addressed in a more recent version.

Hope this answer helps! Please let us know if you have any further queries. I’m happy to assist you further.

---

Please "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.