How to handle cosmos logical partition is exceeding 20 GB

Question

Hi Team,

We've been investigating an issue where application is unable to insert the data into cosmos container due to the logical partition has reached the 20 GB quota limit.

Notably, no exceptions are thrown by Cosmos during data insertions, and the application assumes normal operation. However, after checking recent data insertions in Cosmos Insights, we found that many requests result in a 403 error.

Is there any way we can check the partition's size before inserting the data?

Thanks,

Ajinkya

Answer 1

@ajinkya pathrudkar Thank you for reaching out.

Azure Cosmos DB enforces a maximum logical partition key size of 20 GB. For example, if you have a container/collection partitioned by UserId, the data within the "Alice" logical partition can store up to 20 GB of data.

You can use alerts to monitor if you have any logical partition keys that are approaching the 20 GB logical partition limit. Alerts can send you a notification in the form of an email or execute an action, such as an Azure Function or Logic App, when the condition is triggered.

You can create alerts to monitor if storage for a logical partition key is approaching 20 GB.

Remediation steps

When the 20 GB logical partition size limit is reached, you won't be able to write any more data to that logical partition. As a result, it's recommended to rearchitect your application with a different partition key as a long-term solution.

To help give time for this, you can request a temporary increase in the logical partition key limit for your existing application. File an Azure support ticket and select quota type Temporary increase in container's logical partition key size. Note this is intended as a temporary mitigation and not recommended as a long-term solution, as SLA guarantees are not honored when the limit is increased. To remove the configuration, file a support ticket and select quota type Restore container’s logical partition key size to default (20 GB). This can be done after you have either deleted data to fit the 20 GB logical partition limit or have rearchitected your application with a different partition key.

To learn about best practices for managing workloads that have partition keys requiring higher limits for storage or throughput, see Create a synthetic partition key.

Hope that helps.

Please do not forget to mark as accept answer if the above reply answers your question.

Regards,

Oury