I have written an end to end flow using Java SpringBoot which fetches entries from the container but i see that for this specific container the api fetches entries upto a certain limit say 8 after that it is unable to fetch record.

Question

I am unable to fetch entries from a specific container after a certain limit, although this issue does not occur with other containers. I suspect the problem might be due to a field in this container that stores base64 strings of PDFs. Despite increasing the throughput, the issue persists. The same API flow works perfectly fine when fetching data from other containers. Can you suggest some possible solution and the problem as well?

Answer

Based on the information provided in the context, the issue you are facing might be due to container OOM, which occurs when the available app memory has run out. This issue causes app restart events, which are reported in the Resource Health section of the Azure portal. Normally, container OOM is caused by incorrect memory size configurations. You can try increasing the memory size configuration for the container to see if it resolves the issue. If the issue persists, you can try checking the CPU usage on the host where the app is running. If CPU usage is 50 percent or more, run your app on a host with a higher configuration. Or you can distribute the load on more machines.

Another possibility is that the issue might be due to connection throttling, which can happen because of either a connection limit on a host machine or Azure SNAT (PAT) port exhaustion. Some Linux systems, such as Red Hat, have an upper limit on the total number of open files, which limits the total number of connections. The number of max allowed open files, which are identified as "nofile," needs to be at least double your connection pool size. If your app is deployed on Azure Virtual Machines without a public IP address, by default Azure SNAT ports establish connections to any endpoint outside of your VM. The number of connections allowed from the VM to the Azure Cosmos DB endpoint is limited by the Azure SNAT configuration. You can add your Azure Cosmos DB service endpoint to the subnet of your Azure Virtual Machines virtual network to avoid Azure SNAT limitation.

In summary, you can try increasing the memory size configuration for the container and checking the CPU usage on the host where the app is running. You can also check if the issue is due to connection throttling by verifying the connection limit on a host machine or Azure SNAT (PAT) port exhaustion.

References:

Answer

Hi Abhishek Vidyarthi (IN),

Welcome to Microsoft Q&A forum.

As I understand, you have written a code to fetch data from Azure Cosmos DB container and unable to do so after some records.

Could you please check below pointers for your case:

1). Check if TTL is set for your container or item.

2). Check per-item limits given here: https://learn.microsoft.com/en-us/azure/cosmos-db/concepts-limits#per-item-limits

Large document sizes up to 16 MB are supported with Azure Cosmos DB for MongoDB only. Read the feature documentation to learn more.

There are no restrictions on the item payloads (like number of properties and nesting depth), except for the length restrictions on partition key and ID values, and the overall size restriction of 2 MB. You may have to configure indexing policy for containers with large or complex item structures to reduce RU consumption. See Modeling items in Azure Cosmos DB for a real-world example, and patterns to manage large items.

3). Check per-request limits here:

https://learn.microsoft.com/en-us/azure/cosmos-db/concepts-limits#per-request-limits

Hope this helps, if not please share more information about your account and configuration to investigate more.

Thanks

Share via

I have written an end to end flow using Java SpringBoot which fetches entries from the container but i see that for this specific container the api fetches entries upto a certain limit say 8 after that it is unable to fetch record.

2 answers

Your answer