How to Fix: com.mongodb.MongoExecutionTimeoutException: Request timed out. Retries due to rate limiting: True.

Question

Dear Support team, I am trying to fetch millions of records from DB In 10k batch.

But after some successfully fetching some batches, It throws below Exception:

com.mongodb.MongoExecutionTimeoutException: Request timed out. Retries due to rate limiting: True.

Please help me with a Java Solution, on how can I read all records from DB in any way.

Answer 1

Hi @Deepanshu Goyal ,

For Rate Limiting Timeout errors in Cosmos DB, it is recommended to increase Request Units.

Azure Cosmos DB for MongoDB operations may fail with rate-limiting (16500/429) errors if they exceed a collection's throughput limit (RUs).

You can enable the Server Side Retry (SSR) feature and let the server retry these operations automatically. The requests are retried after a short delay for all collections in your account. This feature is a convenient alternative to handling rate-limiting errors in the client application.

Refer to the detailed documentation for the same here: Prevent rate-limiting errors for Azure Cosmos DB for MongoDB operations

Hope this helps. If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.

Answer 2

this code block I am using to Fetch Products from DB. Which have StoreId = 3.

int PER_BATCH_COUNT = 1000;
List<Product> dataList = new ArrayList<Product>();
int count = 77235; //no of records in DB

int iterations = BigDecimal.valueOf(count).divide(BigDecimal.valueOf(PER_BATCH_COUNT), RoundingMode.UP).intValue();
int iterationNo = 0;
 /*this works as a page no, to calcualte the    next start index*/
int rowCount = 1;

while(iterations != 0) {   
 iterationNo++;

log.info("Getting result for Iteration No: " + iterationNo);
int startIndex = (iterationNo-1)*PER_BATCH_COUNT+1;

Query query = Query.query(Criteria.where(SearchParameters.STORE_ID.getAttributeName()).is("3"));
query.skip(startIndex);
query.limit(PER_BATCH_COUNT);

dataList.addAll(mongoTemplate.find( query, Product.class));
iterations--;
}
return dataList;