CSOM Throttle on websites having more than 3Lakhs lists

Question

I have a website having more than 3Lakhs lists. When I execute the below code, the user gets throttled. Please confirm how to fetch all lists in SharePoint Online CSOM.

Code:
Web web = clientContext.Web;
ListCollection targetlListCollection = web.Lists;
clientContext.Load(targetlListCollection, l => l.Include(x => x.Id, x => x.BaseTemplate, x => x.BaseType, x => x.DefaultViewUrl));
clientContext.RequestTimeout = -1;
clientContext.ExecuteQueryWithIncrementalRetry();

Answer 1

Hi @Amit ,

Build a extension method like this in a static class:

   public static void ExecuteQueryWithIncrementalRetry(this ClientContext clientContext, int retryCount, int delay)  
        {  
            int retryAttempts = 0;  
            int backoffInterval = delay;  
            int retryAfterInterval = 0;  
            bool retry = false;  
            ClientRequestWrapper wrapper = null;  
            if (retryCount <= 0)  
                throw new ArgumentException("Provide a retry count greater than zero.");  
            if (delay <= 0)  
                throw new ArgumentException("Provide a delay greater than zero.");  
  
            // Do while retry attempt is less than retry count  
            while (retryAttempts < retryCount)  
            {  
                try  
                {  
                    if (!retry)  
                    {  
                        clientContext.ExecuteQuery();  
                        return;  
                    }  
                    else  
                    {  
                        //increment the retry count  
                        retryAttempts++;  
  
                        // retry the previous request using wrapper  
                        if (wrapper != null && wrapper.Value != null)  
                        {  
                            clientContext.RetryQuery(wrapper.Value);  
                            return;  
                        }  
                        // retry the previous request as normal  
                        else  
                        {  
                            clientContext.ExecuteQuery();  
                            return;  
                        }  
                    }  
                }  
                catch (WebException ex)  
                {  
                    var response = ex.Response as HttpWebResponse;  
                    // Check if request was throttled - http status code 429  
                    // Check is request failed due to server unavailable - http status code 503  
                    if (response != null && (response.StatusCode == (HttpStatusCode)429 || response.StatusCode == (HttpStatusCode)503))  
                    {  
                        wrapper = (ClientRequestWrapper)ex.Data["ClientRequest"];  
                        retry = true;  
  
                        // Determine the retry after value - use the `Retry-After` header when available  
                        string retryAfterHeader = response.GetResponseHeader("Retry-After");  
                        if (!string.IsNullOrEmpty(retryAfterHeader))  
                        {  
                            if (!Int32.TryParse(retryAfterHeader, out retryAfterInterval))  
                            {  
                                retryAfterInterval = backoffInterval;  
                            }  
                        }  
                        else  
                        {  
                            retryAfterInterval = backoffInterval;  
                        }  
  
                        // Delay for the requested seconds  
                        Thread.Sleep(retryAfterInterval * 1000);  
  
                        // Increase counters  
                        backoffInterval = backoffInterval * 2;  
                    }  
                    else  
                    {  
                        throw;  
                    }  
                }  
            }  
            throw new MaximumRetryAttemptedException($"Maximum retry attempts {retryCount}, has be attempted.");  
        }  
  
        [Serializable]  
        public class MaximumRetryAttemptedException : Exception  
        {  
            public MaximumRetryAttemptedException(string message) : base(message) { }  
        }  

Usage:

5 retries in 10 seconds:

                    Web web = ctx.Web;  
                    ListCollection targetlListCollection = web.Lists;  
                    ctx.Load(targetlListCollection, l => l.Include(x => x.Id, x => x.BaseTemplate, x => x.BaseType, x => x.DefaultViewUrl));  
                    ctx.ExecuteQueryWithIncrementalRetry(5, 10);  

Here is a official document about handing SharePoint Online Throttling in CSOM, I suggest you can have a reference:

Avoid getting throttled or blocked in SharePoint Online

Thanks
Best Regards

---

If an Answer is helpful, please click "Accept Answer" and upvote it.
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.