Forum Discussion
RetryQuery not working after throttling
Hi everyone,
We are using the recommended approach for throttling.
Here is the method we are using to execute the requests:
public void ExecuteQueryWithRetry(ClientContext ctx, int retryCount, int backOffInterval, ILog LOG)
{
DateTime startTimeStamp = DateTime.Now;
try
{
int retryAttempts = 0;
int retryAfterInterval = 0;
bool retry = false;
ClientRequestWrapper wrapper = null;
LOG.Debug($"Client Context: {ctx.GetHashCode()}");
LOG.Debug($"Throttled : {Throttled}");
if (Throttled)
{
while (Throttled)
{
LOG.Debug("Still throttled...");
Thread.Sleep(100);
}
Thread.Sleep(new Random().Next(500));
LOG.Debug("Throttled finished");
}
while (retryAttempts < retryCount)
{
try
{
if (retry && wrapper != null && wrapper.Value != null)
{
LOG.Debug("Execute request with wrapper value...");
ctx.RetryQuery(wrapper.Value);
LOG.Debug("Execute request with wrapper value finished");
Throttled = false;
return;
}
else
{
LOG.Debug("Execute request...");
ctx.ExecuteQuery();
LOG.Debug("Execute request finished");
Throttled = false;
return;
}
}
catch (WebException ex)
{
var response = ex.Response as HttpWebResponse;
// Check for throttling
if (response != null && (response.StatusCode == (HttpStatusCode)429 || response.StatusCode == (HttpStatusCode)503))
{
Throttled = true;
wrapper = (ClientRequestWrapper)ex.Data["ClientRequest"];
retry = true;
string retryAfterHeader = response.GetResponseHeader("Retry-After");
if (!string.IsNullOrEmpty(retryAfterHeader))
{
if (!Int32.TryParse(retryAfterHeader, out retryAfterInterval))
{
retryAfterInterval = backOffInterval;
}
}
else
{
retryAfterInterval = backOffInterval;
}
LOG.Warn($"We got throttled! Will back off for {retryAfterInterval} seconds.");
}
else
{
LOG.Debug(ex.StackTrace);
throw;
}
}
Thread.Sleep(retryAfterInterval * 1000);
retryAttempts++;
}
throw new MaximumRetryAttemptedException($"Maximum retry attempts '{retryCount}' has been attempted.");
}
finally
{
if (LOG.IsDebugEnabled)
{
TimeSpan duration = DateTime.Now - startTimeStamp;
LOG.Debug($"Executed CSOM query in [{duration.TotalMilliseconds.ToString("0.00")}] ms");
}
}
}
We got the 429 error and after that we wait the recommended time until we execute again the request.
The request is re-executed using the RetryQuery method. After the method is called, we got no error but the request is not executed.
Do you have any ideas why we encounter this problem?
2 Replies
- PatrikHellgrenCopper Contributor
cristinadulau If possible you should try to use RateLimit Headers like explained here: https://github.com/OneDrive/samples/tree/master/scenarios/throttling-ratelimit-handling
If that is not possible then have a look at my answer here on using HttpClientWebRequestExecutorFactory: https://github.com/pnp/pnpframework/issues/779#issuecomment-1293277372
This is also used in the RateLimits sample above. - kalpeshvaghelaIron Contributor
cristinadulau
Can you try to use ExecuteQueryRetry Function from PnP.Framework. ?If it's not possible for your to use above Nueget Package then you can copy that function from PnP.Framework repository which you can find code at here
Hope it will helpful to you and if so then Please mark my response as Best Response & Like to help others in this community