Forum Discussion
snkprdhn
Jan 08, 2024Copper Contributor
Resource Graph RateLimiting
When running some resource graph queries inside a Function App I get RateLimiting error:
error in resourceGraphFunction (RateLimiting) Please provide below info when asking for support: timestamp = 2024-01-02T16:10:29.7151093Z, correlationId = bf7ffdce-2c00-49f9-8171-0a682d3e6966. Code: RateLimiting Message: Please provide below info when asking for support: timestamp = 2024-01-02T16:10:29.7151093Z, correlationId = bf7ffdce-2c00-49f9-8171-0a682d3e6966. Exception Details: (RateLimiting) Client application has been throttled and should not attempt to repeat the request until an amount of time has elapsed. Please see Overview of Azure Resource Graph - Azure Resource Graph for help. Code: RateLimiting Message: Client application has been throttled and should not attempt to repeat the request until an amount of time has elapsed. Please see Overview of Azure Resource Graph - Azure Resource Graph for help.
This error is not very frequent and not easily reproducible, I'm using the following python code to reproduce the issue.
import azure.functions as func
import azure.mgmt.resourcegraph as arg
from azure.identity import DefaultAzureCredential
import datetime
def custom_res(pipeline_response, deserialized, *kwargs):
resource = deserialized
quota_remaining = None
quota_resets_after = None
try:
headers = pipeline_response.http_response.internal_response.headers
quota_remaining = headers._store['x-ms-user-quota-remaining']
quota_resets_after = headers._store['x-ms-user-quota-resets-after']
status_code = pipeline_response.http_response.status_code
except AttributeError:
pass
setattr(resource, 'x-ms-user-quota-remaining', quota_remaining)
setattr(resource, 'x-ms-user-quota-resets-after', quota_resets_after)
setattr(resource, 'status_code', status_code)
return resource
def getPagedResources(sub_id, strQuery):
subsList = []
if len(sub_id) > 0:
subsList.append(sub_id)
arg_result_arr = []
skip_num = 0
result_limit = 1000
while(True):
argQueryOptions = arg.models.QueryRequestOptions(result_format="objectArray", top=result_limit, skip=skip_num)
argQuery = arg.models.QueryRequest(subscriptions=subsList, query=strQuery, options=argQueryOptions)
# Run query
argResults = argClient.resources(argQuery, cls=custom_res)
log.info(f"time: {datetime.datetime.now()}, remaining: {getattr(argResults, 'x-ms-user-quota-remaining', None)[1]}, status_code: {argResults.status_code}, reset_time: {getattr(argResults, 'x-ms-user-quota-resets-after', None)[1]}, skip_num: {skip_num}")
if not argResults.data:
break
if not arg_result_arr:
arg_result_arr = argResults.data
else:
arg_result_arr = arg_result_arr + argResults.data
skip_num = skip_num + result_limit
return arg_result_arr
def main(req: func.HttpRequest) -> func.HttpResponse:
qry = "resources"
try:
for i in range(20):
res = getPagedResources('123-456-789-34343',qry)
except Exception as err:
log.error(f"error: {err}")
Output:
Even if the user-quota-remaining
is 0, I don't get the RateLimit error that I sometimes get.
Is there any way to reproduce the issue or any fix for it ?
1 Reply
- JeremyWallaceBrass ContributorPython code is not really in my wheelhouse but based on the code you provided, you may want to consider adding a delay between requests to avoid exceeding the rate limits if that is indeed the issue that is occuring.