Forum Discussion

snkprdhn's avatar
snkprdhn
Copper Contributor
Jan 08, 2024

Resource Graph RateLimiting

When running some resource graph queries inside a Function App I get RateLimiting error:

 

 

error in resourceGraphFunction (RateLimiting) Please provide below info when asking for support: timestamp = 2024-01-02T16:10:29.7151093Z, correlationId = bf7ffdce-2c00-49f9-8171-0a682d3e6966. Code: RateLimiting Message: Please provide below info when asking for support: timestamp = 2024-01-02T16:10:29.7151093Z, correlationId = bf7ffdce-2c00-49f9-8171-0a682d3e6966. Exception Details: (RateLimiting) Client application has been throttled and should not attempt to repeat the request until an amount of time has elapsed. Please see Overview of Azure Resource Graph - Azure Resource Graph  for help. Code: RateLimiting Message: Client application has been throttled and should not attempt to repeat the request until an amount of time has elapsed. Please see Overview of Azure Resource Graph - Azure Resource Graph  for help.

 

 

 This error is not very frequent and not easily reproducible, I'm using the following python code to reproduce the issue.

 

 

import azure.functions as func
import azure.mgmt.resourcegraph as arg
from azure.identity import DefaultAzureCredential
import datetime

def custom_res(pipeline_response, deserialized, *kwargs):
    resource = deserialized
    quota_remaining = None
    quota_resets_after = None
    try:
        headers = pipeline_response.http_response.internal_response.headers
        quota_remaining = headers._store['x-ms-user-quota-remaining']
        quota_resets_after = headers._store['x-ms-user-quota-resets-after']
        status_code = pipeline_response.http_response.status_code
    except AttributeError:
        pass
    setattr(resource, 'x-ms-user-quota-remaining', quota_remaining)
    setattr(resource, 'x-ms-user-quota-resets-after', quota_resets_after)
    setattr(resource, 'status_code', status_code)
    return resource


def getPagedResources(sub_id, strQuery):
    subsList = []
    if len(sub_id) > 0:
        subsList.append(sub_id)

    arg_result_arr = []
    skip_num = 0
    result_limit = 1000
    while(True):
        argQueryOptions = arg.models.QueryRequestOptions(result_format="objectArray", top=result_limit, skip=skip_num)
        argQuery = arg.models.QueryRequest(subscriptions=subsList, query=strQuery, options=argQueryOptions)

        # Run query
        argResults = argClient.resources(argQuery, cls=custom_res)
        log.info(f"time: {datetime.datetime.now()}, remaining: {getattr(argResults, 'x-ms-user-quota-remaining', None)[1]}, status_code: {argResults.status_code}, reset_time: {getattr(argResults, 'x-ms-user-quota-resets-after', None)[1]}, skip_num: {skip_num}")
        if not argResults.data:
            break
        if not arg_result_arr:
            arg_result_arr = argResults.data
        else:
            arg_result_arr = arg_result_arr + argResults.data
        skip_num = skip_num + result_limit
    return arg_result_arr

def main(req: func.HttpRequest) -> func.HttpResponse:
    qry = "resources"
    try:
        for i in range(20):
            res = getPagedResources('123-456-789-34343',qry)
    except Exception as err:
        log.error(f"error: {err}")

 

 

 

Output:

 

Even if the user-quota-remaining

is 0, I don't get the RateLimit error that I sometimes get.

Is there any way to reproduce the issue or any fix for it ?

1 Reply

  • JeremyWallace's avatar
    JeremyWallace
    Brass Contributor
    Python code is not really in my wheelhouse but based on the code you provided, you may want to consider adding a delay between requests to avoid exceeding the rate limits if that is indeed the issue that is occuring.

Resources