Let's be honest. As a cloud engineer or DevOps professional managing a large Azure environment, running even a simple resource inventory query can feel like drinking from a firehose. You hit API limits, face slow performance, and struggle to get the complete picture of your estate—all because the data volume is overwhelming.

But it doesn't have to be this way!

This blog is your practical, hands-on guide to mastering two essential techniques for handling massive data volumes in Azure: using PowerShell and Azure Resource Graph (ARG): Skip Token (for full data retrieval) and Batching (for blazing-fast performance).

Table of Contents

1. Introduction
2. Understanding Data Pagination in Cloud Queries
3. What Is a Skip Token?
   3.1 Concept of Skip Token
   3.2 PowerShell Example Using Skip Token
4. What Is Batching?
   4.1 Concept of Batching
   4.2 PowerShell Example Using Batching
5. Is Skip Token a Generic Concept or Azure-Specific?
   5.1 Skip Token in Generic API Design
   5.2 Skip Token Usage in Azure Resource Graph (ARG)
6. Introduction to Azure Resource Graph (ARG)
   6.1 What Is Azure Resource Graph
   6.2 Why ARG Uses Skip Token and Batching
   6.3 Practical PowerShell Example Using ARG with Skip Token and Batching
7. Summary and References

Introduction: Why Standard Queries Don't Work at Scale

When you query a service designed for big environments, like Azure Resource Graph, you face two limits:

1. Result Limits (Pagination): APIs won't send you millions of records at once. They cap the result size (often 1,000 items) and stop.
2. Efficiency Limits (Throttling): Sending a huge number of individual requests is slow and can cause the API to temporarily block you (throttling).

Skip Token helps you solve the first limit by making sure you retrieve all results.

Batching solves the second by grouping your requests to improve performance.

Understanding Skip Token: The Continuation Pointer

What is a Skip Token?

A Skip Token (or continuation token) is a unique string value returned by an Azure API when a query result exceeds the maximum limit for a single response.

Think of the Skip Token as a “bookmark” that tells Azure where your last page ended — so you can pick up exactly where you left off in the next API call. Instead of getting cut off after 1,000 records, the API gives you the first 1,000 results plus the Skip Token. You use this token in the next request to get the next page of data. This process is called pagination.

Skip Token in Practice with PowerShell

To get the complete dataset, you must use a loop that repeatedly calls the API, providing the token each time until the token is no longer returned.

PowerShell Example: Using Skip Token to Loop Pages

# Define the query
$Query = "Resources | project name, type, location"
$PageSize = 1000 

$AllResults = @()
$SkipToken = $null # Initialize the token

Write-Host "Starting ARG query..."

do {
    Write-Host "Fetching next page. (Token check: $($SkipToken -ne $null))"
    
    # 1. Execute the query, using the -SkipToken parameter
    $ResultPage = Search-AzGraph -Query $Query -First $PageSize -SkipToken $SkipToken

    # 2. Add the current page results to the main array
    $AllResults += $ResultPage.Data

    # 3. Get the token for the next page, if it exists
    $SkipToken = $ResultPage.SkipToken

    Write-Host "  -> Items in this page: $($ResultPage.Data.Count). Total retrieved: $($AllResults.Count)"
    
} while ($SkipToken -ne $null) # Loop as long as a Skip Token is returned

Write-Host "Query finished. Total resources found: $($AllResults.Count)"

This do-while loop is the reliable way to ensure you retrieve every item in a large result set.

Understanding Batching: Grouping Requests

What is Batching?

Batching means taking several independent requests and combining them into a single API call. Instead of making N separate network requests for N pieces of data, you make one request containing all N sub-requests.

Batching is primarily used for performance. It improves efficiency by:

1. Reducing Overhead: Fewer separate network connections are needed.
2. Lowering Throttling Risk: Fewer overall API calls are made, which helps you stay under rate limits.

Batching in Practice with PowerShell

In ARG, batching lets you run up to ten distinct Kusto queries, possibly targeting different scopes (like subscriptions), in one command.

PowerShell Example: Running Multiple Queries in One Call

# Define multiple queries to run together
$BatchQueries = @(
    @{
        Query = "Resources | where type =~ 'Microsoft.Compute/virtualMachines'"
        Subscriptions = "Subscription-A-ID" # Query 1 Scope
    },
    @{
        Query = "Resources | where type =~ 'Microsoft.Network/publicIPAddresses'"
        Subscriptions = "Subscription-B-ID", "Subscription-C-ID" # Query 2 Scope
    }
)

Write-Host "Executing batch of $($BatchQueries.Count) queries..."

# Use the -Batch parameter with Search-AzGraph
$BatchResults = Search-AzGraph -Batch $BatchQueries

Write-Host "Batch complete. Reviewing results..."

# The results are returned in the same order as the input array
$VMCount = $BatchResults[0].Data.Count
$IPCount = $BatchResults[1].Data.Count

Write-Host "Query 1 (VMs) returned: $VMCount results."
Write-Host "Query 2 (IPs) returned: $IPCount results."

Azure Resource Graph (ARG) and Scale

Azure Resource Graph (ARG) is a service built for querying resource properties quickly across a large number of Azure subscriptions using the Kusto Query Language (KQL).

Because ARG is designed for large scale, it fully supports Skip Token and Batching:

• Skip Token: ARG automatically generates and returns the token when a query exceeds its result limit (e.g., 1,000 records).
• Batching: ARG provides a batch API endpoint that allows you to send an array of up to ten distinct queries in a single request for optimization.

Combined Example: Batching and Skip Token Together

This script shows how to use Batching to start a query across multiple subscriptions and then use Skip Token within the loop to ensure every subscription's data is fully retrieved.

# Subscriptions to query
$SubscriptionIDs = @("Sub-Alpha-ID", "Sub-Beta-ID")
$KQLQuery = "Resources | project id, name, type, subscriptionId"

$AllResults = @()

Write-Host "Starting batched query across $($SubscriptionIDs.Count) subscriptions..."

# Create the initial batch request
$CurrentBatch = $SubscriptionIDs | ForEach-Object {
    [PSCustomObject]@{
        Query         = $KQLQuery
        Subscriptions = $_ # Scope is one subscription ID
    }
}

$Cycle = 0
do {
    Write-Host "`n--- Running Batch Cycle: $($Cycle++) ---"
    
    # 1. Run the batch of queries (each query might return a page of results)
    $BatchResponse = Search-AzGraph -Batch $CurrentBatch -First 1000

    # 2. Prepare the batch for the next loop
    $NextBatch = @()
    $NewResultsInCycle = 0

    # 3. Process the results for each sub-query
    foreach ($Result in $BatchResponse) {
        $SubId = $Result.Subscriptions 

        # Accumulate results
        $AllResults += $Result.Data
        $NewResultsInCycle += $Result.Data.Count

        # Check if a Skip Token was returned
        if ($Result.SkipToken) {
            Write-Host "  ✅ Skip Token found for Subscription ID: $SubId. Preparing next page request."
            # Add this query to the next batch with the new Skip Token
            $NextBatch += [PSCustomObject]@{
                Query         = $KQLQuery
                Subscriptions = $SubId
                SkipToken     = $Result.SkipToken # The key to the next page
            }
        } else {
             Write-Host "  🛑 Query complete for Subscription ID: $SubId."
        }
    }

    Write-Host "Total new resources retrieved this cycle: $NewResultsInCycle"

    # Set the batch for the next iteration (only queries with remaining pages)
    $CurrentBatch = $NextBatch

} while ($CurrentBatch.Count -gt 0) # Loop until all pages for all subscriptions are retrieved

Write-Host "`n--- Script Finished ---"
Write-Host "Final total resource count: $($AllResults.Count)"

By combining Skip Token for data completeness and Batching for efficiency, you can confidently query massive Azure estates without hitting limits or missing data.

These two techniques — when used together — turn Azure Resource Graph from a “good tool” into a scalable discovery engine for your entire cloud footprint.

Summary: Skip Token and Batching in Azure Resource Graph

Goal: Efficiently query massive Azure environments using PowerShell and Azure Resource Graph (ARG).

1. Skip Token (The Data Completeness Tool)

2. Batching (The Performance Booster)

3. Best Practice: Combine Them

For maximum efficiency, combine Batching and Skip Token. Use batching to run queries across multiple subscriptions simultaneously, and use the Skip Token logic within the loop to ensure every single subscription's data is fully paginated and retrieved.

Result: Fast, complete, and reliable data collection across your large Azure estate.

References:

• Azure Resource Graph documentation
• Search-AzGraph PowerShell reference