Let's be honest. As a cloud engineer or DevOps professional managing a large Azure environment, running even a simple resource inventory query can feel like drinking from a firehose. You hit API limits, face slow performance, and struggle to get the complete picture of your estate—all because the data volume is overwhelming.

But it doesn't have to be this way!

This blog is your practical, hands-on guide to mastering two essential techniques for handling massive data volumes in Azure: using PowerShell and Azure Resource Graph (ARG): Skip Token (for full data retrieval) and Batching (for blazing-fast performance).

📋 TABLE OF CONTENTS

🚀 GETTING STARTED
   │
   ├─ Prerequisites: PowerShell 7+ & Az.ResourceGraph Module
   └─ Introduction: Why Standard Queries Fail at Scale

📖 CORE CONCEPTS
   │
   ├─ 📑 Skip Token: The Data Completeness Tool
   │  ├─ What is a Skip Token?
   │  ├─ The Bookmark Analogy
   │  ├─ PowerShell Implementation
   │  └─ 💻 Code Example: Pagination Loop
   │
   └─ ⚡ Batching: The Performance Booster
      ├─ What is Batching?
      ├─ Performance Benefits
      ├─ Batching vs. Pagination
      ├─ Parallel Processing in PowerShell
      └─ 💻 Code Example: Concurrent Queries

🔍 DEEP DIVE
   │
   ├─ Skip Token: Generic vs. Azure-Specific
   └─ Azure Resource Graph (ARG) at Scale
      ├─ ARG Overview
      ├─ Why ARG Needs These Techniques
      └─ 💻 Combined Example: Skip Token + Batching

✅ BEST PRACTICES
   │
   ├─ When to Use Each Technique
   └─ Quick Reference Guide

📚 RESOURCES
   └─ Official Documentation & References

Prerequisites

Introduction: Why Standard Queries Don't Work at Scale

When you query a service designed for big environments, like Azure Resource Graph, you face two limits:

1. Result Limits (Pagination): APIs won't send you millions of records at once. They cap the result size (often 1,000 items) and stop.
2. Efficiency Limits (Throttling): Sending a huge number of individual requests is slow and can cause the API to temporarily block you (throttling).

Skip Token helps you solve the first limit by making sure you retrieve all results.

Batching solves the second by grouping your requests to improve performance.

Understanding Skip Token: The Continuation Pointer

What is a Skip Token?

A Skip Token (or continuation token) is a unique string value returned by an Azure API when a query result exceeds the maximum limit for a single response.

Think of the Skip Token as a “bookmark” that tells Azure where your last page ended — so you can pick up exactly where you left off in the next API call. Instead of getting cut off after 1,000 records, the API gives you the first 1,000 results plus the Skip Token. You use this token in the next request to get the next page of data. This process is called pagination.

Skip Token in Practice with PowerShell

To get the complete dataset, you must use a loop that repeatedly calls the API, providing the token each time until the token is no longer returned.

PowerShell Example: Using Skip Token to Loop Pages

# Define the query
$Query = "Resources | project name, type, location"
$PageSize = 1000 

$AllResults = @()
$SkipToken = $null # Initialize the token

Write-Host "Starting ARG query..."

do {
    Write-Host "Fetching next page. (Token check: $($SkipToken -ne $null))"
    
    # 1. Execute the query, using the -SkipToken parameter
    $ResultPage = Search-AzGraph -Query $Query -First $PageSize -SkipToken $SkipToken

    # 2. Add the current page results to the main array
    $AllResults += $ResultPage

    # 3. Get the token for the next page, if it exists
    $SkipToken = $ResultPage.SkipToken

    Write-Host "  -> Items in this page: $($ResultPage.Count). Total retrieved: $($AllResults.Count)"
    
} while ($SkipToken -ne $null) # Loop as long as a Skip Token is returned

Write-Host "Query finished. Total resources found: $($AllResults.Count)"

This do-while loop is the reliable way to ensure you retrieve every item in a large result set.

Understanding Batching: Grouping Requests

What is Batching?

Batching means taking several independent requests and combining them into a single API call. Instead of making N separate network requests for N pieces of data, you make one request containing all N sub-requests.

Batching is primarily used for performance. It improves efficiency by:

1. Reducing Overhead: Fewer separate network connections are needed.
2. Lowering Throttling Risk: Fewer overall API calls are made, which helps you stay under rate limits.

Note: While Azure Resource Graph's REST API supports batch requests, the PowerShell Search-AzGraph cmdlet does not expose a -Batch parameter. Instead, we achieve batching by using PowerShell's ForEach-Object -Parallel (PowerShell 7+) to run multiple queries simultaneously.

Batching in Practice with PowerShell

Using parallel processing in PowerShell, you can efficiently execute multiple distinct Kusto queries targeting different scopes (like subscriptions) simultaneously.

PowerShell Example: Running Multiple Queries in Parallel

# Define multiple queries to run together
$BatchQueries = @(
    @{
        Query = "Resources | where type =~ 'Microsoft.Compute/virtualMachines'"
        Subscriptions = @("SUB_A") # Query 1 Scope
    },
    @{
        Query = "Resources | where type =~ 'Microsoft.Network/publicIPAddresses'"
        Subscriptions = @("SUB_B", "SUB_C") # Query 2 Scope
    }
)

Write-Host "Executing batch of $($BatchQueries.Count) queries in parallel..."

# Execute queries in parallel (true batching)
$BatchResults = $BatchQueries | ForEach-Object -Parallel {
    $QueryConfig = $_
    $Query = $QueryConfig.Query
    $Subs = $QueryConfig.Subscriptions
    
    Write-Host "[Batch Worker] Starting query: $($Query.Substring(0, [Math]::Min(50, $Query.Length)))..." -ForegroundColor Cyan
    
    $QueryResults = @()
    
    # Process each subscription in this query's scope
    foreach ($SubId in $Subs) {
        $SkipToken = $null
        
        do {
            $Params = @{
                Query        = $Query
                Subscription = $SubId
                First        = 1000
            }
            
            if ($SkipToken) {
                $Params['SkipToken'] = $SkipToken
            }
            
            $Result = Search-AzGraph @Params
            
            if ($Result) {
                $QueryResults += $Result
            }
            
            $SkipToken = $Result.SkipToken
            
        } while ($SkipToken)
    }
    
    Write-Host "  [Batch Worker] ✅ Query completed - Retrieved $($QueryResults.Count) resources" -ForegroundColor Green
    
    # Return results with metadata
    [PSCustomObject]@{
        Query = $Query
        Subscriptions = $Subs
        Data = $QueryResults
        Count = $QueryResults.Count
    }
    
} -ThrottleLimit 5

Write-Host "`nBatch complete. Reviewing results..."

# The results are returned in the same order as the input array
$VMCount = $BatchResults[0].Data.Count
$IPCount = $BatchResults[1].Data.Count

Write-Host "Query 1 (VMs) returned: $VMCount results."
Write-Host "Query 2 (IPs) returned: $IPCount results."

# Optional: Display detailed results
Write-Host "`n--- Detailed Results ---"
for ($i = 0; $i -lt $BatchResults.Count; $i++) {
    $Result = $BatchResults[$i]
    Write-Host "`nQuery $($i + 1):"
    Write-Host "  Query: $($Result.Query)"
    Write-Host "  Subscriptions: $($Result.Subscriptions -join ', ')"
    Write-Host "  Total Resources: $($Result.Count)"
    
    if ($Result.Data.Count -gt 0) {
        Write-Host "  Sample (first 3):"
        $Result.Data | Select-Object -First 3 | Format-Table -AutoSize
    }
}

Azure Resource Graph (ARG) and Scale

Azure Resource Graph (ARG) is a service built for querying resource properties quickly across a large number of Azure subscriptions using the Kusto Query Language (KQL).

Because ARG is designed for large scale, it fully supports Skip Token and Batching:

• Skip Token: ARG automatically generates and returns the token when a query exceeds its result limit (e.g., 1,000 records).
• Batching: ARG's REST API provides a batch endpoint for sending up to ten queries in a single request. In PowerShell, we achieve similar performance benefits using ForEach-Object -Parallel to process multiple queries concurrently.

Combined Example: Batching and Skip Token Together

This script shows how to use Batching to start a query across multiple subscriptions and then use Skip Token within the loop to ensure every subscription's data is fully retrieved.

$SubscriptionIDs = @("SUB_A")
$KQLQuery = "Resources | project id, name, type, subscriptionId"

Write-Host "Starting BATCHED query across $($SubscriptionIDs.Count) subscription(s)..."
Write-Host "Using parallel processing for true batching...`n"

# Process subscriptions in parallel (batching)
$AllResults = $SubscriptionIDs | ForEach-Object -Parallel {
    $SubId = $_
    $Query = $using:KQLQuery
    $SubResults = @()
    
    Write-Host "[Batch Worker] Processing Subscription: $SubId" -ForegroundColor Cyan
    
    $SkipToken = $null
    $PageCount = 0
    
    do {
        $PageCount++
        
        # Build parameters
        $Params = @{
            Query        = $Query
            Subscription = $SubId
            First        = 1000
        }
        
        if ($SkipToken) {
            $Params['SkipToken'] = $SkipToken
        }
        
        # Execute query
        $Result = Search-AzGraph @Params
        
        if ($Result) {
            $SubResults += $Result
            Write-Host "  [Batch Worker] Sub: $SubId - Page $PageCount - Retrieved $($Result.Count) resources" -ForegroundColor Yellow
        }
        
        $SkipToken = $Result.SkipToken
        
    } while ($SkipToken)
    
    Write-Host "  [Batch Worker] ✅ Completed $SubId - Total: $($SubResults.Count) resources" -ForegroundColor Green
    
    # Return results from this subscription
    $SubResults
    
} -ThrottleLimit 5  # Process up to 5 subscriptions simultaneously

Write-Host "`n--- Batch Processing Finished ---"
Write-Host "Final total resource count: $($AllResults.Count)"

# Optional: Display sample results
if ($AllResults.Count -gt 0) {
    Write-Host "`nFirst 5 resources:"
    $AllResults | Select-Object -First 5 | Format-Table -AutoSize
}

By combining Skip Token for data completeness and Batching for efficiency, you can confidently query massive Azure estates without hitting limits or missing data.

These two techniques — when used together — turn Azure Resource Graph from a “good tool” into a scalable discovery engine for your entire cloud footprint.

Summary: Skip Token and Batching in Azure Resource Graph

Goal: Efficiently query massive Azure environments using PowerShell and Azure Resource Graph (ARG).

1. Skip Token (The Data Completeness Tool)

2. Batching (The Performance Booster)

3. Best Practice: Combine Them

For maximum efficiency, combine Batching and Skip Token. Use batching to run queries across multiple subscriptions simultaneously and use the Skip Token logic within the loop to ensure every single subscription's data is fully paginated and retrieved.

Result: Fast, complete, and reliable data collection across your large Azure estate.

References:

• Azure Resource Graph documentation
• Search-AzGraph PowerShell reference