How to Visualize Your Azure AI Workloads Usage for Observability

Option 1. Within the Microsoft Foundry Portal (Quickest, No Setup)

If you have models deployed in Microsoft Foundry and would like to monitor its usage, go to the New Foundry Portal → Build → Models → Monitor tab.

View metrics such as:

• Estimated cost
• Total token usage
• Input vs. output tokens
• Number of requests

This is the simplest way to monitor both model and agent usage.

Microsoft Foundry has a built in Monitor tab to view your model/agent usage.

For PAYG plans:

You can also view your total allocated quota (and figure out which Tier you are on) using the Quota Management Screen (New Foundry Portal → Operate → Quota tab). 

This screen shows how much your total allocated quota is, per model in a given subscription + region + Deployment Type (Global, Data Zones or Regional). For eg., in the image below, for gpt-4o, I am allocated 7M total TPM in my subscription. I am only using 150K TPM of the allocated 7M TPM amount.

Which means, my requests will get throttled if I exceed the 150K TPM limit. To avoid throttling, I would need to increase my shared allocation limit.

NOTE: you are charged for usage, so if you allow more capacity, you use more, so you pay more. 

Option 2: Azure Monitor Metrics Explorer

This is already built into the Azure Portal and gives you time-series charts out of the box.

1. Go to Azure Portal → your Azure OpenAI / Foundry resource → Monitoring → Metrics
2. Select a metric like AzureOpenAIRequests or TokenTransaction
3. Set Aggregation to Sum (total) or Max and Time granularity to 1 day
4. Split by ModelDeploymentName to see per-model trends
5. Adjust the time range (e.g., last 30 days) — you'll see day-over-day bars/lines

Tip: You can pin these charts to an Azure Dashboard for a persistent view, or click Share → Download to Excel to get the raw data for your own analysis. 

Option 3: Azure Managed Grafana (Best Pre-Built Dashboard)

This is the best option for a polished, real-time, day-over-day dashboard with no custom code. There's a pre-built AI Foundry dashboard ready to import. [grafana.com], [Create a M...ed Grafana]

How to set it up:

1. Create an Azure Managed Grafana workspace (if you don't have one)
2. In Grafana, go to Dashboards → New → Import → enter dashboard ID 24039 (for Foundry)
3. Select your Azure Monitor data source and point it to your Foundry resource
4. Tip: You can also import this directly from the Azure Portal: Monitor → Dashboards with Grafana → AI Foundry.
5. That's it — the dashboard gives you (per model deployment):

Token trends over time (inference, prompt, completion — day over day)

Request trends over time (AzureOpenAIRequests as a time series)

Latency trends (bonus)

NOTE: Default time range is 7 days — adjust to 30/60/90 days for growth trends

Option 4: Application Insights + KQL Queries (Most Flexible, Custom Reports)

If you want fully custom day-over-day growth calculations (e.g., % change day-to-day), this is the way. [azurefeeds.com]

Setup:

1. Ensure your Foundry project is connected to an Application Insights resource (Foundry → Settings → Connected Resources). 
2. Open up App Insights resource → Logs → New Query or choose a sample query. In the images below, we simply ran 'requests' and set the time range to 24 hours. 
3. There is also a Kusto Query Language (KQL) mode or Simple mode on the right-hand side: 
   • Simple mode will let you run out of the box samples.
   • KQL mode will open up a query window for you to enter custom queries.
4. Below are the results in grid view. 

Same view but showing a chart: 

Export options: 

Another way to get the above graphs are via Log Analytics. Simply enable Diagnostic Settings on your Azure OpenAI resource → send to a Log Analytics workspace. Open Log Analytics → Logs and try our your sample queries. 

Sample KQL for day-over-day token usage (adjust to your needs):

AzureMetrics

| where MetricName in ("TokenTransaction", "ProcessedPromptTokens", "GeneratedTokens")

| where TimeGenerated > ago(30d)

| summarize DailyTokens = sum(Total) by bin(TimeGenerated, 1d), MetricName

| order by TimeGenerated asc

| render timechart

Result: 

Sample KQL for day-over-day growth % (adjust to your needs):

AzureMetrics

| where MetricName == "TokenTransaction"

| where TimeGenerated > ago(30d)

| summarize DailyTokens = sum(Total) by Day = bin(TimeGenerated, 1d)

| sort by Day asc

| extend PrevDay = prev(DailyTokens)

| extend GrowthPct = round((DailyTokens - PrevDay) / PrevDay * 100, 2)

| project Day, DailyTokens, GrowthPct

Option 5: Azure Monitor Workbooks (Custom Dashboards, Shareable)

Workbooks let you build interactive, parameterized dashboards that combine metrics and KQL logs. 

What's more, you can select resources from multiple subscriptions and visualize them all in one place using Workbooks!

1. Go to Azure Portal → Monitor → Workbooks → New
2. Add a Metrics query panel → select your Log Analytics or App Insights or Foundry resource -> Enter the same query you used in Option 4.
3. Do a test run and view the graphs (this can be viewed as charts or a list (grid view)):

You can select different resources (or subscriptions) and view them all in one pane.

      4. Save and share with your team.

Option 6: APIM + Application Insights (Granular Per-Caller/Per-Agent Tracking)

1. If your app routes requests through Azure API Management, you can use the azure-openai-emit-token-metric policy to send per-request token metrics to Application Insights with custom dimensions (User ID, Subscription ID, Agent, etc.). [Azure API...osoft Docs]

This is ideal for scenarios like:

• "Which agent consumed the most tokens last week?"
• "What's the token usage per API consumer/team?"

NOTE: Microsoft Foundry resources do not track usage by users. So, fronting your Foundry resource with an APIM could be a way to track users provided you pass the username/id in the request context. How you implement this is upto your app design. 

Ref: AI-Gateway/labs/token-metrics-emitting/token-metrics-emitting.ipynb at main · Azure-Samples/AI-Gateway · GitHub

Bonus: Check out all other APIM + AI related policies here:

AI-Gateway/labs/semantic-caching at main · Azure-Samples/AI-Gateway

AI-Gateway/labs/token-rate-limiting at main · Azure-Samples/AI-Gateway

Ref: AI-Gateway/labs/token-metrics-emitting/token-metrics-emitting.ipynb at main · Azure-Samples/AI-Gateway · GitHub