Azure Architecture Blog articles

From Policy to Proof: Governing AI to Scale Human Ambition and Machine Intelligence

manasa_ramalinga — Wed, 15 Jul 2026 19:51:51 GMT

Enterprises are moving from AI experiments to AI in production. Copilots are in the flow of daily work, custom applications are built on foundation models, and autonomous agents are beginning to take actions on behalf of the business. As organizations combine human ambition with machine intelligence, trust becomes the defining requirement for scaling AI responsibly and realizing AI's full potential.

That shift changes the governance question from "can we use AI responsibly?" to a harder one: "can we prove, continuously, that every AI system and agent in our estate is safe, compliant, observable, and accountable?"

The organizations getting this right have stopped treating governance as a static document that lives in a policy binder. They treat it as an operating system: a connected set of policies, controls, telemetry, and evidence that runs alongside AI everywhere it operates. Governance defines what should happen. Observability and evaluations verify what is actually happening. Audit and response prove it, and feed what they learn back into policy. The loop, not any single control, is what makes governance real.

Every control in this map ladders up to a principled foundation: Microsoft's Responsible AI Standard, whose six principles (fairness, reliability and safety, privacy and security, inclusiveness, transparency, and accountability) set the bar these controls exist to meet.

This post lays out a practical reference map for that operating system: the domains AI governance must cover, the cross-cutting runtime enforcement layer that operationalizes them, and the Microsoft services that support each one.

The four pillars of AI governance

At its core, AI governance does four things. Skip any one of them and a gap opens up -policy without control is aspirational, control without visibility is blind, and visibility without proof cannot withstand an audit.

Policy - Define the rules, and who owns them. What's acceptable use, which use cases are approved, how each is classified by risk, and -critically -who is accountable for outcomes at each stage.
Control - Enforce the boundaries, proportionate to risk. Turn policy into guardrails, access decisions, and lifecycle gates that constrain systems at runtime -with tighter controls on higher-risk use cases.
Visibility - Observe behavior, including fairness and quality. Capture logs, metrics, traces, token consumption, tool calls, and runtime signals that reveal what AI is truly doing -not just latency and cost, but grounding, drift, and fairness signals.
Proof - Audit, prove, and improve. Produce the evidence to demonstrate compliance and transparency (explainability and traceability you can show regulators and customers), investigate incidents, and feed what you learn back into policy.

What AI governance covers: the domain map

A complete AI governance program spans nine core domains, with runtime enforcement operating as a cross-cutting layer across several of them. Use this taxonomy to map your own requirements to concrete controls:

Policy: Responsible AI policies, approval workflows, and guardrails.
Data governance: Classification, sensitivity labels, DLP, retention, and lineage.
Model governance: Validation, versioning, documentation, and change control.
Observability: Logs, metrics, traces, token usage, and runtime signals.
Evaluations: Quality, safety, grounding, drift, and agent task success.
Security: Threat detection, posture management, proactive AI red teaming, and prompt-injection resistance.
Identity & access: RBAC, least privilege, conditional access, and agent identities.
Audit & compliance: Evidence, eDiscovery, legal hold, and reporting.
Agent governance: Registry, lifecycle, policy enforcement, and fleet visibility.

Cross-cutting runtime enforcement: Runtime enforcement is not a separate governance program. It is the control layer that operationalizes policy, security, identity, data protection, observability, and agent governance while AI systems are running. It governs live interactions between users, agents, models, tools, APIs, MCP servers, A2A agent APIs, and enterprise systems through authentication, authorization, traffic controls, token governance, quotas, policy checks, and operational guardrails.

The Microsoft governance stack

Microsoft's approach is a composed platform rather than a single product — AI lifecycle, data governance, identity, security, observability, and audit working together:

1. Policy and the control plane: define and enforce

Governance starts with a place to define the rules and decide who owns them. This is where responsible AI and acceptable use policies are authored; where new use cases come in through intake, get classified by risk, and are approved or rejected; where you decide up front which decisions and agent actions require human sign-off versus running autonomously within guardrails; and where accountability is assigned across the deploy, update, and retire lifecycle. Enforcing those rules through runtime guardrails, security posture, agent-level controls, and API-level controls is the job of the sections that follow; the work here is defining what “acceptable” means before anything is switched on.

Services that support it:

Microsoft Foundry -Compliance workspace for authoring and housing responsible AI policies and use case approvals.
Azure Policy -Resource level governance to codify those rules as enforceable baselines.
Azure API Management (AI Gateway): Runtime enforcement for AI applications, models, agents, tools, and APIs, including authentication, authorization, token governance, quota management, and policy enforcement.

(Agent specific policy and lifecycle controls via Microsoft Agent 365 are covered in Section 6.)

Practical takeaway: establish a governance baseline before AI and agent adoption scales -including deciding up front where a human stays in the loop. Retrofitting controls onto a sprawling estate is far harder than starting from one.

2. Data governance and compliance: Purview at the center

Most AI risk is ultimately data risk: the wrong information reaching a model, leaving through a response, or being retained without a policy. Microsoft Purview is the primary governance engine for AI interactions and enterprise data controls, and it works in three moves:

Discover - Identify AI usage, sensitive prompts and responses, and overall risk posture with DSPM for AI.
Protect - Apply sensitivity labels, DLP, access boundaries, and retention policies.
Prove - Generate audit trails, eDiscovery evidence, and compliance reporting.

If the question is “how do we govern the data flowing through AI?” - Purview is the anchor.

3. Model governance, safety, and evaluations: quality at build time and runtime

Evaluations are what make governance measurable, and they belong at two points in the lifecycle:

Pre-deployment - Release gating on safety, grounding, quality, fairness, and task success before anything ships.
Post-deployment - Runtime scoring for drift, quality thresholds, and harmful-output alerts once a system is live.

Services that support it:

Microsoft Foundry evaluators: quality, safety, RAG grounding, and agent task metrics
Azure AI Content Safety: detects unsafe or harmful content in prompts and responses. This is also integrated in Microsoft Foundry as guardrails that protect against unsafe content and direct and indirect jailbreak prompt injection attacks.
ASSERT (open source): Microsoft's open-source, policy-driven evaluation framework, short for Adaptive Spec-driven Scoring for Evaluation and Regression Testing. ASSERT turns your specific requirements and organizational policies into targeted, safety-focused test cases rather than generic benchmarks, and runs across frameworks such as LangChain, CrewAI, LiteLLM, and OpenAI, so evaluation is not locked to any single stack. ASSERT also pairs with runtime controls to close the loop: measure a failure rate, apply a control (see Agent Control Specification in section 6), then re-run the same evaluation to prove the rate dropped. That turns "we mitigated it" into a measured before-and-after.

Evaluations are not separate from governance - they are what turn governance from a promise into measurable proof.

4. Observability and runtime monitoring: know what is actually happening

Once AI is in production, you need to see its behavior in real time. Capture four kinds of signal:

Logs: prompts, responses, and decisions
Metrics: latency, token usage, quality scores, request volume, and quota consumption
Traces: agent reasoning paths, tool calls, model calls, and inter-service dependencies
Alerts: policy violations, drift, harmful outputs, and abnormal traffic patterns

Microsoft Foundry provides purpose-built observability for AI applications and agents across three connected capabilities. Evaluation measures quality, safety, and reliability throughout development. Monitoring tracks deployed systems in real-world conditions. Tracing, built on OpenTelemetry standards, captures the execution flow of LLM calls, tool invocations, and agent decisions, so you can debug multi-step reasoning rather than guess at it, across frameworks including LangChain, LangGraph, the OpenAI Agents SDK, and the Microsoft Agent Framework. The Agent Monitoring Dashboard brings these signals together for production traffic: token usage, latency, run success rates, evaluation scores, and red-teaming results in one view, backed by Azure Monitor, Application Insights. From the same surface you can turn on continuous evaluation to score sampled live responses, scheduled evaluations to detect drift against benchmarks, scheduled red team scans to probe deployed agents for emerging risks, and alerts that fire when outputs fail quality thresholds or produce harmful content.

Critically, this coverage is not limited to agents built on Foundry. Agents running elsewhere can be onboarded through the AI Gateway and instrumented with OpenTelemetry semantic conventions to send telemetry to the same Application Insights instance, so one dashboard, one evaluation pipeline, and one alerting surface cover the whole estate.

The broader observability fabric

Foundry observability composes with the rest of the platform: Azure Monitor, Application Insights, Log Analytics, Purview Audit, and Microsoft Agent 365 dashboards. Azure API Management (AI Gateway) extends visibility further by capturing AI traffic patterns, token consumption, API usage, policy violations, and agent-to-tool interactions, providing runtime insight into AI workloads at the network boundary.

Practical takeaway: wire observability before launch, not after the first incident. Continuous evaluation on sampled production traffic is the cheapest early-warning system you can buy, and the trace data it rides on is the same evidence an auditor will ask for later.

5. Security and identity governance: protect the AI estate

AI expands the attack surface, so governance has to include security and identity across the following:

Threat protection -Prompt injection, jailbreak, data exfiltration, and unsafe actions -with Microsoft Defender for Cloud and Microsoft Defender XDR.
Proactive adversarial testing -Find failures before attackers do, with the AI Red Teaming Agent in Microsoft Foundry for automated agent probing and PyRIT, Microsoft’s open-source red-teaming framework.
Security posture -Misconfigurations, exposed endpoints, and risky resources -with Defender for Cloud and Azure Policy.
Identity & access -Who can use AI, and what data, apps, and agents it can reach -with Microsoft Entra ID and RBAC.
Runtime access enforcement: Who can call which model, tool, API, or MCP server, under which conditions, and with what level of throttling, inspection, and monitoring.
Secrets & network -Protecting keys and enforcing private access and network boundaries -with Azure Key Vault and Private Link.
Device management-Discover, monitor, and govern unmanaged AI agents and the endpoints they run on -with Microsoft Intune and Microsoft Defender for Endpoint (MDE).

Cross-cutting runtime enforcement

As AI systems increasingly rely on APIs, tools, MCP servers, enterprise applications, and agent-to-agent interactions, governance must extend beyond model build-time controls to the live interactions that connect those systems. Runtime enforcement helps operationalize the broader AI governance program by enforcing who can call what, under which conditions, at what scale, and with what level of monitoring.

Services that support it :

Azure API Management (AI Gateway): Centralized authentication, authorization, token controls, traffic shaping, quota enforcement, content safety policies, and policy governance for AI models, agents, tools, MCP servers, A2A agent APIs, and enterprise APIs.
Microsoft Entra ID: Identity and access control for applications, users, and agents.
Azure RBAC: Fine-grained authorization for AI resources and workloads.

The mindset shift: treat agents as non-human identities with their own permissions, lifecycle, runtime access and monitoring.

6. Agent governance: the emerging control plane

Agents are a new kind of enterprise actor, and governing them takes two complementary pieces: a control plane to manage the whole fleet, and a portable control standard to enforce safety inside each agent's workflow.

Microsoft Agent 365: govern the agent fleet

Agents introduce a new problem - sprawl. As teams build agents on different platforms, organizations need one way to see and govern all of them. Microsoft Agent 365, now generally available, is the control plane for AI agents: it gives each agent its own identity and manages agents with the admin tools you already use - Microsoft Entra, Defender, Purview, Intune, and the Microsoft 365 admin center. It works across agents built on Microsoft, open-source, and third-party platforms, through five core capabilities:

Registry - Discover and catalog every agent - including shadow agents - and quarantine the unsanctioned ones.
Access control - Give each agent a unique Microsoft Entra Agent ID, and enforce least-privilege, risk-based access with policy templates.
Visualization - Unified dashboards, telemetry, and alerts across the entire agent fleet.
Interoperability - Works with agents from Microsoft, open-source, and third-party frameworks, alongside your Microsoft 365 apps and data.
Security - Threat detection, data protection, and compliance for agents at scale, through Defender, Purview, and Entra.

Agent Control Specification (ACS): portable runtime controls

Governing agents also means enforcing controls consistently, no matter which framework an agent is built on. The Agent Control Specification (ACS) is an open industry standard, part of Microsoft's Agent Governance Toolkit, for placing deterministic safety and security controls at defined checkpoints in an agent's workflow. Just as the Model Context Protocol (MCP) standardized how agents connect to tools, and A2A standardized how agents communicate with each other, ACS aims to standardize how agents are governed: one portable control layer that any framework can adopt, with Microsoft providing reference implementations.

Control checkpoints: validation at defined intervention points across the agent's lifecycle, spanning input, model invocation, state, tool selection and execution, and output
Composable controls: deterministic checks (allow/deny rules, custom filters) alongside model-based checks (classifier endpoints, LLM judges), each placed exactly where it is needed
Portable and auditable: expressed as a declarative manifest with policy-as-code evaluation, so controls are versionable, auditable, and travel with the agent across any stack

Together they cover both altitudes of agent governance: Agent 365 governs the fleet from the outside, while ACS enforces safety inside each agent's workflow. Those checkpoints can also route to a human for approval before a high-impact action -the runtime side of human-in-the-loop -while Agent 365 defines which agents and actions require sign-off.

The governance service map

Use this as a quick reference for matching each governance area to the Microsoft services that support it. Runtime enforcement is shown as a cross-cutting capability because it operationalizes several domains rather than replacing them:

A practical path forward

You do not have to govern everything at once. A pragmatic sequence gets you to a defensible position quickly:

Start with scope: Inventory the AI apps, Copilots, agents, APIs, tools, MCP servers, and custom Microsoft Foundry workloads that exist today.
Classify risk: Determine what data is used, what actions AI can take, what systems it can reach, and which regulations apply.
Apply controls: Bring in Purview, Entra, Defender, Microsoft Foundry guardrails, Azure API Management AI Gateway policies, Agent 365 policies, and portable ACS checkpoints for agents.
Measure and prove: Add evaluations, including policy-driven ASSERT runs and adversarial testing with the AI Red Teaming Agent, plus observability, APIM runtime telemetry, audit evidence, and operational dashboards.

Governance works best when it follows the business rather than leading it. This journey view shows how the controls layer in progressively starting from a high-value business use case and adding only what each next step requires, so governance accelerates safe delivery instead of gating it. The path runs through four stages: align on the use case and its value, stand up a safe foundation of minimum viable controls, build and validate quality and safety, then operate and scale -monitoring, auditing, enforcing runtime controls and governing agents in production.

Microsoft gives you the building blocks to govern AI as a live operating system - not a static policy document. To go deeper, explore Microsoft Foundry, Microsoft Purview, Microsoft Defender for Cloud, Microsoft Entra, Azure API Management and Microsoft Agent 365 -plus the open-source ASSERT evaluation framework and the Agent Control Specification (ACS) for portable, framework-agnostic agent controls.

References

Policy & lifecycle

Microsoft Foundry — https://learn.microsoft.com/en-us/azure/foundry/what-is-foundry
Microsoft Foundry Compliance and Security- Manage compliance and security in Microsoft Foundry - Microsoft Foundry | Microsoft Learn
Azure Policy — https://learn.microsoft.com/en-us/azure/governance/policy/overview
Azure API Management AI Gateway - https://learn.microsoft.com/en-us/azure/api-management/genai-gateway-capabilities
Microsoft Agent 365 — https://learn.microsoft.com/en-us/microsoft-agent-365/overview

Data governance

Microsoft Purview — https://learn.microsoft.com/en-us/purview/purview
DSPM for AI (Data Security Posture Management) — https://learn.microsoft.com/en-us/purview/data-security-posture-management-learn-about
DLP (Data Loss Prevention) — https://learn.microsoft.com/en-us/purview/dlp-learn-about-dlp
Sensitivity labels — https://learn.microsoft.com/en-us/purview/sensitivity-labels

Model safety

Azure AI Content Safety — https://learn.microsoft.com/en-us/azure/ai-services/content-safety/overview
Prompt Shields — https://learn.microsoft.com/en-us/azure/ai-services/content-safety/concepts/jailbreak-detection

Evaluations

Azure AI Foundry evaluators (observability & evaluation) — https://learn.microsoft.com/en-us/azure/foundry/concepts/observability
ASSERT — official announcement (open-source agent evals; no standalone Learn page yet): https://devblogs.microsoft.com/foundry/build-2026-open-trust-stack-ai-agents/

Observability

Azure Monitor — https://learn.microsoft.com/en-us/azure/azure-monitor/fundamentals/overview
Application Insights — https://learn.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview
Log Analytics — https://learn.microsoft.com/en-us/azure/azure-monitor/logs/log-analytics-workspace-overview
Purview Audit — https://learn.microsoft.com/en-us/purview/audit-solutions-overview

Security

Microsoft Defender for Cloud — https://learn.microsoft.com/en-us/azure/defender-for-cloud/defender-for-cloud-introduction
Microsoft Defender XDR — https://learn.microsoft.com/en-us/defender-xdr/microsoft-365-defender
Microsoft Sentinel — https://learn.microsoft.com/en-us/azure/sentinel/sentinel-overview

Identity

Microsoft Entra ID — https://learn.microsoft.com/en-us/entra/identity/
Azure RBAC (role-based access control) — https://learn.microsoft.com/en-us/azure/role-based-access-control/overview
Conditional Access — https://learn.microsoft.com/en-us/entra/identity/conditional-access/overview

Audit & compliance

Purview Audit — https://learn.microsoft.com/en-us/purview/audit-solutions-overview
eDiscovery — https://learn.microsoft.com/en-us/purview/edisc
Compliance Manager — https://learn.microsoft.com/en-us/purview/compliance-manager

Agent governance

Microsoft Agent 365 — https://learn.microsoft.com/en-us/microsoft-agent-365/overview
Agent Control Specification (ACS) — https://microsoft.github.io/agent-governance-toolkit/packages/agent-control-specification/
Agent Governance Toolkit — https://microsoft.github.io/agent-governance-toolkit/
Purview for agents — https://learn.microsoft.com/en-us/purview/purview

Contributors:

This article is maintained by Microsoft. It was originally written by the following contributors.

Manasa Ramalinga | Senior Principal Cloud Solution Architect – US Customer Success 
Mehrnoosh Sameki | Principal PM Manager -AI Governance Product Team

The AI Agent Lifecycle: A Simple Guide

supriyas — Mon, 13 Jul 2026 00:52:49 GMT

The Bigger Picture

Building an AI agent is fundamentally different from building traditional software.

With a website or application, teams typically design, develop, test, and release. Once deployed, the focus shifts primarily to maintenance and feature enhancements. AI agents operate differently. They don't just execute predefined instructions they interpret information, reason, and make decisions.

In banking, those decisions can influence customer experiences, operational efficiency, compliance outcomes, and risk management. As a result, deploying an AI agent is not the finish line; it's the beginning of an ongoing process of learning, monitoring, and improvement.

The lifecycle of an enterprise AI agent reflects this reality.

Stage 1 —> Design: Decide What It Can and Cannot Do

Before writing a single line of code, answer three questions:

What is this agent allowed to do?
What must it never do?
Who is accountable when something goes wrong?

Simple example:

A loan agent is allowed to check credit scores, apply lending policy, and recommend a decision. It is never allowed to approve a loan above $50,000 without a human sign-off. The Head of Credit Risk is accountable.

Design produces one critical output: the risk classification. A low-risk FAQ bot and a high-risk loan decisioning agent need completely different levels of testing, guardrails, and oversight. Getting this wrong at design time is expensive to fix later.

Microsoft helps here with:

Tool	How It Helps
Microsoft Foundry Model Catalog	Browse and compare models and select the right one for the risk level before any code is written
Azure AI Content Safety	Review the built-in risk categories to understand what the platform can enforce, informing the guardrail boundary decisions
Microsoft Responsible AI Impact Assessment	Structured tooling to assess and document harms, likelihood, severity, and mitigations, producing a risk classification artefact.

Stage 2 —> Build: Put the Safety Controls In, Not On

Build the agent but more importantly, build the safety controls at the same time. Not afterwards.

The core stack:

Simple example:

The loan agent is built with: a PII redaction step (strip account numbers before they reach the model), a credit bureau tool, a policy lookup tool, an output checker (does the response cite a real policy?), and a Human in the loop gate (flag any decision over $25k for human review).

Establish the golden dataset during this phase: a representative set of real-world loan scenarios with SME validated expected outcomes. This serves as the ground truth for evaluating accuracy, consistency, and regression performance throughout the agent lifecycle.

Microsoft helps here with:

Tool	How It Helps
Microsoft Foundry Agent Service	The primary platform for building and hosting the agent, tool registration, memory, and orchestration in one place
Azure OpenAI Service	The LLM backbone with configurable built-in content filters on every inference call
Azure AI Content Safety	Input and output guardrails, content moderation and Prompt Shield for injection and jailbreak detection
Azure AI Language	PII detection and redaction across 100+ entity types before data reaches the model
Azure AI Search	The RAG pipeline retrieves verified policy documents to ground every agent response
Azure Functions (Premium)	Hosts custom guardrail logic (e.g. policy compliance checks) inside the bank's private network
Microsoft Foundry Tracing	Instruments every tool call and reasoning step, essential for evaluation and audit

Stage 3 —> Test: Find the Failures Before Customers Do

Testing occurs in three waves:

Automated testing evaluates the agent against the golden dataset, measuring accuracy, groundedness, and safety.
Human review brings in domain experts to assess decision quality, reasoning, and compliance.
Red teaming stress-tests the agent with adversarial prompts to uncover vulnerabilities and safety gaps.

The stage concludes with a quality gate, a formal sign-off that the agent meets the required standards. No sign-off, no deployment.

Simple example:

The loan agent achieves 98% accuracy against the golden dataset. A compliance officer reviews a sample of 50 decisions and confirms that the reasoning meets requirements. During red-team testing, a vulnerability is discovered: the agent can be manipulated through instructions embedded within a PDF. The issue is addressed and remediated before deployment.

Microsoft helps here with:

Tool	How It Helps
Microsoft Foundry Evaluation SDK	Runs the full golden dataset evaluation in parallel structured scores per row, side-by-side comparison between runs
Built-in Safety Evaluators	Out-of-the-box scoring for violence, hate, self-harm, sexual content, and indirect prompt injection
Built-in Quality Evaluators	Groundedness, relevance, coherence, and fluency no configuration needed
Agent Evaluators	TaskAdherence and ToolCallAccuracy checks the agent followed the right process, not just gave the right answer
Microsoft Foundry Versioned Datasets	Locks the golden dataset by version the same benchmark is used for every regression test

Stage 4 —> Deploy: Start Small, Expand Carefully

Do not flip a switch and send all traffic to the new agent. Start in shadow mode.

Shadow mode: Agent processes requests → responses NOT shown to customers Purpose: does it behave in production like it did in test? Pilot (5%): A small slice of real customers get agent responses Watch error rates for 2 weeks Full rollout: Expand only when quality metrics stay within thresholds

Everything must be live before the first customer interaction: monitoring dashboards, alerting, human review queues, and a fallback plan if the agent needs to be pulled.

Simple example:

The loan agent goes live in shadow mode for one week. No unexpected failures. Expands to 5% of applications. Error rate stays below 0.1% for two weeks. Full rollout approved.

Microsoft helps here with:

Tool	How It Helps
Microsoft Foundry Agent Services, Azure Kubernetes Services, Azure Container Apps	Hosts and auto-scales the agent runtime canary deployments enable the staged rollout without a full infrastructure team
Azure API Management	The API gateway enforces rate limits, authentication, and routing before any request reaches the agent
Azure Application Insights	Latency, volume, and error rate dashboards live from the first interaction
Azure Private Endpoints + Managed Identity	All traffic stays inside the bank's network no public endpoints, no passwords in code
Microsoft Foundry Deployment Management	Version-pins the model deployment enables instant rollback if the new version degrades

Stage 5 —> Operate: Watch Everything, Always

A deployed agent is not a finished product. It is a living system. Watch five things continuously:

What to Watch	Why
What's coming in	Are users trying to manipulate the agent?
How fast it responds	Is it meeting SLA?
Quality of outputs	Is it still giving correct answers?
Guardrail trigger rates	Are more things being blocked or slipping through?
Business outcomes	Are loan decisions still aligned with policy?

Sample a portion of live interactions and route them to human reviewers. When a reviewer corrects the agent, that correction is a training signal — collected, annotated, and fed back into the next iteration.

Simple example:

Three weeks after launch, monitoring shows the agent's policy compliance score has dropped from 98% to 94%. Human reviewers identify that a recent policy update was not reflected in the agent's RAG knowledge base. The team is alerted before any customers are affected.

Microsoft helps here with:

Tool	How It Helps
Microsoft Foundry Online Evaluation	Asynchronously samples live traffic and evaluate it, providing continuous quality monitoring without impacting response latency.
Azure AI Content Safety (runtime)	Prompt Shield enforces guardrails on every production interaction in real time
Azure Monitor + KQL	Provides dashboards and alerts across all monitoring signals, including latency, quality, guardrail compliance, and business outcomes.
Microsoft Foundry Tracing	Captures every production trace, including tool interactions and execution history, providing a complete audit trail for review and compliance
Traces to Dataset	Automatically converts production traces into versioned evaluation datasets, feeding seamlessly into the next optimization cycle.

Stage 6 —> Iterate: The Agent Is Never Finished

Every signal from production triggers a loop back into the lifecycle.

Signal	What Happens
Quality score drops	Loop back to Build and update the RAG index
New attack pattern detected	Loop back to Build and patch the guardrail, re-test
Human overrides spiking	Loop back to Design and rethink the HITL threshold
New regulation published	Loop back to Design and full cycle restarts

Simple example:

APRA publishes new guidance on AI in credit decisions. The loan agent must be updated to include a new mandatory disclosure in every decision output. The team loops back to Design, specifies the new requirement, updates the agent in Build, re-tests against an updated golden dataset, and redeploys within three weeks.

Microsoft helps here with:

Tool	How It Helps
Microsoft Foundry Fine-tuning	Fine-tunes the model using human-reviewed annotations, enabling the agent to improve with every feedback cycle.
Microsoft Foundry Dataset Versioning	Promotes newly annotated traces into the next version of the golden dataset, ensuring regression testing remains current.
Microsoft Foundry Experiment Tracking	Maps evaluation outcomes to specific prompt revisions, making it easy to identify the exact change that introduced a regression.
Azure Monitor Alerts	Automatically triggers when quality thresholds are breached, initiating the optimization cycle without requiring manual intervention.

Why the Loop Is the Most Important Part

Most teams focus on stages 1–4. The loop through stage 6 is what separates agents that stay safe from agents that drift into risk over time.

Regulators do not just ask "was it safe when you launched it?" They ask, "is it safe now and can you prove it has been improving?"

The iterate loop, supported by Microsoft Foundry's continuous evaluation and monitoring capabilities, is how you answer yes.

Summary

Design what the agent can and cannot do. Build the safety controls into the system at the same time as the agent itself. Microsoft Foundry Agent Service, Azure AI Content Safety, and Azure AI Search provide the core infrastructure. Test it in three waves before any customer sees it, using Microsoft Foundry's evaluation SDK and built-in evaluators. Deploy it carefully in stages, watching every metric through Azure Monitor and Application Insights. Once live, monitor it continuously using Microsoft Foundry's online evaluation. And when something changes a policy, a regulation, a performance drift loop back to the right stage and run the cycle again. The agent is never finished. The loop is the product.

Beyond the Canvas: The Azure Architecture Diagram Builder Becomes Agent-Ready

arturoqu — Tue, 14 Jul 2026 14:02:30 GMT

AZURE ARCHITECTURE BLOG · 8 MIN READ

Author: Arturo Quiroga, Senior Partner Solutions Architect — Microsoft

Two months ago I published From Prompt to Production: Building Azure Architecture Diagrams with AI, introducing the open-source Azure Architecture Diagram Builder. The response was humbling — thousands of you read it, tried the tool, and filed issues and feature requests. A follow-up on how the Well-Architected Framework scoring works went deep on validation.

You asked, and the tool grew. This post is about what’s new since May — and one change big enough to reframe the whole project: the Azure Architecture Diagram Builder is no longer just an app you click. It’s a partner you chat with, and a tool other agents can call.

TL;DR. Three arcs of new capability: (1) Architecture Chat turns diagram design into a multi-turn conversation over the live canvas; (2) Blueprint Diagrams produce hand-drawn, whiteboard-style deliverables alongside the formal topology; and (3) the app now exposes its capabilities as a Model Context Protocol (MCP) server, so AI agents can generate, validate, cost, and render Azure architectures programmatically. Plus a 13-model fleet, deployment guides grounded in Microsoft Learn, and July output enhancements.

What’s new at a glance

Capability	What it does
Architecture Chat	Refine a diagram by conversation — “add Front Door with WAF,” then“now make it zone-redundant.” Each turn reads the live canvas and auto-saves to history.
Blueprint Diagrams (BETA)	Hand-drawn, whiteboard-style renders with nested zones and numbered flow arrows. Topology, Blueprint, or Both.
A fleet of 14 models	Multi-provider roster — GPT-5.x, DeepSeek, Grok, Mistral, and Kimi — with side-by-side comparison to pick the right brain per task.
MCP server	The app is now a remote MCP server. Agents can list_services, validate_architecture, estimate_costs, generate_bicep and render_diagram with typed, structured outputs.
Microsoft Learn grounding	Deployment guides now cite live Microsoft Learn documentation.
Output enhancements (July 2026)	Cost badges, light/dark render themes, and metadata panels in rendered diagrams.

From clicking to conversing: Architecture Chat

The single most common request after the launch post was some version of “I love the first diagram, but I want to iterate without re-writing the whole prompt.” Regenerating from scratch every time you tweak a requirement is slow and loses context.

Architecture Chat solves this. It’s a conversational panel that sits alongside the canvas and treats your diagram as a living document. Each message is a turn in an ongoing design session:

“Add an Azure Front Door with WAF in front of the app tier.”
“Now make the data layer zone-redundant.”
“Swap the SQL Database for Cosmos DB and update the connections.”

Every turn reads the current state of the canvas — not the original prompt — so refinements compound naturally the way they would with a human architect at a whiteboard. The conversation auto-saves to history, so you can step back through the evolution of a design or branch from an earlier point.

Figure 1. Architecture Chat treats the diagram as a living document. Each message refines the current canvas — adding services, changing SKUs, or reorganizing groups — and the full exchange is saved to history.

The shift is subtle but important: architecture design stops being a one-shot prompt and becomes an iterative dialogue.

The whiteboard deliverable: Blueprint Diagrams (BETA)

Formal topology diagrams with official Azure icons are perfect for documentation and stakeholder decks. But early-stage design conversations often want something looser — the hand-drawn feel of a whiteboard sketch that communicates intent without implying finality.

Blueprint Diagrams generate exactly that: a whiteboard-style render with nested zones (subscription → VNet → subnet), numbered flow arrows, and a deliberately sketchy aesthetic. You choose the output mode:

Topology — the formal, icon-based diagram from the launch post
Blueprint — the hand-drawn whiteboard style
Both — generate the two side by side

Figure 2. The same architecture in two visual languages. Left: the formal, icon-based topology. Right: Blueprint mode — a whiteboard-style render with nested zones and numbered flow steps, plus a numbered legend explaining each hop. Use Blueprint for early design conversations and Topology for final documentation.

It’s the same underlying architecture — two visual languages for two different moments in the design lifecycle.

A fleet of 14 models: pick the right brain per task

The launch post shipped with multi-model support. That fleet has grown to 13 models across five providers, so you can match the model to the job — fast models for iteration, reasoning models for complex designs, code-optimized models for Bicep generation:

OpenAI GPT-5.x — GPT-5.1, GPT-5.2, GPT-5.6 Sol, Terra and Luna, GPT-5.4, GPT-5.4 Mini
DeepSeek — V3.2 Speciale, V4 Pro
xAI Grok — 4.1 Fast, 4.3
Mistral — Large 3
MoonshotAI Kimi — K2.5, K2.7 Code

The Compare Models feature runs the same prompt through any subset of these in parallel and ranks them on service count, token usage, latency, and cost — with Fastest / Cheapest / Most Thorough badges — so you can make an evidence-based choice rather than a guess.

Figure 3. Multi-model comparison across the full 13-model fleet. Top: the results grid ranks every model on service count, connections, token usage, latency, and cost, with Fastest / Cheapest / Most Thorough badges. Bottom: an optional AI Critique uses a critic model to rank the outputs and explain each model’s strengths and gaps.

Adding a model is now a small, well-understood change — a testament to how the multi-provider abstraction has matured since May.

The headline: the Diagram Builder is now an MCP server

Here’s the change that reframes the project. Everything above is about a person using a web app. But the same capabilities — generating a diagram, validating it against WAF, estimating its cost, producing Bicep — are exactly the things an AI agent needs when it reasons about Azure architecture.

So we exposed them. The Azure Architecture Diagram Builder now runs as a Model Context Protocol (MCP) server. Any MCP-capable agent can call its tools with typed inputs and structured outputs:

Tool	What the agent gets
`list_services`	The catalog of supported Azure services and categories
`validate_architecture`	A WAF assessment with pillar scores and findings
`estimate_costs`	Multi-region cost estimates from the Azure Retail Prices API
`generate_bicep`	Infrastructure-as-Code templates for the design
`render_diagram`	A rendered diagram (topology or blueprint) of the architecture

This means an agent can hold a conversation like “design a HIPAA-compliant platform, check it against the Well-Architected Framework, tell me the monthly cost in West Europe, and give me the Bicep” — and the Diagram Builder answers each part programmatically, returning structured data the agent can reason over and chain.

Figure 4. The Diagram Builder as an MCP server inside Microsoft Scout. Top: from a natural-language request, the agent calls the render_diagram tool with structured parameters (title, format, direction, theme, region) and saves the returned SVG to its workspace. Bottom: the rendered architecture — grouped zones, labeled flows, and cost badges — appears inline in the conversation, generated entirely through agent tool calls.

The tool that started as a canvas for humans is now also a building block for agents. That’s the arc: from an app you click, to a partner you chat with, to a tool other agents call.

Grounded in Microsoft Learn, and sharper output

Two smaller-but-meaningful improvements round out the release:

Microsoft Learn grounding. Deployment guides now search official Microsoft Learn documentation at generation time and cite it, so the guidance reflects current, authoritative practice rather than a model’s training snapshot.
Output enhancements (July 2026). Rendered diagrams now carry per-service cost badges, support light and dark render themes, and include metadata panels that summarize the architecture — service counts, regions, and estimated cost — directly on the image.

Highlights

Since the May launch, the Azure Architecture Diagram Builder has grown from a design tool into an agent-ready platform:

Conversational design: iterate on a diagram by chatting over the live canvas, with full history
Two visual languages: formal topology and hand-drawn Blueprint, from the same architecture
13 models, five providers: choose the right brain per task, with evidence-based comparison
Agent-ready: an MCP server exposing generation, validation, costing, and IaC as callable tools
Grounded guidance: deployment guides cite live Microsoft Learn documentation
Still open source: every capability above is available to inspect, extend, and contribute to

Try It Today

Live demo: https://aka.ms/diagram-builder
Source code: GitHub repository
Documentation: See the Getting Started Guide for setup, and the repository’s MCP server directory for agent integration.

If you read the first post and tried the tool — thank you. The features above exist because you told me what you needed. Keep the feedback coming via GitHub Issues.

Tags: artificial intelligence · application · apps & devops · well architected · infrastructure

Golden Paths Are a Product. Treat Them Like One.

KishoreKumarPattabiraman — Tue, 07 Jul 2026 18:26:04 GMT

Audience: Cloud architects, platform engineers, engineering leaders

In my earlier Cloud Native Platforms articles, I focused on how teams build, run, and evolve modern engineering foundations. This article focuses on what keeps those capabilities useful after launch.

Most teams do not lose adoption because the first release was poor. They lose adoption because the path was launched like a project and left to survive like a product.

The pattern is familiar. A capability ships with good intent, reasonable documentation, and early enthusiasm. A few months later, teams start bypassing it, exceptions multiply, support load rises, and the platform team quietly becomes a ticket queue.

If a capability is expected to be adopted, trusted, improved, and measured over time, it needs product discipline from the start.

How this connects to forward-deployed engineering

Forward-deployed engineering (FDE) is a delivery model where an engineer embeds directly with a customer or team, learns the real operating context, and builds a working solution in the field. The model was popularized by Palantir, whose engineers embed with customers. The same pattern now appears inside large engineering organizations, where platform engineers embed with internal product teams. Golden paths are a logical next step. FDE is the front of the funnel, where bespoke solutions get built one team at a time. Between the two sits a graduation decision. When the same pattern shows up across several independent teams with the same friction, it is ready to become a safe default. A golden path is the back end. The part that repeats across teams becomes that default, so you stop solving the same problem by hand every time. FDE does not stop once a path exists. New teams, new domains, and new edges keep producing forward-deployed work in parallel. The loop closes through what the path cannot absorb. Where teams bypass the path or hit a case it does not cover, those gaps become the next forward-deployed engagement, and the cycle repeats.

Figure 1. Forward-deployed work is the continuous front of the funnel. Golden paths are how the repeatable parts scale.

Quick context before we begin

Golden path is the easiest safe default way to deliver a repeated engineering outcome.

Net Promoter Score (NPS) measures how likely someone is to recommend something to others. In this context, internal NPS means how likely one internal engineering team is to recommend a capability to another internal team. It is useful as a directional trust signal, not as a standalone operating metric.

Service Level Objective (SLO) is a quantitative target for a service characteristic such as availability, latency, or error rate.

Service Level Agreement (SLA) is a formal service commitment, often external or contract-backed. Many internal engineering capabilities will use SLOs without using formal SLAs.

Deprecation lane is the planned and time-bound way to retire an older pattern without causing unnecessary disruption.

How to identify a true golden path

Not every good practice is a golden path. A path qualifies when most of the following are true:

Repeated use case across teams
High cost of inconsistency
High cognitive load from scratch
A safe default can be defined
Measurable outcome exists

If those signals are absent, teams are usually looking at a local implementation pattern, not a golden path.

That distinction matters. Internal capability teams need to invest their product energy where standardization meaningfully reduces risk, cost, delay, or complexity.

Two transformation threads that make this real

To keep this practical, this article refers back to two transformation threads throughout the discussion.

The first was an AKS migration effort. The existing hosting model worked until scale, operational control, and predictable performance started pulling in opposite directions. The move to AKS was not treated as a one-time infrastructure shift. Workloads were phased over time, traffic was controlled through a single API entry point, and teams were trained to operate in a container-first model before the migration accelerated. The outcome was not only better control. It created a stronger foundation for observability, scaling, policy-driven operations, and future platform standardization.

The second was a security modernization effort around a more unified authentication and authorization model. Identity logic had grown across services over time. Permissions were fragmented. Operational inconsistency was becoming risk. The move toward a more centralized approach depended as much on rollout safety as on design quality. Feature flags, parallel writes during transition, read-repair, and rollback paths made that shift survivable on a live platform while preserving performance and user continuity.

These two threads matter because they show the same principle in different settings. A capability becomes durable when teams combine the right architecture with the right product mindset, operating model, and feedback loop.

The golden path operating canvas

A useful way to think about a golden path is through five decision areas:

Product framing
Operating model
Engineering guardrails
Adoption and change strategy
Measurement and feedback loop

Figure 2. The measurement and feedback area is what keeps the other four honest over time.

Individually, each area matters. Together, they determine whether a capability becomes the default, stays trusted, and improves over time.

1. Product framing

A common first mistake is treating a capability as an output instead of a product.

Product framing starts with four simple questions:

Who is the consumer
What repeated problem are we solving
What safe default are we standardizing
What measurable outcome proves value

This matters equally for a platform capability and for a shared service. An internal deployment template, a security validation path, a standardized API onboarding flow, or the opinionated consumption path around a reusable identity service can all become golden paths if they remove repeated friction and create better defaults.

In practice

The path gets clearer when teams name the repeated decision they want to remove. In the AKS migration, the real question was not "should we modernize hosting." It was "what is the safest default for workloads that now need predictable performance and stronger operational control." That framing forces sharper scoping. It also prevents vague launch language from hiding the actual default teams are expected to adopt.

If teams are defining a new golden path, product framing is where they start.

If teams already have one and see drift, they should return here first and ask whether the original consumer problem is still sharp, still current, and still visible.

2. Operating model

A golden path does not stay healthy because documentation exists. It stays healthy because ownership exists.

The operating model answers questions like these:

Who owns the path
Who contributes to it
Who makes change decisions
What review cadence exists
How support expectations are communicated
How exceptions are granted and retired

Without this, teams end up with a queue, not a product.

This is where many internal capabilities weaken over time. Platform teams become catch-all owners. Security teams review too late. Consumer teams give feedback, but no one is clearly accountable for converting that feedback into decisions.

Team Topologies describes a similar problem through cognitive load and enabling-team boundaries. When ownership and interaction modes are unclear, friction rises and good patterns fail to scale (Team Topologies).

In practice

The pattern that works is simple: one accountable owner, one recurring decision forum, and one visible exception model. The exception model matters more than teams expect. A path becomes brittle when consumers can go off-path with no review, and it becomes unusable when every exception requires escalation with no expiry. Temporary exceptions need an owner, a review date, and a retirement path.

If teams are starting new, they should define one accountable owner and one recurring review rhythm.

If teams already see drift, they should look for ambiguity in decision rights, ownership, or support expectations.

3. Engineering guardrails

A golden path is only valuable if it makes the safe path the easy path.

Engineering guardrails are the non-negotiables built into the path itself. These typically include:

Security posture
Reliability targets
Cost boundaries
Operational readiness
Observability and telemetry standards
Supportability expectations
Versioning and compatibility expectations

This is one place where product, service, and platform thinking often get separated unnecessarily. They should not be. A shared identity service needs guardrails just as much as a platform deployment path does. A platform path without operational clarity is as incomplete as a service without security consistency.

The unified auth work shows why. Centralizing identity enforcement was not just an architectural choice. It was a way to make security control more consistent, measurable, and operable. Feature-flag strategy, rollback readiness, and performance targets were all part of the guardrail model, not postscript concerns.

Microsoft's Well-Architected guidance reinforces the same principle. Security, reliability, and cost are not side dimensions. They are core decision lenses that shape how capabilities are designed and operated (Microsoft Well-Architected Framework).

In practice

The unified auth work makes this concrete. The technical design mattered, but the real strength came from turning security rules into operating defaults: central enforcement, predictable rollout controls, and a clear rollback posture. The same principle applies elsewhere. If teams do not know the compatibility expectations, cost envelope, or minimum operational standard up front, they invent their own.

If teams are defining a new path, they should decide the minimum guardrails before expansion.

If an existing path is drifting, teams should look for where consumers are stepping outside the guardrails or where the guardrails were never made explicit.

4. Adoption and change strategy

A golden path does not become real because it is published. It becomes real because teams can adopt it safely and progressively.

That means rollout strategy must be designed as carefully as the solution itself. That includes:

Migration sequencing
Consumer onboarding
Enablement and training
Shadow testing where needed
Rollback readiness
Transition governance
Minimum viable self-service

This was one of the clearest lessons from the AKS migration. The technical work mattered, but the people ramp-up mattered just as much. Moving a large engineering group from one operational model to another required structured enablement, practical migration guidance, and a way to build confidence in steps rather than all at once.

This mirrors the intent behind Microsoft's Cloud Adoption Framework. Large-scale change lands better when teams combine technical design with lifecycle planning, capability enablement, and operating model clarity (Microsoft Cloud Adoption Framework).

In practice

The AKS migration worked because adoption was designed, not assumed. Teams had a phased route in, a safe path back, and enough enablement to operate the new model with confidence. Just as important, the path moved toward self-service instead of permanent dependency on the central team. If a so-called path still requires repeated manual intervention for normal use, it is closer to a managed service queue than a true default.

If teams are building a new path, they should design adoption as a first-class workstream.

If teams already have a path that is drifting, they should treat re-adoption as a change program, not a documentation refresh.

5. Measurement and the feedback loop

If feedback does not change decisions, feedback collection becomes theater.

A golden path needs a small, meaningful set of signals that tells teams whether it is healthy. That usually includes some mix of:

Adoption depth
Time to first success
On-path versus off-path incidents
Cost efficiency trend
Reliability trend against SLOs
Internal NPS
Deprecation progress

The metric set does not need to be large. It needs to be actionable.

Internal NPS is especially useful when paired with qualitative input. A team may still be using a capability while quietly warning other teams away from it. That is an early drift signal. It is still only one signal. Mandatory adoption, small sample sizes, and local politics can all distort it.

DORA research reinforces the broader point that improvement depends on disciplined measurement and learning loops, not on collecting more indicators than teams can act on (DORA research).

In practice

The most useful feedback loop is visible. Teams should be able to see what was heard, what changed, and what did not make the cut. That is what turns trust into an operating behavior instead of a sentiment score. Without that trace, even good metrics become noise.

If teams are launching a new path, they should define baseline metrics before expansion.

If teams already have a path and see drift, they should run a focused diagnostic on adoption, incidents, bypass behavior, and team sentiment before redesigning anything.

The mindset shift that makes this work

Project thinking asks whether the capability shipped.

Product thinking asks whether it is being adopted, trusted, improved, and measured over time.

That shift changes how teams define success, stage rollout, involve consumers, and respond to drift. It applies to internal platforms, shared services, reusable components, and enablement models. The operating details are not identical across all of them, but the product discipline is still useful wherever teams expect repeat consumers and recurring outcomes.

The CNCF Platforms white paper captures a related idea through paved paths and shared internal capabilities. The technical foundation matters, but the sustained value comes from how teams shape adoption, ownership, and evolution around it (CNCF Platforms White Paper).

That is the broader point of this article. Golden path thinking is not just for platform teams. It is a repeatability and trust model for internal engineering capability design.

Signals your golden path is drifting

Adoption stalls or declines
Side paths become normal
Support load rises while roadmap clarity drops
Consumers cannot name the owner
Breaking changes surprise teams
Feedback is collected with no decision trail
Teams still use the path, but would not recommend it

These are usually not random symptoms. They point back to weak framing, weak ownership, weak guardrails, weak adoption design, or a weak feedback loop.

Immediate next steps

If you are defining a new golden path

Start where you already embed. Your forward-deployed and high-touch engagements are the best source of golden path candidates.

Pick one repeated, high-friction workflow with high risk if implemented inconsistently.
Define the consumer, safe default, and one measurable success outcome.
Set minimum guardrails for security, reliability, observability, cost, compatibility, and rollback.
Create a 30-day pilot with a named owner, a review cadence, and a visible exception path.
Capture internal NPS plus qualitative feedback, then publish the first improvement decision.

If you already have a golden path and see drift

Identify the top drift signals, such as bypass behavior, adoption drop, or trust erosion.
Diagnose which area is weak: framing, ownership, guardrails, adoption, or feedback loop.
Run a focused 30-day correction cycle rather than a broad redesign.
Reset one decision forum, one owner, and one exception model before changing anything larger.
Publish what changed, then re-measure adoption, sentiment, and operational metrics after one cycle.

Key takeaways

Golden paths reduce repeated risk and effort
Product discipline keeps them useful
Product discipline applies beyond platforms
Safe defaults need explicit guardrails
Trust grows when feedback changes decisions
Golden paths are how the repeatable parts of forward-deployed work scale

What this means

Good engineering capabilities do not fail only because the technology is weak. They often fail because the operating model around them is too thin.

That is why treating a golden path like a product matters. It turns a useful pattern into a dependable default.

It also creates a stronger bridge back to the earlier Cloud Native Platforms Run article and Evolve article. Building, running, and evolving technical foundations is one part of the problem. Sustaining trust and adoption around those foundations is the next one.

The real shift is this: a golden path is not complete when it launches. It is complete when teams still trust it after the first wave of exceptions, change pressure, and scale.

Want to discuss? Drop a comment with patterns you have seen in your environment. I read every reply.

Optimizing GitHub Copilot Cost in the Usage-Based Billing Era

gauravbhardwaj — Tue, 07 Jul 2026 15:43:49 GMT

GitHub Copilot has become a core part of the modern developer workflow. We use it to complete code, explain unfamiliar repositories, write tests, refactor legacy applications, review pull requests, generate documentation, and automate repetitive engineering work.

But as GitHub Copilot moves further into usage-based billing, teams and users are asking a very practical question:

How do we keep getting value from GitHub Copilot without letting costs become unpredictable?

The answer is not to blindly slash GitHub Copilot usage—that would defeat the purpose of adopting AI-assisted development in the first place. The better answer is to use GitHub Copilot more intentionally.

Usage-based billing shifts our mindset from:

“Can I use GitHub Copilot?” to: “Am I using the right GitHub Copilot capability, with the right model, the right context, and the right level of automation for this specific task?”

This guide outlines the major ways developers and organizations can optimize GitHub Copilot costs while continuing to accelerate productivity.

For a strategic look at how organizations are scaling these financial practices across modern infrastructure, read the playbook Token Economics: The New FinOps for Agentic AI | Microsoft Community Hub

Understanding the New Cost Model

Under usage-based billing, GitHub Copilot costs are driven primarily by two factors:

The model used (e.g., frontier vs. lightweight models)
The number of tokens consumed

A token is a small unit of text processed by the AI. This includes what you send to the model (input), what the model generates back (output), and, in some cases, cached context that the model reuses.

This means a short question using a lightweight model consumes very little. Conversely, a long, multi-file agentic session using a frontier model can consume significantly more. GitHub Copilot cost is no longer just about how many people have seats—it is about how those seats are being used.

The Biggest Cost Drivers:

Using expensive models for simple tasks
Overly large context windows
Long chat sessions with repeated context
Agentic workflows that inspect too many files
Enabling tools when they aren't needed
Dumping large terminal logs or full repository scans into the chat
A lack of active budgets and usage monitoring

The good news? Most of these are completely controllable.

1. Set Spending Guardrails First

The first cost-reduction step isn't technical—it’s financial governance. Before teams start heavy usage, set spending caps and budget controls. This is critical for organizations where many developers share a pool of AI credits.

Recommended Actions:

Set a monthly budget and enable hard-stop controls where available.
Segment limits: Create separate limits for standard users and power users.
Track granularly: Monitor consumption by user, team, organization, or cost center.
Review early: Closely analyze the first few weeks of usage and adjust limits based on real consumption patterns.

This prevents a single heavy user, an runaway automated agent session, or an experimental workflow from consuming a massive chunk of shared capacity early in the billing cycle.

For enterprises, avoid a one-size-fits-all budget. A junior developer asking occasional chat questions does not need the same budget as a platform engineer running massive migration tasks across multiple repositories.

A Smarter Budgeting Model:

Standard developer budget
Power user budget
Pilot team budget
Agentic workflow budget
Innovation/experimentation budget

This gives leaders control without blocking productivity, allowing organizations to analyze and adjust budgets frequently.

2. Use the Right Model for the Right Task

Not every task requires the most powerful model. This is one of the biggest mindset shifts in usage-based billing. Developers often default to the strongest model because it feels "safer," but a smaller, cheaper model is often more than enough for daily tasks.

Use Lightweight / Lower-Cost Models For:	Use Stronger / Premium Models For:
Simple code explanations	Architecture & system design decisions
Documentation & comment updates	Complex debugging & multi-file reasoning
Small unit test generation	Security-sensitive reviews
Basic debugging & syntax help	Legacy modernization & code translation
Regex help & simple refactoring	Performance tuning & optimization
Translating code from one style to another	Ambiguous, cross-service production issues
Summarizing small, single files	Large pull request reviews

Team Golden Rule:

Start with the cheapest model that can do the task well. Move to a stronger model only when the task requires deeper reasoning. This keeps high-cost models reserved for where they provide the highest value.

3. Lean More on Code Completions

Inline code completions and next edit suggestions are still some of the most cost-effective GitHub Copilot experiences available. For paid GitHub Copilot plans, these inline experiences are not billed against your AI credits. Developers should aggressively lean on completions when they already know what they want to build.

Ideal Completion Scenarios:

Writing a function after creating the signature
Completing repetitive boilerplate code
Filling in obvious implementation logic or adding similar methods
Writing simple, predictable unit tests
Completing configuration files

The Rule of Thumb: Use chat when you need reasoning. Use completions when you need acceleration.

❌ Bad Habit: Opening a new chat window for every small function.
👉 Better Habit: Write the function name, a descriptive comment, or the first few lines, and let GitHub Copilot fill in the implementation inline. Open chat only if the completion misses the mark or the logic requires deeper discussion.

4. Use Tools Only When Needed

Agents are incredibly powerful because they can interact with tools: reading files, searching repositories, editing code, running commands, inspecting test failures, or connecting to external systems via Model Context Protocol (MCP) servers.

However, tools heavily amplify costs. Every tool call adds more context, more model invocations, and more output back into the token loop. If an agent reads too many files or dumps raw logs into chat, token counts skyrocket.

The goal is not to ban tools, but to scope them correctly:

Don't enable everything: Avoid activating every tool for every agent. Use only what is required.
Specialized agents: Create custom agents with specialized tool access (e.g., a documentation agent needs read/search tools, but likely doesn't need terminal or cloud database access).
Avoid full scans: Ask Copilot to inspect specific files or folders rather than scanning the entire repository.
Summarize logs: Ask Copilot to summarize terminal outputs or run the narrowest relevant unit test instead of dumping pages of raw test logs.

A Cost-Aware Prompting Example:

"Use tools only if needed. Start by reading the files I mention. Do not scan the entire repository. If tests are needed, run only the relevant unit tests and summarize the failure instead of pasting full logs."

5. Narrow the Context Window

Context is useful only when it is relevant. A common cost driver is issuing broad prompts that pull massive chunks of irrelevant code into the session.

❌ Broad Prompt: "Analyze this repo and fix the bug." (Forces the agent to scan, search, and load unnecessary files).
👉 Targeted Prompt: "The bug appears to be in /src/auth/tokenValidator.ts and /src/auth/sessionStore.ts. Review only these files and their related tests. Propose the smallest safe fix."

Practical Ways to Reduce Context:

Explicitly mention exact files, folders, error messages, or failing tests.
Instruct the model on what not to change.
Explicitly exclude generated files, vendor folders, build artifacts, and lock files.
Clear your sessions: Staying in one massive chat session all day causes historical context to stack up. Start a fresh session when your topic/intent changes, when frequent compactions occur, or when different tools are required.
Pro tip: If you are ending a long session but need to carry forward the conclusion, ask Copilot to generate a quick Markdown summary of the session to paste as the starting context of your new, clean session.

6. Use Plan-First Prompts for Complex Work

For large-scale tasks, do not let GitHub Copilot immediately start editing files in agent mode. Always ask it to plan first.

This is crucial for large refactoring, multi-file changes, legacy modernization, dependency upgrades, security fixes, and performance tuning.

A Plan-First Prompting Pattern:

"Before editing files, create a short plan. Identify the files you need to inspect, the likely root cause, the risk level, and the smallest validation test. Do not make changes until the plan is clear."

Why this works:

Prevents wasted work: It ensures the agent doesn't modify the wrong files or run unnecessary commands.
Creates a cost checkpoint: If the plan looks too broad or incorrect, you can narrow the scope before GitHub Copilot kicks off an expensive, multi-step agentic loop.

7. Batch Related Questions in One Session

While giant, day-long sessions are bad for context bloat, opening a brand-new chat for every tiny, consecutive question introduces unnecessary context reload overhead. Finding the right balance is key.

❌ Less Efficient (Fragmented):
- Chat 1: "What does this function do?"
- Chat 2: "Can you write tests for it?"
- Chat 3: "Can you check it for security issues?"
👉 More Efficient (Batched):
- Single Chat: "Review this function for correctness, security, performance, and test coverage. Give me the top issues first, then suggest the smallest safe improvement."

The Balance: Batch highly related work in a single conversation, but hit the "New Chat" button the moment you pivot to an entirely new topic.

8. Keep Custom Instructions Short

Custom instructions (.github/copilot-instructions.md) and AGENT.md files are powerful tools for enforcing team standards, but they are appended as base tokens to every single chat interaction. If they are bloated, they act as a hidden tax on every prompt.

✅ Keep it Concise & Rules-Based:	❌ Avoid Attaching to Every Prompt:
"Prefer small, testable changes."	Massive architecture blueprints
"Do not modify public APIs without calling it out."	Full enterprise coding standards documents
"Use our standard logging pattern."	Entire infrastructure runbooks
"When writing tests, follow the existing test style."	Pages of multi-shot coding examples

The Strategy: Use global/repository instructions only for non-negotiable, high-level rules. For specific workflows, use localized prompt files, specialized skills, or custom agents.

9. Use Skills and Specialized Agents for Repeatable Work

If your team frequently asks Copilot to perform identical workflows (e.g., API reviews, PR summarization, Accessibility audits, Terraform checks), turn them into reusable skills, prompt files, or custom agents.

This ensures prompt formatting remains minimal, highly consistent, and optimized for low token usage. Furthermore, if you discover an excellent workflow during an active chat session, ask Copilot to turn that knowledge into a SKILL file. This prevents future sessions from having to waste tokens "re-learning" the process.

10. Avoid Unnecessary Agentic Mode

Agent mode is incredibly capable, but it shouldn't be the default default mode for standard queries.

🛠️ Use Agent Mode For: Multi-file bug fixes, generating and validating complex test suites, cross-file refactoring, reproducing test failures, or implementing full PR tasks.
💬 Stick to Normal Chat/Completions For: Simple syntax lookups, single-file explanations, writing minor snippets, basic documentation rewrites, or simple formatting tweaks.

The Rule: If a task doesn't explicitly require autonomous file editing, tool execution, or terminal commands, stick to standard chat or inline completions.

11. Monitor Usage Weekly

Optimization without measurement is just guesswork. During your organization's transition into usage-based billing, establish a weekly review cadence to look at your telemetry data.

What to Look For:

Which models are drawing the most volume?
Are premium frontier models being used for simple tasks?
Which teams or workflows are spiking above their allocated credit budgets?
Is tool usage or long context history driving up the average cost per session?

Use these insights to refine default model guidance, create user-level budget overrides for true power users, update prompt templates, and host short, continuous training sessions. The goal is never to shame high usage—high usage is fantastic if it yields high-value code. The goal is to eliminate systemic token waste.

12. Create a Simple Team Policy

To make this stick, give your developers a lightweight checklist they can actually memorize. Here is a great blueprint for an internal team policy:

Completions First: Use inline completions for normal code acceleration.
Cheaper Models First: Default to lightweight models; escalate to frontier models only for deep reasoning.
Scope Wisely: Mention specific files/folders; avoid blind repository scans.
Plan First: Ask for an architectural plan before allowing agents to execute large edits.
Tool Hygiene: Keep tools and custom instructions tightly scoped and concise.
Review Cadence: Check usage metrics weekly and adjust budgets monthly.

Reusable, Cost-Aware Prompt Templates

Copy and paste these templates into your daily workflows to keep token counts down:

For Debugging: "Analyze this error using only the files I mention. Do not scan the whole repository. First explain the likely root cause, then suggest the smallest fix."
For Agent Mode: "Use tools only if needed. Before editing, create a short plan and list the files you need to inspect. Run only the narrowest relevant test."
For Code Review: "Review this pull request for correctness, security, and maintainability. Focus on high-impact issues only. Do not rewrite the code unless necessary."
For Testing: "Generate unit tests for this function using the existing test style. Do not modify production code. Keep the test scope narrow."
For Refactoring: "Refactor this file only. Preserve behavior. Do not change public APIs. Explain the risk before making edits."
For Large Repositories: "Do not analyze the entire repository. Start with /src/payment and /tests/payment. Ask before expanding scope."

Summary: What to Avoid

To keep budgets predictable, coach your teams away from these common anti-patterns:

Setting an expensive frontier model as the default for every single interaction.
Running full agent loops for questions that a simple chat could answer.
Allowing agents to freely scan entire codebases without folder constraints.
Enabling every single available MCP tool by default.
Dumping massive, raw terminal outputs or log dumps directly into the prompt.
Carrying massive, multi-hour chat histories instead of opening fresh sessions.
Disregarding your usage dashboards until the invoice arrives.

The Bigger Picture: Cost Optimization is Workflow Optimization

Optimizing Copilot costs isn't about using AI less—it’s about using AI better. Smaller context windows, tighter tool scoping, cleaner prompts, and proper model selection don't just lower the bill; they make the model significantly more accurate. When you overwhelm a model with irrelevant files and messy logs, you introduce noise that degrades the quality of the output.

Cost optimization is an engineering maturity practice, not a restriction. When managed through a smart operating model, GitHub Copilot will continue to securely accelerate our delivery, eliminate boilerplate toil, and maximize our focus on the work that truly matters.

References:

Token Economics: The New FinOps for Agentic AI | Microsoft Community Hub

Contributors:

This article is maintained by Microsoft. It was originally written by the following contributors.

Gaurav Bhardwaj | Senior Cloud Solution Architect
Dustin Ellis | Senior Cloud Solution Architect

Revolutionizing Document Intelligence: Scaling Construction Industries with AI-Driven Extraction

gauravbhardwaj — Thu, 18 Jun 2026 20:52:34 GMT

Introduction

Generative AI (GenAI) is poised to transform the construction industry by addressing chronic challenges such as low productivity, cost overruns, schedule delays, and labor shortages. By automating the analysis of drawings, specifications, contracts, and project documentation, GenAI can reduce manual effort, accelerate decision-making, and improve coordination across architects, engineers, contractors, and suppliers. Industry studies indicate that AI-powered workflows can increase productivity by 20–40% in planning, engineering, and administrative functions while reducing costly rework and errors. The result is faster project delivery, improved resource utilization, lower costs, and more predictable project outcomes.

A major opportunity for GenAI in construction lies in its ability to unlock the vast amount of information trapped within AutoCAD drawings, architectural plans, BIM models, specifications, and engineering documents. Today, project teams spend countless hours manually reviewing drawings, performing quantity takeoffs, identifying dependencies, and translating design intent into actionable work packages for downstream trades. GenAI can automate this process by extracting and interpreting dimensions, materials, quantities, assemblies, and building components directly from design artifacts, then intelligently distributing that information to foundation, framing, roofing, insulation, MEP, and finish teams. This creates a digital thread from design through execution, eliminating manual handoffs, reducing human error, and ensuring every stakeholder works from a single source of truth. The impact extends beyond productivity gains—GenAI enables more accurate material forecasting, streamlined procurement, reduced waste, faster response to design changes, fewer change orders, and greater confidence that the architect's vision is executed precisely in the field. In an industry where margins are tight and inefficiencies are costly, GenAI has the potential to fundamentally redefine how construction projects are planned, coordinated, and delivered.

This article specifically demonstrates how organizations can leverage Azure AI services—including Azure Content Understanding, Azure foundry, Azure Blob Storage, Azure Open AI—to extract, understand, and operationalize information from construction drawings and project documentation. The solution illustrates how Azure's AI platform can transform unstructured design artifacts into actionable intelligence that improves productivity, reduces risk, accelerates procurement, and enables more efficient execution across the entire construction lifecycle.

This transformation is now achievable through a hybrid AI architecture. By combining structured layout understanding models with Generative AI reasoning capabilities, organizations can build highly scalable, intelligent extraction systems that meet the rigorous safety and compliance standards of the construction sector.

The Evolution from GenAI Approach to Deterministic Precision

Starting with a Generative AI–driven approach to extract structured fields from documents is a fundamentally more effective initial strategy. It accelerates early-stage extraction without requiring large, labeled datasets, while simultaneously enabling structured data collection needed to train deterministic models—which typically require thousands of annotated samples.

This approach delivers immediate value by rapidly identifying relevant data patterns in documents and uncovering key factors that influence extraction accuracy, such as document quality, layout complexity, and multi-section ambiguity. At the same time, it naturally builds the dataset necessary to transition toward a more scalable and repeatable solution.

However, while powerful for contextual reasoning across document sections, Generative AI is inherently probabilistic and sensitive to input variability. For enterprise-grade reliability, precision, and repeatable structured document extraction, a complementary approach is required.

The optimal solution is a hybrid model that combines the strengths of both:

Azure Content Understanding provides precise, consistent field extraction with per-field confidence scores at scale.

Azure OpenAI GPT-5.2 (generative) adds contextual reasoning, validates ambiguous fields, fills extraction gaps, and interprets complex multi-section relationships.

AI Agent (bounded triage) handles exception cases with structured CORRECT/ACCEPT/ESCALATE decisions before human escalation. Together, they form a superior system—delivering higher accuracy, reduced ambiguity, bounded AI cost, and stronger auditability in complex real-world conditions.

Note : AI cannot compensate for inconsistent input data. Standardized document schemas and operational discipline remain prerequisites for reliable automation.

Solution Components and Architecture

The solution follows a modular, event-driven architecture that combines deterministic document understanding and Generative AI to enable scalable, intelligent extraction workflows. At a high level, documents are ingested, deduplicated, processed through Azure Content Understanding for primary extraction, enhanced with GPT-5.2 for gap-fill verification, validated against business rules, and routed through a confidence-based decision system before persistence. The code repository for the solution can be found here

Conceptual Architecture

Azure Architecture: -

The pipeline execution follows this flow: a document is uploaded to Azure Blob Storage, triggering the orchestrator. The pipeline checks for duplicates via SHA-256 hash against Cosmos DB. New documents are submitted to Azure Content Understanding, which returns structured fields with per-field confidence scores. The AI Schema Mapper then identifies gaps—fields that are missing or have confidence below 0.70—and sends only those to GPT-4.1 for verification. Results are normalized, validated against cross-field business rules, and routed based on aggregate confidence.

Throughout the pipeline, built-in feedback loops—quality filtering, validation checks, and confidence gates—ensure that only high-confidence results are persisted automatically, enabling a reliable and production-ready extraction system.

Azure Blob Storage — Primary storage for source PDFs and extraction artifacts. Standard_LRS, Hot tier, HTTPS-only with SAS-secured access for Content Understanding.
Azure Content Understanding — Primary deterministic extractor with custom analyzer supporting 100+ configurable fields. Returns per-field confidence scores (0.0–1.0) plus raw markdown text. Non-LLM, repeatable, and auditable.
Azure AI Foundry / OpenAI (GPT-5.2) — Bounded gap-fill verifier invoked only for missing or low-confidence fields (typically 10–20% of total). Temperature 0.0, JSON response format enforced, schema-aware prompting with domain rules.
Azure Cosmos DB (Serverless)— Document persistence with SHA-256 deduplication, version increment on re-processing, and partition-by-document-type for efficient querying. Pay-per-request scales from zero.
Azure Service Bus (Basic) — Event-driven queue integration with `document-processing` and `human-review` queues for processing triggers and escalation routing.
Application Insights + OpenTelemetry — End-to-end observability with per-stage telemetry events, custom metrics (fill_rate, record_confidence, extraction_duration_ms), and distributed tracing

Cost Impact of Hybrid Approach

Metric	CU-Only	GPT-Only	Hybrid (This Architecture)
Cost per document	~$0.01	$0.15–0.30	$0.03–0.05
Determinism	100%	Variable	95%+
Accuracy	75-80%	80–90%	90-95%
Auditability	Full	Limited	Per-field source attribution

Cost savings: 60–80% reduction compared to GPT-only by limiting LLM to gap fields.

Security and Enterprise Considerations

Azure Blob Storage:

Storage accounts can be secured by minimizing public exposure, enforcing strong identity‑based access, protecting data, and continuously monitoring for threats. Organizations should use Private Endpoints and disable public network access wherever possible, authenticate users and applications with Microsoft Entra ID instead of shared keys, and apply least‑privilege Azure RBAC with managed identities. Data should be encrypted in transit (TLS 1.2+) and at rest using Microsoft‑managed or customer‑managed keys stored in Azure Key Vault, while Microsoft Defender for Storage, logging, soft delete, backups, and Azure Policy should be enabled to detect threats, support recovery, and enforce compliance at scale. Content Safety can be called from the application layer to block uploads based on image content. Staging containers can be used to isolate untrusted uploads. Content Safety provides signals; your app enforces policy.

Azure Content Understanding / AI Vision:

Azure AI services support enterprise-grade security through Microsoft Entra ID–based authentication and Azure RBAC, ensuring only authorized applications can access extraction models. Network isolation can be enforced using Virtual Network (VNet) integration and Private Link to restrict public internet exposure. All data transmitted is encrypted in transit and at rest. Microsoft Defender for Cloud provides continuous security posture visibility across these AI workloads.

Azure OpenAI

Govern which models are approved for use and protect model artifacts and training data from unauthorized access through strong identity, network, encryption, and logging controls. AI applications should be designed with layered defenses, including multi‑stage content filtering, safety meta‑prompts, and least‑privilege permissions for agents and plugins to reduce the risk of prompt injection, data leakage, and unintended actions. High‑risk AI operations should include human‑in‑the‑loop review to prevent autonomous execution of harmful or incorrect outcomes. Organizations must continuously monitor AI systems for misuse, anomalous behavior, and data exfiltration, and they should perform ongoing AI red teaming to identify vulnerabilities such as jailbreaking, adversarial inputs, and model manipulation before they can be exploited.

Azure Cosmos DB

Azure Cosmos enhances network security by supporting access restrictions via Virtual Network (VNet) integrationand secure access through Private Link. Data protection is reinforced by integration with Microsoft Purview, which helps classify and label sensitive data, and Defender for Cosmos DBto detect threats and exfiltration attempts. Cosmos DB ensures all data is encrypted in transit using TLS 1.2+ (mandatory) and at rest using Microsoft-managed or customer-managed keys (CMKs).

Azure Functions / Compute

Secured with Entra ID authentication and managed identities, least-privilege RBAC, HTTPS-only access, private endpoints, VNet integration, and Key Vault for secrets. Hardened with Azure Policy, Defender for Cloud, and centralized logging.

Microsoft Foundry

Microsoft Foundry supports robust identity management using Azure Role-Based Access Control (RBAC) to assign roles within Microsoft Entra ID, and it supports Managed Identities for secure resource access. Conditional Access policies allow organizations to enforce access based on location, device, and risk level. For network security, Azure AI Foundry supports Private Link, Managed Network Isolation, and Network Security Groups (NSGs) to restrict resource access. Data is encrypted in transit and at rest using Microsoft-managed keys or optional Customer-Managed Keys (CMKs). Azure Policy enables auditing and enforcing configurations for all resources deployed in the environment. Additionally, Microsoft Entra Agent ID, which extends identity management and access capabilities to AI agents. AI agents created within Microsoft Foundry are automatically assigned identities in a Microsoft Entra directory centralizing agent and user management in one solution. AI Security Posture Management can be used to assess the security posture of AI workloads. Defender for AI Services provides threat protection and insights for you AI resources. Purview APIs enable Azure AI Foundry and developers to integrate data security and compliance controls into custom AI apps and agents. This includes enforcing policies based on how users interact with sensitive information in AI applications. Purview Sensitive Information Types can be used to detect sensitive data in user prompts and responses when interacting with AI applications.

DevOps Security

Security is further “shifted left” by integrating automated controls directly into CI/CD pipelines. GitHub Advanced Security for Azure DevOps, which provides dependency scanning, CodeQL-based static application security testing (SAST), and secret scanning to identify vulnerabilities and exposed credentials in code and third-party libraries. Infrastructure-as-code templates can be validated with Azure Policy and Microsoft Defender for Cloud, while pipeline protections such as protected branches and approvals reduce the risk of unauthorized changes. DevOps environments can be hardened using Azure Key Vault for secrets management, Managed Identities and Microsoft Entra ID for least-privilege access, and monitoring through Azure Monitor . Microsoft Defender for Cloud DevOps Security provides centralized code‑to‑cloud visibility across Azure DevOps, GitHub, and GitLab, identifying risks in code, secrets, dependencies, and IaC and helping teams prioritize fixes early in CI/CD pipelines

Related and Future Scenarios

Although document extraction serves as the initial use case, this architecture establishes a scalable pattern for many applications:

Insurance Claims Processing: Swap schema to claim fields; update CU analyzer for claim forms
Legal Contract Analysis: Schema for clauses, parties, dates; add NER in normalization
Healthcare Medical Records: HIPAA-compliant Cosmos; schema for diagnoses, medications, vitals
Financial Document Processing: Schema for transactions, accounts; add currency normalization
Engineering/Construction Plans: Schema for dimensions, materials, specifications
Digital Twin Integration: Feed extracted data into asset models for real-time facility visualization
Predictive Analytics: Track extracted values over time for trend detection and forecasting

Conclusion

Modernizing document extraction is not simply about applying AI—it requires aligning technology, operational discipline, and data quality. Early exploration using Generative AI enabled rapid learning and feasibility validation. However, a production-grade solution must be built on structured layout understanding models supported by standardized schema definitions and operational controls.

By combining primary structured extraction with Generative AI reasoning for bounded gap-fill verification, organizations can achieve scalable, repeatable, and auditable extraction processes. This hybrid approach enables reduced manual effort, lower error rates, and the transition from batch manual processing to intelligent, automated workflows.

The result is not just an automated extraction tool, but a scalable AI architecture for modern document intelligence—adaptable to any industry, any document type, and any structured data need.

Contributors:

This article is maintained by Microsoft. It was originally written by the following contributors.

Gaurav Bhardwaj | Senior Cloud Solution Architect – US Customer Success 
Manasa Ramalinga | Senior Principal Cloud Solution Architect – US Customer Success 
Abed Sau | Principal Cloud Solution Architect – US Customer Success

Announcing the Path to Production for Agents Webinar Series

brauerblogs — Tue, 07 Jul 2026 14:54:02 GMT

Many organizations have made significant progress exploring AI—building pilots, prototypes, and proofs of concept. Yet a common challenge remains: how do you move from promising experiments to production-ready systems that are secure, scalable, and trusted? Join us for the Path to Production Webinar Series on July 27-28, a two-day deep dive designed to help technical teams operationalize AI and agent-based solutions using proven architecture patterns, governance models, and engineering practices.

This simulive event will be available in two time zones, making it easier to participate.

Use this link to register: https://aka.ms/AccelerateThePathToProduction

Why this series matters

A large percentage of AI initiatives never make it to production—not because of lack of ambition, but because organizations struggle to establish trustworthy, governed AI systems; build scalable architectural foundations; manage risk, cost, and operational complexity; and ensure reliability in non-deterministic systems. This webinar series addresses those challenges head-on with concrete, actionable guidance spanning the full lifecycle of production AI systems.

What attendees will learn

This series delivers an implementation-focused roadmap for building, deploying, and operating AI agents at enterprise scale. Each session dives deep into governance, architecture patterns, orchestration, security, evaluation, and observability - with reference architectures and real-world engineering examples. Learn how to design scalable agent systems, integrate with enterprise data and services, and apply best practices for reliability and performance.

Attendees will leave with practical techniques and proven patterns to confidently ship production-grade agent solutions. After the workshop, customers who have Unified Contracts are eligible for a packaged set of engagements that will implement this guidance with your Microsoft cloud solution architects. Otherwise, contact your partner to learn more about taking advantage of the Frontier Transformation Offer through the Frontier Accelerate program.

Session overview

Day	Session	Focus	Speaker
July 27	Frontier Center of Excellence (CoE) & Governance	Create a governance framework with quality gates that helps organizations deliver secure, responsible, trustworthy AI at scale.	Akriti Mehta Divye Sheth
July 27	AI Landing Zones	Build a production-ready reference architecture for AI applications and agents with guardrails for networking, identity, security, and cost governance.	Nadeem Ishqair Bilal Amjad
July 27	Agentic Architecture	Adopt a governance-first, multi-agent architecture blueprint that embeds controls from user channels and orchestration through integration layers, data, and models.	Yeliz Kilinc Nour Shaker
July 28	AgentOps	Apply DevOps principles to production AI, including evaluation, CI/CD quality gates, observability, monitoring, red teaming, and incident response.	Paulo Lacerda Richard Healy
July 28	AI Security, Trust & Observability	Address prompt injection, data leakage, autonomous tool misuse, and AI-specific observability requirements for traceability, safety, and auditability.	Yuening Chen Raaid Mahbub
July 28	Solution Optimization	Reduce token cost, cut latency, tune RAG, optimize multi-agent coordination, and apply FinOps practices for sustainable scale.	Tanuja Bhamidipati Fatos Ismali

Day 1: Establishing the foundation for production AI

July 27

AI Center of Excellence (CoE) & Governance

Why do so many AI initiatives die in the PoC graveyard? Because organizations cannot trust the AI. This session shows how an AI CoE plus governance framework creates a uniform quality gate at every layer of your AI application in an organization for delivering a single, organization-wide view of secure, responsible, trustworthy AI that is ready to scale.

AI Landing Zones

Scaling AI from experimentation to production demands a secure, governed, and scalable foundation. This session explores how AI Landing Zones provide a production-ready reference architecture for deploying AI applications and agents with the right guardrails for networking, identity, security, and cost governance, aligned with the Cloud Adoption Framework and Well-Architected best practices. Attendees will learn how to design AI platforms that balance innovation with compliance, accelerate time-to-production using validated architectures and infrastructure-as-code, and integrate AI services into enterprise environments.

Agentic Architecture

Many enterprise AI pilots stall not for lack of technology, but because they lack a trustworthy architecture. This session introduces a governance-first, multi-agent architecture blueprint that closes the trust gap by embedding uniform controls and quality checks at every level, from user channels and agent orchestration through integration layers to core data and models, under a common governance and security framework. Attendees will learn how this layered agentic architecture creates a reliable, enterprise-wide AI fabric that organizations can adopt with confidence, aligning AI initiatives with high standards of trust, interoperability, and scale.

Day 2: Operating and scaling AI in production

July 28

AgentOps

This session covers the full lifecycle of deploying and operating agentic AI solutions in production. We will explore how teams can move from successful prototypes to production-ready agents using evaluation, CI/CD quality gates, observability, continuous monitoring, scheduled red teaming, and incident response practices. We will also cover how to apply DevOps principles to the unique challenges of AI systems, including non-deterministic behavior, prompt regression, model drift, tool-calling risk, and changing user behavior. Attendees will learn a practical AgentOps operating model for improving release confidence, detecting regressions earlier, and connecting agent operations back to Microsoft Foundry and Azure Monitor.

AI Security, Trust & Observability

This session focuses on securing AI systems in production, addressing risks beyond traditional application security such as prompt injection, data leakage, and autonomous tool misuse. It applies a defense-in-depth approach across identity, data protection, orchestration, and runtime controls. It also introduces AI-specific observability for trust and compliance, including traceability, safety and security monitoring, and auditability, ensuring AI systems are secure, controllable, and compliant at scale.

Solution Optimization

Getting AI to production is only half the battle. Once agentic workloads are live, organizations face compounding challenges including rising token costs, latency that degrades user trust, RAG pipelines that return noise instead of signal, and orchestration overhead that multiplies with every agent added to the mesh. This session provides a practical engineering playbook for optimizing agentic AI across the full stack, from model selection and inference routing through prompt compression, RAG tuning, caching strategies, and multi-agent coordination. It also covers the FinOps discipline required to control cost at scale, including capacity sizing, batch processing, and intelligent model routing. Attendees will leave with actionable patterns for reducing inference cost, cutting latency, and scaling reliably across regions.

Who should attend

Cloud and solution architects
AI and ML engineers and developers
Platform engineering and infrastructure teams
Technical decision-makers driving AI transformation initiatives

If your team is working to move AI beyond prototypes into production-scale systems, this series will provide directly applicable guidance for architecture, governance, operations, and optimization.

Next steps after the webinar series:

We will conduct a personalized assessment of your organization’s readiness to adopt AI agents at scale.

Call to action

Join us on July 27-28 to accelerate your path from AI experimentation to trusted, enterprise-scale production systems. Registration details can be added to this announcement before publication.

Use this link to register: Path to Production for Agents

Cloud Native Platforms: Build

KishoreKumarPattabiraman — Thu, 21 May 2026 22:41:29 GMT

Audience: Cloud architects, platform engineers, engineering leaders making design decisions

Reading time: 8 minutes

Series: Cloud Native Platforms. Build, Run, Evolve. This is Part 1 of 3.

Most engineering teams can build systems.

Few can scale them without rebuilding them.

As platforms grow, complexity does not increase linearly. It multiplies across users, services, tenants, regions, and integrations. The systems that struggle and the systems that scale are rarely separated by which cloud they run on. They are separated by a handful of design choices made early and applied consistently.

This post is about those choices.

The differentiator is not the cloud

Scalable platforms are not built with the right tools. They are built with the right design choices.

Cloud services have closed the gap on infrastructure. The differentiator is no longer which managed service a team picks. It is whether the platform is designed to absorb change, tolerate failure, and support visibility from day one. Five engineering disciplines determine whether a platform scales gracefully or collects technical debt while it grows.

Figure 1. The five disciplines compound into platform scale. Any one neglected becomes the constraint that forces a rewrite later.

1. Flexibility is the foundation of scale

Hard-coded systems work until they do not. The first request to add a tenant, a region, a SKU (a sellable product variant), or a regulatory variant is the moment a rigid design starts to bend. Each subsequent request adds weight.

Scalable platforms move behavior out of code:

Configuration replaces conditional logic
Feature flags enable safer, tenant-scoped rollouts
APIs evolve through versioning, not breaking changes
Schemas evolve additively. Breaking changes go through versioned contracts with a deprecation window long enough that consumers can migrate without downtime.

In practice

The pattern that works: configuration in a managed store, feature flags with tenant scope, and APIs versioned per consumer contract. Cost is the discipline of treating configuration as code (versioned, reviewed, audited). The return is that releases stop being events and start being routine. A change that previously needed a coordinated deployment can be executed in minutes, gated to a single tenant for verification, and rolled out broadly only after the signal is clean. Most platforms reach this state by retrofit, not by design. Doing it earlier costs less than waiting.

If a change requires a redeploy, it should require a very good reason.

2. Failures are normal. Resilience is a choice.

Distributed systems will fail in unpredictable ways. The real question is not how to prevent failure. It is how the system responds when failure happens.

Resilience is engineered, not inherited from the platform. The patterns that move the needle are well known and consistently applied:

Idempotent operations (safe to call multiple times with the same result) that make retries safe
Reliable messaging patterns such as the transaction outbox (writing the message to the same database transaction as the business change, then publishing asynchronously) to avoid lost or duplicated events
Decoupled services that contain blast radius (the scope of damage when one component fails)
Timeouts, retries, and circuit breakers (a wrapper around a dependency that stops calling it for a cool-off window after repeated failures) tuned per dependency
Bulkheads (isolation pools, often a separate compute or queue lane per workload class) that keep noisy neighbours from starving critical paths of resources

In practice

The pattern that works: every write that can be retried carries an idempotency key, every queue consumer is safe to replay, every event published goes through an outbox in the same transactional unit as the business change. When peak load triggers retries, duplicates collapse cleanly instead of producing duplicate orders, double-charged customers, or split-brain state. The contract changes outwards: callers can retry without thinking, queues can be at-least-once instead of exactly-once, and recovery moves from a manual cleanup task to a property of the system. Most teams that adopt this pattern stop seeing certain classes of incident entirely.

Implementation note

An idempotent API is not just a design preference. It changes how the rest of the system can be built. Once writes are safe to repeat, retries become cheap, queues become trustworthy, and recovery becomes automatic.

The naive implementation (read the key, if absent process and save) has a race. Two concurrent requests with the same key both miss the lookup, both call the processor, and both attempt to save. That is the failure mode idempotency exists to prevent. The pattern that survives production is an atomic reserve-then-execute: insert a row keyed by the idempotency key with a unique constraint before doing any work. The first writer wins. Concurrent callers either wait for the original to complete and read its result, or they receive a conflict response.

// Contract for the idempotency store. The two key methods are TryReserveAsync // (atomic insert with unique-key constraint) and CompleteAsync (record the // result of the first writer). GetCompletedResultAsync polls until the first // writer commits or returns 409 Conflict if the in-flight window exceeds the // configured deadline. public interface IIdempotencyStore { Task<Reservation> TryReserveAsync( string idempotencyKey, string requestHash, CancellationToken ct); Task CompleteAsync( string idempotencyKey, OrderResult result, CancellationToken ct); Task<OrderResult> GetCompletedResultAsync( string idempotencyKey, CancellationToken ct, TimeSpan? maxWait = null); } public readonly record struct Reservation( bool IsFirstWriter, string RequestHash); // Idempotency via atomic reserve-then-execute. // First writer wins; replays return the original result; concurrent // duplicates lose the race and read the winner's outcome (or get 409). public async Task<OrderResult> CreateOrderAsync( Order order, string idempotencyKey, CancellationToken ct) { var requestHash = StableHash(order); // canonical content hash // Atomic insert: succeeds for the first caller, fails for the rest. var reserved = await _store.TryReserveAsync( idempotencyKey, requestHash, ct); if (!reserved.IsFirstWriter) { if (reserved.RequestHash != requestHash) throw new IdempotencyKeyReusedException(); // A previous run committed (return its result) or is in-flight // (poll with a bounded deadline; 409 if exceeded). return await _store.GetCompletedResultAsync( idempotencyKey, ct, maxWait: TimeSpan.FromSeconds(5)); } // We are the first writer. Execute, persist, mark complete. var result = await _processor.ProcessAsync(order, ct); await _store.CompleteAsync(idempotencyKey, result, ct); return result; }

Three production details matter:

TTL or compaction on the idempotency record. Without it, the store grows forever. Most teams retain records for the request retry window plus a safety margin (commonly 24 to 72 hours).
Stable content hash, not the default object hash code. The request hash detects key reuse with a different body, so a client that reuses an idempotency key with a different payload receives IdempotencyKeyReusedException rather than silently getting the wrong result. Canonicalise field ordering, locale, and null handling before hashing.
Bound the in-flight window explicitly. The genuinely hard case is when the processor succeeded but the store write failed. Production-grade implementations either run the side-effect and the store write in the same transaction (when the processor and store share a database) or use the transaction outbox pattern to bridge them. The poll-with-deadline in GetCompletedResultAsync handles the duplicate-arrives-mid-flight case; the transactional boundary handles everything else.

3. Observability is not optional

Without observability, teams operate blind. As systems grow, the price of guessing rises faster than the price of seeing.

At build time, observability is a design property. The decisions made before the system reaches production are what determine whether it can be operated at all. The dashboards, alerts, and incident practices covered in Part 2 of this series rely on instrumentation choices made here.

The build-time work that pays off in production:

Request identifiers propagated through every service hop, every queue, every async boundary, so a single user action can be traced end to end
Structured logging with a consistent schema (event name, correlation id, tenant, severity) rather than free-form strings
Metrics emitted at the boundaries that matter (every external call, every queue read or write, every database operation), not only at the entry point
Tracing libraries integrated at the framework or middleware layer so coverage is automatic, not opt-in
Schemas designed so business signals (orders, sessions, transactions) and system signals (CPU, latency, errors) share the same identifiers and can be correlated later

In practice

The pattern that works: a single request id flowing through every service hop, every queue, every async boundary, propagated automatically at the framework layer rather than per-call. Add one structured logging schema across services (event name, correlation id, tenant, severity), so that a single query joins business events with system events. The investment is hours of upfront framework wiring. The return is that production diagnosis stops being archaeology. Cross-service questions become single dashboards; postmortems shrink from days to hours; and the dashboards in Part 2 actually work because the data underneath is shaped to support them.

4. Delivery practices set the ceiling

Scaling teams requires scaling delivery. Small inefficiencies in pipelines, environments, and release coordination compound into measurable drag.

Delivery maturity that pays off at scale:

Pipelines as code, reviewed and versioned like application code
Parallel deployments across services and regions where dependencies allow
Infrastructure as code with shared modules, not hand-managed environments
Automated quality gates: tests, security scans, dependency checks
Trunk-based development (developers commit to a single shared branch many times a day) with short-lived feature branches and progressive delivery. Important caveat: trunk-based works only when test automation and feature flags are already in place. Adopting it before those foundations exist tends to amplify production incidents rather than reduce them.

In practice

The pattern that works: pipelines run in parallel where dependencies allow, infrastructure provisioning is templated rather than per-environment, and quality gates run automatically rather than as discretionary steps. Sequential deployment of a multi-service platform across three environments takes hours; parallelised deployment of the same change takes minutes. The payback is not only release speed. It is the compounding cost reduction of every wait state for every engineer on every release. Teams that treat pipelines as a product feature, not an afterthought, ship more confidently and recover from bad changes faster because the rollback path was exercised, not invented during an incident.

Slow pipelines are not a tooling problem. They are a design problem.

5. Cost discipline is engineering work

Cloud platforms can become expensive quickly when cost is treated as someone else's problem. Cost is a property of the design, not a quarterly review.

The teams that get this right treat cost the same way they treat performance:

Elastic compute and storage tiers chosen per workload pattern
Non-production environments with automated scale-down windows (the easiest savings to leave on the table)
Tagging discipline so cost can be attributed to a service, a feature, a tenant
Egress and data-tier choices, not compute, dominate cloud bills past a certain scale. Right-size storage tiers (hot vs cool vs archive), eliminate cross-region chatter, and watch egress on the data plane more closely than compute on the request path.
Budgets and usage alerts wired into the same channels as reliability alerts
Cost reviews built into design discussions, not deferred to FinOps (Financial Operations: the practice of managing cloud spend as an engineering concern)

In practice

The pattern that works: non-production environments scale down automatically outside business hours, storage tiers match access patterns (hot, cool, archive), and tagging is enforced so every dollar can be attributed to a service or feature. Cost reviews happen at design time, not after the bill arrives. The biggest savings come from data plane decisions, not compute: cross-region egress, oversized storage tiers, and forgotten test environments dominate cloud bills past a certain scale. Treat cost as a first-class non-functional requirement, alongside latency and availability, and the discipline compounds in every design discussion that follows.

A scenario that ties it together

Figure 2. A reference architecture that puts the disciplines into one shape. The request path is decoupled, the data layer is purpose-fit, identity is brokered by managed identity throughout, private endpoints isolate the data tier from public networks, and observability runs as a first-class lane.

Picture a multi-tenant platform at a growth inflection. Onboarding a new tenant takes weeks because tenant-specific behaviour is hard-coded across services. Every release carries risk because there is no way to roll out a change to one tenant without affecting the rest. Incidents linger because logs and metrics live in different tools and nobody can correlate them in production.

Do not start with a rewrite. Start with the smallest set of changes that unlocks the next year of growth: extract configuration out of code, introduce tenant-aware feature flags, wire a unified observability view into the existing services, and parallelise the pipelines. None of these are architectural revolutions. They are design choices applied with discipline, in the order the disciplines compound.

Eighteen months in, onboarding a tenant takes hours instead of weeks. Releases move from monthly events to weekly increments. Incidents are caught earlier and resolved faster. The platform did not get bigger. It got more capable. The five disciplines did the work; the team made the choice to apply them.

What teams get wrong

The common pattern is architecting for the system you have, not the system you are growing into. It looks like progress because the current sprint ships. Pillars get postponed because they feel like overhead.

The cost surfaces later. Each shortcut becomes a constraint. The constraints compound, and three releases later the team is debating a rewrite.

The fix is not premature abstraction. It is small, deliberate investments in flexibility, resilience, observability, delivery, and cost from day one. The discipline is to make these investments before they are urgent.

Where to start when you cannot do everything at once

Five disciplines is a wall, and real teams cannot fund all five at once. The right order depends on whether the platform is being built fresh or already running.

For a system already in production and already in pain, the SRE community's hierarchy of reliability needs gives the most defensible starting order: monitoring and observability first (you cannot fix what you cannot see), then incident response (close the bleeding cleanly), then resilience patterns (idempotency, retries, decoupling) so the bleeding has fewer reasons to start, then flexibility and delivery so safe change can travel at speed. Cost discipline runs alongside throughout, never as the headline.

For a system being built fresh, the order in this post (flexibility, resilience, observability, delivery, cost) reflects the Azure Well-Architected Framework's emphasis on designing for change, failure, and visibility before scaling teams or workloads. Both orders are defensible. What is not defensible is leaving any of the five for later.

The most concrete starter from this post: request id propagation. A single correlation identifier travelling through every service hop, every queue, every async boundary, costs hours up front and pays back every time someone has to debug production for the rest of the platform's life. It is the smallest unit of the observability discipline and the foundation that the dashboards, traces, and incident response in Part 2 all depend on.

The shift

The most important transformation in scaling a platform is not technical. It is mindset.

The shift is from project thinking to platform thinking:

Build reusable capabilities, not one-off solutions
Design systems for long-term evolution, not the next release
Enable other teams, not just deliver for one team

Tools change. Cloud services evolve. The architectural fashions of this year will not be the architectural fashions of the next. What persists is the discipline behind the choices. Scalable systems are not built by tools. They are built by teams that treat design as continuous work. The same discipline shows up again in Part 2 (operating these systems) and Part 3 (using AI to augment that work). The tools change. The disciplines do not.

Want to discuss? What single design choice has paid the most dividends in the platforms you run? Drop a comment with patterns you have seen in your environment. Every reply gets read.

Next in this series: Running Cloud Native Platforms: Why Day 2 Decides Everything. Building is half the journey. The next post looks at what it takes to operate these platforms once they are in production.

Cloud Native Platforms: Run

KishoreKumarPattabiraman — Thu, 21 May 2026 22:41:09 GMT

Audience: SREs (Site Reliability Engineers), platform engineers, engineering managers running production systems

Reading time: 8 minutes

Series: Cloud Native Platforms. Build, Run, Evolve. This is Part 2 of 3.

Most systems are designed thoughtfully.

Most operations are inherited reactively.

The systems that survive are not the ones built with the most care. They are the ones operated with the most discipline. Production has a way of revealing every shortcut taken during design and every assumption left unverified.

This post is about what it takes to operate a platform once the build is done.

How they are run, not how they are built

Systems are not defined by how they are built. They are defined by how they are run.

A well-designed system that is operated reactively will fail in production. A modestly designed system that is operated with discipline will outperform it. Five operational disciplines decide which side of that line a platform lives on. Each one is engineering work, not a checklist for someone else to handle.

Figure 1. The incident lifecycle as a state machine. The states are not optional steps. They are the contract between the team and the system.

1. Observability is the backbone of reliability

Without observability, every operation becomes a guess. As systems grow, the cost of guessing rises faster than the cost of seeing.

Part 1 of this series argued that observability is a design property: instrumentation contracts, request id propagation, structured logging schemas. Production is where those design choices either pay off or do not. Strong observability in production is a contract that lets any engineer answer three questions in minutes: what failed, why it failed, and what the impact was. The shape of that contract matters more than the tool that implements it. (This three-question framing is community-popularised through the SRE community and writers such as Charity Majors. See Honeycomb's What is Observability for the canonical articulation of the three-pillars and question framing; the substance is older than the framing.)

Dashboards organised around user journeys, not infrastructure components
Service level indicators (SLIs: the specific measurements you care about, e.g., success rate, p99 latency) chosen from the user's perspective, not the database's
Alerts that page only on burn-rate against an SLO (Service Level Objective: the target value of an SLI, e.g., 99.9% of requests complete in under 800ms over a rolling month) using a multi-window strategy. A short window catches fast burns; a long window catches slow drifts. This is what makes SLOs operational rather than decorative.
Sampling and retention tuned for cost, but never for blind spots
The distinction between MTTA (mean time to acknowledge: how fast someone notices) and MTTR (mean time to restore: how fast service returns) tracked separately. Conflating them hides whether the team's bottleneck is detection, response, or fix.

In practice

The pattern that works: rebuild the operational view around two or three user journeys (sign-in, place order, view history) rather than per-component charts. Tie alerts to error budget burn rather than raw threshold crossings. Track MTTA and MTTR separately so the team's actual bottleneck (detection, response, or fix) is visible. The investment is rethinking what to measure, not buying a new tool. The return is that incidents stop being discovered by customer complaints first. Teams that make this shift typically find their existing telemetry was sufficient; only the questions being asked of it were wrong.

If a dashboard cannot answer "what is the user experiencing right now", it is not an observability dashboard. It is decoration.

2. Alerts are signals, not notifications

More alerts do not mean better monitoring. In practice, the opposite is true. Once alerts outpace the team's ability to act, important signals start getting missed.

Effective alerting works to a small set of rules:

Severity that maps to action, not to technical category
Ownership baked in, never inferred at runtime
Thresholds tied to user impact, not raw metric values
Noise treated as a defect, with a regular review cadence
Suppression and grouping for known multi-alert patterns

In practice

The pattern that works: audit every alert against one test, "what action would I take in the next five minutes if this fires now?" Demote alerts with no answer to dashboards. Remove alerts where the answer is the same as another alert's. Group related alerts so one incident produces one page, not twelve. Most teams discover their alert volume drops by an order of magnitude after a thorough audit, and the alerts that remain start getting trusted again. Trust is the precondition for every other operational practice. Without it, on-call rotations decay into noise filtering and the real signals get missed.

Figure 2. From raw events to pages, in approximate orders of magnitude. The numbers vary by team and workload; what does not vary is that each stage needs to remove one to two orders of magnitude of noise. Teams that page on raw events end up with on-call rotations nobody trusts.

3. Incident response is a practiced muscle

Failures are inevitable. Unstructured response is not.

The teams that recover quickly do not improvise during incidents. They follow a structure that has been practiced when nothing was on fire. The structure is intentionally simple, because incident time is the worst time to negotiate roles.

Clear roles: incident lead, communications lead, scribe, subject matter expert (the RACI model, Responsible-Accountable-Consulted-Informed, adapted for incident response)
Defined escalation paths with clear handoff criteria. Escalation means re-paging to a higher tier or specialist, not returning to detection. The lifecycle diagram in Figure 1 makes the distinction explicit.
Runbooks for the top failure modes, kept short enough to actually be read
Status communication on a fixed cadence, even when there is nothing new to say. Customer comms and internal comms are tracked separately.
Blameless postmortems (focus on the system that allowed the failure, not the person who pushed the button) that produce action items the team actually completes
Game days: scheduled exercises that simulate failure modes (region outage, dependency unavailability, traffic spike) under controlled conditions, so gaps in runbooks are found before incidents do

In practice

The pattern that works: name the incident lead and the comms lead before the first message goes out. Write runbooks short enough to be scannable at 3 AM. Run blameless postmortems with action items that actually get tracked to completion. Schedule game days quarterly so the runbooks are exercised before real incidents. Teams that operate with this structure do not have more engineers; they have engineers who are not single points of failure during recovery. The deepest experts stay the deepest experts, but the platform stops depending on whether they happen to be online.

Implementation note

A short, well-structured runbook outperforms a long, exhaustive one. The goal during an incident is not to think. It is to act on a procedure that has been thought through in calmer times.

# Runbook header pattern (keep it scannable in incident time) title: High latency on order API slo_protected: # this runbook protects two SLOs - order-completion-success - order-completion-latency severity: # derived from burn rate, not declared fast_burn: P1 # 14.4x budget burn over 1 hour => page now slow_burn: P2 # 6x budget burn over 6 hours => investigate owner: payments-team indicators: # triggers for evaluation, not severity - p99 (99th-percentile) latency exceeds the SLO target for 5 min - error rate exceeds the SLO target for 3 min on order-completion first_actions: - Open the order-journey dashboard. Confirm impact in business terms. - Check Service Bus queue depth and dead-letter rate (the most common cause of API latency under load is downstream backpressure) - Verify Cosmos DB RU/s saturation and partition hotspots - Inspect the most recent deployment for behavioural changes escalate_if: - Latency does not recover in 15 min - Error rate exceeds 5% (fast burn against the SLO) - Customer reports arrive before our own signals do rollback_path: - Feature flag "new-order-pipeline" can be disabled per-tenant - Last known good deployment id is in the release tracker note_on_scaling: # CPU is rarely the cause of latency in this service. Scale only after # confirming the bottleneck is compute, not a downstream dependency or # queue depth. Adding capacity to a saturated downstream amplifies the # incident; it does not resolve it.

The general principle behind that last note travels beyond this runbook: scale-out is the right remediation for compute saturation, not for downstream saturation. When latency rises because a database, queue, or external dependency is saturated, adding capacity in front of the bottleneck moves more requests into the bottleneck and makes the incident worse. This is one of the most common operational mistakes when the dashboard shows red and the on-call instinct says "add more".

4. Release confidence is engineered

Releases get harder as systems grow. The platforms that ship confidently at scale have engineered the path, not learned to fear it.

The patterns that change the math:

Feature flags that allow change without deploy
Canary deployments (releasing the new version to a small slice of traffic first, watching error budget burn before continuing) that surface problems on a small slice
Gradual rollouts with automated rollback triggers
Database migrations split from application releases
Release coordination that scales with services, not with team size

In practice

The pattern that works: every change ships behind a feature flag, canary deployments take a small slice of traffic first, and rollback is a one-click step in the pipeline rather than a procedure to be invented during an incident. The cost is the discipline of building rollback paths and exercising them. The return is releases that stop being events. Issues that previously triggered full rollbacks get isolated to a slice and rolled back automatically before they reach most users. The willingness to ship smaller, more frequent changes follows directly from the confidence that bad changes can be undone fast.

Big releases feel safe because they are rare. They are actually risky because every change rides together.

5. Reliability is continuous, not a milestone

Reliability is not achieved through tools alone. It requires continuous refinement, feedback-driven improvement, and a budget that the team can spend on operational work without negotiating each time.

The disciplines that keep systems reliable over years are codified well in the SRE-book framing of service level objectives and error budgets (the canonical reference is the Google SRE Book chapter on Service Level Objectives, with the operational follow-up in the SRE Workbook chapter on alerting on SLOs). The names matter less than the practice they enable.

SLOs chosen from the user's perspective, with two or three per service rather than ten. More SLOs means none of them shape behaviour.
Error budgets: the inverse of the SLO, expressing how much unreliability the team is willing to spend in a window. Used up early in the month means slow down on releases. Healthy means feature work keeps moving.
Multi-window burn-rate alerting turns SLOs from dashboards into pages: short window catches catastrophic failures, long window catches slow drift. Without burn-rate alerting, SLOs are observation, not operation. (The pattern is documented in the SRE Workbook.)
Reliability work has its own backlog, prioritised against features. Not a wishlist after every incident.
Regular game days that exercise failure modes (region failover, dependency outage, traffic spike) before they happen for real
Capacity planning informed by data, not by anxiety

In practice

The pattern that works: define two or three SLOs per service, expressed from the user's perspective. Compute the error budget weekly. When the budget is healthy, ship feature work. When the budget is burning fast, slow down and fix the cause. The conversation about which incidents matter and which can wait becomes possible because there is a shared number to point at. Reliability becomes a quantified property of the platform, not an opinion debated at every retrospective. Teams that adopt this discipline stop having the recurring "how reliable do we need to be?" argument and start having data-grounded trade-off discussions instead.

A scenario that ties it together

A platform was launching a new region. The build had gone well. Day 1 was clean. Two weeks in, latency started creeping up during peak hours. Alerts fired on raw thresholds, but no one could tell which ones to trust. Incident calls turned into long debugging sessions because three different teams owned overlapping pieces of the request path.

The team did not start by buying a new tool. They started by treating operations as engineering work. The dashboard was redesigned around the user journey. Alerts were audited and most were demoted or removed. Roles for incident response were written down. A short runbook covered the top failure modes. Releases were broken into canary slices behind feature flags.

None of this was new. It was discipline applied consistently to work that was previously assumed to be someone else's. The next region launch took half the effort, and the team's mean time to restore on the failures that did happen was measurably lower.

What teams get wrong

The common pattern is treating Day 2 as the cost of Day 1. Teams design beautifully, ship fast, then quietly absorb the operational debt. Dashboards proliferate. Alerts grow louder. Postmortems pile up.

The fix is not more dashboards. It is treating operations as engineering work with the same rigour as feature delivery. Operability is a property the system either has or does not. It is not earned by adding monitoring. It is earned by designing for visibility and operating with discipline.

Where to start

The most concrete starter from this post: an alert audit. List every alert that fires in the next week and apply a single test to each one: "what action would I take in the next five minutes?" Demote the alerts that have no answer. Remove the alerts where the answer is the same as another alert's. The audit takes a morning. The result usually halves alert volume and lifts trust on what remains, which is the precondition for every other operational practice in this post.

The shift

The most important shift in maturity is not technical. It is in stance.

The shift is from shipping software to operating systems:

Operations is not a phase that follows engineering. It is engineering.
Reliability is not a milestone reached. It is a discipline practiced.
Incidents are not interruptions to the work. They are the work.

The teams that internalise this shift run platforms that are smaller, calmer, and more trusted. They do not have fewer incidents because their systems are more advanced. They have fewer incidents because their operational discipline is more consistent. Part 3 of this series argues that the same discipline applies again, in a different domain: the practices that make platforms operable are the practices that make AI useful in delivery.

Want to discuss? What is the one operational practice your team adopted that changed how you sleep at night? Drop a comment with patterns you have seen in your environment. Every reply gets read.

Previously in this series: Building Cloud Native Platforms That Scale: Patterns That Actually Work. The first post covered the design choices that make scale possible.

Next in this series: AI-First Platform Engineering: From Copilot to Agentic Delivery. Cloud helped us scale infrastructure. The next post looks at how AI is now changing how we build and run platforms.

Cloud Native Platforms: Evolve

KishoreKumarPattabiraman — Thu, 21 May 2026 22:40:33 GMT

Audience: Engineering leaders, platform architects, senior developers exploring how to operationalise AI in their teams

Reading time: 8 minutes

Series: Cloud Native Platforms. Build, Run, Evolve. This is Part 3 of 3.

Cloud helped us scale infrastructure.

AI is starting to do the same thing for the work around the code: the planning, the testing, the release communication, the incident triage, the writing that surrounds writing software.

The conversation about AI in software has narrowed too quickly to "Copilot in the editor". The bigger story is happening across the lifecycle. Planning, design, development, testing, release, and operations are all being augmented at once. The platforms that adopt AI well are not the ones with the most usage. They are the ones with the clearest discipline around how it is used.

This post is about that discipline.

AI is changing how we engineer, not how we type

AI is not changing how we write code. It is changing how we engineer software.

Code generation is the surface. Underneath it, AI is reshaping the unit of leverage. The question is no longer how fast a developer can type. It is how well a workflow can be expressed as a reusable engineering asset. Six disciplines determine whether AI moves the needle on outcomes or just adds another tool to the stack.

Figure 1. AI across the SDLC. Each phase has clear AI assist points and clear human-owned validations. The boundary is not negotiable. It is the design.

1. From assistance to augmentation

Early AI tools focused on assisting individual developers. Code suggestions. Autocomplete. Quick refactors. The value was real but bounded by the editor.

The shift now is into structured workflows that span the lifecycle. The unit of leverage is no longer a single suggestion. It is a sequence of actions executed reliably across phases. ("Agentic" later in this post means a system that makes its own next-step decisions inside guardrails. A workflow follows a fixed sequence; an agent chooses the path.)

Code generation has become baseline, not differentiator
Workflow generation is where the largest gains live
Multi-step assistance with explicit human checkpoints
Context that travels across tools, not just within one

In practice

The pattern that works: start with the single highest-volume writing task on the team (commit messages, code review comments, release notes, postmortem first drafts) and turn the AI assist for that task into a shared workflow rather than each individual's private trick. The cost is one engineer's afternoon documenting the workflow and the eval set. The return is that every engineer on the team inherits the work, and the task that used to consume an engineer's morning every two weeks becomes a background step in the release process. Workflow generation, not faster typing, is where the gains compound across a team.

Code suggestions help one developer. Reusable workflows help the next ten.

2. AI across the SDLC, with guardrails

AI now has a useful role at every phase of delivery. The role is different at each phase, and the guardrails are different too.

Phase	What AI helps with	What humans must validate
Plan	Breaking down requirements, drafting acceptance criteria	Domain context, business priorities, customer impact
Build	Code generation, refactoring, scaffolding	Architectural fit, security boundaries, performance
Test	Test case generation, edge case discovery	Coverage of business-critical paths, regulatory cases
Release	Release notes, changelog summaries, communication drafts	Accuracy, tone, customer-facing claims
Operate	Log triage, incident summaries, runbook drafts	Root cause attribution, action item ownership

The guardrails are not optional decoration. They are the design.

In practice

The pattern that works: stage AI assists for release communication (changelog drafting, customer-facing release notes, internal release announcements) and require a human review before anything goes out. The draft arrives consistently, faster than a human could produce, and easier to compare across releases. The reviewer is not eliminated; the reviewer is moved from author to editor, which is where their judgment actually matters. Teams that adopt this pattern stop missing release-note deadlines and stop publishing inconsistent communication across products.

3. From prompts to reusable assets

Many teams begin with prompt experimentation. Individuals find techniques that work for their tasks. The result is a patchwork of personal practices that do not survive a team change.

The compounding value comes when prompts mature into reusable engineering assets.

Figure 2. The maturity model from prompts to agents. The value compounds at the workflow stage and accelerates at the agent stage. The disciplines that make agents safe are the same ones that made workflows reliable.

The maturity stages, in order of leverage:

Prompts: ad-hoc, individual, hard to share
Templates: parameterised prompts versioned with the project
Workflows: multi-step sequences with clear inputs, outputs, checkpoints
Agents: autonomous task chains operating within explicit guardrails

The diagram is a maturity ladder, not a graduation. In practice teams operate at all four stages simultaneously for different tasks. A senior engineer may use a one-off prompt to explore a refactor, run a versioned template for commit messages, hand off to a workflow for release notes, and trigger an agent for routine PR triage, all in the same hour. The point of the ladder is not to leave earlier stages behind. It is to know which stage a given task belongs to and to invest accordingly.

In practice

The pattern that works: pick the three prompts your team uses every week, codify them as parameterised templates in the same repository as the application code, and treat them as engineering artefacts (reviewed, versioned, owned). New engineers inherit the team's accumulated practice instead of building their own from scratch. Quality becomes consistent because the variance between individuals shrinks. Investment pays back in weeks, not quarters, and the maturity ladder keeps producing returns as the team moves from templates to workflows to agents.

4. Agentic delivery, with guardrails that survive a security review

The next stage is agentic. AI executes sequences of tasks within a defined scope. The risk is not that the agent will fail. It is that the system around the agent will not catch the failure, and that the failure modes are different in kind from traditional automation. Agents are non-deterministic, they can be manipulated through their inputs, and their actions can have side effects in systems the team does not own.

Five guardrails make agentic delivery safe. The first four are necessary. The fifth is what carries the agent through a security review at a regulated enterprise.

Identity and scope: the agent runs as a managed identity (or scoped service principal) with the smallest set of permissions that lets it do its job. Permissions are expressed as allowlists, not denylists. Tools fetched at runtime are subject to the same identity boundary as the agent itself.
Input quarantine: anything the agent reads from a user-controlled source (work item bodies, PR descriptions, customer tickets) is treated as untrusted text. The agent does not execute instructions found in fetched content, and tool calls are validated against an output schema before execution. This is the prompt-injection mitigation, and it is the most common gap in agentic systems shipped today.
Cost and blast-radius caps: every run has a maximum token budget, a maximum number of tool calls, and a maximum spend. Exceeding any cap aborts the run cleanly. Without caps, scoped credentials are not enough to bound the damage.
Evaluations and traceability: agents are evaluated against a fixed test set before deployment, and on every prompt or model change. Every action is logged with inputs, outputs, the model and prompt versions used, and the reasoning trace where the model exposes one. Logs are redacted for secrets and personally identifiable information at write time.
Reversibility taxonomy: actions are categorised by reversibility, not asserted to be reversible in general. A draft write to a private store is reversible. A post to a customer-facing channel is not reversible (deletion does not unsend). A database update may be reversible by a compensating transaction or not at all. Irreversible actions require human approval at the boundary, before they happen, not after. The agent is allowed to draft and stage. The human is the only one who is allowed to make the move that cannot be undone.

In practice

The pattern that works: start with one low-risk agent (release-notes drafter, PR triage assistant) running on read-only inputs, write-only-to-drafts permissions, and a hard cost cap per run. Require explicit human approval at the irreversible step. Wire up an evaluation set on day one, and rerun it on every prompt or model change. Treat regressions as failures, not warnings. The first agent the team ships is rarely the most valuable; it is the rehearsal that establishes the controls every later agent inherits. Teams that skip this rehearsal end up with an agent in production that no one feels safe extending.

Implementation note

An agent without a reversibility taxonomy and a regression eval set is a liability. The discipline is the same one that made workflows reliable: scoped identity, idempotency, traceability, and a clear boundary between machine action and human decision. The YAML below is illustrative, not a runtime contract; it is meant to show the shape of the controls a real agent definition would carry, not the syntax of any specific platform.

# Agent run definition (illustrative; not a specific platform's syntax) name: release-notes-drafter trigger: pre-release identity: type: managed-identity scope: tenant=<tenant-id> resource=release-tools/<app-id> permissions: allow: - read: work-items in milestone (filter: state=Done) - read: pull-requests in milestone (filter: merged) - write: drafts/release-notes/${run-id} # Production channels are NOT in the allowlist. The agent cannot post. limits: max_tokens_per_run: 80000 max_tool_calls_per_run: 20 max_runtime_seconds: 300 max_cost_usd: 0.40 on_exceeded: abort_with_partial_artifact input_handling: treat_fetched_content_as: untrusted # Indirect prompt injection is mitigated by the layered discipline below, # not by a single feature flag. Each item is a separate control. enforce_instruction_hierarchy: true validate_tool_args_against_schema: true validate_outputs_against_schema: true steps: - fetch: completed work items in milestone - draft: release notes from items - validate: required fields present - request-review: from: release-manager idempotency_key: ${milestone-id}-${draft-hash} - on-approval: action: post-to-internal-channel reversibility: not-reversible requires: explicit-human-click # the agent does NOT click this audit: log_inputs: true log_outputs: true redact: - secrets # Pattern-based: handles structured PII like emails, phones, IDs. - pii_patterns: [email, phone, national-id, payment-card, ip-address] # Entity-based: required for unstructured PII like names. Pattern alone # cannot redact a customer name without an entity-recognition step. - pii_entities: ner-based # names, locations, organisations retain: 365_days # tune to your audit policy, not to the demo evaluation: test_set: tests/release-notes/eval-v3.jsonl on_prompt_change: rerun on_model_change: rerun fail_threshold: 5_percent_regression

5. Where AI still needs human judgment

AI has clear boundaries. The boundaries are not embarrassing. They are the design.

What must stay human-owned:

Architectural trade-offs and design decisions
Security validation and threat modelling
Correctness for business-critical and regulatory paths
Domain context that has not been written down
Accountability for outcomes, not just outputs

The goal is collaboration, not replacement. The teams that get the most value from AI are not the ones with the most automation. They are the ones with the clearest sense of where automation ends and judgment begins.

In practice

The pattern that works: name the human-owned items explicitly in the team's working agreement (architecture, security, regulatory correctness, accountability) and audit every AI workflow against that list. When a workflow asks the AI to make a decision in any of those categories, redesign it so the AI prepares the analysis and a human makes the call. Most teams over-trust AI for one of these areas in their first six months and learn the hard way. Naming the boundary up front prevents the lesson from being paid in production. The clarity is the value; the model behind the workflow is interchangeable.

6. Responsible AI is engineering work

The first five disciplines decide whether AI moves the needle. The sixth decides whether the platform can defend the choices it makes with AI. Responsible AI is the engineering practice of building systems whose AI behaviour is fair, transparent, accountable, and safe by design, not by audit after the fact. Treating it as a compliance checkbox at the end of the project is how teams end up shipping AI workflows that fail security review, embarrass the company, or harm users.

Six controls turn responsible AI from a policy into engineering work. These map directly onto the practices Microsoft and the broader industry have converged on, but the names matter less than the practice they enable.

Fairness in inputs and outputs. The training data, eval set, and prompts are reviewed for systematic bias against any group the system serves. The eval set covers under-represented cases by design, not by accident, and regressions on those cases fail the build.
Transparency to end users. When a user sees AI-generated content, they are told. When a decision is AI-assisted, the path from input to output is explainable in plain language, not just in a model card buried in documentation.
Content safety filters. Inputs and outputs pass through safety classifiers (prompt injection, prohibited content, jailbreak patterns) before reaching the model and before reaching the user. Filtering decisions are logged and reviewable.
Accountability ownership. Every AI workflow has a named owner who is accountable for its outcomes, not just its uptime. The owner has the authority to pause or roll back the workflow when harm is detected.
Data minimisation and residency. The AI sees only the data it needs to do the task. Personally identifiable information and customer data are scoped, redacted, and kept inside the boundary the customer agreed to. Cross-tenant leakage is treated as a P1 incident, not a feature request.
Harm evaluation alongside quality evaluation. The eval set measures harm potential (toxicity, hallucination on factual queries, leakage of confidential context) with the same rigour as it measures correctness. Both must pass for a release to ship.

Figure 3. Responsible AI as a set of engineering controls around the AI workflow. The six controls fall into four categories: data discipline (fairness, data minimisation), model discipline (content safety, harm evaluation), deployment discipline (transparency to users), and governance (accountability ownership). All six are necessary; none is sufficient on its own.

In practice

The pattern that works: write the responsible AI plan before the first agent ships, not after the first incident. Pick one workflow that touches user data or generates customer-facing content, and use it as the reference implementation: fairness review on the eval set, content safety filters wrapping the model call, transparency annotation in the UI, redaction of identifying details in logs, harm evals running alongside quality evals on every change, and a named owner with explicit pause authority. The first such workflow takes longer to ship than the unconstrained version. Every workflow after it inherits the controls and ships faster than it would have without them. Teams that defer responsible AI to a future quarter end up retrofitting it under pressure, which is the most expensive way to do it.

A scenario that ties it together

Picture a platform team several months into using Copilot. Adoption is high. Productivity dashboards show gains. But defect rates are not improving and lead time is flat. Leadership asks the obvious question: is AI actually helping, or just feeling like help?

The answer is not to stop using AI. It is to change how AI is measured. Move adoption metrics to the background. Move outcome metrics to the front: defect escape rate, lead time for change, change failure rate, mean time to recovery. In parallel, promote the individual prompts that have proved themselves to shared templates, and the templates to versioned workflows. Retrofit responsible AI controls onto the workflows that shipped first: content safety filters, harm evaluations alongside quality evaluations, transparency annotations on customer-facing output, and a named owner for each workflow.

Six months later, the picture is different. Defect rate improves on the parts of the codebase where reusable workflows were introduced. Onboarding for new engineers is visibly faster. Release notes are consistent across teams. The shift is from celebrating use to tracking outcomes, and once the team measures what matters, the tooling decisions start making themselves.

What teams get wrong

The common pattern is measuring AI by usage, not by outcome. Adoption metrics tell you who tried Copilot. They do not tell you whether defects dropped, lead time improved, or release notes got better.

The fix is not less AI. It is better measurement. The four metrics named in the scenario above (defect escape rate, lead time for change, change failure rate, mean time to recovery) come from the DORA research on software delivery performance and have become a useful default. Two warnings travel with them. First, attribution is hard: an AI workflow rolled out alongside a test refactor and a CI pipeline change cannot claim credit cleanly. Second, baselines matter more than headlines: a single quarter's improvement is not a trend, and a single team's gain is not the platform's gain. Outcome measurement done well needs a baseline window, an attribution discipline, and a kill criterion for workflows that are not paying back. Done poorly, it is just adoption metrics with better names.

There is also the question of cost. AI usage carries a per-run token bill, an evaluation bill on every change, and (for agents) a cost cap that limits damage when something goes wrong. None of these are large compared to the engineering time saved when the workflow works. All of them are visible enough that a finance-aware reader will ask. Track them.

Where to start

The most concrete starter from this post: promote one personal prompt to a shared template. Pick the prompt that gets used most often (commit messages, code reviews, release notes, debugging assist), move it from someone's notes into the repository where the team versions everything else, and watch what changes when the next person on the team runs it. That is the smallest unit of the workflow shift this post argues for, and it is the step where prompts stop being individual practice and start becoming engineering assets.

The shift

The shift is from building systems to building smarter systems:

AI does not replace engineers. It changes what an engineer's leverage looks like.
The unit of value is the workflow, not the suggestion.
The discipline that made platforms operable is the same discipline that makes AI useful.
Responsible AI is not a compliance step. It is the sixth engineering discipline that lets the other five compound safely.

The series ends here, but the arc is consistent across all three posts. The disciplines that make platforms scale are the same disciplines that make AI useful. Build with discipline. Run with discipline. Evolve with discipline. The tools change. The disciplines do not.

Want to discuss? Where has AI moved the needle most in your delivery, and where has it disappointed you? Drop a comment with patterns you have seen in your environment. Every reply gets read.

Previously in this series:

Building Cloud Native Platforms That Scale: Patterns That Actually Work. Part 1 covered the design choices that make scale possible.
Running Cloud Native Platforms: Why Day 2 Decides Everything. Part 2 covered the operational disciplines that decide production outcomes.

This is the third and final post in the series.

WAR, Azure Advisor, and Us (Azure Arch Diagram Builder): Three Ways to Score an Azure Architecture

arturoqu — Thu, 21 May 2026 18:06:15 GMT

Author: Arturo Quiroga, Azure AI services Engineer - Senior Partner Solutions Architect — Microsoft

A few days ago I published From Prompt to Production: Building Azure Architecture Diagrams with AI, introducing the open-source Azure Architecture Diagram Builder. One feature got more follow-up questions than any other: the Well-Architected Framework (WAF) validation. Architects from partners and customers — many of whom already use Azure Advisor and the Well-Architected Review — wanted to know exactly what scoring algorithm we use, how it compares to Microsoft's official tools, and whether they should be using all three.

This post is that answer. It's a deep dive into how design-time WAF validation works, how Microsoft's two official WAF assessment algorithms work, and where each fits in the architecture lifecycle.

TL;DR. Microsoft ships two WAF assessment vehicles — the Well-Architected Review (questionnaire, scored from human answers) and the Azure Advisor score (healthy-resources-÷-applicable-resources weighted per subcategory, with Defender Secure Score for Security and cost-weighted math for Cost). Both require either a human filling in a form or live Azure telemetry. Our app runs at design time on a diagram, before anything is deployed, using a hybrid pipeline: a deterministic rule pre-scan followed by an LLM refinement pass. Same five WAF pillars, different lifecycle stage. Complementary, not competitive.

Why design-time validation matters

Every cost overrun, reliability gap, and security incident I've ever debugged was cheaper to fix on a whiteboard than in production. Yet most WAF tooling assumes the architecture already exists — either because there are deployed resources to scan (Advisor) or because someone has built enough of it to answer 60 specific questions about it (WAR).

That leaves a gap. Between "rough sketch" and "deployed resource group" there is no algorithmic WAF feedback loop. That's the gap the Diagram Builder fills.

Microsoft's two official WAF assessment algorithms

Before describing our approach, it's worth being precise about what Microsoft already ships, because the term "WAF assessment algorithm" can mean either of two very different things.

1. Azure Well-Architected Review (WAR) — questionnaire-based

The Well-Architected Review is a free self-assessment hosted on Microsoft Learn.

Aspect	Detail
Input	Human answers to ~60 questions mapped to the WAF pillar checklists
Workload variants	Core WAR, plus AI/ML, IoT, SAP on Azure, Azure Stack Hub, SaaS, Mission Critical
Scoring	Derived from the answers — each "no" or unanswered question subtracts from the pillar score
Output	Per-pillar maturity score + prioritized recommendations + optional Advisor integration
Improvement tracking	"Milestones" (point-in-time snapshots)
When to use	Periodic deep reviews; greenfield design baselining; brownfield audits

WAR is human-driven. The algorithm is essentially "how many of the recommended practices have you confirmed you do?" — which is exactly the right algorithm when the assessor is the workload team itself.

2. Azure Advisor Score — telemetry-based

The Advisor score is the closest thing Microsoft ships to a real, deterministic WAF algorithm. It runs continuously over your deployed Azure resources.

The math:

Pillar-specific overrides:

Security uses Microsoft Defender for Cloud's Secure Score model.
Cost weights by retail $ cost of healthy resources, plus age-of-recommendation weighting; postponed/dismissed items are removed from the denominator.
Reliability / Performance / Operational Excellence use the healthy-resources ratio above.

Key terms:

Healthy resource — a deployed resource with no open Advisor recommendation against it for that pillar.
Total applicable — resources Advisor was able to evaluate (excludes dismissed/snoozed).

Advisor is the right tool once you're in production. It cannot help you before deployment, because there is nothing to count as "healthy" or "applicable."

The missing stage: design time

Here's the lifecycle, with each tool's domain shaded:



Design / Diagram — Diagram Builder validation runs here.

Operate / Observe — Azure Advisor runs here continuously.
Periodic Review — WAR runs here, typically quarterly or at major milestones.

These three stages are sequential and complementary. Our app does not replace Advisor or WAR — it adds a feedback loop earlier in the lifecycle, where corrections are cheapest.

How design-time validation works in the Azure Architecture Diagram Builder

The validator is a two-phase hybrid pipeline: deterministic local rules first, then LLM refinement. The full source lives in three files:

src/services/architectureValidator.ts — orchestrator and prompt
src/services/wafPatternDetector.ts — topology + service rule engine
src/data/wafRules.ts — the rule knowledge base

Phase 1 — Deterministic rule pre-scan (~1 ms, no LLM)

When you click Validate Architecture, the validator runs a fully client-side rule engine against the diagram's services, connections, and groups. There are two kinds of rules:

Architecture-pattern rules

These fire when a topology anti-pattern is detected:

Pattern	Detection trigger
`single-region`	No global LB (Traffic Manager / Front Door) with ≥3 services
`single-database`	Exactly one database service, no replication signal
`no-cache`	Compute + database present, no Redis/CDN
`no-monitoring`	No Azure Monitor / App Insights / Log Analytics
`no-identity`	No Microsoft Entra ID
`no-waf`	Public web tier without WAF / Front Door / App Gateway
`direct-db-access`	An edge from a frontend service directly into a database
`no-key-vault`	4+ services and no Key Vault
`no-backup`	Database present, no Azure Backup / Recovery Services
`no-api-gateway`	2+ compute services and no APIM / App Gateway / Front Door

Service-specific rules

Every service in the in the generated Azure Architecture diagram is matched against SERVICE_SPECIFIC_RULES by normalized type — App Service, Functions, AKS, Cosmos DB, SQL Database, Storage, Key Vault, and 22 more.

The knowledge base at a glance

Metric	Count
Total rules	73
Architecture-pattern rules	10
Service-specific rules	63
Distinct Azure services covered	29
Rules tagged Reliability	18
Rules tagged Security	34
Rules tagged Cost Optimization	5
Rules tagged Operational Excellence	7
Rules tagged Performance Efficiency	9

The preliminary score

Each finding has a severity, and severity drives a fixed point deduction from a starting score of 100:

Severity	Deduction
critical	−12
high	−7
medium	−3
low	−1

Result is floored at 10 (so even a deliberately bad architecture scores at least 10) and ceilinged at 95 (no findings ≠ perfect — there's always something the model might still catch). This is the deterministic baseline before the LLM ever sees the architecture, and it's what makes the pipeline reproducible.

Phase 2 — LLM contextual refinement

The pre-scan output, the topology, and the optional natural-language description are folded into a focused prompt sent to one of seven Azure OpenAI models (GPT-5.1 through 5.4, GPT-5.x Codex variants, DeepSeek V3.2 Speciale, Grok 4.1 Fast). The system prompt gives the model explicit scoring guardrails:

Score based on what IS present, not what COULD be added.

A well-connected architecture with appropriate services should score 60–80.

Score below 50 only for critical gaps (no auth, no monitoring, single points of failure).

Findings are improvement suggestions, not reasons to penalize the score severely.

The model returns strict JSON:

{
  "overallScore": 0-100,
  "summary": "2–3 sentence assessment",
  "pillars": [
    {
      "pillar": "Reliability | Security | Cost Optimization | Operational Excellence | Performance Efficiency",
      "score": 0-100,
      "findings": [
        {
          "severity": "critical | high | medium | low",
          "category": "...",
          "issue": "...",
          "recommendation": "...",
          "resources": ["service-name-1", "service-name-2"],
          "source": "rule-based | ai-analysis"
        }
      ]
    }
  ],
  "quickWins": [ /* same shape as findings */ ]
}

Two things to call out:

Every finding is tagged rule-based or ai-analysis. That tag is the credibility lever. You can always see what the deterministic engine produced versus what the model contributed on top. If you don't trust the AI layer, you can ignore it entirely — the rule layer still stands.
The LLM is given pattern hints, not the entire rule catalog. The prompt stays small and focused, which is roughly 3–5× faster and cheaper than asking the LLM to do everything from scratch.

What the user sees

On every run the modal reports:

Overall WAF score (0–100)
Per-pillar score × 5 (0–100 each)
Severity breakdown — counts of critical / high / medium / low across all findings
Quick wins — high-impact, low-effort items the model surfaces separately
Hybrid metadata — local findings count, patterns detected, KB rules used, preliminary score, local elapsed ms
AI metrics — model used, reasoning effort, prompt/completion/total tokens, elapsed time
App Insights telemetry — an Architecture_Validated event with model, overall score, finding count, elapsed time

Worked example

Take this prompt, which I've used in demos with partners:

"A multi-region web application: Azure Front Door in front of two App Service instances in West US 2 and East US 2, both reading from an Azure SQL Database with geo-replication, with Application Insights for telemetry. No Entra ID, no Key Vault."

After generation, Validate Architecture runs:

Phase 1 — pre-scan (deterministic), ~1 ms

Patterns detected: no-identity, no-key-vault
Findings produced: 8 (1 critical, 1 high, 3 medium, 3 low)
Preliminary score: 100 − 12 − 7 − (3×3) − (1×3) = 69

Phase 2 — LLM refinement, ~6–9 s depending on model

The model accepts the two pattern hints, validates them in context, and adds three more findings of its own:

Finding	Source	Pillar	Severity
No Microsoft Entra ID for authentication	rule-based	Security	critical
No Key Vault for secret management	rule-based	Security	high
App Service slots not used for safe deploys	ai-analysis	Operational Excellence	medium
SQL DB geo-replication present but RTO/RPO not documented	ai-analysis	Reliability	medium
No CDN for static assets behind Front Door	ai-analysis	Performance Efficiency	low

Final scores returned by the model:

Pillar	Score
Reliability	78
Security	52
Cost Optimization	80
Operational Excellence	70
Performance Efficiency	75
Overall	71

The Security score is the lowest because two of the highest-severity findings landed there — exactly what a human reviewer would flag first.

Multi-model comparison

Because the deterministic floor is identical across runs, the Validation Comparison view becomes a fair shootout of what each LLM adds on top of the same baseline. The same diagram is scored by all seven models, and the UI surfaces:

Overall score per model
Per-pillar score per model
Severity-count deltas
Number of ai-analysis findings each model contributed
Quick wins each model identified

This is genuinely useful for two reasons. First, it shows that LLM scores vary — typically by ±5–10 points on the same architecture — which is exactly why we publish the rule-based vs ai-analysis tag. Second, it lets architects pick the model whose review style matches their own.

How we align with Microsoft's algorithms

Alignment point	What it means
Same five pillars	Identical names and scope to the official WAF
Same source material	Rules derived from WAF docs and Azure Architecture Center service guides
Severity-graded findings	Map conceptually to Advisor's high/medium/low impact recommendations
Per-pillar + overall scoring	Mirrors WAR/Advisor output shape, so the results feel familiar

Where we deliberately differ — and why

Concern	Microsoft	Diagram Builder	Why we differ
Needs deployed resources	Advisor: yes	No — works on a diagram	We're a design-time tool; the architecture doesn't exist yet
Needs human Q&A	WAR: yes	No — derived from the diagram	One-click validation inside the design flow
Healthy/Applicable ratio	Advisor: yes	No	No resource-health signal exists pre-deployment
Subcategory fixed weights	Advisor: yes	No explicit weights	Severity is the de-facto weight (12/7/3/1)
Defender Secure Score for Security	Advisor: yes	No	Defender requires deployed resources
Cost-weighted scoring	Advisor: yes	No (separate Cost Estimation feature)	Cost is a separate pipeline in our app
AI/LLM refinement	Neither	Yes	Catches context-specific issues a static catalog misses, and explains findings in natural language
Multi-model comparison	Neither	Yes	Lets architects see scoring variance across models

Honest limitations

I'd rather you hear these from me than discover them in production:

LLM scores drift. ±5–10 points across models on the same diagram is normal. Treat the score as directional, the findings as actionable. The rule-based tag is your anchor.
No live telemetry. We can't know if your App Service is actually using availability zones — only that you have App Service in the diagram. Advisor will tell you the truth post-deployment.
Generic ruleset. No specialized workload branches yet (AI/ML, IoT, SAP, SaaS). WAR has those.
No milestone tracking. Each validation run is independent. Compare runs manually using the Validation Comparison view.
Rule coverage is finite. 29 services and 73 rules is a strong start but not exhaustive — the LLM layer exists in part to compensate for that gap.

How to use all three together

A lifecycle that actually works:

Design — Use the Diagram Builder to sketch the architecture and validate at design time. Iterate until the per-pillar scores look reasonable and the critical/high findings are addressed.
Deploy — Generate Bicep from the diagram, deploy, and let Azure Advisor start scoring real resources.
Operate — Use Azure Advisor continuously. Use Defender Secure Score for security posture.
Periodic review — Run a Core WAR every quarter or at major milestones to capture the things only humans know (business context, tradeoffs, planned debt).

None of these three replace the others. They cover different stages of the same loop.

What's next

A few things on the roadmap I'd love feedback on:

Milestone tracking so design-time scores can be compared over time the way WAR milestones work.
Workload-specific rulesets mirroring WAR's branches — starting with AI/ML.
Direct Advisor handoff — once a diagram is deployed, surface the corresponding Advisor recommendations in the same UI to close the loop.

Try it, fork it, tell me where it's wrong

Live app: https://aka.ms/diagram-builder
Source: github.com/Arturo-Quiroga-MSFT/azure-architecture-diagram-builder

Useful references:

If you're a partner or customer architect who's already living in Advisor and WAR, I'd genuinely value your reaction — does the design-time stage feel like a real gap to you, or are you already covering it some other way? Open an issue on the repo or reply on LinkedIn.

Posted on the Azure Architecture Blog · Comments and issues welcome on the repo.

From Prompt to Production: Building Azure Architecture Diagrams with AI

arturoqu — Fri, 22 May 2026 18:35:07 GMT

Author: Arturo Quiroga, Senior Partner Solutions Architect — Microsoft

Cloud architects spend significant time translating ideas into architecture diagrams. They toggle between Visio, draw.io, pricing calculators, and documentation. According to the 2024 Stack Overflow Developer Survey, 61% of developers spend more than 30 minutes a day searching for answers or solutions, time lost to context-switching rather than design. What if you could describe your architecture in plain English and get a diagram, cost estimate, and deployment guide in minutes?

The Challenge: Fragmented Architecture Workflows

Designing Azure architectures today typically involves multiple disconnected steps:

Sketch the architecture in a diagramming tool
Look up official Azure icons and drag them into place
Research pricing across regions using the Azure Pricing Calculator
Validate the design against the Well-Architected Framework (WAF)
Write deployment documentation and Infrastructure as Code templates
Compare alternative designs manually

Each step lives in a different tool, and keeping them in sync as designs evolve is costly. The Azure Architecture Diagram Builder brings these workflows together in a single browser-based experience.

How It Works

Describe your architecture in natural language, for example "A HIPAA-compliant healthcare platform with FHIR APIs, event-driven processing, and multi-region disaster recovery", and the AI generates a diagram with grouped services, data flow connections, and logical organization.

Figure 1. Enter a natural-language prompt describing your architecture. Curated example prompts help you get started, and you can optionally upload an existing diagram for the AI to analyze.

The tool uses Azure OpenAI to power generation across multiple models, enabling you to choose the model that best fits your scenario — from fast iterations to deeper reasoning.

Key Features

AI-Powered Architecture Generation

Describe what you need in plain English, and the AI creates an architecture diagram with:

714 official Azure service icons across 29 categories
Smart grouping: services are logically organized (Frontend, Backend, Data, Security)
Data flow connections: labeled edges showing how data moves through the system
13 curated example prompts: from simple web apps to complex enterprise scenarios like Zero Trust networks, Industrial IoT with 5,000+ sensors, and global multiplayer gaming backends

Figure 2. A generated industrial IoT architecture. Top: the clean diagram view as initially produced. Bottom: the same diagram with per-service monthly cost overlays toggled on, plus a running subscription total in the toolbar.

Architecture Image Import

Already have an architecture on a whiteboard or in a screenshot? Upload the image and let the AI analyze it, mapping services to official Azure icons and recreating the architecture as an editable, interactive diagram.

Figure 3. Upload a photo of a whiteboard sketch (top-right reference panel) and the AI recreates it as an editable diagram with official Azure service icons and labeled data flow connections.

ARM Template Import

Import existing ARM templates to visualize your current infrastructure. The AI parses resource definitions and dependencies, groups related resources into logical layers, and produces a meaningful diagram of what you actually have deployed — a fast way to document an inherited environment or sanity-check a template before deployment.

Figure 4. ARM template import in action. Top: the parser status banner while resources and dependencies are being analyzed. Bottom: the resulting diagram, with resources auto-grouped into logical layers (Web Tier, Data Layer, Container Platform, Observability & Logging) and a Generated from: ARM Template badge linking the diagram back to its source file.

Well-Architected Framework Validation

Validate your architecture against all five WAF pillars — Security, Reliability, Performance Efficiency, Cost Optimization, and Operational Excellence. The validator provides:

An overall WAF score with pillar-level breakdowns
Specific findings with severity levels
Actionable recommendations you can select and apply

Select the recommendations you agree with, and the AI regenerates an improved architecture incorporating those changes.

Figure 5. WAF validation results showing the overall score, per-pillar breakdowns, and individual findings with severity badges. Tick the recommendations you want and the AI rebuilds the diagram with those changes applied.

Multi-Model Comparison

Run the same architecture prompt through multiple AI models side-by-side and compare:

Architecture Comparison: service counts, connection counts, groups, token usage, and latency
Validation Comparison: WAF scores across models, severity breakdowns, and finding counts
Apply Winner: pick the best result and apply it to the canvas with one click
Present Critique: a talking avatar narrates the AI-generated ranking with live closed captions

Figure 6. Multi-model comparison. Top: select the models and reasoning effort, then enter the prompt. Bottom: side-by-side results across all selected models with service counts, latency, token usage, and Fastest / Cheapest / Most Thorough badges.

Multi-Region Cost Estimation

Get cost estimates from the Azure Retail Prices API across 8 Azure regions: East US 2, Australia East, Canada Central, Brazil South, Mexico Central, West Europe, Sweden Central, and Southeast Asia. Features include:

Color-coded cost legend (green / yellow / red thresholds)
SKU and tier information for each service
Export options: CSV, JSON, plain-text summary, and an analysis report with top cost drivers, Reserved Instance flags, and a ranked multi-region comparison table

Figure 7. The cost legend overlay shows per-service pricing with color-coded thresholds. The region selector in the toolbar lets you re-price the entire architecture in any of eight Azure regions.

Deployment Guide Generation with Bicep

Generate step-by-step deployment documentation including:

Prerequisites and Azure resource requirements
Step-by-step deployment instructions
Bicep templates for each service (Infrastructure as Code)
Post-deployment verification steps
Security configuration recommendations

Figure 8. Each generated Deployment Guide opens with the architecture name, an estimated deployment time, and a prerequisites checklist covering subscription roles, CLI versions, Microsoft Entra ID permissions, and region requirements, followed by numbered, copy-ready deployment steps.

Figure 9. The Infrastructure as Code section produces a main.bicep orchestrator plus a per-service module (Log Analytics, Key Vault, Cosmos DB, SQL Database, Event Hubs, Azure Functions, and more). The Download All Templates button packages everything into a ready-to-deploy folder.

Workflow Animation & Avatar Presenter

Visualize how data flows through your architecture with step-by-step animations that highlight services on the canvas as each step plays. When the Azure Speech Service is configured, a photorealistic talking avatar can narrate the workflow or present model comparison results, with live word-by-word closed captions in a draggable, resizable panel.

Figure 10. A workflow step is highlighted on the canvas as the Avatar Presenter narrates that step. Live word-by-word closed captions appear in a draggable, resizable panel, useful for accessibility and stakeholder demos.

Export Options

Figure 11. A single-slide PowerPoint export, available in dark or light theme, ready to drop straight into a stakeholder deck.

Format	Use Case
PNG	Documentation, presentations
SVG	Scalable vector graphics
PPTX	Single PowerPoint slide (dark or light theme)
Draw.io	Edit in diagrams.net
JSON	Backup, version control
CSV / ZIP	Cost analysis with multi-region comparison

Highlights

The Azure Architecture Diagram Builder unifies the architecture design lifecycle in a single tool:

End-to-end workflow: from natural-language description to deployable Bicep templates without tool switching
Official Azure icons: 714 icons across 29 categories, mapped directly from the Azure service catalog
Live pricing: queries the Azure Retail Prices API at design time rather than relying on static estimates
WAF-integrated validation: architectural best practices built into the design loop rather than applied after the fact
Multi-model flexibility: choose the AI model that best suits each task, with fast models for iteration and reasoning models for complex designs
Open source: the source code is available for customization and contribution

One-Command Deploy with Azure Developer CLI

The fastest way to get your own instance running is with azd:

# Install azd (once)
brew tap azure/azd && brew install azd   # macOS
winget install microsoft.azd             # Windows

# Clone, configure, and deploy
git clone https://github.com/Arturo-Quiroga-MSFT/azure-architecture-diagram-builder
cd azure-architecture-diagram-builder
azd auth login
azd env set AZURE_OPENAI_ENDPOINT "https://your-resource.openai.azure.com/"
azd env set AZURE_OPENAI_API_KEY  "your-key"
azd up   # Provisions infrastructure + builds + deploys (~8 min)

azd up provisions the following via Bicep:

Resource	Purpose
Azure Container Registry	Stores the Docker image
Azure Container Apps	Runs the app (nginx + token server)
Log Analytics + Application Insights	Monitoring and telemetry
Azure Speech (S0)	Avatar Presenter (optional, keyless auth via managed identity)

Try It Today

The Azure Architecture Diagram Builder is available now:

Live demo: https://aka.ms/diagram-builder
Source code: GitHub repository
Documentation: See the Getting Started Guide for detailed setup instructions

We welcome feedback and contributions. Use the GitHub Issues page to report bugs, suggest features, or share your experience.

Tags: artificial intelligence · application · apps & devops · well architected · infrastructure

How to Secure Azure Databricks without Public Exposure using WAF + Private Endpoints

FaizaanMerchant — Mon, 11 May 2026 23:37:12 GMT

While first thing that comes up to mind is that lets configure IP Access List with keeping Azure Databricks in Hybrid Connectivity. This approach is technically doable, but not the best approach for organizations which follows Zero Trust Architecture Framework.

This is where organizations has to design a solution which follows CAF Principles and is fully secured with Azure Application Gateway with Web Application Firewall (WAF) combined with Private Endpoints (Azure Private Link) becomes critical to enforce Zero Trust Architecture.

In this blog, I’ll walk through:

Why Zero Trust is essential for Databricks
Traffic Flow For Securing Databricks
Architecture Components
Key Considerations
Conclusion

Why Zero Trust for Azure Databricks?

Azure Databricks is a SaaS-managed service—but in many enterprise environments, data sensitivity demands full network isolation.

Azure Private Link enables organizations to:

Eliminate public internet exposure
Ensure traffic remains within private networks
Reduce risk of data exfiltration

Additionally, organizations prefer:

Controlled access via corporate network (VPN/ExpressRoute)
Full audit and inspection of inbound traffic
Strong compliance alignment (RBI, SEBI, PCI-DSS, etc.)

This drives the need for private-only access models, where public access is completely disabled.

Traffic Flow For Securing Databricks –

Organizations following Zero Trust Architecture often has concerns about accessibility in secured manner, while majority being accessed through the intranet but there are scenarios where in subset of users / partners / vendor needs access which are outside of your organization network and they cant be incorporated in the organization network, instead they have to access over Internet.

This becomes very challenging to consider the flow and secure your connectivity. Below chart would explain how the traffic will flow happen entirely for Databricks and how we will secure it –

External User Access (Red Flow)
1. External user connects via internet
2. Request hits Application Gateway (WAF)
3. Traffic is:
  - Inspected
  - Validated
4. Routed to Databricks via Private Endpoint

Ensures all external traffic is secured and inspected

Internal User Access (Green Flow)
1. Internal user connects via VPN / ExpressRoute
2. Traffic enters Hub VNet
3. Routed directly to:
  - Databricks Private Endpoint (in Spoke VNet)

Ensures private, low-latency, secure access without internet exposure

Architecture Components –

In this section of the blog, I’ll go through the architecture and the components that are required for design considerations.

As we are focused on Zero Trust Architecture, we will consider having a secured Hub & Spoke networking model.

Key Services & Considerations –

Application Gateway with WAF
WAF with Custom Rules
Secured Internal Network Connectivity
Databricks Workspace – Public Disabled
Databricks Workspace – Private Endpoint Enabled

1. Application Gateway with WAF

Acts as the single entry point to Databricks, deployed in the Hub VNet.
Provides SSL termination, routing, and traffic inspection, ensuring all access is centralized and backend services are not directly exposed.

2. WAF with Custom Rules

Protects against OWASP Top 10 threats and runs in Prevention mode.
Custom rules enable restriction based on IP, geo-location, or request patterns, giving fine-grained security and compliance control for specific networks only.

3. Secured Internal Network Connectivity

External users → access via WAF (inspected traffic)
Internal users → access via VPN/ExpressRoute (private flow)

Supported by NSGs, VPN Gateway, and firewall controls to enforce secure, segmented access.

4. Databricks Workspace – Public Access Disabled

Public access is completely disabled, ensuring the workspace is not exposed to the internet and only accessible via approved private paths.

5. Databricks Workspace – Private Endpoint Enabled

Exposed via Private Endpoint in the Spoke VNet, mapped to a private IP with DNS integration.
Ensures traffic remains within Azure backbone and is accessible only through authorized networks.

Key Considerations

Application Gateway Listener should be configured with FQDN with which external users will be accessing the Databricks.
Listener FQDN (Example - databricks.example.com) will be resolving to Public Ip of Application Gateway (Frontend Ip).
SSL Certificate & Public DNS Configurations has to be considered for the configuration to work.
Backend Pool for Application Gateway should be configured with FQDN of Databricks Workspace. Note – If you add Private Ip of workspace it will be shown as health backend, but web request will fail
Ensure that DNS Resolution for Private Endpoint is configured appropriately.
Application Gateway should be able to resolve FQDN of Databricks workspace to Private Endpoint.
End-to-End SSL Configuration at Application Gateway Level.
Custom Rules for WAF should be configured for allowing particular Public Network while others should be blocked.

Conclusion

As organizations continue adopting Azure Databricks for critical analytics and AI workloads, securing access becomes non-negotiable. A traditional approach with hybrid network model with IP Access List is no longer sufficient for enterprise-grade security.

By combining Application Gateway (WAF) with Private Endpoints, and leveraging a Hub-Spoke architecture, we can achieve a true Zero Trust design—where every request is validated, inspected, and routed securely without exposing backend services.

This architecture not only reduces the attack surface but also ensures compliance with strict regulatory standards, while still enabling seamless access for both internal and external users.

Useful Links -

Azure WAF Custom Rules - https://learn.microsoft.com/en-us/azure/web-application-firewall/ag/custom-waf-rules-overview

Azure Databricks Private Link - https://learn.microsoft.com/en-us/azure/databricks/security/network/concepts/private-link#choose-the-right-private-link-implementation

Azure Databricks Architecture - https://learn.microsoft.com/en-us/azure/databricks/security/network/concepts/architecture

Azure Databricks VNet - https://learn.microsoft.com/en-us/azure/databricks/security/network/classic/vnet-inject#overview

Azure Hub & Spoke Model - https://learn.microsoft.com/en-us/azure/architecture/networking/architecture/hub-spoke

Configure DNS forwarding for Azure NetApp Files

mkachare — Mon, 11 May 2026 04:20:10 GMT

This post has been written with the collaboration of Rizul Khanna

Applies to: Azure NetApp Files — SMB, dual-protocol, and NFSv4.1 Kerberos volumes deployed in hub-spoke or Azure Virtual WAN topologies using an external private DNS forwarder.

Overview

Azure NetApp Files (ANF) has a hard dependency on DNS for all volume types that integrate with Active Directory (AD): SMB, dual-protocol (SMB + NFS), and NFSv4.1 with Kerberos. Unlike most Azure PaaS services, ANF does not use Azure Private Link and has no privatelink.* zone. Its volumes attach directly to a delegated subnet, and their hostnames are registered into AD-integrated DNS via Secure Dynamic DNS (DDNS). This architecture means DNS design decisions for the ANF delegated subnet are fundamentally different from those that apply to storage accounts, SQL databases, or other services that use private endpoints.

This article documents what DNS resolution ANF requires, how to correctly configure an external private DNS forwarder in hub-spoke and Virtual WAN deployments, and the specific undocumented requirements that cause volume creation failures and SMB permission errors in practice. Several requirements covered here are not present in the official Azure NetApp Files documentation and have been identified through field support cases.

ANF does not inherit the VNET DNS server setting. It queries only the two DNS server IPs configured in the Active Directory connection on the NetApp account. This is not documented in the ANF networking or AD connection articles. The VNET DNS server setting is irrelevant to ANF volume creation and AD join behavior — only the AD connection DNS IPs matter.

Architecture overview

The following diagram shows the two separate DNS paths that must be configured when ANF is deployed in a hub-spoke or Virtual WAN topology with an external private DNS forwarder. The client resolution path (VNET DNS setting) and the ANF internal resolution path (AD connection DNS fields) are distinct and must not be conflated.

Note:

ANF AD connection DNS IPs must point to the external DC IPs directly — not to the private DNS forwarder. The forwarder handles client-side resolution only and must have both forward and reverse rulesets for the AD domain.
Figure 1: DNS resolution paths for ANF with an external private DNS forwarder. Client VMs use the forwarder (VNET DNS setting). ANF uses the external AD DC IPs directly (AD connection DNS fields). Both forward and reverse lookup rulesets are required on the forwarder.

What DNS must provide for Azure NetApp Files

Outbound resolution — ANF querying DNS

ANF must be able to resolve the following records from the DNS IPs specified in the AD connection:

AD domain controller SRV records: _ldap._tcp.<site>._sites.dc._msdcs.<domain>, _kerberos._tcp.dc._msdcs.<domain>, and site-scoped equivalents

Kerberos KDC records: _kerberos._tcp.<domain> and _kerberos-master._tcp/udp.<domain>

DC A records: Forward lookup for each DC hostname to its IP

PTR (reverse) records: IP-to-hostname for each DC — required for dual-protocol volume creation, NFSv4.1 Kerberos, LDAP-over-TLS certificate validation, and NTFS ACL operations on SMB shares.

Note:

_kerberos-master._tcp and _kerberos-master._udp SRV records are not created automatically by Active Directory DNS. They must be added manually in the DNS zone. Their absence causes Kerberos failures that do not clearly identify DNS as the root cause. This requirement is not documented in any ANF article.

ANF performs Secure Dynamic DNS (DDNS) using GSS-TSIG to register SMB and dual-protocol volume hostnames in AD DNS. This requires that the DNS IPs in the AD connection belong to Microsoft AD-integrated DNS servers. External private DNS forwarders (Infoblox, BIND, Unbound, and similar appliances) do not support GSS-TSIG and will silently discard DDNS updates — volume hostnames will not appear in DNS and SMB mounts will fail. No error is surfaced in the ANF portal or activity log when DDNS is silently dropped.

Inbound resolution — clients resolving ANF hostnames

SMB and dual-protocol volumes are accessed via a hostname of the form <smb-prefix>-XXXX.<ad-dns-domain>, where the four-character suffix is assigned by ANF and cannot be overridden. Clients must resolve this hostname to the volume IP via the VNET DNS server setting, which in enterprise environments points to the external private DNS forwarder. The forwarder must have a forward lookup ruleset for the AD domain pointing to the external DC IPs. NFSv3 mounts use the volume IP directly and do not require hostname resolution.

Note:

NFSv3 volume creation success does not indicate SMB readiness. NFSv3 mounts use the volume IP directly and require no AD join, Kerberos exchange, or reverse DNS. SMB and dual-protocol volumes require all three. Using NFSv3 as a connectivity proxy during SMB troubleshooting produces false confidence. This distinction is not documented in ANF troubleshooting guidance.

Configuring the external private DNS forwarder

The two DNS paths — client resolution vs ANF internal

In environments using an external private DNS forwarder (Infoblox, BIND, Windows DNS VM, or similar appliance), two distinct DNS paths must be kept separate. The VNET DNS server setting governs client resolution of ANF SMB hostnames and should point to the external forwarder. The ANF AD connection DNS fields govern ANF's own resolution of DCs and DDNS registration and must point directly to writable Microsoft AD-integrated DC IPs.

DNS Path	Used By	Correct Target
Client resolution (VNET DNS setting)	VMs, Citrix, application servers resolving ANF SMB hostnames.	External private DNS forwarder, which forwards AD zone queries to external DC IPs
ANF internal resolution (AD connection DNS fields)	ANF service — DDNS, Kerberos, LDAP, SRV lookup	Writable AD-integrated external DC IPs directly — not the forwarder

Required rulesets on the external private DNS forwarder

The external private DNS forwarder must have both of the following rulesets configured. Missing either one produces failures that are difficult to diagnose because forward DNS tests pass while the actual failure occurs in a different path.

References:
Understanding Private DNS resolver endpoints & rulesets
How to create Private Reverse DNS records

Forward lookup ruleset

Forward all queries for the AD domain to the external DC IPs:

Zone: ad.contoso.com
Targets: <DC-IP-1>:53, <DC-IP-2>:53 (writable external DC IPs)

Reverse lookup ruleset (most commonly missing)

Forward reverse lookup queries for the DC IP range to the external DC IPs:

Zone: <reverse-octets>.in-addr.arpa.
Targets: <DC-IP-1>:53, <DC-IP-2>:53 (same external DC IPs)

Critical:

The reverse lookup ruleset is the most commonly missing configuration item and causes a failure that forward DNS tests do not detect. Without it, Windows clients cannot resolve DC IPs to hostnames. This produces the following error when provisioning NTFS permissions on an ANF SMB share: 'The program cannot open the required dialog box because it cannot determine whether the computer named is joined to a domain.' All connectivity tests pass. Forward DNS passes. The volume was created successfully. Only the reverse lookup fails — and only when NTFS ACL operations are attempted. This failure mode and its root cause are not documented in any ANF article.

GSS-TSIG constraint — why the forwarder cannot be in the ANF AD connection

External private DNS forwarders (including Infoblox, BIND, Unbound, and third-party appliances) do not support GSS-TSIG, the protocol ANF uses to securely register SMB volume hostnames into AD DNS. If a forwarder IP is placed in the ANF AD connection DNS fields, ANF sends DDNS update packets to the forwarder, which discards them silently. The volume hostname never appears in DNS. Clients cannot mount by name. No error is returned in the ANF portal.

The correct design: external DC IPs in the ANF AD connection, external private DNS forwarder as the VNET DNS server for clients only.

Role of 168.63.129.16

168.63.129.16 is the Azure-provided internal resolver. It should be configured as the upstream forwarder target on the external private DNS forwarder for all queries not covered by AD or other conditional forwarders. This allows Azure-hosted DNS zones (such as any privatelink.* zones linked to your VNET) to resolve correctly through the forwarder.

IMPORTANT:
168.63.129.16 must never be placed in the ANF AD connection DNS fields. It is not AD-aware, cannot answer SRV queries for your domain, cannot accept DDNS updates, and is unreachable from on-premises over ExpressRoute or VPN. Its correct position is as an upstream target on the external private DNS forwarder — not in ANF's AD connection. This is not stated anywhere in the ANF documentation set.

DNS forwarder pattern comparison

Pattern	DDNS Support	Reverse DNS	Ops overhead	Best Fit
AD DNS on DCs + upstream 168.63.129.16	Yes, Native	Yes, if reverse zones on DC's	Medium	Default; simplest topology
External private DNS forwarder (VMs only) + external DC IPs in ANF AD connection	Yes (ANF bypasses forwarder)	Yes, if reverse ruleset on forwarder.	Medium	Enterprise with existing DNS infrastructure
External DNS forwarder in ANF AD connection (incorrect)	No — DDNS silently dropped	N/A — config is wrong	High	Not supported
Azure DNS Private Resolver in ANF AD connection (incorrect)	No — DDNS not accepted	N/A — config is wrong	High	Not supported

Azure Virtual WAN considerations

When ANF is deployed in a spoke VNET connected to an Azure Virtual WAN hub, a routing requirement applies that directly causes Kerberos and LDAP failures — which appear to be DNS or AD failures — when not addressed. This is one of the most common misdiagnoses in ANF deployments using Virtual WAN with NVA inspection.

ANF subnet prefix must be in Routing Intent additional prefixes

Azure Virtual WAN with Routing Intent routes private traffic through an NVA or Azure Firewall in the hub. For return traffic from external AD domain controllers to reach the ANF data plane IP, the hub must have an explicit routing entry for the ANF delegated subnet prefix. If the ANF delegated subnet is a /26 inside a larger VNET (/21 or /16), the broader VNET prefix alone is not sufficient — the specific /26 must be added explicitly.

Action Required:
In Azure Virtual WAN: Hub -> Routing -> Routing Intent -> Private Traffic -> Additional Prefixes. Add the ANF delegated subnet prefix (for example, 10.x.x.0/26) explicitly. Without this, Kerberos and LDAP reply traffic from external domain controllers is dropped before reaching the ANF data plane. The symptom is TCP port 88 connects succeeding followed by KRB5_KDC_UNREACH — which looks like a Kerberos or DNS problem but is a routing problem.

Use availability zone switching to surface detailed error messages

When ANF volume creation fails with the generic 'context deadline exceeded' error from the XMLrequest_filer endpoint, the error does not identify root cause. Redeploying the volume to a different availability zone (AZ1 to AZ2, or AZ2 to AZ3) forces a different backend assignment and consistently produces a more descriptive error that distinguishes routing failures (KRB5_KDC_UNREACH) from Kerberos authentication failures, DNS lookup failures, and LDAP errors.

TIP:
If the detailed error shows 'Successfully connected to ip <DC-IP>, port 88 using TCP' followed by 'Cannot contact any KDC for requested realm', the outbound path works but reply packets are dropped — this is a routing problem, not a DNS or Kerberos problem. Check vWAN Routing Intent, NVA firewall rules, and UDRs for the ANF subnet prefix. This diagnostic technique is not documented in ANF troubleshooting guidance.

Required DNS records

The following records must exist in the AD DNS zone served by the external DC DNS servers and must be resolvable from the ANF delegated subnet. Records marked * are not created automatically by AD DNS and must be added manually.

Forward lookup zone

Record	Type	Notes
_ldap._tcp.dc._msdcs.<domain>	SRV	Domain-wide DC discovery
_kerberos._tcp.dc._msdcs.<domain>	SRV	Domain-wide KDC discovery
_ldap._tcp.<site>._sites.dc._msdcs.<domain>	SRV	Site-scoped — preferred when AD site is specified in ANF
_kerberos._tcp.<site>._sites.dc._msdcs.<domain>	SRV	Site-scoped Kerberos
_kerberos-master._tcp.<domain> *	SRV	NOT auto-created by AD DNS — must be added manually
_kerberos-master._udp.<domain> *	SRV	NOT auto-created by AD DNS — must be added manually
<dc-hostname>	A	Forward A record for each external DC in the AD site
<anf-smb-hostname>	A	Registered by ANF via DDNS — must not be scavenged or blocked

Reverse lookup zone (<reverse-octets>.in-addr.arpa)

Record	Type	Notes
PTR for each external DC IP	PTR	Required for dual-protocol, NFSv4.1 Kerberos, LDAP-over-TLS, and NTFS ACL operations
PTR for each ANF volume IP *	PTR	Required for NFSv4.1 Kerberos reverse-lookup clients — create manually or via DDNS if supported

How ANF internally fails when reverse DNS is missing

When the reverse DNS ruleset is absent from the external private DNS forwarder, the failure does not surface as a DNS error in the ANF portal. Instead, it propagates through ANF's internal security daemon (secd) and presents as a generic InternalServerError or a Kerberos authentication failure. Understanding the internal failure chain explains why reverse DNS is non-negotiable and why the symptom is so misleading.

The secd service list mechanism

ANF uses an internal process called secd (Security Daemon) to manage all Active Directory communication — Kerberos ticket exchange, LDAP binds, and DC discovery. secd maintains a service list of discovered DCs. When a DC communication attempt fails for any reason, secd marks that DC as UNUSABLE and records a forgive time after which it will retry. If all DCs in the service list are simultaneously marked UNUSABLE, secd returns RESULT_ERROR_SECD_NO_SERVER_AVAILABLE, which propagates to the portal as InternalServerError.

The reverse PTR lookup inside secd

A critical and undocumented behavior: before secd completes a SASL/GSSAPI bind to an LDAP server, it performs a reverse PTR lookup of the DC's IP address. This lookup is used to validate the DC identity as part of Kerberos mutual authentication. If the PTR lookup fails — because the external private DNS forwarder has no reverse ruleset for the DC IP range — secd logs the failure and marks that DC UNUSABLE immediately, even though TCP connectivity on ports 88 and 389 succeeded.

The following is the exact failure sequence from ANF backend logs when reverse DNS is absent:

Stage 1 — TCP connects succeed, PTR lookup fails

Successfully connected to ip 10.x.x.60, port 389 using TCP
Entry for host-address: 10.x.x.60 not found in the current source: FILES
Source: DNS unavailable. Entry for host-address: 10.x.x.60 not found in any of the available sources

secd successfully opens the TCP connection to the DC on port 389 (LDAP), then immediately attempts a reverse lookup of that IP. The forwarder has no reverse ruleset, so DNS returns NXDOMAIN. secd logs 'DNS unavailable' and proceeds to mark the DC UNUSABLE.

Stage 2 — GSSAPI bind fails as a consequence

Unable to SASL bind to LDAP server using GSSAPI: Local error
Unable to connect to LDAP (Active Directory) service on dc01.ad.contoso.com (Error: Local error)

Because the PTR lookup failed, secd cannot complete the GSSAPI mutual authentication context. The 'Local error' is not a Kerberos configuration problem — it is the direct result of the identity validation step failing due to missing reverse DNS.

Stage 3 — All DCs marked UNUSABLE

10.x.x.27 UNUSABLE  Wed Apr  8 00:19:24 2026
10.x.x.28 UNUSABLE  Wed Apr  8 00:19:24 2026
10.x.x.29 UNUSABLE  Wed Apr  8 00:19:25 2026
10.x.x.30 UNUSABLE  Wed Apr  8 00:19:25 2026 
... (all DCs in the service list)

secd cycles through every DC in the discovered service list. Each DC fails the same PTR lookup. Each is marked UNUSABLE. Once the list is exhausted:

Stage 4 — Service list exhausted, error propagates

No servers in the service list which aren't marked bad
Unable to select any server in the current serviceList
RESULT_ERROR_SECD_NO_SERVER_AVAILABLE:6940
No servers available for MS_LDAP_AD, domain: ad.contoso.com

This internal error code propagates up through the ANF volume creation stack and is presented to the operator as the generic 'context deadline exceeded (Client.Timeout exceeded while awaiting headers)' error in the portal. The actual cause — missing PTR records — is completely obscured.

Stage 5 — Kerberos pre-auth error (secondary, misleading)

Received error from KDC: -1765328359 / Additional pre-authentication required

This Kerberos error code (KRB5KDC_ERR_PREAUTH_REQUIRED) appears in logs and can mislead investigation toward Kerberos configuration, encryption type mismatches, or clock skew. It is a downstream consequence of the failed PTR-based GSSAPI context — not a root cause. Chasing this error without first verifying reverse DNS is a common and time-consuming dead end.

KEY INSIGHT:
The complete failure chain is: missing reverse PTR ruleset on DNS forwarder → secd PTR lookup returns DNS unavailable → GSSAPI mutual auth cannot complete → DC marked UNUSABLE → all DCs exhausted → InternalServerError at portal. TCP connectivity on ports 88 and 389 succeeds at every stage. Only the PTR lookup fails. This is why all standard connectivity tests pass and the issue remains invisible until reverse DNS is specifically tested.

Why this failure is invisible to standard troubleshooting ?

Standard ANF DNS troubleshooting checks forward SRV records, forward A record resolution, and TCP port connectivity to DCs. All of these pass when only the reverse ruleset is missing. The secd PTR lookup is an internal step that occurs after TCP connectivity is confirmed and is not tested by any of the standard nslookup or Test-NetConnection commands used during initial validation. The only reliable way to surface this failure without access to backend logs is to explicitly test reverse PTR resolution from the ANF VNET — as documented in the verification section below.

Verify DNS configuration

Run the following commands from a test VM in the same VNET as the ANF delegated subnet. Use the external private DNS forwarder IP for client-side tests and an external DC IP for ANF-side tests.

Forward SRV lookup — site-scoped

 nslookup -type=SRV _ldap._tcp.<SITE>._sites.dc._msdcs.<domain> <forwarder-IP>
 nslookup -type=SRV _kerberos._tcp.<SITE>._sites.dc._msdcs.<domain> <forwarder-IP>

Forward SRV lookup — domain-wide

 nslookup -type=SRV _ldap._tcp.dc._msdcs.<domain> <forwarder-IP>
 nslookup -type=SRV _kerberos._tcp.dc._msdcs.<domain> <forwarder-IP>

Reverse PTR lookup — use the external forwarder IP

 nslookup <external-DC-IP> <forwarder-IP>

Expected output:

 Server:  <forwarder-IP>
 <reverse-arpa>  name = dc01.ad.contoso.com.

If reverse lookup returns NXDOMAIN or times out while forward lookup succeeds, add the reverse DNS ruleset to the external private DNS forwarder. This is the most common cause of NTFS permission failures after a volume is successfully created.

Port connectivity from ANF VNET

Test-NetConnection -ComputerName <external-DC-IP> -Port 88    # Kerberos
Test-NetConnection -ComputerName <external-DC-IP> -Port 389   # LDAP
Test-NetConnection -ComputerName <forwarder-IP>  -Port 53     # DNS

Common issues and resolutions

Symptom	Likely cause	Resolution
InternalServerError: context deadline exceeded (XMLrequest_filer)	Generic ANF backend timeout — routing or Kerberos root cause not visible at this level	Switch deployment availability zone for a detailed error. Check vWAN Routing Intent for ANF /26 prefix.
KRB5_KDC_UNREACH — TCP port 88 connects succeed but auth fails	Return traffic from external DCs dropped before reaching ANF NIC — routing issue, not DNS	Add ANF subnet /26 to vWAN Hub Routing Intent > Additional Prefixes > Private Traffic
'Cannot determine whether the computer is joined to a domain' — NTFS permissions	Reverse DNS (PTR) lookup failing on external forwarder for DC IPs	Add reverse lookup ruleset to external private DNS forwarder: <reverse-zone>.in-addr.arpa. > external DC IPs
DDNS fails — SMB hostname not in DNS after volume creation	ANF AD connection DNS IPs point to external forwarder — GSS-TSIG not supported by forwarder	Set AD connection DNS IPs to writable Microsoft AD-integrated external DC IPs directly
'Failed to validate LDAP configuration' during dual-protocol creation	Missing PTR records for external DCs, or reverse zone unreachable	Add PTR records for all external DCs. Verify reverse ruleset is present on the forwarder.
NFSv4.1 Kerberos: 'Cannot determine realm for numeric host address'	Missing PTR for ANF volume IP or external DC IPs	Add PTR records for ANF volume IPs and all external DC IPs in the reverse zone.
SMB hostname resolves on-premises but not from Azure VMs	External private DNS forwarder missing forward ruleset for AD zone, or targeting wrong DC IPs	Verify forward ruleset is present and targeting reachable writable external DC IPs.
Volume creation fails after external DC IP change	External DNS forwarder (especially BIND) caching stale DC IPs — default TTL up to 7 days	Flush forwarder cache. Set short TTLs on DC A records. Consider Microsoft AD-integrated DNS for AD zones.

Summary of key requirements:

ANF AD connection DNS IPs must point to writable Microsoft AD-integrated DNS servers (external DC IPs) — not the external private DNS forwarder, not Azure DNS Private Resolver, not 168.63.129.16.
The external private DNS forwarder must have both a forward ruleset (AD domain > external DC IPs) and a reverse ruleset (in-addr.arpa. zone for DC IP ranges > external DC IPs). The reverse ruleset is required for NTFS ACL operations on SMB shares and is not mentioned in ANF documentation.
168.63.129.16 is the upstream forwarder target on the external DNS forwarder — not a target in the ANF AD connection. It is unreachable from on-premises and is not AD-aware.
External private DNS forwarders (Infoblox, BIND, Unbound) do not support GSS-TSIG. Placing a forwarder IP in the ANF AD connection causes silent DDNS failure with no portal error.
In Virtual WAN deployments, add the ANF delegated subnet /26 to the hub Routing Intent under Additional Prefixes for Private Traffic. The broader VNET prefix alone is not sufficient.
NFSv3 volume creation success does not indicate SMB readiness — NFSv3 uses the IP directly and bypasses AD, Kerberos, and reverse DNS.
_kerberos-master SRV records are not created automatically by AD DNS and must be added manually.
DNS scavenging should be disabled on zones containing ANF records, or records pre-created as static entries, as ANF does not aggressively refresh DDNS registrations.
When volume creation fails with a generic 'context deadline exceeded' error, switch the deployment availability zone before deep troubleshooting to surface a more descriptive error.

Related documentation:

Governing Agent Sprawl: A Multi‑Region AI Agent Landing Zone on Azure (Reference Architecture)

KimVaddi — Thu, 07 May 2026 05:19:02 GMT

It doesn’t take long for AI agents to get out of hand.

In most enterprises, the first few agents are celebrated. A chatbot here. A document summarizer there. Then another team ships an agent that calls APIs. Someone else connects one to internal data. Within months, IT is staring at dozens—or hundreds—of autonomous systems running across subscriptions, regions, and tools.

At that point, the questions stop being about model quality and start being uncomfortable operational ones:

Who owns this agent?
What data can it access?
What happens if it misbehaves?
Why did it just consume half our monthly token budget in a day?

Developers can build an AI agent in minutes—the difficult part is understanding what agents are doing, how they perform, and whether they comply with organizational policy. Signals scatter across tools, context is lost, and governance becomes reactive.

This reference architecture exists to solve that problem.

It describes a multi‑region AI agent landing zone on Azure that treats agents as first‑class, governable workloads—provisioned automatically, constrained by policy, and observable from day one.

The architectural principle: separate control from execution

The design starts with a simple but non‑negotiable rule:

Control plane concerns must be separated from runtime concerns.

Azure landing zones already follow this model. Management groups, Azure Policy, and RBAC are global constructs. Workloads run in regions. This architecture applies the same discipline to AI agents.

The runtime plane is where agents execute, models infer, and data flows—often in multiple Azure regions.
The control plane is where identity, policy, safety, evaluation, and oversight live—independent of region.

This separation is what allows teams to scale agents without losing control.

Layer 1: Azure AI Gateway — governing every request

The first control layer sits directly in the request path.

The AI gateway in Azure API Management provides a policy‑enforcement and observability layer in front of AI models, agents, and tools. It is not a separate service—it extends Azure API Management.

Everything flows through it:

Microsoft Foundry model deployments
Azure AI Model Inference API endpoints
OpenAI‑compatible third‑party models
Self‑hosted models
MCP servers and A2A agent APIs (preview)

What the gateway actually enforces

This layer is intentionally narrow and operational:

Token quotas and rate limits
The llm-token-limit policy (GA) enforces tokens‑per‑minute or quota ceilings per consumer before requests reach the backend. This prevents one application—or one agent—from exhausting shared capacity.
Content safety at ingress
The llm-content-safety policy (GA) integrates Azure AI Content Safety to moderate prompts automatically. Unsafe requests never reach the model.
Traffic routing and resiliency
Azure API Management supports multi‑region gateway deployment (Premium tier). If a region fails, traffic routes to the next closest gateway automatically.
Token usage, prompts, and completions are logged to Azure Monitor and Application Insights using built‑in policies such as llm-emit-token-metric.

The gateway does not understand agent intent or business context. That is by design. It governs traffic, not behavior.

Layer 2: Azure AI Foundry Control Plane — governing behavior at scale

The second layer governs what agents do, not just how requests flow.

Azure AI Foundry Control Plane provides a unified management surface for AI agents, models, and tools across projects and subscriptions. It is designed specifically for agentic systems.

Foundry Control Plane is currently in public preview.

What Foundry Control Plane adds

Fleet‑wide inventory
Every agent, model, and tool appears in a single, searchable view across projects.
Continuous evaluation on production traffic
Foundry runs evaluations that measure task adherence, groundedness, tool‑call accuracy, sensitive data exposure, and other agent‑specific risk dimensions.
Centralized guardrails
Policy is enforced across inputs, outputs, and tool interactions—not just prompts. Bulk remediation can be applied across the fleet.
Security integration
Foundry integrates with:
- Microsoft Entra for agent identity (Entra Agent ID)
- Microsoft Defender for threat signals
- Microsoft Purview for data protection and compliance visibility

Foundry Control Plane also requires an AI Gateway to be configured for advanced governance scenarios—reinforcing the layered approach.

Layer 3: Microsoft Agent 365 — enterprise oversight, not just Azure oversight

The third layer exists because Azure governance alone is not enough.

Agents don’t just call APIs. They act on behalf of users. They access enterprise data. They operate inside Microsoft 365 workflows.

Microsoft Agent 365 is the tenant‑level control plane for AI agents. It brings agents under the same administrative model used for users and applications.

Status: Frontier Preview
General availability: May 1, 2026

Why this layer matters

Agent 365 introduces controls that Azure alone cannot provide:

Agent registry
A single inventory of all agents in the tenant—including sanctioned and shadow agents. Unsanctioned agents can be quarantined.
Identity‑first access control
Every agent is issued an Entra agent ID. Conditional Access policies apply to agents the same way they do to users.
Human‑in‑the‑loop oversight
Agents surface in Microsoft 365 admin workflows, not just Azure portals.
Security and compliance
Defender and Purview extend threat detection and data protection policies to agent activity.

Agent 365 does not replace Foundry Control Plane. It complements it—connecting agent operations to enterprise identity, compliance, and productivity systems.

How the pieces work together

Individually, these services are powerful. The architecture works because they are deliberately layered.

External approval → automated provisioning

When a use case is approved in an external governance system, it triggers an Azure DevOps pipeline using the REST API.

That pipeline:

Provisions subscriptions and resource groups
Deploys Foundry projects
Configures Azure API Management with AI Gateway policies
Enables monitoring and logging

Governance is applied before the first request is made.

One policy model, many regions

Azure landing zones are region‑agnostic at the governance layer. This architecture follows that guidance.

Policies and RBAC apply globally
AI Gateway enforces limits locally in each region
Runtime services scale region by region

Expanding to a new region does not introduce a new governance model—only new capacity.

A single operational view

Signals flow upward:

AI Gateway emits traffic and usage metrics
Foundry Control Plane correlates evaluations, guardrail enforcement, and security alerts
Agent 365 aggregates tenant‑level identity, compliance, and threat signals

Operations teams no longer hunt across dashboards. They work from one prioritized view, with context intact.

What this architecture deliberately does not promise

This is a reference architecture, not a silver bullet.

It does not eliminate the need for:

Clear agent ownership
Business‑level approval processes
Ongoing evaluation of agent usefulness

What it does provide is a foundation—one that lets organizations scale agentic AI without accepting chaos as the cost of innovation.

Closing thoughts

Agent sprawl is not a tooling failure. It’s an architectural one.

By separating control from execution, layering governance where it belongs, and aligning AI operations with existing Azure and Microsoft 365 control planes, this architecture gives enterprises a way to move fast without losing sight of what their agents are doing.

That’s the difference between experimentation—and production.

Co-Contributor:

Jorge Pena Alarcon-Sr. Cloud & AI Specialist

References (official Microsoft sources)

Architecture to Resilience: A Decision Guide

varghesejoji — Mon, 04 May 2026 21:24:03 GMT

Start with the framework, accelerate with the tool

Watch the video walkthrough

The Application Resilience Framework originated from a practical gap we saw in resilience reviews: teams had architecture diagrams, monitoring data, incident history, and runbooks, but no consistent way to connect them into a measurable resilience model.

The framework is intended to close that gap by turning architecture context into a structured lifecycle for risk identification, mitigation validation, health modeling, and governance. It aligns closely with the Reliability pillar of the Azure Well-Architected Framework, especially the guidance around identifying critical flows, performing Failure Mode Analysis, defining reliability targets, and building health models.

Application Resilience Framework flow from artifact import to measurable operational resilience.

The Application Resilience Framework Tool helps teams apply this framework faster by starting with artifacts they already have, such as data flow diagrams or sequence diagrams in Mermaid or image format. The tool extracts workflows, application components, platform components, dependencies, and initial failure modes, then guides the team through the decisions needed to make resilience measurable.

From those artifacts, the tool creates the first version of a resilience model by extracting workflows, application components, platform components, dependencies, and initial failure modes. It then guides the team through one import step followed by four phases:

Import Artifacts -> Phase 1: Failure Mode Analysis -> Phase 2: Mitigation and Validation -> Phase 3: Health Model Mapping -> Phase 4: Operations and Governance

It is not a replacement for WAF guidance or Resilience Hub style assessments. It is a practical way to operationalize those concepts at the workload and workflow level, producing prioritized risks, mitigation plans, validation paths, health signals, dashboards, reports, and governance ownership.

How to use this guide

This guide follows the same flow as the tool. For each step, it covers:

The decision: What needs to be decided?
The options: What paths are available?
The guidance: When each option fits

Use this with the video walkthrough. The video shows the tool in action. This guide explains the choices behind each step.

Question 1: What artifact should you import first?

The import step creates the starting point for the model. Regardless of the input path, the output is the same: workflows that move into Phase 1: Failure Mode Analysis.

Options

Import option	Best for	What happens
Data flow diagram	System, module, data movement, and dependency views	If imported as an image, the tool breaks it into sequence-style flows. Selected flows become workflows.
Sequence diagram	Transaction flow and service interaction views	Converted directly into workflows.
Mermaid input	Diagrams maintained as code in Mermaid format	Converted directly into workflows.
Image input	JPG or PNG diagrams	Azure Foundry Vision models interpret the image and convert it into workflows.
Manual entry	Missing or incomplete diagrams	User creates or corrects workflows manually.

When to pick which

Use data flow for system and dependency views. Use sequence diagrams for transaction or interaction views. Regardless of import path, the output is the same: workflows, components, dependencies, and initial failure modes ready for Phase 1.

Question 2: Which workflows should be analyzed first?

Phase 1 is Failure Mode Analysis. This is where the tool identifies what can fail and how important each failure is.

Options

Critical user flows: Login, checkout, payment, onboarding, request processing.
High-risk platform flows: Database writes, queue processing, storage access, identity, messaging, external APIs.
Known issue areas: Workflows with recent incidents, recurring alerts, or customer impact.

When to pick which

Start where failure creates the highest customer or business impact. The goal is not to model everything at once. The goal is to model the right thing first.

Deliverables

Failure Mode Analysis catalog
RPV risk scores
Criticality classification

Question 3: How should failure modes be prioritized?

After workflows and components are imported, the tool helps score each failure mode using Risk Priority Value or RPV, which uses the four factors of Impact, Likelihood, Detectability and Outage severity.

Options

Use generated failure modes and scores: Best for a fast first pass.
Tune the RPV scores with engineering input: Best when workload context matters.
Add custom failure modes: Best when known risks come from incidents, reviews, or customer experience.

When to pick which

Use the generated model to accelerate the first pass, then adjust it with real system knowledge. The goal is not to create the longest list of risks. The goal is to identify the risks that deserve attention first.

Deliverables

Failure Mode Catalog
RPV Risk Scores
Prioritized criticality list

Question 4: Are mitigations defined or validated?

Phase 2 is Mitigation and Validation. This is where each failure mode gets a response plan.

Options

Detection only: The team can detect the failure, but the response is not defined.
Defined mitigation: The response is documented, such as retry, fallback, failover, scaling, restore, or rebalance.
Validated mitigation: The response has been tested through a controlled validation or chaos test.

When to pick which

For low-risk items, documented mitigation may be enough. For critical and high-risk items, validation is the key. A mitigation that has not been tested is still an assumption.

Deliverables

Mitigation playbooks
Chaos test plans
Support playbooks

Question 5: Which risks need health signals?

Phase 3 is Health Model Mapping. This is where the tool connects risks to observability.

A failure mode should not just sit in a document. It should map to a signal that can show whether the system is healthy, degraded, or unhealthy.

Options

Map all failure modes: Best for small systems or highly critical workloads.
Map critical and high-risk failure modes first: Best for large systems.
Track unmapped risks as gaps: Best when observability coverage is still improving.

When to pick which

Start with the highest RPV items. Every critical failure mode should have at least one signal, such as a metric, log, alert, availability check, or dependency signal.

Deliverables

Health model
Signal definitions
Coverage report
Bicep templates

Question 6: Should the health model be exported or deployed?

Once the health model is built, the next decision is how to use it.

Options

Export for review: Best when the team needs to validate the model first.
Generate monitoring templates: Best when the team wants repeatable implementation.
Deploy to Azure: Best when the model is ready to become part of operations.
Use outputs in downstream tools: Best when support, SRE, or incident response workflows need structured playbooks.

When to pick which

Export first if the model is still being reviewed. Deploy when component relationships, signals, and coverage are accurate enough for operational use.

Question 7: How will governance keep the model current?

Phase 4 is Operations and Governance. This is where the resilience model becomes an ongoing practice.

Options

One-time assessment: Useful for quick discovery but limited long term.
Recurring review: Best for production workloads that change regularly.
Closed-loop governance: Best when incidents, failed validations, and monitoring gaps feed back into the model.

When to pick which

For production systems, use a recurring governance cadence. Assign owners, track gaps, review dashboards, and update the model as the system changes.

Deliverables

Governance model
Dashboards
Reports and exports
Runbooks

Putting it together: three adoption patterns

Once governance is defined, the tool can be used in different ways depending on the team’s maturity and objective. The three common adoption patterns are:

Pattern A: Quick resilience review

Import one critical workflow
Generate failure modes
Review RPV scores
Identify top risks
Export findings

Best for fast architecture reviews or early customer conversations.

Pattern B: Full workload assessment

Import multiple workflows
Build a full Failure Mode Catalog
Define mitigations and recovery steps
Create chaos test plans
Map risks to signals
Produce coverage reports

Best for structured resilience assessments.

Pattern C: Operational health model

Build and tune the health model
Export or deploy monitoring artifacts
Track risk and signal coverage
Review mitigation effectiveness
Assign governance ownership
Feed findings back into the model

Best when the goal is continuous operational improvement.

A short checklist before using the tool

Which workflow should we import first?
Do we have a data flow diagram, sequence diagram, or Mermaid file?
What components and dependencies should be included?
Which failure modes matter most?
How should RPV be adjusted for this workload?
Do critical failure modes have mitigations?
Have those mitigations been validated?
Are failure modes mapped to health signals?
What coverage gaps remain?
Should the health model be exported or deployed?
Who owns ongoing review?
How often should the model be updated?

Closing thought

The Application Resilience Framework Tool provides a practical way to move from architecture artifacts to measurable, continuously improving resilience.

It starts with data flow or sequence diagrams, builds a structured view of the system, and guides teams through the decisions that matter: what can fail, how severe it is, how it is mitigated, how it is detected, and how it is governed.

Tool repo: Application Resilience Framework Tool

How MS Discovery Is Empowering Scientists to Do More

sameeraman — Sun, 03 May 2026 12:05:56 GMT

Research and development has traditionally been a slow, sequential, and largely manual endeavour. Scientists formulate hypotheses, design experiments, run computations in constrained environments, and document results, each stage dependent on the last, each transition requiring human review and intervention. Knowledge is fragmented across systems, insights are bottlenecked by individual capacity, and the gap between hypothesis and actionable outcome can span weeks or months.

For organisations tackling complex scientific and operational challenges, from drug discovery to industrial process optimisation, this pace of iteration is simply no longer acceptable.

At Microsoft, we recently introduced Microsoft Discovery, a platform that I believe fundamentally changes this model. Much like Microsoft 365 transformed the way knowledge workers collaborate and create, Microsoft Discovery is designed to simplify and empower the way scientists and researchers work. It provides a unified, end-to-end platform that integrates advanced artificial intelligence, high-performance computing, and knowledge management to support the full scientific reasoning lifecycle: knowledge gathering, hypothesis generation, experiment design, simulation, results analysis, and documentation.

In this article, I want to share how we used Microsoft Discovery to automate a real-world simulation workflow for a mining organisation and what that experience taught our team about the future of AI-augmented science.

What Is Microsoft Discovery?

Microsoft Discovery is Microsoft's scientific AI platform, a solution designed to accelerate research and experimentation across the full innovation lifecycle. Rather than replacing scientific judgement, Discovery is designed to amplify human expertise, embedding AI assistance at each stage of the R&D process while maintaining governance, traceability, and scientific rigour.

From Traditional R&D to AI-Augmented Science

To appreciate what Discovery enables, it is important to understand where it fits in.

In the traditional R&D model, knowledge discovery centres on manual literature reviews and historical data analysis. Researchers individually search, read, and synthesise information which is a time-intensive process where discovery is limited by each person's capacity to locate and interpret relevant material. Hypothesis generation and experimental design are expert-led and largely manual. Computational experimentation, where it exists, runs in fixed or constrained environments with limited parallelism. Analysis and iteration follow the same sequential pattern: execute, review, document, repeat.

Microsoft Discovery changes this fundamentally. In the AI-cloud-enabled model it provides:

Knowledge synthesis at scale — Researchers can explore literature, historical experiments, and organisational knowledge through a single interface, with intelligent indexing surfacing insights faster than manual search could ever achieve.
AI-assisted hypothesis generation — Collaborative human-and-AI workflows support hypothesis exploration and feasibility assessment, while final decisions remain with the scientist.
Cloud-scale experimentation — Elastic compute and parallel processing allow simulations and experiments to run at scale, with integrated tracking and reproducibility built in.
Continuous feedback and human-in-the-loop governance — Results are analysed and compared more rapidly, enabling faster iteration, with AI-generated insights reviewed and validated by researchers before action.
Governed knowledge assets — Experiment lineage, outcomes, and best practices are captured as reusable, governed assets, supporting long-term organisational learning.

The net effect is a transition from slow, manual, and fragmented research processes to an agile, automated, and data-driven R&D model — one that improves research efficiency, increases the return on innovation investment, and enables faster, higher-impact solutions to complex challenges. In high level, the research and deveolopment loop we discussed and how Microsoft Discovery enriches it show in the following diagram.

The Real-World Problem: Screening Thousands of Molecules

To bring this to life, let me walk you through a real-world use case we worked on recently. A mining organisation needed to identify the best-performing oxidant compounds for a chemical reaction central to their operations. We will be talking about only a workflow that sits squarely in the simulation phase of the scientific loop — and it is a perfect example of the kind of work that Microsoft Discovery can strongly transform.

How Scientists Did It Before

In the traditional process, scientists would begin by selecting candidate molecules from established molecular libraries based on characteristics identified through literature review. These libraries can contain thousands of molecules, each defined in standard molecular file formats (such as XYZ or CIF files) that describe their three-dimensional atomic structures.

From there, a researcher would manually work through a multi-step pipeline:

Pre-processing and preparation: The selected molecular files are processed and prepared for quantum mechanical (QM) calculations. This involves filtering molecules based on properties like the types of metals present, electron count, and atomic weight — criteria that directly affect both the scientific relevance and the computational cost of the simulations. The output is a set of prepared input files (known as GJF files) ready for simulation.
Running quantum mechanical simulations: The prepared input files are submitted to a computational chemistry tool (Gaussian 16) to perform Density Functional Theory (DFT) calculations. These simulations compute the electronic structure and energy states of each molecule across different charge and multiplicity configurations. Crucially, each molecule requires multiple independent simulation runs, and the computational cost scales rapidly with molecular complexity. With thousands of candidate molecules, this step alone can involve thousands of individual simulation jobs.
Collecting and post-processing results: Once all simulations complete, the output log files are collected and processed. For each molecule, the lowest-energy charge and multiplicity combination is identified, and a set of quantum mechanical descriptors and classical molecular descriptors are extracted. These descriptors are then fed into a trained machine learning model to predict the redox potential of each compound, a key metric that indicates how effectively a molecule can act as an oxidant in the target reaction.
Summarisation and filtering: Finally, the predicted redox potentials and other relevant characteristics are compiled into a summary, enabling researchers to identify the most promising candidates for further investigation and experimental validation.

Every step in this pipeline required manual intervention: writing and adjusting scripts, verifying input and output files, monitoring job queues, handling failures, and stitching results together. A single researcher could easily spend days or weeks moving through this process — and any error at one stage meant going back and re-running subsequent steps.

How We Automated This with Microsoft Discovery Agents

When we looked at this workflow through the lens of Microsoft Discovery, the opportunity was clear. The scientific reasoning, selecting which molecules to test, interpreting redox potential results, deciding what to investigate next, should remain with the researcher. But the operational overhead of preparing files, submitting simulations, monitoring jobs, collecting results, and assembling summaries? That could be orchestrated by a team of AI agents.

A Team of Agents, Working Together

We designed a multi-agent architecture within Microsoft Discovery to automate this simulation workflow end to end. Here is how the team of agents operates:

Router Agent: The entry point. When a researcher submits a request for example, asking to run QM calculations on a set of candidate molecules the Router Agent interprets the intent and orchestrates the downstream workflow.

Planner Agent: Once the Router Agent identifies the task, the Planner Agent examines the input files provided by the researcher and formulates a step-by-step execution plan. It determines what needs to happen, in what order, and with what parameters, much like a project manager scoping out a piece of work.

Gaussian Prep Agent: This agent handles the preparation step. It is intelligent enough to inspect the current molecular files, apply the necessary filtering criteria, and prepare them for simulation, generating the input files that the computational chemistry tool requires. What previously involved manual scripting and file-by-file verification is now handled autonomously. We used Microsoft Discovery tools to do the underlying execution with this agent.

MPI Gaussian Agent: This is where the power of cloud-scale computing comes in. The Gaussian Agent submits the prepared simulation jobs and manages their execution using an MPI-based master-worker pattern. This approach enables massive parallel execution scaling out across the cloud to run thousands of simulations concurrently rather than sequentially. Given that the candidate molecule libraries can contain thousands of entries, and each molecule may require multiple simulation runs, this parallel execution capability is transformative. What might have taken days in a constrained local environment can now complete in a fraction of the time.

Redox Potential Agent: Once the simulations are complete, this agent takes over. It processes the simulation outputs, identifies the optimal charge and multiplicity state for each molecule, extracts the relevant QM and classical descriptors, and runs them through the trained machine learning model to predict redox potentials.

Summariser Agent: The final agent in the chain. It maps the predicted redox potentials back to the original molecules, applies any additional filtering criteria, and produces a clean, structured summary a JSON file that the researcher can immediately use to identify the most promising candidates and take them forward into the next phase of their work.

What the Researcher Experiences

From the scientist's perspective, the transformation is striking. Instead of spending days writing scripts, babysitting job queues, and manually stitching results together, they provide their input files and describe what they need. The agents take it from there planning, preparing, executing, processing, and summarising and deliver a curated output ready for scientific interpretation.

The researcher's time is freed to focus on what matters most: thinking critically about the science. Which molecules look most promising? What does the redox potential distribution tell us? Should we adjust the filtering criteria and run another round? These are the high-value questions that require human expertise and now scientists can spend their time on exactly that, rather than on operational mechanics.

The Bigger Picture: Accelerating the Entire Scientific Loop

It is important to note that this simulation workflow is just one piece of the broader scientific loop. The full cycle of scientific research, from initial knowledge gathering and literature review, through hypothesis generation, experimental design, simulation, results analysis, and documentation involves many stages, each of which can benefit from the same kind of AI-augmented approach.

Microsoft Discovery is designed to support this entire cycle. In our project, we did not stop at simulation. We also explored how agents can accelerate the knowledge gathering phase, helping researchers navigate vast bodies of literature and surface relevant prior work more efficiently. We looked at how AI can assist with hypothesis generation and evaluation, helping scientists reason about which directions are most promising before committing to expensive computations. And we examined how agents can support the analysis and reporting phases comparing results against hypotheses, generating visualisations, and even assisting with drafting research documents.

What excites me most about Microsoft Discovery is not any single capability, but the cumulative effect of embedding AI assistance across every stage of the research process. Each phase that gets faster and more efficient creates a multiplier effect on the phases that follow. When knowledge gathering takes hours instead of weeks, researchers generate better hypotheses sooner. When simulations run at cloud scale in parallel, results arrive faster. When analysis is augmented by AI, iteration cycles tighten. The entire loop accelerates.

Conclusion

The way we approach scientific research is undergoing a fundamental shift. Large language models and the AI agents built from them are not replacing scientists, they are empowering them to work at a pace and scale that was previously unimaginable.

Microsoft Discovery represents a new operating model for R&D. By combining advanced AI, high-performance cloud computing, and intelligent workflow orchestration, it enables researchers to offload the repetitive, time-consuming operational work to agents and invest their expertise where it has the greatest impact: in asking better questions, interpreting complex results, and pushing the boundaries of what we know.

In the use case I have shared here, a team of six AI agents automated a simulation pipeline that would have taken a single researcher days of manual work. They prepared molecular input files, scaled out thousands of quantum mechanical simulations in parallel across the cloud, processed the results, predicted redox potentials using machine learning, and delivered a structured summary all with minimal human intervention.

This is just the beginning. As AI agents become more capable and the tools surrounding them more mature, the potential to accelerate discovery across every scientific domain is immense. Whether you are in materials science, pharmaceuticals, energy, agriculture, or any field where complex R&D is central to progress, Microsoft Discovery offers a platform to do more, faster, and with greater confidence.

The future of science is not about working harder. It is about working smarter with AI as your partner in discovery.

Running Diffusion Models at Scale on AKS

PrabalDeb — Thu, 30 Apr 2026 04:38:08 GMT

Diffusion workloads are simple at prototype scale and unforgiving in production. A single demo can run on one GPU-backed VM, but a real platform has to handle bursty demand, long-running jobs, model artifact distribution, secure public access, rollout safety, and hardware-level observability.

Azure Kubernetes Service (AKS) is a strong fit when the requirement is not just to run a model, but to operate a repeatable platform for GPU inference. The reusable pattern is straightforward: keep the API and control layer on CPU nodes, buffer work through a dispatch layer, run inference on isolated GPU capacity, push results to durable storage, and treat security, telemetry, and deployment automation as first-class platform features.

AKS reference architecture for diffusion workloads

The architecture above shows the core operating model. DNS and an edge layer: Application Gateway with WAF, and optionally Front Door for global entry: route traffic to an AKS CPU pool that hosts the API tier. GPU jobs run on a separate GPU pool, while shared add-ons and CSI drivers run on a system pool. Teams can keep dispatch inside Kubernetes or externalize it through Service Bus plus KEDA, and Azure dependencies should be reached over Private Link with Azure Monitor covering both app and hardware telemetry.

The storage block serves two purposes: durable output storage and, if needed, a shared Hugging Face model cache exposed to GPU pods through PV and PVC mounts.

The reference pattern

The core architecture separates control-plane traffic from GPU execution:

A lightweight API tier on the CPU node pool receives requests, validates identity, and hands execution work to a dispatch layer.
That dispatch layer can stay inside Kubernetes using native queueing and controller patterns, or it can publish work to Azure Service Bus for external queue-backed dispatch.
Scaling can likewise stay AKS-native through cluster and workload autoscaling, or it can use KEDA to react directly to queue backlog.
GPU work runs on a dedicated GPU node pool, isolated from the API and cluster add-ons.
GPU workers should mount persistent storage for model caches so Hugging Face assets can survive pod restarts and repeated job submissions.
Results are stored outside the pod lifecycle in blob-backed storage and returned through a stable status API.
Edge routing, TLS termination, and WAF inspection happen at the ingress layer, while token validation is typically enforced in the API tier or a dedicated upstream auth component.

This split lets each lane scale on the right signal: request traffic for the API tier, backlog for dispatch, and job demand for GPU workers. It also keeps tuning simpler for cost, latency, and reliability. For single-region deployments, Application Gateway or Application Gateway for Containers is often enough; Azure Front Door becomes more useful for global entry, multi-region failover, or shared edge policy.

In the reference architecture, the CPU pool hosts the externally reachable APIs and control-plane components that submit work into the dispatch layer. The GPU pool hosts the actual model execution components, including short-lived diffusion jobs and longer-lived worker runtimes. A separate system pool hosts shared cluster services such as AGIC and the Secret Store and Blob CSI drivers, while KEDA is added only when teams choose the Service Bus pattern. That keeps platform plumbing off the application and GPU lanes.

The persistence layer is most useful as a model cache rather than as general-purpose application state. There are two practical ways to back it:

Node-local persistence keeps the cache close to the GPU worker and is the simplest option when jobs benefit from warm data already present on the same node.
Azure Storage backed persistence is more useful when model download times are long enough that keeping artifacts on shared durable storage materially reduces job startup latency.

Choose the dispatch model

The important design decision is not whether to use one branded queueing technology. It is whether the platform needs a fully Kubernetes-native control loop or an explicit external queue with backlog-driven scaling.

The Kubernetes-native option keeps dispatch inside the cluster:

The API creates or signals internal Kubernetes work objects.
A Kubernetes-native queue or controller pattern manages admission and dispatch.
AKS workload autoscaling and cluster autoscaling handle most scale changes.
This path is simpler when the team wants fewer external dependencies and the workload shape is already well understood.

The Azure Service Bus plus KEDA option externalizes the control loop:

The API publishes work to Azure Service Bus.
Queue consumers or schedulers materialize GPU execution from that queue.
KEDA scales the scheduling or worker path directly from queue depth.
This path is better when backlog visibility, queue durability, or burst-driven autoscaling needs to be explicit and independently observable.

Both models can fit the same AKS platform. The GPU isolation, security boundaries, storage pattern, and observability expectations remain the same.

A simple way to choose is:

Start with Kubernetes-native dispatch when the team wants the fewest moving parts and the job profile is already predictable.
Choose Azure Service Bus plus KEDA when durable backlog, explicit queue depth, and burst-driven worker scaling are important operating requirements.
Consider KAITO or the AI toolchain operator add-on when the primary need is managed serving of supported models rather than custom diffusion job orchestration.

Scale by workload lane, not by one generic pool

Not every GPU workload should share the same execution path. Keep short-lived inference, queue-backed workers, and longer-running runtimes in separate lanes so one class does not block another. Where supported, GPU multi-instance configurations can further improve utilization for lighter jobs while leaving full GPUs available for heavier ones.

On AKS, the better pattern is to define separate operating lanes:

An API admission lane on CPU nodes for authentication, validation, and request submission.
A scheduling lane that can use either Kubernetes-native queueing with AKS autoscaling or Azure Service Bus with KEDA.
A GPU execution lane for diffusion jobs and longer-lived worker runtimes.
Dedicated labels, taints, autoscaling bounds, and dashboards per lane.

Within the GPU execution lane, teams can go one step further and define capacity classes for full-GPU and fractional-GPU jobs. That is useful when some models need the memory and throughput of a whole device, while others can run efficiently on a smaller GPU partition. The NVIDIA device plugin DaemonSet shown in the GPU pool is what advertises those GPU resources (and MIG slices) to the kube-scheduler so pods can request them like any other resource.

That gives platform teams clean capacity isolation and avoids letting one workload class starve another.

Secure the edge, the workload, and the secret path

GPU platforms should treat security as a day-one requirement, not a later add-on.

At the edge, use DNS with Application Gateway and WAF, and add Front Door when global routing is needed. Store public TLS certificates in Azure Key Vault and project them into the cluster through the Secrets Store CSI Driver so renewals do not require redeployment. For protected APIs, validate Microsoft Entra ID tokens in the service or a dedicated auth layer, and keep health probe endpoints separate from business routes.

Inside the cluster, keep authorization scoped tightly:

Separate system, CPU, and GPU node pools.
Use namespace boundaries for tenant or environment isolation.
Give the API only the Kubernetes RBAC it needs to create and monitor jobs, plus Azure permissions only when the external queue option is enabled.
Prefer Microsoft Entra Workload ID over long-lived credentials for workload access to Azure resources such as Key Vault and Blob Storage, and extend that to Service Bus when the external queue pattern is used.

For operator access, keep the cluster management path separate from the public request path. In the reference architecture, developers come in through Azure Bastion rather than broad direct exposure of cluster endpoints.

For secrets, move away from cluster-local secrets as early as possible. A production-ready path uses Azure Key Vault with the Secrets Store CSI Driver so credentials are not baked into images, manifests, or CI pipelines. If the platform uses Azure Service Bus, queue access should use managed identity as well. Blob-backed result storage should likewise use managed identity and CSI-based integration instead of embedding long-lived credentials into workloads.

For the network path between the cluster and its Azure dependencies, prefer Private Endpoints over public service endpoints. The diagram uses a single PE icon as shorthand for this pattern: in practice, teams usually create private endpoints per service and pair them with private DNS so ACR pulls, Key Vault reads, Blob I/O, and optional Service Bus traffic resolve to private IPs inside the VNet, which keeps platform traffic off the public internet and simplifies firewall and DNS policy.

Observe both the application and the hardware

GPU workloads need two telemetry views: application behavior and hardware behavior. The first tracks request IDs, job IDs, latency, and failures; the second tracks GPU utilization, memory pressure, and device-level performance. Together they show whether the problem is code, dispatch pressure, or hardware saturation.

On Azure, that split maps well to:

Structured application logs and OpenTelemetry exported to Application Insights.
Azure Monitor dashboards that include internal queue pressure or Service Bus backlog, plus AKS autoscale or KEDA scale activity depending on the chosen pattern.
NVIDIA DCGM exporter metrics scraped into Azure Managed Prometheus and visualized in Azure Managed Grafana.

This model is what turns raw GPU hosting into an operable platform. Without it, teams can see requests failing but not whether the root cause is code, dispatch saturation, scheduling, or hardware contention.

The diagram reflects that split clearly. Application Insights and Azure dashboards track service and dispatch behavior, while Prometheus, Grafana, the NVIDIA device plugin, and DCGM exporter track cluster and GPU health. That combination is what allows teams to correlate dispatch delay, AKS or KEDA scale-out, execution time, and failure rates with actual GPU utilization and memory pressure.

Keep CI/CD small, secretless, and reversible

The deployment model does not need to be complex to be production-grade. A practical AKS pattern is:

Pull request validation for code quality, tests, Dockerfiles, and secret scanning.
Immutable container tags built from the commit SHA.
GitHub Actions with OpenID Connect and Azure workload identity federation.
ACR as the image source of truth.
Environment-based promotion with approval gates for production.
Rollout verification with Kubernetes health checks and smoke tests.

The key principle is separation of concerns. CI/CD should roll forward application images and validated configuration, not rebuild the whole platform on every deploy. Shared components such as ingress, node pools, identity, storage, monitoring, and optional KEDA or Service Bus should remain under controlled infrastructure change management.

What makes this pattern reusable

This AKS pattern generalizes well beyond one model family or one product surface because it is built on fundamentals:

Separate API admission from dispatch and GPU execution.
Choose the dispatch boundary that fits the workload: Kubernetes-native queueing with AKS autoscaling, or Service Bus plus KEDA.
Isolate workload classes into different scaling lanes.
Scale worker capacity from the most useful signal for the chosen model: workload pressure inside AKS or external queue backlog through KEDA.
Put authentication, TLS, and routing at the edge.
Use workload identity and externalized secrets.
Instrument both software behavior and GPU behavior.
Keep deployments automated, traceable, and easy to roll back.

That is the real architecture story. The model can change. The runner can change. Even the queue and gateway choices can evolve. But the engineering fundamentals stay stable, and that stability is what makes diffusion workloads viable at scale.

Possible alternatives: KAITO and the AI toolchain operator add-on

Some teams do not need a fully custom GPU execution platform. On AKS, two adjacent options are KAITO and the AI toolchain operator add-on.

KAITO is the lighter-weight choice for rapid experimentation with supported model presets. The AI toolchain operator add-on is the more managed option for standardized LLM or multimodal serving with AKS-native operational features. Both are less suitable when the platform needs custom diffusion pipelines, queue-backed job orchestration, artifact-heavy workflows, or application-specific dispatch logic.

Reference documents

AKS node pools: Create node pools for a cluster in Azure Kubernetes Service (AKS)
Microsoft Entra Workload ID for AKS: Use Microsoft Entra Workload ID with Azure Kubernetes Service (AKS)
Key Vault integration for AKS: Use the Azure Key Vault provider for Secrets Store CSI Driver in an Azure Kubernetes Service (AKS) cluster
Azure Blob and other CSI storage drivers on AKS: Use Container Storage Interface (CSI) drivers on Azure Kubernetes Service (AKS)
AKS GPU multi-instance support: Use multi-instance GPUs in Azure Kubernetes Service (AKS)
KEDA on AKS: Simplified application autoscaling with Kubernetes Event-driven Autoscaling (KEDA) add-on
Application Gateway Ingress Controller: What is Application Gateway Ingress Controller?
AKS monitoring with Azure Monitor, managed Prometheus, and Grafana: Enable monitoring for Azure Kubernetes Service (AKS) clusters
Application telemetry with Application Insights: Introduction to Application Insights - OpenTelemetry observability
Hugging Face Diffusers: Diffusers documentation
NVIDIA DCGM exporter: dcgm-exporter
NVIDIA device plugin DaemonSet: NVIDIA k8s-device-plugin

Closing thought

Running diffusion models in production is not mainly a model-hosting problem. It is a platform engineering problem with GPUs in the middle. Teams that treat AKS as the control surface for isolation, observability, identity, and repeatable rollout discipline end up with a system that can scale beyond a benchmark and survive real operational demand.

Transforming Video Content into Structured SOPs Using Graph-based RAG

dikshashakya — Wed, 29 Apr 2026 01:04:53 GMT

Introduction

In today’s digital-first environments, a large portion of enterprise knowledge lives inside video content, training sessions, onboarding walkthroughs, and recorded operational procedures.

While videos are great for learning, they are not ideal for quick reference, compliance, or repeatable processes. Converting that knowledge into structured documentation like Standard Operating Procedures (SOPs) is often manual and time-consuming.

What if this process could be automated using AI?

The Problem

Transcripts alone don’t solve the problem.

When videos are converted into text, the output typically lacks:

Clear structure (sections, headings, hierarchy)
Context (relationships between steps, tools, and roles)
Completeness (definitions and dependencies spread across the content)

This leads to a common challenge:

Teams spend significant effort manually reading transcripts, interpreting context, and restructuring them into usable documentation.

As seen in modern architecture challenges, manual and repetitive configurations don’t scale well and increase maintenance effort

Enter Graph-based RAG (GraphRAG)

GraphRAG extends traditional RAG by building a knowledge graph instead of treating content as disconnected chunks.

What GraphRAG Does

Extracts entities (tools, systems, roles, concepts)
Maps relationships between them
Groups related concepts into logical sections
Preserves context across the entire document

Architecture Overview

Below is the high-level pipeline:

Video → Transcription → Knowledge Graph → LLM Generation → Structured SOP

Implementation Approach (Step-by-Step)

Stage 1: Knowledge Graph Construction

Convert video to transcript
Split transcript into chunks
Feed chunks into GraphRAG

GraphRAG performs:

Text Unit Extraction
Entity Recognition
Relationship Mapping
Community Detection

Result: A structured knowledge graph representation of the transcript

Stage 2: Structure Extraction

From the knowledge graph:

Sequential Steps

Preserve procedural flow from transcript order

Logical Sections

Derived using community detection

Key Concepts

Identified using graph centrality (importance via connections)

This creates a framework for the SOP

Stage 3: Intelligent Document Generation

Using Azure OpenAI, each SOP section is generated:

Section	Generated From
Title & Purpose	High-level concepts
Scope	Entity boundaries
Definitions	Entity descriptions
Responsibilities	Role-based entities
Procedures	Sequential steps
References	Linked content

The key advantage: LLM is grounded in graph structure not raw text

Key Benefits

Context Preservation - Relationships between concepts are maintained across sections.

Comprehensive Coverage - Community detection ensures important topics are not missed.

Reduced Hallucination - LLM generation is grounded in structured knowledge.

Scalability- Works for: 30-minute tutorials, 3-hour training sessions and Enterprise knowledge bases

Real-World Impact (Example)

In enterprise scenarios like pharmaceutical SOP generation:

Processing time: ~15–20 minutes for a multi-hour video
Output quality: 8–10 structured SOP sections
Consistency: Terminology and relationships preserved
Coverage: Minimal missing topics

Where This Approach Works Best

Training videos → SOPs
Meeting recordings → action summaries
Technical demos → documentation
Interview recordings → knowledge bases
Tutorials → reference guides

Key Takeaway

This approach represents a shift from text processing → knowledge understanding.

By combining:

Knowledge graphs (structure)
LLMs (language generation)

We can transform raw, unstructured content into usable, enterprise-grade knowledge assets.

Resources

https://microsoft.github.io/graphrag/index/overview/

Final Thoughts

Have you explored GraphRAG or similar approaches in your projects?

What challenges did you face?
How did you handle unstructured knowledge?

Share your experiences — let’s learn together.

Flexible Cooling for AI Growth: How Zonal Architecture Supports Diverse Hardware Needs

stsolo — Fri, 29 May 2026 20:01:33 GMT

By: Ricardo Bianchini, Steve Solomon, Brijesh Warrier, Martin Herbert, Jay Jochim, Husam Alissa, Pulkit Misra, Eric Peterson and Cam Turner

Context -

Microsoft is pioneering zonal cooling in its next-generation AI datacenters, enabling flexible, performant, efficient, and sustainable thermal management for diverse workloads.

The unprecedented growth of artificial intelligence (AI) is transforming datacenter infrastructure. Modern facilities must now support a diverse array of IT equipment, each with distinct cooling requirements. For example, modern GPUs and other AI accelerators require liquid cooling as air cooling is impractical at power draws exceeding 1 kW per accelerator due to the limited heat capacity of air to remove the resulting thermal load. Meanwhile, non-AI-accelerator (i.e., general-purpose) hardware deployments such as CPU-based compute, storage, and networking are expected to mostly remain air-cooled for the foreseeable future.

Furthermore, liquid cooling offers a significant efficiency advantage: its superior heat dissipation allows coolant supply temperatures at the chip as high as 45°C without sacrificing peak performance. In contrast, air-cooled equipment requires much lower supply temperatures—around 30 °C—for optimal efficiency.

The divergence in hardware cooling requirements creates a complex landscape that demands a strategy that is both flexible and adaptive. As shown in Figure 1, relying on a unified facility water system (FWS) introduces major inefficiencies. For example, liquid-cooled GPU racks may receive coolant below their required operating temperature when served by a single-temperature loop. This inefficiency becomes even more pronounced as the proportion of liquid- to air-cooled equipment increases (e.g., 90:10 liquid-to-air ratio for NVIDIA GB300 servers) since a larger share of the equipment is unnecessarily overcooled.

Beyond operational efficiency, sustainability is a key priority for Microsoft even as we grow our AI infrastructure. Among our sustainability commitments, Microsoft has set goals to become carbon negative and eliminate water evaporation as a cooling method in its next-generation datacenters. A key lever for reducing carbon emissions is improving PUE (Power Usage Effectiveness, i.e., total power divided by IT power), a standard measure of datacenter power and energy efficiency. Achieving this requires dynamically matching cooling delivery to the specific needs of each equipment type, ensuring optimal performance, reduced energy consumption, and enhanced sustainability.

Zonal Cooling: Flexible by Design

Zonal cooling is a facility design that introduces multiple independent water loops, each supplying coolant at different temperatures. Figure 2 illustrates a specific implementation of the zonal concept with two facility-level zones: one loop serves air-cooled equipment, maintaining lower temperatures for human comfort and general-purpose hardware, and the other loop caters to liquid-cooled IT AI accelerators, which can operate efficiently at higher supply temperatures. This separation enables datacenter operators to precisely match cooling supply to the requirements of each zone, avoiding the inefficiency of over-cooling all equipment to the lowest common denominator.

A key strength of zonal cooling is its flexibility. As new generations of IT hardware emerge, with varying thermal profiles, zonal cooling allows datacenters to adapt without major infrastructure overhauls. For example, future AI accelerators may need different liquid temperature ranges (see 30℃ Coolant - A Durable Roadmap for the Future) or technological improvements, such as microfluidics, may enable operating at even higher coolant temperatures, while general-purpose equipment requirements may remain unchanged. Zonal cooling’s architecture supports these changes by enabling operators to adjust loop temperatures and reconfigure cooling assignments as needed.

Forms of Zonal Cooling

Liquid cooling expands the allowable coolant supply temperature range and enables temperature-specific zones. This zonal approach can be applied at multiple layers:

Facility-level: Two distinct temperature zones within a datacenter—one for air-cooled equipment and another for liquid-cooled equipment.
Row-level: Tailor coolant temperature for each row based on deployed hardware (e.g., general-purpose vs GPU servers).
Rack-level: Enable multiple temperature zones within a single rack for fine-grained optimization across servers.
Chip-level: Apply zonal cooling inside the server. For example, use colder coolant for a GPU’s high-bandwidth memory (HBM) while supplying warmer coolant for the SoC and CPUs. This fine-grained approach can enable higher HBM stacking for improved performance, while avoiding unnecessary cooling overhead.

Microsoft is building facility-level zonal cooling in the next generation of its AI datacenters going live in 2028 and beyond, while exploring the other three approaches in the lab. Facility-level zonal cooling is expected to reduce PUEs by up to 10%.

Benefits from Zonal Cooling

Zonal cooling is a strategic enabler for performance and efficiency. It can deliver:

Improved energy efficiency and sustainability: By reducing the load on datacenter cooling infrastructure, zonal cooling improves energy efficiency as captured by annualized PUE, which measures average efficiency across all operating conditions. Lower annualized PUE means energy savings and lower carbon emissions.
Increased server density: Tailored zonal cooling reduces peak cooling power demand during the hottest days, which in turn lowers peak PUE. Designers can leverage this reduction to reserve power for lower water temperatures (anticipating future accelerator needs), add more servers within the same utility power envelope, or contract less utility power per datacenter.
Higher performance: Strategic control of coolant temperatures unlocks higher chip performance without sacrificing efficiency. For example, colder loops allow GPUs and CPUs to sustain elevated clock speeds via safe overclocking, while optimized memory cooling supports greater stacking density and increased bandwidth.
Improved flexibility: With independent zones, operators can easily adjust coolant supply temperatures or reconfigure zones as new generations of hardware with varied cooling requirements emerge. This flexibility ensures compatibility with future innovations while maintaining optimal performance.

Looking Ahead

Zonal cooling represents a paradigm shift in datacenter thermal management. Its flexible, zone-specific approach to cooling air- and liquid-cooled IT equipment positions datacenters to efficiently adapt to future hardware innovations and workload diversity. As the industry continues to push boundaries in performance and sustainability, zonal cooling will be a foundational strategy for building performance and efficient infrastructure that meets tomorrow’s challenges.