Azure Architecture Blog articles

Choosing the Right Agent in Microsoft Foundry

supriyas — Wed, 19 Aug 2026 22:52:13 GMT

Many discussions about Microsoft Foundry Agent Service eventually arrive at the same question: should this workload be implemented as a Prompt Agent or a Hosted Agent? While the documentation explains both options well, the architectural decision usually comes down to something much simpler: where do you want the orchestration logic to live?

First, what actually makes something an agent?

A basic AI assistant generates an answer. An agent can also decide what to do next, call tools, access data, maintain context and complete work across multiple steps.

At the center of most agents are three building blocks:

Model: provides language understanding, generation, and reasoning.

Instructions: define the job, boundaries, role, and expected behaviour.

Tools: connect the agent to knowledge and actions such as search, APIs, databases, code execution, MCP servers or business systems.

For enterprise use, that is only the starting point. You also need identity, authorization, network controls, content safety, session management, evaluation, tracing, versioning, rollback, and cost controls. Foundry Agent Service provides the surrounding platform capabilities, while letting you choose how much runtime logic your team owns.

Where should orchestration logic live and who should own the runtime?

Understanding the Runtime Boundary

When evaluating Foundry Agent Service, many teams focus on models. In practice, models are rarely the architectural differentiator.

Most architecture reviews eventually come down to three questions:

Who owns orchestration?
Who owns state?
Who owns operations?

Foundry Agent Service provides a managed platform for these concerns, but the amount of control retained by engineering teams depends on the selected agent type.

For most teams, the architectural decision usually comes down to one of two operating models

Prompt agents: declarative agents defined by a model, instructions, and tools, with a managed runtime.

Hosted agents: code-based agents that you package and run in Foundry, while the service manages the endpoint, identity, scaling, sessions, and observability.

Prompt Agents

With a Prompt Agent, engineering teams focus primarily on defining the model, instructions, tools, knowledge sources and identity configuration, while Foundry takes responsibility for the surrounding runtime.

Why teams start here

Prompt agents are usually the fastest route from an idea to a working, governed agent. They are a good fit when the behaviour can be expressed clearly through instructions and supported tools.

You need to deliver quickly.

The agent follows a fairly straightforward reasoning and tool-use loop.

Foundry-supported tools cover the required integrations.

You do not need custom libraries, middleware, or orchestration code.

You want Foundry to own compute, scaling, and patching.

Reviewers need an agent definition that is easy to inspect.

Good examples

Enterprise knowledge assistant. Employees ask about policies, engineering standards, procedures, or product information. The agent retrieves approved content and cites its sources.

Document review assistant. The agent checks a proposal or design against an approved rubric and returns structured findings, while a human keeps responsibility for the final decision.

Employee self-service agent. The agent answers questions and performs a small number of tightly scoped actions, such as checking request status or creating a support case.

A useful warning sign A Prompt agent is probably becoming the wrong fit when the prompt starts looking like application code. Large branching instructions, retry logic written in prose, state-machine behaviour, custom payload handling, framework middleware or real-time media are all signs that runtime logic belongs in code instead.

Hosted Agents

Hosted Agents move the responsibility boundary. Instead of defining behaviour through configuration alone, engineers deploy an actual application into Foundry Agent Service. Hosted Agents are framework-agnostic. Whether your team builds with Agent Framework, LangGraph, Semantic Kernel, OpenAI Agents SDK, or a custom runtime, Foundry can host the application while managing the surrounding operational services.

When Hosted agents make sense

You need a particular agent framework or custom orchestration engine.

The flow includes branching, parallel work, fan-out and fan-in, or human approvals.

Business rules require a deterministic state machine around model reasoning.

You need custom packages, middleware, algorithms, retries, caching, or error handling.

The client sends custom payloads or webhooks.

The session needs persistent files or custom state.

The design includes multi-agent orchestration or real-time voice.

Good examples

A bank onboarding workflow where uploaded documents must be validated, checked against multiple systems, and routed to a human when confidence drops below a threshold.

A fraud investigation agent that gathers transaction history, enriches data from multiple internal systems, applies bank-specific risk rules, requests additional evidence when required, and generates a recommended outcome for an investigator. The process involves long-running workflows, branching logic and audit requirements that are better suited to code-based orchestration.

A lending workflow that coordinates document collection, credit bureau checks, income verification, affordability assessments, policy exceptions, and approval routing. The process spans multiple systems and often requires deterministic decision paths that extend beyond prompt-driven orchestration.

A security operations agent that aggregates alerts from SIEM platforms, enriches incidents with threat intelligence, executes automated containment actions, opens tickets, requests approvals for high-impact remediation steps, and maintains a complete audit trail of decisions and actions.

The trade-off

More control also means more ownership. Your team must secure and patch the code and dependencies, test the runtime, manage supply-chain risk, and think about compute sizing, cold starts, session lifecycle, and cost. Hosted agents reduce platform plumbing, but they do not remove application engineering.

Choosing Prompt Agent/Hosted Agents

1) Runtime Control Is Usually the Real Requirement

A pattern I see quite often is teams arriving at the solution before they've fully articulated the requirement. The conversation usually starts with "We need a Hosted Agent," but after digging into the workload, the real requirements turn out to be things like persistent state, webhook processing, custom orchestration, background execution, framework-specific capabilities, or human approval workflows.

These are runtime concerns, not agent concerns and they're usually the factors that determine whether a Hosted Agent is necessary. Hosted Agents are valuable because they give engineering teams control over those aspects of execution while still offloading much of the operational infrastructure to Foundry.

This is also where teams most commonly choose the wrong agent type. A frequent assumption is that existing investments in frameworks such as LangGraph or Semantic Kernel automatically imply a Hosted Agent architecture. In practice, many of these workloads are relatively simple orchestration scenarios that can be implemented effectively as Prompt Agents, with lower operational overhead and less infrastructure to manage.

My advice is usually to start by identifying the runtime requirements rather than selecting an agent type. Once those requirements are clear, the right architecture often becomes obvious.

2) When Hosted Agents become mandatory

The moment you need custom Python packages, long-running workflows, external SDKs, deterministic orchestration or framework-specific capabilities, the conversation shifts from Prompt Agents to Hosted Agents.

What I would choose today

If I were starting a new project today, I'd begin with a Prompt Agent unless there was a clear reason not to. In my experience, Prompt Agents cover far more enterprise use cases than many teams initially expect. The best projects tend to start simple, prove value, learn where the limitations are, and then introduce Hosted Agents only when runtime customization becomes a genuine requirement. That progression is usually far less risky than leading with a fully custom solution.

Quickstart: Create a prompt agent - Microsoft Foundry | Microsoft Learn

Hosted agents in Foundry Agent Service - Microsoft Foundry | Microsoft Learn

From Features to Flow: How Real-World Adoption Reshaped the Azure Architecture Diagram Builder

arturoqu — Thu, 13 Aug 2026 22:10:46 GMT

In May, I introduced the open-source Azure Architecture Diagram Builder as a way to move from a natural-language prompt to an Azure architecture diagram, cost estimate, Well-Architected assessment, and deployment guidance. In July, I shared how the project had become agent-ready through Model Context Protocol (MCP).

Those posts described what the tool could do. The more interesting story came next: what happened when people actually used it.

As adoption grew, the central product question changed. It was no longer simply, Can AI generate an Azure architecture? It became:

How do we help an architect choose how to begin, improve a result without losing their work, validate it responsibly, and turn it into something another person can use?

That question reshaped the Azure Architecture Diagram Builder from a collection of capabilities into a guided workflow:

Create → Refine → Validate & Improve → Share or Build

This post explains what we learned, what changed in the product, and why the hardest part of AI-assisted architecture is not the first diagram. It is everything that comes after it.

TL;DR. Growing adoption created a feedback loop. Aggregate usage showed that people moved beyond generation into validation, recommendations, exports, and deployment guidance. Privacy-safe feedback revealed recurring problems with diagram integrity, preservation of human edits, cost credibility, export quality, and validation continuity. Those signals led to a four-stage architecture journey that keeps human judgment and professional review at the center. The same lesson now shapes agent access and the next product boundary: distinguish logical proposals from evidence-backed physical architecture.

Adoption created a product feedback loop

As of August 5, 2026, the first two Azure Architecture Blog articles had accumulated approximately 12,100 combined views. A refreshed view of deduplicated application telemetry through August 13 recorded:

Activity	Aggregate count
Architecture generation and refinement events	5,023
Well-Architected validations	960
Recommendations applied	175
Diagram exports	2,020
Deployment guides generated	212

As of August 13, the public repository had reached 45 stars and 14 forks. In GitHub’s current rolling 14-day window, the repository recorded 277 unique visitors and 67 unique cloners.

These numbers measure different things and should not be added together. Article views are not unique readers. Application activity uses anonymous telemetry identifiers, not verified people. GitHub traffic is a rolling aggregate window. The signals are useful because of the pattern they reveal, not because they can be combined into one headline user count.

Activity also accelerated during the period following the second article. Compared with the May 19–July 9 baseline, daily activity from July 10 through August 13 was approximately 7.0 times higher for architecture generation and refinement, 5.6 times higher for Well-Architected validation, and 5.9 times higher for recommendation application. The timing coincided with publication; it does not prove that the article alone caused the growth.

The important product lesson was simpler: people were not stopping after the first diagram.

They were testing alternatives, validating designs, applying recommendations, exporting artifacts, and asking how to move toward implementation. Generation was the entry point, not the complete job.

The first guided-journey signals reinforce the need for more than one starting path. Through August 13, the new journey instrumentation recorded 880 interactions from 174 anonymous identifiers across 241 sessions. At first start, structured brief/image generation and Guided Chat were selected at almost the same frequency (158 and 156 events), while template and live-Azure import added another 68 selections. These are interaction counts, not unique people or conversion rates, and the window is still too early to claim that the journey improves completion. They are enough to show that architecture work does not begin in one uniform way.

Figure 1. The in-product Start Here panel explains one complete architecture loop. The stages are recommendations, not gates, and direct access to every tool remains available.

Stage 1: Create — make the starting choice explicit

As capabilities accumulated, the first screen became harder to interpret. Architecture Chat and structured generation were both useful, but they competed for attention. Importing an existing architecture was available, yet easy to miss.

The new starting experience makes three paths explicit:

Starting path	Best suited for
Guided Chat	Exploring requirements conversationally and refining them over multiple turns
Generate Diagram	Providing a structured brief or image and producing a first architecture quickly
Import Existing	Opening an existing architecture or infrastructure artifact for analysis and editing

This is not a marketing landing page placed in front of the tool. It is a small decision point inside the authoring experience. Once a path is selected, the user lands on the real canvas.

The distinction matters because different architecture tasks begin with different levels of certainty. Sometimes the architect knows the target services. Sometimes the problem needs discovery. Sometimes the architecture already exists and the work is to understand or improve it.

The product should acknowledge those differences instead of pretending every design starts with a perfect prompt.

Figure 2. Three starting paths reflect three different architecture situations: discovery, structured generation, and analysis of an existing design.

Stage 2: Refine — preserve human work

One of the clearest feedback themes was not about adding another AI capability. It was about preventing AI from casually undoing human effort.

An architect might spend time arranging a one-page diagram for a review, resizing groups, moving labels, or emphasizing a specific boundary. A subsequent AI refinement could improve the service selection while disrupting that carefully prepared layout.

The design principle that emerged was straightforward:

AI acceleration should preserve deliberate human work by default.

Refinement now retains existing node positions, group geometry, sizes, and viewport context whenever possible. The model can change the architecture without treating every turn as permission to redraw the entire document.

The same principle applies beyond geometry:

Preserve the prior validation result when recommendations change the architecture.
Preserve the active light or dark theme in exported artifacts.
Preserve the distinction between the authoring canvas and the presentation deliverable.
Preserve user-configured pricing assumptions rather than replacing them with one fixed estimate.

This is a broader lesson for AI-assisted tools. A generated result is not the only source of value. The edits, judgments, and communication choices a person adds afterward are part of the artifact too.

Figure 3. In this controlled synthetic refinement, all eight existing service positions and four group anchors remained unchanged. The containing Application group expanded to accommodate the new edge tier, so preservation does not imply that every group dimension stays fixed.

Quality is structural, not only visual

A diagram can look polished while still being architecturally confusing. Early feedback exposed cases where a generated service appeared disconnected because a model referenced a display name instead of the service identifier used by the canvas.

The correction was not another prompt instruction alone. The application now resolves connection endpoints across identifiers, normalized service names, and service-type aliases. It repairs valid edges, drops invalid or self-referential edges, detects remaining orphan nodes, and records aggregate integrity signals.

That creates a more useful definition of diagram quality:

Are the services connected as intended?
Were any generated edges repaired or dropped?
Are there orphaned nodes?
Did refinement preserve the existing layout?
Did an architecture change receive a fresh validation?

Visual polish still matters, especially when an artifact leaves the editor. But structural integrity gives the product something deterministic to test and monitor.

Stage 3: Validate & Improve — treat validation as a lifecycle

The Azure Well-Architected Framework is most useful when validation becomes iterative rather than ceremonial.

The Diagram Builder can assess a proposed design across the five Well-Architected pillars, surface findings, and apply selected recommendations. But that workflow exposed an important state-management problem: when the architecture changed, the prior validation result disappeared along with the obvious route back to revalidation.

The updated experience keeps the previous report, marks it Revalidate Needed, and makes clear that the score describes an earlier state of the architecture. A new validation replaces it only after the updated design has been assessed.

This distinction prevents a stale score from looking current.

It also clarifies what an architecture-level assessment can and cannot prove. A diagram may show that a WAF, cache, backup service, or secondary region exists. It usually cannot prove that purge protection, diagnostic routing, encryption settings, role assignments, health probes, or failover policies are configured correctly.

That is why validation findings need to distinguish between:

Pattern-level gaps — missing or misplaced architectural components
Configuration-level gaps — required settings that must be verified in Infrastructure as Code or the deployed environment

Generated scores and recommendations help architects review a design; they do not replace an Azure Well-Architected Review, security review, deployment validation, or professional judgment.

Figure 4. Architecture changes make a previous validation historical, not useless. The result remains available while the interface clearly asks for a fresh validation.

The editing canvas and the final deliverable serve different purposes.

Canvas dots, handles, navigation controls, and selection states help during authoring. They can make an exported diagram feel unfinished. The Diagram Builder now separates those concerns with Plain, Dots, and Grid export backgrounds while preserving the active light or dark theme.

Figure 5. Authoring and delivery are different contexts. The upper view shows the editable canvas and its real export controls; the lower view is the Plain PNG produced from that same canvas, without editing chrome.

Cost language is deliberately qualified. Azure services often combine fixed, usage-based, and configuration-dependent charges. A baseline that includes six numerically priced services but excludes 20 usage-based items is not the total cost of the architecture. The output identifies those exclusions rather than treating missing values as zero.

The final stage also includes deployment guides and Infrastructure as Code. Here, honesty about artifact coverage is essential. A generated Bicep file may be a useful starter while still omitting private endpoints, diagnostic settings, failover configuration, or service-specific resources. The artifact should state what it implements, what remains conceptual, and whether Azure Resource Manager validation passed.

AI-generated diagrams, costs, validation results, deployment guides, and Infrastructure as Code should all be reviewed and validated before production use.

The same journey now extends to agents

The MCP server introduced in the previous article makes the Diagram Builder available to agent experiences such as Microsoft Scout. The four-stage journey provides a useful way to think about agent orchestration too:

Import or create one canonical architecture.
Refine it without silently changing the intended topology.
Validate it, apply supported improvements, and revalidate.
Render or generate artifacts with explicit coverage and limitations.

The current MCP surface exposes 12 tools, three resources, and three reusable prompts. It can normalize an existing architecture, validate and harden it deterministically, estimate regional costs from a dated pricing snapshot, render presentation/technical/cost views, and generate Bicep, Terraform, and deployment guidance. The calling agent still owns orchestration and reasoning; the MCP server is intended to remain a deterministic architecture capability, not a second hidden agent.

The native MCP renderer can project one canonical architecture into three communication profiles:

Presentation emphasizes the primary request path, reduces supporting labels, and removes pricing.
Technical preserves complete connection detail for engineering inspection.
Cost retains the focused composition while adding service-level pricing assumptions, a fixed-priced baseline, and explicit exclusions.

These are MCP-generated SVG views, not Blueprint diagrams or screenshots of the editable web canvas. The services, connections, and groups remain the same; only the information treatment changes.

Figure 6. Native AADB MCP output from one 8-service, 9-connection, 4-group architecture. Presentation prioritizes the story, Technical exposes connection detail, and Cost foregrounds pricing assumptions and exclusions.

Recent work on the MCP renderer added purpose-built presentation, technical, and cost profiles. More importantly, testing agent-generated artifacts reinforced an accountability principle: a polished diagram and a compiled Bicep file do not prove deployability.

An agent workflow should report whether topology changed, whether validation improved, which services are represented only conceptually, and whether the generated IaC passed Azure preflight. That is more useful than an unsupported claim that a design is production-ready.

Trust also includes the tool boundary itself. The hosted MCP endpoints now require a bearer token for real session operations; missing or incorrect credentials are rejected. A shared token is appropriate for the current controlled integration, but it is not the end state for enterprise multi-user access. Entra ID/OAuth, per-client authorization, rotation, and revocation remain future hardening work.

Figure 7. The guided lifecycle extends beyond the web application. In this synthetic Scout run with GPT-5.6 Sol, the agent used authenticated AADB MCP tools to validate, harden, cost, render, and generate starter artifacts while explicitly reporting coverage gaps and that nothing was deployed.

Learning from adoption without identifying people

Product learning does not require reconstructing individual identities.

The findings behind this article use aggregate, deduplicated application telemetry, public article counters, public repository totals, and paraphrased feedback themes. They do not correlate Application Insights identifiers, feedback records, GitHub accounts, or email addresses.

Written feedback remains submittable without contact information. When someone explicitly opts into follow-up, the email address is stored with the feedback record in Cosmos DB and is not sent to normal product telemetry. The current 180-day expiry field is a retention marker; automated deletion must be implemented and verified before describing that retention period as enforced.

Those boundaries matter for both product design and public writing:

Aggregate activity rather than profiling individuals.
Paraphrase themes rather than publishing comments without permission.
Keep optional contact consent separate from telemetry.
Avoid presenting anonymous identifiers as confirmed people.
Avoid claiming that publication timing proves acquisition causality.

This is not a claim of legal compliance. It is a product discipline: collect less, preserve user agency, and make only the claims the evidence supports.

What changed

The guided journey is the visible result, but the deeper change is how the project now evaluates progress.

Earlier question	Better question
Did the model generate a diagram?	Did it generate a connected and understandable architecture?
Did the user click Validate?	Was the current architecture validated, and was it revalidated after changes?
Did export start?	Did a professional artifact finish generating successfully?
Does the IaC compile?	What does it actually implement, and does Azure preflight pass?
How many features exist?	Can an architect understand the next useful step?

The model portfolio continued to evolve as well. The production selector now contains 15 configured entries, including MAI-Thinking-1 (Public Preview). But the more consequential changes in this article are deliberately model-independent: preserve human work, keep state and provenance explicit, qualify generated artifacts, and authenticate the tools agents can call.

The goal is not to remove flexibility. Architects can still open any tool directly, rearrange the canvas, reject recommendations, change pricing assumptions, or export at any point.

The goal is to make the workflow coherent without pretending architecture itself is linear.

The next boundary: logical versus physical architecture

Recent feedback points to a harder problem than adding another model or export format. Architects working with private Azure AI landing zones need to distinguish shared platform resources from project-owned resources, preserve VNet and subnet boundaries, and reason about CIDRs, NSGs, route tables, private endpoints, DNS, and managed identities.

The current Topology mode can show services and relationships, but it should not imply exact physical fidelity when those facts are absent. A useful logical diagram answers what exists and how it interacts. A physical or low-level design must answer where it is deployed, how it is isolated, and which values came from evidence.

That is the next technical direction I am exploring: an evidence-aware Physical Architecture view backed by deterministic reconstruction from Terraform plan/state, ARM, or a live Azure inventory. Exact fields would be labeled as observed or resolved; AI suggestions would remain explicitly proposed; unsupported or missing inputs would be reported instead of silently invented.

This capability is not shipped today, and it will require its own schema, validation rules, layout, security review, and evaluation set. That distinction matters. The lesson from adoption is not to put every architecture concern into one crowded canvas. It is to make each artifact’s purpose and evidence boundary clear.

Try it, challenge it, help shape what comes next

The Azure Architecture Diagram Builder remains open source, and the live experience is available today:

Live app: https://aka.ms/diagram-builder
Source code: github.com/Arturo-Quiroga-MSFT/azure-architecture-diagram-builder
Getting started: Documentation and deployment guidance

The next phase is to measure whether the guided journey helps people complete the full loop, especially recommendation-to-revalidation and artifact-generation success. In parallel, I am beginning the narrower physical-architecture investigation described above. Both efforts will use aggregate signals, reviewed fixtures, and sufficiently large cohorts rather than individual journey reconstruction.

Try the workflow with a real architecture problem. Tell me where the handoffs are unclear, where the diagram loses intent, or where an artifact claims more than it implements. Those are the gaps worth fixing next.

Measurement note: Article views are rounded public counters observed August 5, 2026. Application figures use deduplicated retained telemetry through August 13 and anonymous identifiers. GitHub totals and rolling 14-day traffic were observed August 13. The comparison windows are May 19–July 9 and July 10–August 13. These signals have different populations and must not be added together. Timing comparisons show concurrent activity, not causal attribution.

Skill or Sub-Agent. Choosing AI Capabilities You Will Actually Reuse

KishoreKumarPattabiraman — Thu, 30 Jul 2026 20:50:02 GMT

Audience: Cloud architects, platform engineers, engineering leaders

The wrong first question

Most teams building AI capabilities start with the wrong question. They ask which model to use.

The model matters less than the shape of the capability around it. The first real fork is this. Are you building a skill or a sub-agent? Get that wrong and no model choice will save you. A skill and a sub-agent are two different delivery shapes, and each one fails at the other one's job.

The insight

The choice between a skill and a sub-agent is not about model power. It comes down to four checks. How the work iterates, whether the output carries a voice, how far an early wrong turn spreads, and how often it recurs. Score each, count which way they lean, and the shape falls out. An even split means build both, and let the skill drive the sub-agent.

The three sections below take the checks worth a pause. Frequency is the plain one: a one-off craft piece leans to a skill, a repeatable batch job to a sub-agent.

A skill lives inside the conversation. It reads files, asks a question, refines with the author, and keeps a human in the loop mid-flight. A sub-agent takes one prompt, runs to completion, and returns one report. Both are useful, for different work.

Dimension	Skill	Sub-agent
Iteration	Conversation, many turns	One hand-off, one pass
Voice	Holds a style profile and applies it	Drifts toward generic by design
Human gate	Every turn	Once, at the end
Best for	Craft, subjective output	Bounded, structured output

Table 1. The same three dimensions decide the shape every time.

1. Decide the iteration model first

Before anything else, architects should ask how the work actually happens. Is it a conversation or a hand-off? That single answer removes most of the ambiguity.

Craft work needs back and forth
Batch work needs one clean pass
Conversations need memory of the thread
Hand-offs need a bounded input and a clear output

A skill is right when the value comes from iteration. A blog post, a design review, a tricky refactor. A sub-agent is right when the work is well defined and the output is the deliverable.

In practice

The pattern that works: use a skill when the team expects three or four rounds of "close, but change this". The trade-off: a skill costs more attention per run because a human stays involved. The trap to avoid: forcing iterative craft into a one-shot agent and then editing the output by hand every time.

2. Voice fidelity decides craft work

Some outputs have a voice. An article, a customer email, an architecture narrative. Others do not. A query result, a data export, a status summary. The line between them is not cosmetic. It decides which shape survives review.

Voice-heavy work favours a skill
Voice-neutral work favours a sub-agent
Skills can hold a style profile and apply it
Sub-agents drift toward generic by design

When the output carries a name, fidelity is the whole game. A capable model with no voice anchor produces text that reads like it came from a committee.

In practice

The pattern that works: give a skill an explicit voice profile with banned phrases and cadence rules. Let it self-check before it shows anyone anything. The trade-off: the profile takes real effort to write once. The trap to avoid: expecting a stateless agent to match a personal style from a single prompt.

Implementation note

A voice profile is not documentation. It lives in the skill definition, an executable contract the skill checks itself against before a draft is ever shown. A small profile goes a long way.

# voice-profile (excerpt) banned_phrases: [seamless, robust, game-changing, leverage the power] forbid: [em-dash, semicolon, exclamation in body] max_avg_sentence_words: 20 require: - one "In practice" block per section - a closing discussion question self_check: run before any draft is shown to a human

3. Put the human gate where the risk is

Every AI capability needs a human review gate. The design question is where that gate sits. Placement is the difference between catching a problem early and unpicking it later.

Skills gate continuously, turn by turn
Sub-agents gate once, at the end
Continuous gates catch drift early
End gates are cheaper but riskier for craft

If a wrong turn early corrupts everything after it, the team wants a skill. If the work is bounded and a bad output is easy to spot and discard, an end gate is fine.

In practice

The pattern that works: match the gate to the blast radius. High blast radius and subjective quality point to a skill. Low blast radius and objective output point to a sub-agent. The trap to avoid: a one-shot agent doing forty minutes of unattended work that a human then has to unpick.

4. The pattern that scales is both

The mature answer is not one or the other. It is a skill on top of a sub-agent. The two shapes compose cleanly when each one keeps to its own job.

The skill orchestrates and holds the voice
The sub-agent executes bounded sub-tasks
The human reviews at the skill layer
Each layer does what it is good at

Figure 1. In the combined pattern the human reviews at the skill layer, and the sub-agent only touches the bounded task.

The skill runs the conversation and keeps quality. When it needs a bounded, repeatable job done, it delegates to a sub-agent. The result is iteration where craft lives and automation where the work is mechanical.

In practice

The pattern that works: a skill drafts and refines an article with the author, and calls a sub-agent to fetch and summarise reference material. The trade-off: two layers are more to build than one. The trap to avoid: collapsing both into a single agent and losing either the voice or the automation.

The operational trade-offs

Shape is not only a design choice. It shows up in cost, latency, and how you debug a bad run. Architects should price these in before committing to a pattern.

A skill spends more tokens and more human minutes per run
A sub-agent spends compute once and returns fast
A skill fails in small, visible steps you can correct
A sub-agent fails as one block you inspect after the fact

The cost of a skill is attention. Someone stays in the loop and that time is real. The cost of a sub-agent is rework. When a one-shot run goes wrong, the whole output is suspect and someone redoes it. Observability follows the same split. A skill leaves a turn-by-turn trail you can read. A sub-agent leaves one input and one output. You instrument the boundary and log the prompt and the result. Pick the shape whose failure mode your team can afford. The wrong shape does not announce itself. It shows up later as a cost line or a rewrite.

Two capabilities, one team

Consider a team standardising its engineering work with AI. Two capabilities land on the backlog in the same week. The first is recurring status queries. Well defined input, structured output, no voice. A stateless sub-agent fits. One prompt in, one report out, gate at the end. It works on day one and keeps working.

The second is authored technical content. Subjective, voice-heavy, many rounds of refinement. The reflex is to reuse the sub-agent that just shipped. That reflex is the mistake. The queries stay clean. The content reads flat and generic, and every draft needs a heavy human rewrite. Rebuilt as a skill with a voice profile and a turn-by-turn gate, the same work compounds instead of fighting back. Same team, same models, two different shapes of work, and only one right tool for each.

What teams get wrong

The common pattern is defaulting to whichever shape the team built first. A team ships one sub-agent, likes it, and forces every new problem into a sub-agent. Or it builds one skill and runs everything as a conversation, including batch work that should be automated.

It looks like consistency. It feels like reuse. But it leads to craft work that reads generic and batch work that needs babysitting. The fix is not a better model. It is naming the shape of the work before picking the tool.

The three shapes to watch for in your own stack:

A voiced deliverable coming out of a one-shot agent, rewritten by hand every run. A skill wearing a sub-agent costume.
A batch job run as a conversation and babysat turn by turn. A sub-agent wearing a skill costume.
A large workflow forced into one agent that holds neither the voice nor the automation. Two shapes collapsed into one.

Name which one you are looking at, and the fix picks itself.

A quick way to decide

When a new capability lands on the backlog, run four checks before picking a tool.

Iteration: conversation or one hand-off
Output: subjective and voiced, or structured and neutral
Blast radius: does an early wrong turn corrupt the rest
Frequency: a one-off craft piece, or a repeatable batch job

Three or more answers leaning subjective and iterative point to a skill. Three or more leaning structured and repeatable point to a sub-agent. A split answer usually means a skill orchestrating a sub-agent underneath.

Figure 2. Score the four checks and count the leanings. Three or four one way pick the shape. An even split means a skill orchestrating a sub-agent.

Where to start depends on what you have already built

The framework is the destination. Where you start depends on what your team has shipped so far. Find your stage and take the one first move for it this week.

Stage	First move, this week	Watch out for
Just starting, nothing built yet	Pick the single task you repeat most and write a one-paragraph capability brief for it, iteration, output, blast radius, and frequency, before you build. The brief names the shape, and the shape names the tool.	Building a general assistant before you have named one concrete job.
One capability, reused for everything	List every job you push through the one tool, find the one whose shape does not match, and rebuild just that one in the right shape. You do not need to replace what works.	Forcing new work into the tool you already have.
A small fleet, a handful of capabilities	Take your largest layered workflow and split it, a skill that holds the voice and the human gate on top, a sub-agent that does the bounded work underneath.	Capabilities that duplicate each other with no composition between them.

Table 2. Same framework, different first move. What you have already built decides where the leverage is this week.

Your setup also shapes the answer. A solo builder should optimise for their own voice and iteration speed, where one strong skill beats three thin ones. A platform team should standardise the capability brief and a shared voice profile, so the fleet stays consistent as more people add to it, and a new capability inherits the house style instead of drifting from it.

Figure 3. Whatever you have built so far, the first move has the same shape. Name the work before the tool, then match the shape to the tool.

The shift

The shift is from "what can the model do" to "what shape is the work". Model capability is table stakes now. The advantage is in matching the capability to the work. Our own capability fleet is built this way, interactive skills and autonomous sub-agents in separate places with an orchestrator on top, and that split is what keeps it maintainable as it grows.

Iterative and voice-heavy points to a skill. Bounded and mechanical points to a sub-agent. Large and layered points to a skill orchestrating sub-agents. Decide that first, and the model becomes a detail the team can change later without rebuilding anything.

Most teams collapse both ideas into "automation" and end up with neither. The teams that separate them build capabilities they actually reuse.

Want to discuss? Drop a comment with patterns you have seen in your environment. I read every reply.

Mastering GitHub Copilot Budgets: How to Prevent Surprise Overages Without Blocking Devs

gauravbhardwaj — Wed, 29 Jul 2026 17:33:31 GMT

1. The Core Mental Model: The Water Park Analogy

To understand Copilot billing, think of your enterprise as a water park:

The Shared Pool (Included Usage): Every Copilot Business ($19/mo) and Enterprise ($39/mo) seat license adds a set number of free AI credits into a shared corporate water tank (1,900 or 3,900 credits/seat, respectively).
Phase 1 (Pool Phase): All licensed developers drink from this central pool for free until it runs empty.
Phase 2 (Metered Overage): Once the pool is dry, extra water costs $0.01 per credit. This phase only activates if your enterprise explicitly enables the "AI Credit Paid Usage" policy.
User-Level Budgets (ULBs): Personal wristbands limiting how much water one person can consume across both Phase 1 and Phase 2 combined.
Group / Enterprise Budgets: Group bar tabs that kick in only during Phase 2 to cap overage spending.

2. How GitHub Evaluates Requests (The 3-Step Decision Flow)

Every time a developer uses an AI credit feature (like GitHub Copilot Chat or custom Agents), GitHub evaluates the request through a 3-step hierarchy:

Step 1: The Personal Guardrail (User-Level Budgets)

ULBs cap total consumption (Pool + Metered). These are always hard stops. The system selects the most specific rule:

Individual ULB: Overrides everything (e.g., $50 for a Lead Data Scientist).
Cost Center ULB: Overrides Universal (e.g., $20/dev for Engineering, $5/dev for Marketing).
Universal ULB: The global baseline for all licensed users.

Step 2: The Included Pool Check

If the user hasn't hit their ULB, the system checks if pooled credits remain. You can also configure Included Usage Controls on Cost Centers to ring-fence a team's pool draw to match their contributed seats.

Step 3: Group & Enterprise Overage Caps

Once the pool is empty, metered charges begin ($0.01/credit). The system checks overage budgets in order:

Cost Center Budget
Organization Budget
Enterprise Budget

⚠️ Critical FinOps Note: Group/Enterprise budgets do NOT stop usage by default. You must explicitly enable the setting:

Stop usage when budget limit is reached

Without this setting toggled ON, overage charges will accrue uncapped!

3. Key Takeaways & Common Pitfalls

Budget Control	What It Limits	Active Phase	Hard Stop Built-In?
Individual / Cost Center / Universal ULB	Single user total credits	Pool + Metered	Yes (Always)
Cost Center Budget	Team overage spend	Metered Only	Only if Stop Usage is ON
Org Budget	Organization overage spend	Metered Only	Only if Stop Usage is ON
Enterprise Budget	Enterprise overage spend	Metered Only	Only if Stop Usage is ON

The "Lowest Headroom Wins" Rule

If a developer has $10 remaining on their personal ULB, but the Enterprise Overage Budget only has $1 left before hitting its limit, the Enterprise limit will block the user. Whichever budget hits its ceiling first wins.

What Gets Blocked vs. What Keeps Working?

When a developer hits a budget limit:

❌ Blocked: Copilot Chat, CLI, Agents, and complex reasoning models (anything consuming AI credits).
✅ Unblocked: Standard inline code completions and next edit suggestions (included free with seat licenses).

Conclusion: Best Practices for Admins

Establish a Universal ULB First: Set a reasonable baseline (e.g., $10–$15/user) to ensure fair pool access.
Use Cost Center ULBs for Group Granularity: Avoid managing thousands of individual budgets by setting per-user caps at the Cost Center level.
Always Turn On "Stop Usage": Ensure your enterprise and cost center overage budgets have the hard stop toggle enabled to prevent unexpected monthly bills.
Isolate R&D with Cost Center Exclusion: Use exclusion rules for teams that require independent spending authority outside the general enterprise cap.

Token Economics in Practice

prateekwrites — Tue, 28 Jul 2026 17:57:41 GMT

Introduction: The cheap-token trap

Token prices alone are a poor economic model for agents. The price of reaching a fixed capability has fallen sharply — In a 2025 Report Stanford's AI Index reported a roughly 280-fold drop in the cost of GPT-3.5-level inference between late 2022 and late 2024, and Epoch AI tracks steep (if uneven) per-benchmark price declines. The intuitive conclusion is that agents are getting cheaper to run. The operational reality is the opposite.

Agents turn cheaper inference into longer, stochastic trajectories: growing context windows, repeated tool schemas, retries, reflection loops, and sub-agent fan-out. In one study of agentic coding, repeated runs of the same agent on the same task varied in token cost by as much as 30× for coding agents. When a single logical task can cost you thirty times more depending on the path the agent takes, optimizing average cost per token will happily make the wrong system look efficient. Similar argument can be made for other agentic systems where we may need more than one tries, more than one MCP Calls, Reasoning or use of multiple skills, hooks or tool calls to arrive at a completed task.

So, the leading question of token economics isn't "what's the token price?" It's "what does it cost to get one accepted unit of useful work — and how confident can we be in that number before the agent runs?"

The unit that actually matters: Cost per accepted task

I use token economics to mean managing the unit economics of useful AI work under uncertainty. The meaningful unit is cost per accepted task, not cost per token.

Let A = 1 mean a task passed its acceptance rubric. The long-run unit cost of a policy π is approximately:

The numerator is expected task cost; the denominator is the probability the output is actually acceptable. This follows the FinOps distinction between successful and unsuccessful AI outputs and the recommendation to connect cost with workload value. It is a working definition for this project, not a quoted standard — but it reframes the engineering problem immediately. A "cheaper" policy that halves cost while dropping acceptance from 95% to 70% is more expensive per accepted task, and only this ratio makes that visible.

That reframing turns "pick the cheapest model" into a five-step discipline:

Forecast a distribution, not a single token estimate.
Select a cost policy that is plausible for the task and its risk.
Enforce routing, context, cache, and budget controls during execution.
Evaluate whether the output still clears a workload-specific quality floor.
Revert unsafe savings, reconcile predicted vs. actual usage, and calibrate the next forecast.

From a metric to a controller

If cost is a random variable, the objective is a stochastic one. Minimize expected task cost subject to two constraints — a quality floor on every workload segment, and a bound on how often you blow the budget

subject to a per-segment quality floor:

and a chance constraint on budget breach:

Here π is the policy; C_task is total task cost; Q_s is quality for a supported segment s with floor Q_min; B is the budget; and ε is the tolerated breach probability. The pieces are all borrowed — stochastic optimization for the expected-cost objective; FrugalGPT and Confident Adaptive Language Modeling for the LLM precedent of cutting cost while preserving performance; SRE service-level objectives for treating "acceptable service" as an action-driving threshold and Group DRO for the insight that averages hide group failures; and Charnes–Cooper chance-constrained programming for the probabilistic budget limit. The synthesis — wiring them into one agent controller — is the contribution.

Two honest caveats travel with this controller: Q_s needs a confidence-adjusted lower bound (sparse segments shouldn't trigger changes on two samples), and the chance constraint is not a guarantee until your forecast's percentile coverage is calibrated against real traces. A modeled P95 is a planning estimate, not a promised 5% breach bound.

Two halves of the loop: feed-forward and feedback

The current work is result of two self-prototypes — FutureTokenPredictor and TokenGov — built to make agent unit economics operable on Azure. These are reusable implementation patterns and experiments.

The controller splits cleanly into a planning half and a runtime half.

FutureTokenPredictor is the feed-forward side. It models workflow archetypes and uncertain iteration counts to produce P50/P95-style planning estimates before execution and recommends a policy. It stays outside the request path.
TokenGov is the feedback side. Its request path applies the admitted cost policy; an out-of-band control plane evaluates outcomes and changes externalized policy when quality regresses. Runtime telemetry then flows back to the predictor as calibration data for the next forecast.

Neither half is sufficient alone. Prediction without control is a spreadsheet. Control without quality feedback silently degrades your hardest segments. The value is the wire between them: a forecast that becomes an enforceable policy, an eval verdict that can reverse a cost action, and actuals that sharpen the next forecast.

How the equation lands on Azure

This is where token economics stops being a metric and becomes architecture. Each term in the controller maps to a concrete Azure control:

Controller term	Azure control in practice
π (policy)	Externalized in Azure App Configuration; enforced by API Management GenAI gateway (routing, context, cache, token policies)
E[C_task \| π] (expected cost)	Reconstructed from APIM gateway, model, and Application Insights telemetry
Q_s (segment quality)	Azure AI Foundry evaluation over golden sets and sampled production traces
B, ε (budget, breach tolerance)	Forecast-informed limits and Azure Monitor alerts; Cost Management for allocation
Reversion	A Monitor-triggered Azure Function tightens or reverts policy in App Configuration — closing the eval-to-enforcement loop without a code deployment

Most of these primitives already exist and are individually documented: APIM provides token quotas, semantic caching, and token metrics; Foundry Model Router offers cost/balanced/quality routing modes; Foundry cloud evaluation scores datasets and sampled traces. The interesting gap they don't close on their own is the connected mechanism — an evaluation verdict that can constrain or reverse a cost-saving action, and actual usage that improves the next forecast.

Here is the full two-plane view. FutureTokenPredictor forecasts and recommends before execution; TokenGov owns runtime enforcement and quality-triggered reversion; prediction IDs join forecasts to actual telemetry so calibration can improve the next estimate.

From Concept to Implementation

Version 1 release the Token Prediction and forecast ability using a local mcp server called FutureTokenPredictor using a local MCP server modeled behind a simple UI, where you can create an assessment for your UI Workload. It lets you simple describe the AI / Agentic Solution you want to build and suggested a topology for it. From there , depending on your model selection, the studio, helps you predict the range of token usage and its estimated costs.

In full version, this forecast is used to build a policy and govern your AI Spend accordingly.

If you want to read more about the FutureTokenPredictor and how it works, check out my earlier blog Agentic Currency – Tokens and AI Infra: Full-Stack Cost Prediction for Autonomous Agents

Version 2 with full governance and control will be released soon.

TokenEconomics is available in the GitHub Repo TokenEconomics

Clone it, experiment and test it out. Please provide feedback via a pull request on the repo or directly here via comments

Happy Reading!

References

The 2025 AI Index Report | Stanford HAI

Chance-Constrained Programming | JSTOR

How are AI agents spending your tokens? - Stanford Digital Economy Lab

FinOps for AI Overview

AI gateway capabilities in Azure API Management | Microsoft Learn

Model router for Microsoft Foundry concepts - Microsoft Foundry | Microsoft Learn

Preserve a Legacy IP During Azure Migration with Private Link Service Direct Connect

kumarshashikaushal — Mon, 27 Jul 2026 12:31:30 GMT

Why this matters

Phased migrations to Azure frequently stall on a single, unglamorous constraint: an application with a hard-coded dependency address. The usual options are to re-address the application or change its configuration, and both add development effort, regression testing, and risk to a migration that is already time-boxed.

This article demonstrates a pattern that removes that constraint from the critical path. Using Azure Private Link Service Direct Connect together with a Private Endpoint, the migrated application continues to reach its dependency on exactly the same private address it has always used, while that dependency remains on premises during the interim phase.

The practical benefits are:

Reduced migration risk. No application re-addressing or code change, so no regression cycle and a significantly smaller blast radius.
Faster migration. Web and application tiers can move now rather than waiting for the backend dependency to be modernised, so phase one is no longer blocked by phase two.
Avoided remediation cost. No development effort to change hard-coded addressing, and none of the associated testing spend.
Business continuity preserved. The application connects to the same address throughout, which removes addressing-related cutover risk mid-migration.
Overlapping address space handled. One of the most common reasons enterprise datacentre exits stall is addressed directly, without VNet peering.

The pattern is deliberately a bridge rather than a destination. It buys time to modernise properly; it does not remove the need to do so.

Customer challenge

The customer was planning a phased migration of a legacy application to Azure. The web and application tiers could move first, but a dependent database service needed to remain on premises during the initial phase. The application used a hard-coded private address for this dependency, and changing the application configuration or addressing would have increased migration risk and required additional development and regression testing.

Customer requirements

The customer needed transitional architecture that would allow the web and application tiers to run in Azure while continuing to reach the on-premises database through its existing application-facing address. The solution is needed to:

Preserve the existing application-facing database address.
Migrate the web and application tiers to Azure without immediate application code or configuration changes.
Keep the database on-premises until a later migration phase.
Provide private, controlled connectivity between the Azure-hosted application tiers and the on-premises database.
Validate the complete traffic path before considering the pattern for production use.

Solution and lab overview

This lab evaluates Azure Private Link Service Direct Connect as a transitional connectivity pattern for a phased migration. The capability can connect a Private Link service directly to a privately routable destination address without requiring the destination to sit behind a traditional load balancer. That makes it relevant to direct IP-based scenarios such as database connections, custom applications, and privately reachable on-premises resources.

In the lab, the migrated web and application virtual machines connect to a Private Endpoint in an extended prod VNet. The endpoint maps to a Private Link Service Direct Connect resource in a hub VNet. The hub then routes the traffic across a site-to-site VPN to the simulated on-premises database server. The detailed IP values used to reproduce the lab are introduced in the architecture and deployment sections rather than in the customer narrative.

Preview notice
Private Link Service Direct Connect is in public preview and available only in selected regions. Review the documented requirements, limitations, and regional availability before enabling it in a subscription or using it in a production design.

Why this pattern works

Legacy applications often depend on fixed private addresses that cannot be changed safely during the first migration wave.
Reusing or overlapping address space complicates conventional routed hybrid connectivity.
Private Link Service Direct Connect can route traffic to a specific privately routable destination address and bypass the traditional load-balancer requirement.
The pattern provides a temporary abstraction that lets the Azure-hosted application tier keep using its expected destination while the actual dependency remains on-premises.

When this pattern fits

Consider this pattern when:

An application has a hard-coded or otherwise fixed dependency address that cannot be changed in the current migration phase.
The application tier is ready to migrate ahead of its backend dependency.
On-premises and Azure address spaces overlap, which rules out straightforward VNet peering.
You need a controlled, reversible transition step rather than a single large cutover.

Look at other options when:

The application can be reconfigured or modernised within the migration window. Where that is possible, resolving the dependency properly is the better outcome.
You need a permanent end-state architecture. This pattern is transitional by design.
Your target region does not yet support Private Link Service Direct Connect, which is in public preview with limited regional availability.

Architecture

Figure 1. Logical architecture and traffic path

Component	Address space / IP	Location	Purpose
Hub VNet	10.230.0.0/21	West US 2	Hosts the VPN Gateway, Azure Bastion, test subnet, and Private Link Service subnet.
On-premises VNet	10.220.0.0/21	North Central US	Simulates the on-premises network with production and development subnets.
On-premises production subnet	10.220.1.0/24	North Central US	Hosts vm-db at 10.220.1.6. IIS on this VM simulates the test service on TCP port 80.
Extended production VNet	10.220.1.0/24	West US 2	Hosts the migrated web and application VMs that retain the original application-facing address space.
Private Link Service subnet	10.230.3.0/24	Hub VNet	Hosts the Private Link Service NAT IP configurations 10.230.3.10 and 10.230.3.11.
Private Endpoint	10.220.1.6	Extended production subnet	Provides the local endpoint used by the migrated application tiers to reach the on-premises dependency.

Traffic flow

Testing note: In this lab, vm-db represents the on-premises database server. To validate the end-to-end network path without deploying a database engine, we install IIS on vm-db and use its web server on TCP port 80 as a lightweight test service. A successful HTTP response confirms that traffic reaches vm-db through the intended private connectivity path; it does not validate database functionality.

The web or application VM in the extended production subnet connects to 10.220.1.6 on TCP port 80. For this test, that port is served by IIS running on vm-db.
Because 10.220.1.6 is assigned to the Private Endpoint in the same subnet, the connection is sent to the Private Endpoint locally.
The Private Endpoint connection maps to the Private Link Service Direct Connect resource in the hub VNet.
The Private Link Service Direct Connect resource uses the destination IP address 10.220.1.6.
The hub VNet routes 10.220.0.0/21 through the VPN Gateway to the simulated on-premises network.
The simulated on-premises vm-db receives the request through IIS, and the HTTP response returns through the same private connectivity path.

Prerequisites and assumptions

An Azure subscription where the Microsoft.Network/AllowPrivateLinkserviceUDR feature flag can be registered.
Azure CLI and Az PowerShell modules available in the execution environment. The commands below assume PowerShell syntax for variables.
Permission to create resource groups, VNets, subnets, NSGs, public IPs, VPN gateways, local network gateways, Private Link Service, Private Endpoint, NICs, and VMs.
A region supported by Private Link Service Direct Connect. This lab uses West US 2 and North Central US.
This lab intentionally uses overlapping application-facing IP space in the extended Azure VNet and the on-premises production subnet. Do not peer the hub VNet and the extended production VNet directly.

Credential hygiene
The original lab used a sample shared key and local administrator password. For publication, replace all secrets with strong unique values or Key Vault-backed automation. This document uses placeholders only.

Deploy the lab

Run the following sequence from an authenticated PowerShell session with Azure CLI available. Review every variable before execution. The lab uses IIS on the simulated database VM only as a lightweight TCP and HTTP target; the objective is to validate the network path rather than database behavior.

# -----------------------------
# Variables
# -----------------------------
$RG="rg-pls-demo"
$LOCATION="westus2"
$VNET="vnet-hub"
$VNET_PREFIX="10.230.0.0/21"
$GW_SUBNET="GatewaySubnet"
$GW_PREFIX="10.230.1.0/24"
$BASTION_SUBNET="AzureBastionSubnet"
$BASTION_PREFIX="10.230.2.0/24"
$PLS_SUBNET="snet-pls"
$PLS_PREFIX="10.230.3.0/24"
$TEST_SUBNET="snet-test"
$TEST_PREFIX="10.230.4.0/24"
$ONPREM_RG="onprem-group"
$ONPREM_LOCATION="northcentralus"
$ONPREM_VNET="vnet-onprem"
$ONPREM_VNET_PREFIX="10.220.0.0/21"
$ONPREM_GW_SUBNET="GatewaySubnet"
$ONPREM_GW_PREFIX="10.220.0.0/24"
$ONPREM_PROD_SUBNET="snet-prod"
$ONPREM_PROD_PREFIX="10.220.1.0/24"
$ONPREM_DEV_SUBNET="snet-dev"
$ONPREM_DEV_PREFIX="10.220.2.0/24"
$ONPREM_DB_IP="10.220.1.6"
$VNET_EXTENDED="Extended-VNET"
$VNET_PREFIX_EXTENDED="10.220.1.0/24"
$EXT_SUBNET="snet-extended-prod"
$EXT_PREFIX="10.220.1.0/24"
$PLS_NAME="pls-directconnect-db"
$PLS_IP1="10.230.3.10"
$PLS_IP2="10.230.3.11"
$DESTINATION_IP=$ONPREM_DB_IP
$PE_NAME="pe-db"
$PE_CONNECTION_NAME="pe-to-pls"
$PLS_JSON="pls-ipconfigs.json"
$PE_JSON="pe-ipconfig.json"
$LNG_ONPREM="lng-onprem"
$LNG_AZURE="lng-azure"
$PIP_ONPREM_VPNGW="pip-onprem-vpngw"
$ONPREM_VPNGW="onprem-vpngw"
$PIP_VPNGW="pip-vpngw"
$VPNGW_HUB="vpngw-hub"
$SHAREDKEY="<replace-with-strong-shared-key>"
$ADMINUSER="adminuser"
$ADMIN_PASSWORD="<replace-with-strong-password>"

# -----------------------------
# Create hub resource group, VNet, NSGs, and subnets
# -----------------------------
az group create --name $RG --location $LOCATION
az network vnet create --resource-group $RG --name $VNET --location $LOCATION --address-prefixes $VNET_PREFIX
az network nsg create -g $RG -n nsg-gateway -l $LOCATION
az network nsg create -g $RG -n nsg-bastion -l $LOCATION
az network nsg create -g $RG -n nsg-pls -l $LOCATION
az network nsg create -g $RG -n nsg-test -l $LOCATION
az network vnet subnet create -g $RG --vnet-name $VNET -n $GW_SUBNET --address-prefixes $GW_PREFIX
az network vnet subnet create -g $RG --vnet-name $VNET -n $BASTION_SUBNET --address-prefixes $BASTION_PREFIX
az network vnet subnet create -g $RG --vnet-name $VNET -n $PLS_SUBNET --address-prefixes $PLS_PREFIX --network-security-group nsg-pls --disable-private-link-service-network-policies true
az network vnet subnet create -g $RG --vnet-name $VNET -n $TEST_SUBNET --address-prefixes $TEST_PREFIX --network-security-group nsg-test

# -----------------------------
# Configure Azure Bastion NSG rules and associate with the Azure bastion Subnet
# -----------------------------
$NSG="nsg-bastion"
$nsg=Get-AzNetworkSecurityGroup -ResourceGroupName $RG -Name $NSG
$nsg.SecurityRules.Add((New-AzNetworkSecurityRuleConfig -Name "AllowHttpsInbound" -Priority 120 -Direction Inbound -Access Allow -Protocol Tcp -SourceAddressPrefix Internet -SourcePortRange * -DestinationAddressPrefix * -DestinationPortRange 443))
$nsg.SecurityRules.Add((New-AzNetworkSecurityRuleConfig -Name "AllowGatewayManagerInbound" -Priority 130 -Direction Inbound -Access Allow -Protocol Tcp -SourceAddressPrefix GatewayManager -SourcePortRange * -DestinationAddressPrefix * -DestinationPortRange 443))
$nsg.SecurityRules.Add((New-AzNetworkSecurityRuleConfig -Name "AllowAzureLoadBalancerInbound" -Priority 140 -Direction Inbound -Access Allow -Protocol Tcp -SourceAddressPrefix AzureLoadBalancer -SourcePortRange * -DestinationAddressPrefix * -DestinationPortRange 443))
$nsg.SecurityRules.Add((New-AzNetworkSecurityRuleConfig -Name "AllowBastionHostCommunication" -Priority 150 -Direction Inbound -Access Allow -Protocol Tcp -SourceAddressPrefix VirtualNetwork -SourcePortRange * -DestinationAddressPrefix VirtualNetwork -DestinationPortRange 8080,5701))
$nsg.SecurityRules.Add((New-AzNetworkSecurityRuleConfig -Name "AllowSshRdpOutbound" -Priority 100 -Direction Outbound -Access Allow -Protocol Tcp -SourceAddressPrefix * -SourcePortRange * -DestinationAddressPrefix VirtualNetwork -DestinationPortRange 22,3389))
$nsg.SecurityRules.Add((New-AzNetworkSecurityRuleConfig -Name "AllowAzureCloudOutbound" -Priority 110 -Direction Outbound -Access Allow -Protocol Tcp -SourceAddressPrefix * -SourcePortRange * -DestinationAddressPrefix AzureCloud -DestinationPortRange 443))
$nsg.SecurityRules.Add((New-AzNetworkSecurityRuleConfig -Name "AllowBastionCommunicationOutbound" -Priority 120 -Direction Outbound -Access Allow -Protocol Tcp -SourceAddressPrefix VirtualNetwork -SourcePortRange * -DestinationAddressPrefix VirtualNetwork -DestinationPortRange 8080,5701))
$nsg.SecurityRules.Add((New-AzNetworkSecurityRuleConfig -Name "AllowHttpOutbound" -Priority 130 -Direction Outbound -Access Allow -Protocol Tcp -SourceAddressPrefix * -SourcePortRange * -DestinationAddressPrefix Internet -DestinationPortRange 80))
Set-AzNetworkSecurityGroup -NetworkSecurityGroup $nsg
az network vnet subnet update -g $RG --vnet-name $VNET -n $BASTION_SUBNET --network-security-group nsg-bastion

# -----------------------------
# Deploy Azure Bastion and hub VPN Gateway
# -----------------------------
az network public-ip create -g $RG -n pip-bastion -l $LOCATION --sku Standard --zone 1 2 3
az network bastion create -g $RG -n bastion-hub --public-ip-address pip-bastion --vnet-name $VNET -l $LOCATION --sku Standard
az network public-ip create -g $RG -n $PIP_VPNGW -l $LOCATION --sku Standard --zone 1 2 3
az network vnet-gateway create -g $RG -n $VPNGW_HUB --public-ip-addresses $PIP_VPNGW --vnet $VNET --gateway-type Vpn --vpn-type RouteBased --sku VpnGw2AZ

# -----------------------------
# Create extended Azure prod VNet and migrated web/app VMs(+ vm-test in hub VNET)
# -----------------------------
az network vnet create --resource-group $RG --name $VNET_EXTENDED --location $LOCATION --address-prefixes $VNET_PREFIX_EXTENDED
az network nsg create -g $RG -n nsg-extended -l $LOCATION
az network vnet subnet create --resource-group $RG --vnet-name $VNET_EXTENDED --name $EXT_SUBNET --address-prefixes $EXT_PREFIX --network-security-group nsg-extended
az network nic create -g $RG -n nic-testvm --vnet-name $VNET --subnet $TEST_SUBNET
az vm create -g $RG -n vm-test --nics nic-testvm --image Win2022Datacenter --admin-username $ADMINUSER --admin-password $ADMIN_PASSWORD
az network nic create -g $RG -n nic-web --vnet-name $VNET_EXTENDED --subnet $EXT_SUBNET --private-ip-address 10.220.1.4
az vm create -g $RG -n vm-web --nics nic-web --image Win2022Datacenter --admin-username $ADMINUSER --admin-password $ADMIN_PASSWORD
az network nic create -g $RG -n nic-app --vnet-name $VNET_EXTENDED --subnet $EXT_SUBNET --private-ip-address 10.220.1.5
az vm create -g $RG -n vm-app --nics nic-app --image Win2022Datacenter --admin-username $ADMINUSER --admin-password $ADMIN_PASSWORD

# -----------------------------
# Create simulated on-premises VNet and VPN Gateway
# -----------------------------
az group create --name $ONPREM_RG --location $ONPREM_LOCATION
az network vnet create --resource-group $ONPREM_RG --name $ONPREM_VNET --location $ONPREM_LOCATION --address-prefixes $ONPREM_VNET_PREFIX
az network vnet subnet create --resource-group $ONPREM_RG --vnet-name $ONPREM_VNET --name $ONPREM_GW_SUBNET --address-prefixes $ONPREM_GW_PREFIX
az network vnet subnet create --resource-group $ONPREM_RG --vnet-name $ONPREM_VNET --name $ONPREM_PROD_SUBNET --address-prefixes $ONPREM_PROD_PREFIX
az network vnet subnet create --resource-group $ONPREM_RG --vnet-name $ONPREM_VNET --name $ONPREM_DEV_SUBNET --address-prefixes $ONPREM_DEV_PREFIX
az network public-ip create --resource-group $ONPREM_RG --name $PIP_ONPREM_VPNGW --location $ONPREM_LOCATION --sku Standard
az network vnet-gateway create --resource-group $ONPREM_RG --name $ONPREM_VPNGW --public-ip-addresses $PIP_ONPREM_VPNGW --vnet $ONPREM_VNET --gateway-type Vpn --vpn-type RouteBased --sku VpnGw2AZ

# -----------------------------
# Create local network gateways and VPN connections
# -----------------------------
$AZURE_VPN_PUBLICIP = az network public-ip show -g $RG -n $PIP_VPNGW --query ipAddress -o tsv
$ONPREM_VPN_PUBLICIP = az network public-ip show -g $ONPREM_RG -n $PIP_ONPREM_VPNGW --query ipAddress -o tsv
Write-Host "Azure VPN IP : $AZURE_VPN_PUBLICIP"
Write-Host "OnPrem VPN IP : $ONPREM_VPN_PUBLICIP"
az network local-gateway create -g $RG -n $LNG_ONPREM --gateway-ip-address $ONPREM_VPN_PUBLICIP --local-address-prefixes 10.220.0.0/21
az network local-gateway create -g $ONPREM_RG -n $LNG_AZURE --gateway-ip-address $AZURE_VPN_PUBLICIP --local-address-prefixes 10.230.0.0/21
az network vpn-connection create -g $RG -n conn-to-onprem --vnet-gateway1 $VPNGW_HUB --local-gateway2 $LNG_ONPREM --shared-key $SHAREDKEY
az network vpn-connection create -g $ONPREM_RG -n conn-to-azure --vnet-gateway1 $ONPREM_VPNGW --local-gateway2 $LNG_AZURE --shared-key $SHAREDKEY
az network vpn-connection show -g $RG -n conn-to-onprem --query "{Name:name,Status:connectionStatus}" -o table
az network vpn-connection show -g $ONPREM_RG -n conn-to-azure --query "{Name:name,Status:connectionStatus}" -o table

# -----------------------------
# Create on-premises DB VM
# -----------------------------
az network nic create -g $ONPREM_RG -n nic-db --vnet-name $ONPREM_VNET --subnet $ONPREM_PROD_SUBNET --private-ip-address $ONPREM_DB_IP
az vm create -g $ONPREM_RG -n vm-db --nics nic-db --image Win2022Datacenter --admin-username $ADMINUSER --admin-password $ADMIN_PASSWORD

# -----------------------------
# Register PLS Direct Connect preview feature
# -----------------------------
az feature register --namespace Microsoft.Network --name AllowPrivateLinkserviceUDR
az feature show --namespace Microsoft.Network --name AllowPrivateLinkserviceUDR --query properties.state -o tsv
az provider register --namespace Microsoft.Network

# -----------------------------
# Create PLS Direct Connect IP configuration JSON
# -----------------------------
$PLS_SUBNET_ID = az network vnet subnet show -g $RG --vnet-name $VNET -n $PLS_SUBNET --query id -o tsv
@"
[
{
    "name": "ipconfig1",
    "primary": true,
    "private-ip-allocation-method": "Static",
    "private-ip-address": "$PLS_IP1",
    "subnet": { "id": "$PLS_SUBNET_ID" }
},
{
    "name": "ipconfig2",
    "primary": false,
    "private-ip-allocation-method": "Static",
    "private-ip-address": "$PLS_IP2",
    "subnet": { "id": "$PLS_SUBNET_ID" }
}
]
"@ | Out-File -FilePath $PLS_JSON -Encoding ascii

az network private-link-service create -g $RG -n $PLS_NAME -l $LOCATION --destination-ip-address $DESTINATION_IP --ip-configurations "@$PLS_JSON"
az network private-link-service show -g $RG -n $PLS_NAME --query "name:name,provisioningState:provisioningState,destinationIpAddress:destinationIPAddress}" -o table

# -----------------------------
# Create Private Endpoint in the extended Azure production subnet
# -----------------------------
az network vnet subnet update -g $RG --vnet-name $VNET_EXTENDED -n $EXT_SUBNET --private-endpoint-network-policies Disabled
$PLS_ID = az network private-link-service show -g $RG -n $PLS_NAME --query id -o tsv
@"
[
{
    "name": "pe-ipconfig1",
    "private-ip-address": "$ONPREM_DB_IP"
}
]
"@ | Out-File -FilePath $PE_JSON -Encoding ascii

az network private-endpoint create -g $RG -n $PE_NAME -l $LOCATION --vnet-name $VNET_EXTENDED --subnet $EXT_SUBNET --private-connection-resource-id $PLS_ID --connection-name $PE_CONNECTION_NAME --ip-configs "@$PE_JSON"
az network private-link-service show -g $RG -n $PLS_NAME --query privateEndpointConnections -o table

# -----------------------------
# Install IIS on the on-premises DB VM and publish a test page
# -----------------------------
az vm run-command invoke -g $ONPREM_RG -n vm-db --command-id RunPowerShellScript --scripts "Install-WindowsFeature Web-Server -IncludeManagementTools"
az vm run-command invoke -g $ONPREM_RG -n vm-db --command-id RunPowerShellScript --scripts "Set-Content -Path 'C:\inetpub\wwwroot\index.html' -Value '<h1>OnPrem DB Server IIS Page</h1>'"
az vm run-command invoke -g $ONPREM_RG -n vm-db --command-id RunPowerShellScript --scripts "Get-WindowsFeature Web-Server; netstat -ano | findstr :80"

Validate the solution

Checkpoint	Command	Expected result
VPN tunnel status	az network vpn-connection show -g $RG -n conn-to-onprem --query connectionStatus -o tsv az network vpn-connection show -g $ONPREM_RG -n conn-to-azure --query connectionStatus -o tsv	Both connections show Connected.
DB web service	az vm run-command invoke -g $ONPREM_RG -n vm-db --command-id RunPowerShellScript --scripts "netstat -ano \| findstr :80"	The DB VM is listening on TCP port 80.
Private Endpoint connection	az network private-link-service show -g $RG -n $PLS_NAME --query "privateEndpointConnections[].{Connection:name,Status:privateLinkServiceConnectionState.status}" -o table az network private-endpoint show -g $RG -n $PE_NAME --query "{Name:name,ProvisioningState:provisioningState}" -o table	The connection status is Approved, and the Private Endpoint provisioning state is Succeeded.
Hard-coded IP test from web and app tiers	az vm run-command invoke -g $RG -n vm-web --command-id RunPowerShellScript --scripts "Test-NetConnection 10.220.1.6 -Port 80" az vm run-command invoke -g $RG -n vm-app --command-id RunPowerShellScript --scripts "Test-NetConnection 10.220.1.6 -Port 80"	TcpTestSucceeded is True.
HTTP test from web and app tiers	az vm run-command invoke -g $RG -n vm-web --command-id RunPowerShellScript --scripts "Invoke-WebRequest http://10.220.1.6 -UseBasicParsing" az vm run-command invoke -g $RG -n vm-app --command-id RunPowerShellScript --scripts "Invoke-WebRequest http://10.220.1.6 -UseBasicParsing"	The response contains the sample IIS page.

Production considerations

Do not peer the extended prod VNet with the Azure hub VNet. The extended VNet uses 10.220.1.0/24, which overlaps with the 10.220.0.0/21 on-premises prefix that the hub routes through the VPN Gateway. The repeated database IP is intentionally exposed to the web and app tiers only through the Private Endpoint abstraction, not through direct VNet peering.
The PLS subnet must have Private Link service network policies disabled. The Private Endpoint subnet must have Private Endpoint network policies disabled for this lab pattern.
Private Link Service Direct Connect requires at least two IP configurations. The lab uses 10.230.3.10 and 10.230.3.11 for high availability alignment.
The destination IP must be privately routable and reachable from the PLS path. In this lab, the hub routes 10.220.0.0/21 to the on-premises side through VPN Gateway.
For production, validate return path, NSG rules, route tables, firewall policies, DNS behavior, logging, and operational ownership before adopting the pattern.

Clean up

When the lab is no longer required, remove the lab resource groups. Validate that no shared resources exist in these groups before deletion.

az group delete --name rg-pls-demo --yes --no-wait
az group delete --name onprem-group --yes --no-wait

Conclusion

This lab shows how a Private Endpoint and Private Link Service Direct Connect can provide a controlled transition path when an application tier moves to Azure before a fixed-address dependency. The pattern can reduce immediate application change, but it should remain an interim architecture: validate routing symmetry, network security, monitoring, regional availability, operational ownership, and a clear plan to remove the legacy address dependency before production adoption.

Used well, this approach reduces migration risk, avoids application remediation cost, and keeps a datacentre exit moving when it would otherwise stall. In a recent engagement it unblocked a phased migration for an enterprise customer whose application carried a hard-coded dependency address, without a single change to application code.

Private Link Service Direct Connect is in public preview and available in selected regions, so confirm regional availability, routing, DNS behaviour, and operational ownership before any production adoption.

If you try the lab, I would welcome your feedback, and any variations you find useful in your own migrations.

References

About the author

Kumar Kaushal is a Senior Digital Cloud Solutions Architect at Microsoft, focused on helping customers design and validate practical Azure architectures for cloud migration, hybrid connectivity, networking, resiliency, and modernization.

Reminder: Path to Production for Agents Webinar Series Starts Next Week

brauerblogs — Thu, 23 Jul 2026 00:15:07 GMT

Next week, join Microsoft for the Path to Production for Agents webinar series—a free, six-session technical training designed to help organizations move from AI experimentation to secure, scalable, production-ready agent solutions. The series takes place July 27–28 and is aimed at architects, technical leaders, engineers, and AI practitioners looking to operationalize AI at enterprise scale.

Across six expert-led sessions, you'll learn how to:

Establish governance foundations for AI at scale
Design production-ready AI platforms and landing zones
Build reliable multi-agent architectures
Implement AgentOps practices for deployment and observability
Secure and govern AI systems
Optimize performance, cost, and scalability

Each session includes proven Microsoft architecture patterns, real-world engineering guidance, and practical techniques you can apply immediately to accelerate your path from prototype to production.

Register today: Path to Production for Agents Registration

Don't miss this opportunity to gain the knowledge and frameworks needed to confidently deploy production-grade AI agents at scale. We look forward to seeing you next week!

Hypervelocity Engineering: Accelerating Enterprise AI with Azure AI Landing Zones

VimalVerma — Fri, 17 Jul 2026 14:57:27 GMT

Artificial Intelligence is evolving at unprecedented speed. The challenge for enterprises is no longer building AI solutions—it is engineering AI platforms that can adapt, scale, and govern innovation continuously. Hypervelocity Engineering (HVE) provides the engineering operating model that enables this transformation.

Executive Summary

Artificial Intelligence is transforming how enterprises design and operate digital platforms. Traditional Enterprise Architecture practices—built around static documentation, periodic governance reviews, and manual implementation—are struggling to keep pace with the rapid evolution of AI workloads.

HyperVelocity Engineering (HVE) introduces a disciplined engineering operating model that combines AI-assisted decision making, platform engineering, automation, and continuous governance. When applied to Azure AI Landing Zone, HVE enables architects to evolve from producing architecture documents to continuously engineering secure, governed, and scalable AI platforms.

This article demonstrates how HVE Core principles map naturally to Azure AI Landing Zone and how Enterprise Architects can use RPIR (Research, Plan, Implement, Review) as a continuous architecture lifecycle.

Why Enterprise AI Needs a New Engineering Model

Organizations across every industry are rapidly adopting Generative AI, intelligent agents, and AI-assisted business processes. Yet many enterprise AI initiatives encounter the same obstacles:

AI projects are developed independently across business units.
Platform capabilities evolve slower than AI innovation.
Governance and security become reactive rather than proactive.
Infrastructure is treated as a one-time deployment instead of a continuously evolving product.
Engineering teams spend excessive time provisioning environments instead of delivering business value.

Traditional cloud engineering practices were designed for application modernization. Enterprise AI introduces new demands—rapid experimentation, scalable model deployment, secure data access, and continuous compliance. Meeting these demands requires a fundamentally different engineering approach.

Hypervelocity Engineering (HVE) addresses this challenge by enabling organizations to build platforms that evolve at the speed of AI while maintaining enterprise-grade governance, security, and operational excellence.

From Traditional Engineering to Hypervelocity Engineering

Enterprise architecture is shifting from project-centric delivery to platform-centric engineering.

Traditional Engineering	Hypervelocity Engineering
Project-based delivery	Product & platform engineering
Manual provisioning	Infrastructure as Code
Governance after deployment	Governance by Design
Static architecture	Continuous evolution
Infrastructure owned by IT	Self-service engineering platforms
Periodic releases	Continuous delivery
Manual operations	AI-assisted engineering

This transformation allows engineering organizations to focus less on repetitive operational tasks and more on innovation.

What is HyperVelocity Engineering?

Hypervelocity Engineering (HVE) is Microsoft's engineering operating model designed to accelerate software and platform delivery through automation, reusable engineering patterns, platform engineering, AI-assisted development, and continuous feedback.

Unlike traditional methodologies, HVE is not another architecture framework or software product. Instead, it defines how engineering organizations operate to deliver secure, governed, and continuously improving solutions.

Its core philosophy is straightforward:

Engineer platforms, automate everything practical, measure continuously, and improve through rapid feedback while keeping humans accountable for critical decisions.

HVE combines several modern engineering disciplines into a unified operating model:

Platform Engineering
Infrastructure as Code (IaC)
Policy as Code
Security by Design
AI-assisted engineering
Continuous observability
DevSecOps
Human-in-the-loop governance

Together, these capabilities enable organizations to innovate faster without compromising security or compliance.

Azure AI Landing Zone as the Reference Implementation

Azure AI Landing Zone provides the foundational platform required to deploy enterprise AI workloads securely and consistently.

Typical capabilities include:

Microsoft Entra ID
Management Groups
Azure Policy
Hub-and-Spoke Networking
Private Endpoints
Azure Firewall
Azure API Management
Azure AI Foundry
Azure AI Search
Azure Kubernetes Service (AKS)
Azure Monitor
Defender for Cloud
Infrastructure as Code (Bicep/Terraform)

HVE provides the engineering operating model that continuously evolves this platform.

The Core Principles of Hypervelocity Engineering

Hypervelocity Engineering is built around six complementary principles.

Outcome-Driven Engineering

Every engineering decision should align with measurable business outcomes rather than technology adoption alone. Success is measured by customer value, not infrastructure deployment.

Platform Engineering

Rather than building infrastructure repeatedly for every project, organizations create reusable, self-service platforms that accelerate application delivery while maintaining consistency.

Automation Everywhere

Automation extends beyond deployments. Infrastructure provisioning, governance, security validation, policy enforcement, testing, and operations should all be automated wherever possible.

Security by Design

Security is integrated into every engineering stage—from identity and networking to deployment pipelines and operational monitoring—reducing risk while improving delivery speed.

AI-Assisted Engineering

AI enhances developer productivity by generating code, documentation, infrastructure templates, architecture recommendations, and testing artifacts. Humans remain responsible for validation and governance.

Continuous Observability and Feedback

Operational insights, telemetry, cost analysis, and user feedback continuously improve future engineering decisions.

Applying RPIR to Enterprise Architecture

Rather than treating architecture as a one-time activity, HVE applies the RPIR cycle continuously.

Phase	Enterprise Architecture Activities	Azure Capabilities	Expected Outcome
Research	Gather business requirements, architecture standards, security baselines, and reference guidance	Azure AI Search, CAF, Well-Architected Framework	Evidence-based architecture decisions
Plan	Define target architecture, evaluate options, create ADRs, prioritize roadmap	Azure Landing Zones, Azure Policy, Management Groups	Governed architecture blueprint
Implement	Deploy infrastructure, policies, networking, AI services, and automation	Bicep, Terraform, Azure DevOps, GitHub Actions	Repeatable, automated platform deployment
Review	Validate security, compliance, reliability, cost, and operational readiness	Defender for Cloud, Azure Monitor, Azure Advisor	Continuous improvement and governance

Instead of ending after deployment, the Review phase feeds directly back into Research, enabling continuous architecture evolution.

Mapping HVE Principles to Azure AI Landing Zone

HVE Principle	Azure AI Landing Zone Implementation
Outcome Driven	Align landing zone design with business outcomes and AI strategy
Platform Engineering	Shared AI Hub, reusable landing zone modules, self-service provisioning
Automation Everywhere	Infrastructure as Code, CI/CD pipelines, Policy as Code
Security by Design	Zero Trust, Microsoft Entra ID, Key Vault, Defender for Cloud
Observability & Feedback	Azure Monitor, Log Analytics, cost insights, operational metrics
AI-Assisted Engineering	Azure AI Foundry, GitHub Copilot, AI Search, architecture assistants

These principles ensure that governance, security, and operational excellence are embedded throughout the platform lifecycle rather than applied only during reviews.

Reference Architecture Walkthrough

The architecture consists of four logical layers:

AI Hub

The AI Hub provides centralized ingress and shared platform services using Azure Application Gateway, Azure Firewall, and Azure API Management. These services enforce security, routing, and governance before requests reach AI workloads.

Shared Platform Services

Shared Azure Kubernetes Service (AKS), Azure AI Foundry, and Azure AI Search provide reusable AI capabilities for multiple business units. Centralizing these services improves scalability, operational consistency, and cost efficiency.

AI Spoke Environments

Each business unit deploys isolated AI workloads into dedicated spoke environments. Private networking, isolated data stores, and workload-specific resources maintain tenant separation while consuming shared platform capabilities.

Platform Governance

Infrastructure as Code, Azure Policy, Microsoft Entra ID, Defender for Cloud, Azure Monitor, and FinOps capabilities operate across every layer to provide continuous governance, security, and operational visibility.

This architecture reflects HVE's platform engineering philosophy by separating shared capabilities from workload-specific implementations while maintaining centralized governance.

Benefits for Enterprise Architects

Adopting HVE alongside Azure AI Landing Zone provides measurable advantages:

Accelerates architecture delivery through AI-assisted design.
Standardizes landing zone deployments using reusable platform modules.
Embeds governance through Policy as Code and Infrastructure as Code.
Improves security with Zero Trust and continuous compliance validation.
Enables continuous architecture evolution using the RPIR lifecycle.
Reduces operational overhead through automation and observability.
Supports FinOps practices with integrated cost monitoring and optimization.
Creates auditable architecture decisions through Architecture Decision Records (ADRs).

Rather than replacing architects, HVE allows architects to focus on strategic decisions while AI assists with repetitive engineering activities.

Common Anti-Patterns to Avoid

Organizations should avoid several common pitfalls when adopting enterprise AI:

Building separate AI platforms for every project
Treating governance as a post-deployment activity
Relying on manual infrastructure provisioning
Operating without continuous telemetry and feedback
Deploying AI services without standardized platform engineering

Hypervelocity Engineering addresses these anti-patterns by embedding automation, governance, and continuous improvement into the engineering lifecycle.

Design Recommendations

When adopting Hypervelocity Engineering for Azure AI Landing Zones:

Start with measurable business outcomes rather than technology selection.
Treat your AI Landing Zone as a reusable platform product.
Automate infrastructure, security, and governance using Infrastructure as Code and Policy as Code.
Integrate AI-assisted engineering while maintaining human oversight for architectural decisions.
Use the RPIR loop to continuously evolve your platform based on operational insights and business feedback.
Embed security, observability, and FinOps from the outset to support sustainable growth.

Conclusion

Enterprise AI success depends on more than deploying advanced models—it requires an engineering operating model capable of delivering secure, scalable, and continuously evolving platforms.

Hypervelocity Engineering provides that model by combining platform engineering, automation, AI-assisted development, and continuous feedback into a unified approach. When paired with Azure AI Landing Zones, it transforms static infrastructure into a living platform that accelerates innovation while preserving governance, security, and operational excellence.

For enterprise architects, the value is clear: Azure AI Landing Zones establish the foundation, and Hypervelocity Engineering ensures that foundation continuously adapts to changing business needs and the rapid pace of AI innovation.

In the era of enterprise AI, success belongs to organizations that engineer for continuous evolution—not just initial deployment. Hypervelocity Engineering offers the operating model to achieve exactly that.

From Policy to Proof: Governing AI to Scale Human Ambition and Machine Intelligence

manasa_ramalinga — Wed, 15 Jul 2026 19:51:51 GMT

Enterprises are moving from AI experiments to AI in production. Copilots are in the flow of daily work, custom applications are built on foundation models, and autonomous agents are beginning to take actions on behalf of the business. As organizations combine human ambition with machine intelligence, trust becomes the defining requirement for scaling AI responsibly and realizing AI's full potential.

That shift changes the governance question from "can we use AI responsibly?" to a harder one: "can we prove, continuously, that every AI system and agent in our estate is safe, compliant, observable, and accountable?"

The organizations getting this right have stopped treating governance as a static document that lives in a policy binder. They treat it as an operating system: a connected set of policies, controls, telemetry, and evidence that runs alongside AI everywhere it operates. Governance defines what should happen. Observability and evaluations verify what is actually happening. Audit and response prove it, and feed what they learn back into policy. The loop, not any single control, is what makes governance real.

Every control in this map ladders up to a principled foundation: Microsoft's Responsible AI Standard, whose six principles (fairness, reliability and safety, privacy and security, inclusiveness, transparency, and accountability) set the bar these controls exist to meet.

This post lays out a practical reference map for that operating system: the domains AI governance must cover, the cross-cutting runtime enforcement layer that operationalizes them, and the Microsoft services that support each one.

The four pillars of AI governance

At its core, AI governance does four things. Skip any one of them and a gap opens up -policy without control is aspirational, control without visibility is blind, and visibility without proof cannot withstand an audit.

Policy - Define the rules, and who owns them. What's acceptable use, which use cases are approved, how each is classified by risk, and -critically -who is accountable for outcomes at each stage.
Control - Enforce the boundaries, proportionate to risk. Turn policy into guardrails, access decisions, and lifecycle gates that constrain systems at runtime -with tighter controls on higher-risk use cases.
Visibility - Observe behavior, including fairness and quality. Capture logs, metrics, traces, token consumption, tool calls, and runtime signals that reveal what AI is truly doing -not just latency and cost, but grounding, drift, and fairness signals.
Proof - Audit, prove, and improve. Produce the evidence to demonstrate compliance and transparency (explainability and traceability you can show regulators and customers), investigate incidents, and feed what you learn back into policy.

What AI governance covers: the domain map

A complete AI governance program spans nine core domains, with runtime enforcement operating as a cross-cutting layer across several of them. Use this taxonomy to map your own requirements to concrete controls:

Policy: Responsible AI policies, approval workflows, and guardrails.
Data governance: Classification, sensitivity labels, DLP, retention, and lineage.
Model governance: Validation, versioning, documentation, and change control.
Observability: Logs, metrics, traces, token usage, and runtime signals.
Evaluations: Quality, safety, grounding, drift, and agent task success.
Security: Threat detection, posture management, proactive AI red teaming, and prompt-injection resistance.
Identity & access: RBAC, least privilege, conditional access, and agent identities.
Audit & compliance: Evidence, eDiscovery, legal hold, and reporting.
Agent governance: Registry, lifecycle, policy enforcement, and fleet visibility.

Cross-cutting runtime enforcement: Runtime enforcement is not a separate governance program. It is the control layer that operationalizes policy, security, identity, data protection, observability, and agent governance while AI systems are running. It governs live interactions between users, agents, models, tools, APIs, MCP servers, A2A agent APIs, and enterprise systems through authentication, authorization, traffic controls, token governance, quotas, policy checks, and operational guardrails.

The Microsoft governance stack

Microsoft's approach is a composed platform rather than a single product — AI lifecycle, data governance, identity, security, observability, and audit working together:

1. Policy and the control plane: define and enforce

Governance starts with a place to define the rules and decide who owns them. This is where responsible AI and acceptable use policies are authored; where new use cases come in through intake, get classified by risk, and are approved or rejected; where you decide up front which decisions and agent actions require human sign-off versus running autonomously within guardrails; and where accountability is assigned across the deploy, update, and retire lifecycle. Enforcing those rules through runtime guardrails, security posture, agent-level controls, and API-level controls is the job of the sections that follow; the work here is defining what “acceptable” means before anything is switched on.

Services that support it:

Microsoft Foundry -Compliance workspace for authoring and housing responsible AI policies and use case approvals.
Azure Policy -Resource level governance to codify those rules as enforceable baselines.
Azure API Management (AI Gateway): Runtime enforcement for AI applications, models, agents, tools, and APIs, including authentication, authorization, token governance, quota management, and policy enforcement.

(Agent specific policy and lifecycle controls via Microsoft Agent 365 are covered in Section 6.)

Practical takeaway: establish a governance baseline before AI and agent adoption scales -including deciding up front where a human stays in the loop. Retrofitting controls onto a sprawling estate is far harder than starting from one.

2. Data governance and compliance: Purview at the center

Most AI risk is ultimately data risk: the wrong information reaching a model, leaving through a response, or being retained without a policy. Microsoft Purview is the primary governance engine for AI interactions and enterprise data controls, and it works in three moves:

Discover - Identify AI usage, sensitive prompts and responses, and overall risk posture with DSPM for AI.
Protect - Apply sensitivity labels, DLP, access boundaries, and retention policies.
Prove - Generate audit trails, eDiscovery evidence, and compliance reporting.

If the question is “how do we govern the data flowing through AI?” - Purview is the anchor.

3. Model governance, safety, and evaluations: quality at build time and runtime

Evaluations are what make governance measurable, and they belong at two points in the lifecycle:

Pre-deployment - Release gating on safety, grounding, quality, fairness, and task success before anything ships.
Post-deployment - Runtime scoring for drift, quality thresholds, and harmful-output alerts once a system is live.

Services that support it:

Microsoft Foundry evaluators: quality, safety, RAG grounding, and agent task metrics
Azure AI Content Safety: detects unsafe or harmful content in prompts and responses. This is also integrated in Microsoft Foundry as guardrails that protect against unsafe content and direct and indirect jailbreak prompt injection attacks.
ASSERT (open source): Microsoft's open-source, policy-driven evaluation framework, short for Adaptive Spec-driven Scoring for Evaluation and Regression Testing. ASSERT turns your specific requirements and organizational policies into targeted, safety-focused test cases rather than generic benchmarks, and runs across frameworks such as LangChain, CrewAI, LiteLLM, and OpenAI, so evaluation is not locked to any single stack. ASSERT also pairs with runtime controls to close the loop: measure a failure rate, apply a control (see Agent Control Specification in section 6), then re-run the same evaluation to prove the rate dropped. That turns "we mitigated it" into a measured before-and-after.

Evaluations are not separate from governance - they are what turn governance from a promise into measurable proof.

4. Observability and runtime monitoring: know what is actually happening

Once AI is in production, you need to see its behavior in real time. Capture four kinds of signal:

Logs: prompts, responses, and decisions
Metrics: latency, token usage, quality scores, request volume, and quota consumption
Traces: agent reasoning paths, tool calls, model calls, and inter-service dependencies
Alerts: policy violations, drift, harmful outputs, and abnormal traffic patterns

Microsoft Foundry provides purpose-built observability for AI applications and agents across three connected capabilities. Evaluation measures quality, safety, and reliability throughout development. Monitoring tracks deployed systems in real-world conditions. Tracing, built on OpenTelemetry standards, captures the execution flow of LLM calls, tool invocations, and agent decisions, so you can debug multi-step reasoning rather than guess at it, across frameworks including LangChain, LangGraph, the OpenAI Agents SDK, and the Microsoft Agent Framework. The Agent Monitoring Dashboard brings these signals together for production traffic: token usage, latency, run success rates, evaluation scores, and red-teaming results in one view, backed by Azure Monitor, Application Insights. From the same surface you can turn on continuous evaluation to score sampled live responses, scheduled evaluations to detect drift against benchmarks, scheduled red team scans to probe deployed agents for emerging risks, and alerts that fire when outputs fail quality thresholds or produce harmful content.

Critically, this coverage is not limited to agents built on Foundry. Agents running elsewhere can be onboarded through the AI Gateway and instrumented with OpenTelemetry semantic conventions to send telemetry to the same Application Insights instance, so one dashboard, one evaluation pipeline, and one alerting surface cover the whole estate.

The broader observability fabric

Foundry observability composes with the rest of the platform: Azure Monitor, Application Insights, Log Analytics, Purview Audit, and Microsoft Agent 365 dashboards. Azure API Management (AI Gateway) extends visibility further by capturing AI traffic patterns, token consumption, API usage, policy violations, and agent-to-tool interactions, providing runtime insight into AI workloads at the network boundary.

Practical takeaway: wire observability before launch, not after the first incident. Continuous evaluation on sampled production traffic is the cheapest early-warning system you can buy, and the trace data it rides on is the same evidence an auditor will ask for later.

5. Security and identity governance: protect the AI estate

AI expands the attack surface, so governance has to include security and identity across the following:

Threat protection -Prompt injection, jailbreak, data exfiltration, and unsafe actions -with Microsoft Defender for Cloud and Microsoft Defender XDR.
Proactive adversarial testing -Find failures before attackers do, with the AI Red Teaming Agent in Microsoft Foundry for automated agent probing and PyRIT, Microsoft’s open-source red-teaming framework.
Security posture -Misconfigurations, exposed endpoints, and risky resources -with Defender for Cloud and Azure Policy.
Identity & access -Who can use AI, and what data, apps, and agents it can reach -with Microsoft Entra ID and RBAC.
Runtime access enforcement: Who can call which model, tool, API, or MCP server, under which conditions, and with what level of throttling, inspection, and monitoring.
Secrets & network -Protecting keys and enforcing private access and network boundaries -with Azure Key Vault and Private Link.
Device management-Discover, monitor, and govern unmanaged AI agents and the endpoints they run on -with Microsoft Intune and Microsoft Defender for Endpoint (MDE).

Cross-cutting runtime enforcement

As AI systems increasingly rely on APIs, tools, MCP servers, enterprise applications, and agent-to-agent interactions, governance must extend beyond model build-time controls to the live interactions that connect those systems. Runtime enforcement helps operationalize the broader AI governance program by enforcing who can call what, under which conditions, at what scale, and with what level of monitoring.

Services that support it :

Azure API Management (AI Gateway): Centralized authentication, authorization, token controls, traffic shaping, quota enforcement, content safety policies, and policy governance for AI models, agents, tools, MCP servers, A2A agent APIs, and enterprise APIs.
Microsoft Entra ID: Identity and access control for applications, users, and agents.
Azure RBAC: Fine-grained authorization for AI resources and workloads.

The mindset shift: treat agents as non-human identities with their own permissions, lifecycle, runtime access and monitoring.

6. Agent governance: the emerging control plane

Agents are a new kind of enterprise actor, and governing them takes two complementary pieces: a control plane to manage the whole fleet, and a portable control standard to enforce safety inside each agent's workflow.

Microsoft Agent 365: govern the agent fleet

Agents introduce a new problem - sprawl. As teams build agents on different platforms, organizations need one way to see and govern all of them. Microsoft Agent 365, now generally available, is the control plane for AI agents: it gives each agent its own identity and manages agents with the admin tools you already use - Microsoft Entra, Defender, Purview, Intune, and the Microsoft 365 admin center. It works across agents built on Microsoft, open-source, and third-party platforms, through five core capabilities:

Registry - Discover and catalog every agent - including shadow agents - and quarantine the unsanctioned ones.
Access control - Give each agent a unique Microsoft Entra Agent ID, and enforce least-privilege, risk-based access with policy templates.
Visualization - Unified dashboards, telemetry, and alerts across the entire agent fleet.
Interoperability - Works with agents from Microsoft, open-source, and third-party frameworks, alongside your Microsoft 365 apps and data.
Security - Threat detection, data protection, and compliance for agents at scale, through Defender, Purview, and Entra.

Agent Control Specification (ACS): portable runtime controls

Governing agents also means enforcing controls consistently, no matter which framework an agent is built on. The Agent Control Specification (ACS) is an open industry standard, part of Microsoft's Agent Governance Toolkit, for placing deterministic safety and security controls at defined checkpoints in an agent's workflow. Just as the Model Context Protocol (MCP) standardized how agents connect to tools, and A2A standardized how agents communicate with each other, ACS aims to standardize how agents are governed: one portable control layer that any framework can adopt, with Microsoft providing reference implementations.

Control checkpoints: validation at defined intervention points across the agent's lifecycle, spanning input, model invocation, state, tool selection and execution, and output
Composable controls: deterministic checks (allow/deny rules, custom filters) alongside model-based checks (classifier endpoints, LLM judges), each placed exactly where it is needed
Portable and auditable: expressed as a declarative manifest with policy-as-code evaluation, so controls are versionable, auditable, and travel with the agent across any stack

Together they cover both altitudes of agent governance: Agent 365 governs the fleet from the outside, while ACS enforces safety inside each agent's workflow. Those checkpoints can also route to a human for approval before a high-impact action -the runtime side of human-in-the-loop -while Agent 365 defines which agents and actions require sign-off.

The governance service map

Use this as a quick reference for matching each governance area to the Microsoft services that support it. Runtime enforcement is shown as a cross-cutting capability because it operationalizes several domains rather than replacing them:

A practical path forward

You do not have to govern everything at once. A pragmatic sequence gets you to a defensible position quickly:

Start with scope: Inventory the AI apps, Copilots, agents, APIs, tools, MCP servers, and custom Microsoft Foundry workloads that exist today.
Classify risk: Determine what data is used, what actions AI can take, what systems it can reach, and which regulations apply.
Apply controls: Bring in Purview, Entra, Defender, Microsoft Foundry guardrails, Azure API Management AI Gateway policies, Agent 365 policies, and portable ACS checkpoints for agents.
Measure and prove: Add evaluations, including policy-driven ASSERT runs and adversarial testing with the AI Red Teaming Agent, plus observability, APIM runtime telemetry, audit evidence, and operational dashboards.

Governance works best when it follows the business rather than leading it. This journey view shows how the controls layer in progressively starting from a high-value business use case and adding only what each next step requires, so governance accelerates safe delivery instead of gating it. The path runs through four stages: align on the use case and its value, stand up a safe foundation of minimum viable controls, build and validate quality and safety, then operate and scale -monitoring, auditing, enforcing runtime controls and governing agents in production.

Microsoft gives you the building blocks to govern AI as a live operating system - not a static policy document. To go deeper, explore Microsoft Foundry, Microsoft Purview, Microsoft Defender for Cloud, Microsoft Entra, Azure API Management and Microsoft Agent 365 -plus the open-source ASSERT evaluation framework and the Agent Control Specification (ACS) for portable, framework-agnostic agent controls.

References

Policy & lifecycle

Microsoft Foundry — https://learn.microsoft.com/en-us/azure/foundry/what-is-foundry
Microsoft Foundry Compliance and Security- Manage compliance and security in Microsoft Foundry - Microsoft Foundry | Microsoft Learn
Azure Policy — https://learn.microsoft.com/en-us/azure/governance/policy/overview
Azure API Management AI Gateway - https://learn.microsoft.com/en-us/azure/api-management/genai-gateway-capabilities
Microsoft Agent 365 — https://learn.microsoft.com/en-us/microsoft-agent-365/overview

Data governance

Microsoft Purview — https://learn.microsoft.com/en-us/purview/purview
DSPM for AI (Data Security Posture Management) — https://learn.microsoft.com/en-us/purview/data-security-posture-management-learn-about
DLP (Data Loss Prevention) — https://learn.microsoft.com/en-us/purview/dlp-learn-about-dlp
Sensitivity labels — https://learn.microsoft.com/en-us/purview/sensitivity-labels

Model safety

Azure AI Content Safety — https://learn.microsoft.com/en-us/azure/ai-services/content-safety/overview
Prompt Shields — https://learn.microsoft.com/en-us/azure/ai-services/content-safety/concepts/jailbreak-detection

Evaluations

Azure AI Foundry evaluators (observability & evaluation) — https://learn.microsoft.com/en-us/azure/foundry/concepts/observability
ASSERT — official announcement (open-source agent evals; no standalone Learn page yet): https://devblogs.microsoft.com/foundry/build-2026-open-trust-stack-ai-agents/

Observability

Azure Monitor — https://learn.microsoft.com/en-us/azure/azure-monitor/fundamentals/overview
Application Insights — https://learn.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview
Log Analytics — https://learn.microsoft.com/en-us/azure/azure-monitor/logs/log-analytics-workspace-overview
Purview Audit — https://learn.microsoft.com/en-us/purview/audit-solutions-overview

Security

Microsoft Defender for Cloud — https://learn.microsoft.com/en-us/azure/defender-for-cloud/defender-for-cloud-introduction
Microsoft Defender XDR — https://learn.microsoft.com/en-us/defender-xdr/microsoft-365-defender
Microsoft Sentinel — https://learn.microsoft.com/en-us/azure/sentinel/sentinel-overview

Identity

Microsoft Entra ID — https://learn.microsoft.com/en-us/entra/identity/
Azure RBAC (role-based access control) — https://learn.microsoft.com/en-us/azure/role-based-access-control/overview
Conditional Access — https://learn.microsoft.com/en-us/entra/identity/conditional-access/overview

Audit & compliance

Purview Audit — https://learn.microsoft.com/en-us/purview/audit-solutions-overview
eDiscovery — https://learn.microsoft.com/en-us/purview/edisc
Compliance Manager — https://learn.microsoft.com/en-us/purview/compliance-manager

Agent governance

Microsoft Agent 365 — https://learn.microsoft.com/en-us/microsoft-agent-365/overview
Agent Control Specification (ACS) — https://microsoft.github.io/agent-governance-toolkit/packages/agent-control-specification/
Agent Governance Toolkit — https://microsoft.github.io/agent-governance-toolkit/
Purview for agents — https://learn.microsoft.com/en-us/purview/purview

Contributors:

This article is maintained by Microsoft. It was originally written by the following contributors.

Manasa Ramalinga | Senior Principal Cloud Solution Architect – US Customer Success 
Mehrnoosh Sameki | Principal PM Manager -AI Governance Product Team

The AI Agent Lifecycle: A Simple Guide

supriyas — Mon, 13 Jul 2026 00:52:49 GMT

The Bigger Picture

Building an AI agent is fundamentally different from building traditional software.

With a website or application, teams typically design, develop, test, and release. Once deployed, the focus shifts primarily to maintenance and feature enhancements. AI agents operate differently. They don't just execute predefined instructions they interpret information, reason, and make decisions.

In banking, those decisions can influence customer experiences, operational efficiency, compliance outcomes, and risk management. As a result, deploying an AI agent is not the finish line; it's the beginning of an ongoing process of learning, monitoring, and improvement.

The lifecycle of an enterprise AI agent reflects this reality.

Stage 1 —> Design: Decide What It Can and Cannot Do

Before writing a single line of code, answer three questions:

What is this agent allowed to do?
What must it never do?
Who is accountable when something goes wrong?

Simple example:

A loan agent is allowed to check credit scores, apply lending policy, and recommend a decision. It is never allowed to approve a loan above $50,000 without a human sign-off. The Head of Credit Risk is accountable.

Design produces one critical output: the risk classification. A low-risk FAQ bot and a high-risk loan decisioning agent need completely different levels of testing, guardrails, and oversight. Getting this wrong at design time is expensive to fix later.

Microsoft helps here with:

Tool	How It Helps
Microsoft Foundry Model Catalog	Browse and compare models and select the right one for the risk level before any code is written
Azure AI Content Safety	Review the built-in risk categories to understand what the platform can enforce, informing the guardrail boundary decisions
Microsoft Responsible AI Impact Assessment	Structured tooling to assess and document harms, likelihood, severity, and mitigations, producing a risk classification artefact.

Stage 2 —> Build: Put the Safety Controls In, Not On

Build the agent but more importantly, build the safety controls at the same time. Not afterwards.

The core stack:

Simple example:

The loan agent is built with: a PII redaction step (strip account numbers before they reach the model), a credit bureau tool, a policy lookup tool, an output checker (does the response cite a real policy?), and a Human in the loop gate (flag any decision over $25k for human review).

Establish the golden dataset during this phase: a representative set of real-world loan scenarios with SME validated expected outcomes. This serves as the ground truth for evaluating accuracy, consistency, and regression performance throughout the agent lifecycle.

Microsoft helps here with:

Tool	How It Helps
Microsoft Foundry Agent Service	The primary platform for building and hosting the agent, tool registration, memory, and orchestration in one place
Azure OpenAI Service	The LLM backbone with configurable built-in content filters on every inference call
Azure AI Content Safety	Input and output guardrails, content moderation and Prompt Shield for injection and jailbreak detection
Azure AI Language	PII detection and redaction across 100+ entity types before data reaches the model
Azure AI Search	The RAG pipeline retrieves verified policy documents to ground every agent response
Azure Functions (Premium)	Hosts custom guardrail logic (e.g. policy compliance checks) inside the bank's private network
Microsoft Foundry Tracing	Instruments every tool call and reasoning step, essential for evaluation and audit

Stage 3 —> Test: Find the Failures Before Customers Do

Testing occurs in three waves:

Automated testing evaluates the agent against the golden dataset, measuring accuracy, groundedness, and safety.
Human review brings in domain experts to assess decision quality, reasoning, and compliance.
Red teaming stress-tests the agent with adversarial prompts to uncover vulnerabilities and safety gaps.

The stage concludes with a quality gate, a formal sign-off that the agent meets the required standards. No sign-off, no deployment.

Simple example:

The loan agent achieves 98% accuracy against the golden dataset. A compliance officer reviews a sample of 50 decisions and confirms that the reasoning meets requirements. During red-team testing, a vulnerability is discovered: the agent can be manipulated through instructions embedded within a PDF. The issue is addressed and remediated before deployment.

Microsoft helps here with:

Tool	How It Helps
Microsoft Foundry Evaluation SDK	Runs the full golden dataset evaluation in parallel structured scores per row, side-by-side comparison between runs
Built-in Safety Evaluators	Out-of-the-box scoring for violence, hate, self-harm, sexual content, and indirect prompt injection
Built-in Quality Evaluators	Groundedness, relevance, coherence, and fluency no configuration needed
Agent Evaluators	TaskAdherence and ToolCallAccuracy checks the agent followed the right process, not just gave the right answer
Microsoft Foundry Versioned Datasets	Locks the golden dataset by version the same benchmark is used for every regression test

Stage 4 —> Deploy: Start Small, Expand Carefully

Do not flip a switch and send all traffic to the new agent. Start in shadow mode.

Shadow mode: Agent processes requests → responses NOT shown to customers Purpose: does it behave in production like it did in test? Pilot (5%): A small slice of real customers get agent responses Watch error rates for 2 weeks Full rollout: Expand only when quality metrics stay within thresholds

Everything must be live before the first customer interaction: monitoring dashboards, alerting, human review queues, and a fallback plan if the agent needs to be pulled.

Simple example:

The loan agent goes live in shadow mode for one week. No unexpected failures. Expands to 5% of applications. Error rate stays below 0.1% for two weeks. Full rollout approved.

Microsoft helps here with:

Tool	How It Helps
Microsoft Foundry Agent Services, Azure Kubernetes Services, Azure Container Apps	Hosts and auto-scales the agent runtime canary deployments enable the staged rollout without a full infrastructure team
Azure API Management	The API gateway enforces rate limits, authentication, and routing before any request reaches the agent
Azure Application Insights	Latency, volume, and error rate dashboards live from the first interaction
Azure Private Endpoints + Managed Identity	All traffic stays inside the bank's network no public endpoints, no passwords in code
Microsoft Foundry Deployment Management	Version-pins the model deployment enables instant rollback if the new version degrades

Stage 5 —> Operate: Watch Everything, Always

A deployed agent is not a finished product. It is a living system. Watch five things continuously:

What to Watch	Why
What's coming in	Are users trying to manipulate the agent?
How fast it responds	Is it meeting SLA?
Quality of outputs	Is it still giving correct answers?
Guardrail trigger rates	Are more things being blocked or slipping through?
Business outcomes	Are loan decisions still aligned with policy?

Sample a portion of live interactions and route them to human reviewers. When a reviewer corrects the agent, that correction is a training signal — collected, annotated, and fed back into the next iteration.

Simple example:

Three weeks after launch, monitoring shows the agent's policy compliance score has dropped from 98% to 94%. Human reviewers identify that a recent policy update was not reflected in the agent's RAG knowledge base. The team is alerted before any customers are affected.

Microsoft helps here with:

Tool	How It Helps
Microsoft Foundry Online Evaluation	Asynchronously samples live traffic and evaluate it, providing continuous quality monitoring without impacting response latency.
Azure AI Content Safety (runtime)	Prompt Shield enforces guardrails on every production interaction in real time
Azure Monitor + KQL	Provides dashboards and alerts across all monitoring signals, including latency, quality, guardrail compliance, and business outcomes.
Microsoft Foundry Tracing	Captures every production trace, including tool interactions and execution history, providing a complete audit trail for review and compliance
Traces to Dataset	Automatically converts production traces into versioned evaluation datasets, feeding seamlessly into the next optimization cycle.

Stage 6 —> Iterate: The Agent Is Never Finished

Every signal from production triggers a loop back into the lifecycle.

Signal	What Happens
Quality score drops	Loop back to Build and update the RAG index
New attack pattern detected	Loop back to Build and patch the guardrail, re-test
Human overrides spiking	Loop back to Design and rethink the HITL threshold
New regulation published	Loop back to Design and full cycle restarts

Simple example:

APRA publishes new guidance on AI in credit decisions. The loan agent must be updated to include a new mandatory disclosure in every decision output. The team loops back to Design, specifies the new requirement, updates the agent in Build, re-tests against an updated golden dataset, and redeploys within three weeks.

Microsoft helps here with:

Tool	How It Helps
Microsoft Foundry Fine-tuning	Fine-tunes the model using human-reviewed annotations, enabling the agent to improve with every feedback cycle.
Microsoft Foundry Dataset Versioning	Promotes newly annotated traces into the next version of the golden dataset, ensuring regression testing remains current.
Microsoft Foundry Experiment Tracking	Maps evaluation outcomes to specific prompt revisions, making it easy to identify the exact change that introduced a regression.
Azure Monitor Alerts	Automatically triggers when quality thresholds are breached, initiating the optimization cycle without requiring manual intervention.

Why the Loop Is the Most Important Part

Most teams focus on stages 1–4. The loop through stage 6 is what separates agents that stay safe from agents that drift into risk over time.

Regulators do not just ask "was it safe when you launched it?" They ask, "is it safe now and can you prove it has been improving?"

The iterate loop, supported by Microsoft Foundry's continuous evaluation and monitoring capabilities, is how you answer yes.

Summary

Design what the agent can and cannot do. Build the safety controls into the system at the same time as the agent itself. Microsoft Foundry Agent Service, Azure AI Content Safety, and Azure AI Search provide the core infrastructure. Test it in three waves before any customer sees it, using Microsoft Foundry's evaluation SDK and built-in evaluators. Deploy it carefully in stages, watching every metric through Azure Monitor and Application Insights. Once live, monitor it continuously using Microsoft Foundry's online evaluation. And when something changes a policy, a regulation, a performance drift loop back to the right stage and run the cycle again. The agent is never finished. The loop is the product.

Beyond the Canvas: The Azure Architecture Diagram Builder Becomes Agent-Ready

arturoqu — Tue, 14 Jul 2026 14:02:30 GMT

AZURE ARCHITECTURE BLOG · 8 MIN READ

Author: Arturo Quiroga, Senior Partner Solutions Architect — Microsoft

Two months ago I published From Prompt to Production: Building Azure Architecture Diagrams with AI, introducing the open-source Azure Architecture Diagram Builder. The response was humbling — thousands of you read it, tried the tool, and filed issues and feature requests. A follow-up on how the Well-Architected Framework scoring works went deep on validation.

You asked, and the tool grew. This post is about what’s new since May — and one change big enough to reframe the whole project: the Azure Architecture Diagram Builder is no longer just an app you click. It’s a partner you chat with, and a tool other agents can call.

TL;DR. Three arcs of new capability: (1) Architecture Chat turns diagram design into a multi-turn conversation over the live canvas; (2) Blueprint Diagrams produce hand-drawn, whiteboard-style deliverables alongside the formal topology; and (3) the app now exposes its capabilities as a Model Context Protocol (MCP) server, so AI agents can generate, validate, cost, and render Azure architectures programmatically. Plus a 13-model fleet, deployment guides grounded in Microsoft Learn, and July output enhancements.

What’s new at a glance

Capability	What it does
Architecture Chat	Refine a diagram by conversation — “add Front Door with WAF,” then“now make it zone-redundant.” Each turn reads the live canvas and auto-saves to history.
Blueprint Diagrams (BETA)	Hand-drawn, whiteboard-style renders with nested zones and numbered flow arrows. Topology, Blueprint, or Both.
A fleet of 14 models	Multi-provider roster — GPT-5.x, DeepSeek, Grok, Mistral, and Kimi — with side-by-side comparison to pick the right brain per task.
MCP server	The app is now a remote MCP server. Agents can list_services, validate_architecture, estimate_costs, generate_bicep and render_diagram with typed, structured outputs.
Microsoft Learn grounding	Deployment guides now cite live Microsoft Learn documentation.
Output enhancements (July 2026)	Cost badges, light/dark render themes, and metadata panels in rendered diagrams.

From clicking to conversing: Architecture Chat

The single most common request after the launch post was some version of “I love the first diagram, but I want to iterate without re-writing the whole prompt.” Regenerating from scratch every time you tweak a requirement is slow and loses context.

Architecture Chat solves this. It’s a conversational panel that sits alongside the canvas and treats your diagram as a living document. Each message is a turn in an ongoing design session:

“Add an Azure Front Door with WAF in front of the app tier.”
“Now make the data layer zone-redundant.”
“Swap the SQL Database for Cosmos DB and update the connections.”

Every turn reads the current state of the canvas — not the original prompt — so refinements compound naturally the way they would with a human architect at a whiteboard. The conversation auto-saves to history, so you can step back through the evolution of a design or branch from an earlier point.

Figure 1. Architecture Chat treats the diagram as a living document. Each message refines the current canvas — adding services, changing SKUs, or reorganizing groups — and the full exchange is saved to history.

The shift is subtle but important: architecture design stops being a one-shot prompt and becomes an iterative dialogue.

The whiteboard deliverable: Blueprint Diagrams (BETA)

Formal topology diagrams with official Azure icons are perfect for documentation and stakeholder decks. But early-stage design conversations often want something looser — the hand-drawn feel of a whiteboard sketch that communicates intent without implying finality.

Blueprint Diagrams generate exactly that: a whiteboard-style render with nested zones (subscription → VNet → subnet), numbered flow arrows, and a deliberately sketchy aesthetic. You choose the output mode:

Topology — the formal, icon-based diagram from the launch post
Blueprint — the hand-drawn whiteboard style
Both — generate the two side by side

Figure 2. The same architecture in two visual languages. Left: the formal, icon-based topology. Right: Blueprint mode — a whiteboard-style render with nested zones and numbered flow steps, plus a numbered legend explaining each hop. Use Blueprint for early design conversations and Topology for final documentation.

It’s the same underlying architecture — two visual languages for two different moments in the design lifecycle.

A fleet of 14 models: pick the right brain per task

The launch post shipped with multi-model support. That fleet has grown to 13 models across five providers, so you can match the model to the job — fast models for iteration, reasoning models for complex designs, code-optimized models for Bicep generation:

OpenAI GPT-5.x — GPT-5.1, GPT-5.2, GPT-5.6 Sol, Terra and Luna, GPT-5.4, GPT-5.4 Mini
DeepSeek — V3.2 Speciale, V4 Pro
xAI Grok — 4.1 Fast, 4.3
Mistral — Large 3
MoonshotAI Kimi — K2.5, K2.7 Code

The Compare Models feature runs the same prompt through any subset of these in parallel and ranks them on service count, token usage, latency, and cost — with Fastest / Cheapest / Most Thorough badges — so you can make an evidence-based choice rather than a guess.

Figure 3. Multi-model comparison across the full 13-model fleet. Top: the results grid ranks every model on service count, connections, token usage, latency, and cost, with Fastest / Cheapest / Most Thorough badges. Bottom: an optional AI Critique uses a critic model to rank the outputs and explain each model’s strengths and gaps.

Adding a model is now a small, well-understood change — a testament to how the multi-provider abstraction has matured since May.

The headline: the Diagram Builder is now an MCP server

Here’s the change that reframes the project. Everything above is about a person using a web app. But the same capabilities — generating a diagram, validating it against WAF, estimating its cost, producing Bicep — are exactly the things an AI agent needs when it reasons about Azure architecture.

So we exposed them. The Azure Architecture Diagram Builder now runs as a Model Context Protocol (MCP) server. Any MCP-capable agent can call its tools with typed inputs and structured outputs:

Tool	What the agent gets
`list_services`	The catalog of supported Azure services and categories
`validate_architecture`	A WAF assessment with pillar scores and findings
`estimate_costs`	Multi-region cost estimates from the Azure Retail Prices API
`generate_bicep`	Infrastructure-as-Code templates for the design
`render_diagram`	A rendered diagram (topology or blueprint) of the architecture

This means an agent can hold a conversation like “design a HIPAA-compliant platform, check it against the Well-Architected Framework, tell me the monthly cost in West Europe, and give me the Bicep” — and the Diagram Builder answers each part programmatically, returning structured data the agent can reason over and chain.

Figure 4. The Diagram Builder as an MCP server inside Microsoft Scout. Top: from a natural-language request, the agent calls the render_diagram tool with structured parameters (title, format, direction, theme, region) and saves the returned SVG to its workspace. Bottom: the rendered architecture — grouped zones, labeled flows, and cost badges — appears inline in the conversation, generated entirely through agent tool calls.

The tool that started as a canvas for humans is now also a building block for agents. That’s the arc: from an app you click, to a partner you chat with, to a tool other agents call.

Grounded in Microsoft Learn, and sharper output

Two smaller-but-meaningful improvements round out the release:

Microsoft Learn grounding. Deployment guides now search official Microsoft Learn documentation at generation time and cite it, so the guidance reflects current, authoritative practice rather than a model’s training snapshot.
Output enhancements (July 2026). Rendered diagrams now carry per-service cost badges, support light and dark render themes, and include metadata panels that summarize the architecture — service counts, regions, and estimated cost — directly on the image.

Highlights

Since the May launch, the Azure Architecture Diagram Builder has grown from a design tool into an agent-ready platform:

Conversational design: iterate on a diagram by chatting over the live canvas, with full history
Two visual languages: formal topology and hand-drawn Blueprint, from the same architecture
13 models, five providers: choose the right brain per task, with evidence-based comparison
Agent-ready: an MCP server exposing generation, validation, costing, and IaC as callable tools
Grounded guidance: deployment guides cite live Microsoft Learn documentation
Still open source: every capability above is available to inspect, extend, and contribute to

Try It Today

Live demo: https://aka.ms/diagram-builder
Source code: GitHub repository
Documentation: See the Getting Started Guide for setup, and the repository’s MCP server directory for agent integration.

If you read the first post and tried the tool — thank you. The features above exist because you told me what you needed. Keep the feedback coming via GitHub Issues.

Tags: artificial intelligence · application · apps & devops · well architected · infrastructure

Golden Paths Are a Product. Treat Them Like One.

KishoreKumarPattabiraman — Tue, 07 Jul 2026 18:26:04 GMT

Audience: Cloud architects, platform engineers, engineering leaders

In my earlier Cloud Native Platforms articles, I focused on how teams build, run, and evolve modern engineering foundations. This article focuses on what keeps those capabilities useful after launch.

Most teams do not lose adoption because the first release was poor. They lose adoption because the path was launched like a project and left to survive like a product.

The pattern is familiar. A capability ships with good intent, reasonable documentation, and early enthusiasm. A few months later, teams start bypassing it, exceptions multiply, support load rises, and the platform team quietly becomes a ticket queue.

If a capability is expected to be adopted, trusted, improved, and measured over time, it needs product discipline from the start.

How this connects to forward-deployed engineering

Forward-deployed engineering (FDE) is a delivery model where an engineer embeds directly with a customer or team, learns the real operating context, and builds a working solution in the field. The model was popularized by Palantir, whose engineers embed with customers. The same pattern now appears inside large engineering organizations, where platform engineers embed with internal product teams. Golden paths are a logical next step. FDE is the front of the funnel, where bespoke solutions get built one team at a time. Between the two sits a graduation decision. When the same pattern shows up across several independent teams with the same friction, it is ready to become a safe default. A golden path is the back end. The part that repeats across teams becomes that default, so you stop solving the same problem by hand every time. FDE does not stop once a path exists. New teams, new domains, and new edges keep producing forward-deployed work in parallel. The loop closes through what the path cannot absorb. Where teams bypass the path or hit a case it does not cover, those gaps become the next forward-deployed engagement, and the cycle repeats.

Figure 1. Forward-deployed work is the continuous front of the funnel. Golden paths are how the repeatable parts scale.

Quick context before we begin

Golden path is the easiest safe default way to deliver a repeated engineering outcome.

Net Promoter Score (NPS) measures how likely someone is to recommend something to others. In this context, internal NPS means how likely one internal engineering team is to recommend a capability to another internal team. It is useful as a directional trust signal, not as a standalone operating metric.

Service Level Objective (SLO) is a quantitative target for a service characteristic such as availability, latency, or error rate.

Service Level Agreement (SLA) is a formal service commitment, often external or contract-backed. Many internal engineering capabilities will use SLOs without using formal SLAs.

Deprecation lane is the planned and time-bound way to retire an older pattern without causing unnecessary disruption.

How to identify a true golden path

Not every good practice is a golden path. A path qualifies when most of the following are true:

Repeated use case across teams
High cost of inconsistency
High cognitive load from scratch
A safe default can be defined
Measurable outcome exists

If those signals are absent, teams are usually looking at a local implementation pattern, not a golden path.

That distinction matters. Internal capability teams need to invest their product energy where standardization meaningfully reduces risk, cost, delay, or complexity.

Two transformation threads that make this real

To keep this practical, this article refers back to two transformation threads throughout the discussion.

The first was an AKS migration effort. The existing hosting model worked until scale, operational control, and predictable performance started pulling in opposite directions. The move to AKS was not treated as a one-time infrastructure shift. Workloads were phased over time, traffic was controlled through a single API entry point, and teams were trained to operate in a container-first model before the migration accelerated. The outcome was not only better control. It created a stronger foundation for observability, scaling, policy-driven operations, and future platform standardization.

The second was a security modernization effort around a more unified authentication and authorization model. Identity logic had grown across services over time. Permissions were fragmented. Operational inconsistency was becoming risk. The move toward a more centralized approach depended as much on rollout safety as on design quality. Feature flags, parallel writes during transition, read-repair, and rollback paths made that shift survivable on a live platform while preserving performance and user continuity.

These two threads matter because they show the same principle in different settings. A capability becomes durable when teams combine the right architecture with the right product mindset, operating model, and feedback loop.

The golden path operating canvas

A useful way to think about a golden path is through five decision areas:

Product framing
Operating model
Engineering guardrails
Adoption and change strategy
Measurement and feedback loop

Figure 2. The measurement and feedback area is what keeps the other four honest over time.

Individually, each area matters. Together, they determine whether a capability becomes the default, stays trusted, and improves over time.

1. Product framing

A common first mistake is treating a capability as an output instead of a product.

Product framing starts with four simple questions:

Who is the consumer
What repeated problem are we solving
What safe default are we standardizing
What measurable outcome proves value

This matters equally for a platform capability and for a shared service. An internal deployment template, a security validation path, a standardized API onboarding flow, or the opinionated consumption path around a reusable identity service can all become golden paths if they remove repeated friction and create better defaults.

In practice

The path gets clearer when teams name the repeated decision they want to remove. In the AKS migration, the real question was not "should we modernize hosting." It was "what is the safest default for workloads that now need predictable performance and stronger operational control." That framing forces sharper scoping. It also prevents vague launch language from hiding the actual default teams are expected to adopt.

If teams are defining a new golden path, product framing is where they start.

If teams already have one and see drift, they should return here first and ask whether the original consumer problem is still sharp, still current, and still visible.

2. Operating model

A golden path does not stay healthy because documentation exists. It stays healthy because ownership exists.

The operating model answers questions like these:

Who owns the path
Who contributes to it
Who makes change decisions
What review cadence exists
How support expectations are communicated
How exceptions are granted and retired

Without this, teams end up with a queue, not a product.

This is where many internal capabilities weaken over time. Platform teams become catch-all owners. Security teams review too late. Consumer teams give feedback, but no one is clearly accountable for converting that feedback into decisions.

Team Topologies describes a similar problem through cognitive load and enabling-team boundaries. When ownership and interaction modes are unclear, friction rises and good patterns fail to scale (Team Topologies).

In practice

The pattern that works is simple: one accountable owner, one recurring decision forum, and one visible exception model. The exception model matters more than teams expect. A path becomes brittle when consumers can go off-path with no review, and it becomes unusable when every exception requires escalation with no expiry. Temporary exceptions need an owner, a review date, and a retirement path.

If teams are starting new, they should define one accountable owner and one recurring review rhythm.

If teams already see drift, they should look for ambiguity in decision rights, ownership, or support expectations.

3. Engineering guardrails

A golden path is only valuable if it makes the safe path the easy path.

Engineering guardrails are the non-negotiables built into the path itself. These typically include:

Security posture
Reliability targets
Cost boundaries
Operational readiness
Observability and telemetry standards
Supportability expectations
Versioning and compatibility expectations

This is one place where product, service, and platform thinking often get separated unnecessarily. They should not be. A shared identity service needs guardrails just as much as a platform deployment path does. A platform path without operational clarity is as incomplete as a service without security consistency.

The unified auth work shows why. Centralizing identity enforcement was not just an architectural choice. It was a way to make security control more consistent, measurable, and operable. Feature-flag strategy, rollback readiness, and performance targets were all part of the guardrail model, not postscript concerns.

Microsoft's Well-Architected guidance reinforces the same principle. Security, reliability, and cost are not side dimensions. They are core decision lenses that shape how capabilities are designed and operated (Microsoft Well-Architected Framework).

In practice

The unified auth work makes this concrete. The technical design mattered, but the real strength came from turning security rules into operating defaults: central enforcement, predictable rollout controls, and a clear rollback posture. The same principle applies elsewhere. If teams do not know the compatibility expectations, cost envelope, or minimum operational standard up front, they invent their own.

If teams are defining a new path, they should decide the minimum guardrails before expansion.

If an existing path is drifting, teams should look for where consumers are stepping outside the guardrails or where the guardrails were never made explicit.

4. Adoption and change strategy

A golden path does not become real because it is published. It becomes real because teams can adopt it safely and progressively.

That means rollout strategy must be designed as carefully as the solution itself. That includes:

Migration sequencing
Consumer onboarding
Enablement and training
Shadow testing where needed
Rollback readiness
Transition governance
Minimum viable self-service

This was one of the clearest lessons from the AKS migration. The technical work mattered, but the people ramp-up mattered just as much. Moving a large engineering group from one operational model to another required structured enablement, practical migration guidance, and a way to build confidence in steps rather than all at once.

This mirrors the intent behind Microsoft's Cloud Adoption Framework. Large-scale change lands better when teams combine technical design with lifecycle planning, capability enablement, and operating model clarity (Microsoft Cloud Adoption Framework).

In practice

The AKS migration worked because adoption was designed, not assumed. Teams had a phased route in, a safe path back, and enough enablement to operate the new model with confidence. Just as important, the path moved toward self-service instead of permanent dependency on the central team. If a so-called path still requires repeated manual intervention for normal use, it is closer to a managed service queue than a true default.

If teams are building a new path, they should design adoption as a first-class workstream.

If teams already have a path that is drifting, they should treat re-adoption as a change program, not a documentation refresh.

5. Measurement and the feedback loop

If feedback does not change decisions, feedback collection becomes theater.

A golden path needs a small, meaningful set of signals that tells teams whether it is healthy. That usually includes some mix of:

Adoption depth
Time to first success
On-path versus off-path incidents
Cost efficiency trend
Reliability trend against SLOs
Internal NPS
Deprecation progress

The metric set does not need to be large. It needs to be actionable.

Internal NPS is especially useful when paired with qualitative input. A team may still be using a capability while quietly warning other teams away from it. That is an early drift signal. It is still only one signal. Mandatory adoption, small sample sizes, and local politics can all distort it.

DORA research reinforces the broader point that improvement depends on disciplined measurement and learning loops, not on collecting more indicators than teams can act on (DORA research).

In practice

The most useful feedback loop is visible. Teams should be able to see what was heard, what changed, and what did not make the cut. That is what turns trust into an operating behavior instead of a sentiment score. Without that trace, even good metrics become noise.

If teams are launching a new path, they should define baseline metrics before expansion.

If teams already have a path and see drift, they should run a focused diagnostic on adoption, incidents, bypass behavior, and team sentiment before redesigning anything.

The mindset shift that makes this work

Project thinking asks whether the capability shipped.

Product thinking asks whether it is being adopted, trusted, improved, and measured over time.

That shift changes how teams define success, stage rollout, involve consumers, and respond to drift. It applies to internal platforms, shared services, reusable components, and enablement models. The operating details are not identical across all of them, but the product discipline is still useful wherever teams expect repeat consumers and recurring outcomes.

The CNCF Platforms white paper captures a related idea through paved paths and shared internal capabilities. The technical foundation matters, but the sustained value comes from how teams shape adoption, ownership, and evolution around it (CNCF Platforms White Paper).

That is the broader point of this article. Golden path thinking is not just for platform teams. It is a repeatability and trust model for internal engineering capability design.

Signals your golden path is drifting

Adoption stalls or declines
Side paths become normal
Support load rises while roadmap clarity drops
Consumers cannot name the owner
Breaking changes surprise teams
Feedback is collected with no decision trail
Teams still use the path, but would not recommend it

These are usually not random symptoms. They point back to weak framing, weak ownership, weak guardrails, weak adoption design, or a weak feedback loop.

Immediate next steps

If you are defining a new golden path

Start where you already embed. Your forward-deployed and high-touch engagements are the best source of golden path candidates.

Pick one repeated, high-friction workflow with high risk if implemented inconsistently.
Define the consumer, safe default, and one measurable success outcome.
Set minimum guardrails for security, reliability, observability, cost, compatibility, and rollback.
Create a 30-day pilot with a named owner, a review cadence, and a visible exception path.
Capture internal NPS plus qualitative feedback, then publish the first improvement decision.

If you already have a golden path and see drift

Identify the top drift signals, such as bypass behavior, adoption drop, or trust erosion.
Diagnose which area is weak: framing, ownership, guardrails, adoption, or feedback loop.
Run a focused 30-day correction cycle rather than a broad redesign.
Reset one decision forum, one owner, and one exception model before changing anything larger.
Publish what changed, then re-measure adoption, sentiment, and operational metrics after one cycle.

Key takeaways

Golden paths reduce repeated risk and effort
Product discipline keeps them useful
Product discipline applies beyond platforms
Safe defaults need explicit guardrails
Trust grows when feedback changes decisions
Golden paths are how the repeatable parts of forward-deployed work scale

What this means

Good engineering capabilities do not fail only because the technology is weak. They often fail because the operating model around them is too thin.

That is why treating a golden path like a product matters. It turns a useful pattern into a dependable default.

It also creates a stronger bridge back to the earlier Cloud Native Platforms Run article and Evolve article. Building, running, and evolving technical foundations is one part of the problem. Sustaining trust and adoption around those foundations is the next one.

The real shift is this: a golden path is not complete when it launches. It is complete when teams still trust it after the first wave of exceptions, change pressure, and scale.

Want to discuss? Drop a comment with patterns you have seen in your environment. I read every reply.

Optimizing GitHub Copilot Cost in the Usage-Based Billing Era

gauravbhardwaj — Tue, 07 Jul 2026 15:43:49 GMT

GitHub Copilot has become a core part of the modern developer workflow. We use it to complete code, explain unfamiliar repositories, write tests, refactor legacy applications, review pull requests, generate documentation, and automate repetitive engineering work.

But as GitHub Copilot moves further into usage-based billing, teams and users are asking a very practical question:

How do we keep getting value from GitHub Copilot without letting costs become unpredictable?

The answer is not to blindly slash GitHub Copilot usage—that would defeat the purpose of adopting AI-assisted development in the first place. The better answer is to use GitHub Copilot more intentionally.

Usage-based billing shifts our mindset from:

“Can I use GitHub Copilot?” to: “Am I using the right GitHub Copilot capability, with the right model, the right context, and the right level of automation for this specific task?”

This guide outlines the major ways developers and organizations can optimize GitHub Copilot costs while continuing to accelerate productivity.

For a strategic look at how organizations are scaling these financial practices across modern infrastructure, read the playbook Token Economics: The New FinOps for Agentic AI | Microsoft Community Hub

Understanding the New Cost Model

Under usage-based billing, GitHub Copilot costs are driven primarily by two factors:

The model used (e.g., frontier vs. lightweight models)
The number of tokens consumed

A token is a small unit of text processed by the AI. This includes what you send to the model (input), what the model generates back (output), and, in some cases, cached context that the model reuses.

This means a short question using a lightweight model consumes very little. Conversely, a long, multi-file agentic session using a frontier model can consume significantly more. GitHub Copilot cost is no longer just about how many people have seats—it is about how those seats are being used.

The Biggest Cost Drivers:

Using expensive models for simple tasks
Overly large context windows
Long chat sessions with repeated context
Agentic workflows that inspect too many files
Enabling tools when they aren't needed
Dumping large terminal logs or full repository scans into the chat
A lack of active budgets and usage monitoring

The good news? Most of these are completely controllable.

1. Set Spending Guardrails First

The first cost-reduction step isn't technical—it’s financial governance. Before teams start heavy usage, set spending caps and budget controls. This is critical for organizations where many developers share a pool of AI credits.

Recommended Actions:

Set a monthly budget and enable hard-stop controls where available.
Segment limits: Create separate limits for standard users and power users.
Track granularly: Monitor consumption by user, team, organization, or cost center.
Review early: Closely analyze the first few weeks of usage and adjust limits based on real consumption patterns.

This prevents a single heavy user, an runaway automated agent session, or an experimental workflow from consuming a massive chunk of shared capacity early in the billing cycle.

For enterprises, avoid a one-size-fits-all budget. A junior developer asking occasional chat questions does not need the same budget as a platform engineer running massive migration tasks across multiple repositories.

A Smarter Budgeting Model:

Standard developer budget
Power user budget
Pilot team budget
Agentic workflow budget
Innovation/experimentation budget

This gives leaders control without blocking productivity, allowing organizations to analyze and adjust budgets frequently.

2. Use the Right Model for the Right Task

Not every task requires the most powerful model. This is one of the biggest mindset shifts in usage-based billing. Developers often default to the strongest model because it feels "safer," but a smaller, cheaper model is often more than enough for daily tasks.

Use Lightweight / Lower-Cost Models For:	Use Stronger / Premium Models For:
Simple code explanations	Architecture & system design decisions
Documentation & comment updates	Complex debugging & multi-file reasoning
Small unit test generation	Security-sensitive reviews
Basic debugging & syntax help	Legacy modernization & code translation
Regex help & simple refactoring	Performance tuning & optimization
Translating code from one style to another	Ambiguous, cross-service production issues
Summarizing small, single files	Large pull request reviews

Team Golden Rule:

Start with the cheapest model that can do the task well. Move to a stronger model only when the task requires deeper reasoning. This keeps high-cost models reserved for where they provide the highest value.

3. Lean More on Code Completions

Inline code completions and next edit suggestions are still some of the most cost-effective GitHub Copilot experiences available. For paid GitHub Copilot plans, these inline experiences are not billed against your AI credits. Developers should aggressively lean on completions when they already know what they want to build.

Ideal Completion Scenarios:

Writing a function after creating the signature
Completing repetitive boilerplate code
Filling in obvious implementation logic or adding similar methods
Writing simple, predictable unit tests
Completing configuration files

The Rule of Thumb: Use chat when you need reasoning. Use completions when you need acceleration.

❌ Bad Habit: Opening a new chat window for every small function.
👉 Better Habit: Write the function name, a descriptive comment, or the first few lines, and let GitHub Copilot fill in the implementation inline. Open chat only if the completion misses the mark or the logic requires deeper discussion.

4. Use Tools Only When Needed

Agents are incredibly powerful because they can interact with tools: reading files, searching repositories, editing code, running commands, inspecting test failures, or connecting to external systems via Model Context Protocol (MCP) servers.

However, tools heavily amplify costs. Every tool call adds more context, more model invocations, and more output back into the token loop. If an agent reads too many files or dumps raw logs into chat, token counts skyrocket.

The goal is not to ban tools, but to scope them correctly:

Don't enable everything: Avoid activating every tool for every agent. Use only what is required.
Specialized agents: Create custom agents with specialized tool access (e.g., a documentation agent needs read/search tools, but likely doesn't need terminal or cloud database access).
Avoid full scans: Ask Copilot to inspect specific files or folders rather than scanning the entire repository.
Summarize logs: Ask Copilot to summarize terminal outputs or run the narrowest relevant unit test instead of dumping pages of raw test logs.

A Cost-Aware Prompting Example:

"Use tools only if needed. Start by reading the files I mention. Do not scan the entire repository. If tests are needed, run only the relevant unit tests and summarize the failure instead of pasting full logs."

5. Narrow the Context Window

Context is useful only when it is relevant. A common cost driver is issuing broad prompts that pull massive chunks of irrelevant code into the session.

❌ Broad Prompt: "Analyze this repo and fix the bug." (Forces the agent to scan, search, and load unnecessary files).
👉 Targeted Prompt: "The bug appears to be in /src/auth/tokenValidator.ts and /src/auth/sessionStore.ts. Review only these files and their related tests. Propose the smallest safe fix."

Practical Ways to Reduce Context:

Explicitly mention exact files, folders, error messages, or failing tests.
Instruct the model on what not to change.
Explicitly exclude generated files, vendor folders, build artifacts, and lock files.
Clear your sessions: Staying in one massive chat session all day causes historical context to stack up. Start a fresh session when your topic/intent changes, when frequent compactions occur, or when different tools are required.
Pro tip: If you are ending a long session but need to carry forward the conclusion, ask Copilot to generate a quick Markdown summary of the session to paste as the starting context of your new, clean session.

6. Use Plan-First Prompts for Complex Work

For large-scale tasks, do not let GitHub Copilot immediately start editing files in agent mode. Always ask it to plan first.

This is crucial for large refactoring, multi-file changes, legacy modernization, dependency upgrades, security fixes, and performance tuning.

A Plan-First Prompting Pattern:

"Before editing files, create a short plan. Identify the files you need to inspect, the likely root cause, the risk level, and the smallest validation test. Do not make changes until the plan is clear."

Why this works:

Prevents wasted work: It ensures the agent doesn't modify the wrong files or run unnecessary commands.
Creates a cost checkpoint: If the plan looks too broad or incorrect, you can narrow the scope before GitHub Copilot kicks off an expensive, multi-step agentic loop.

7. Batch Related Questions in One Session

While giant, day-long sessions are bad for context bloat, opening a brand-new chat for every tiny, consecutive question introduces unnecessary context reload overhead. Finding the right balance is key.

❌ Less Efficient (Fragmented):
- Chat 1: "What does this function do?"
- Chat 2: "Can you write tests for it?"
- Chat 3: "Can you check it for security issues?"
👉 More Efficient (Batched):
- Single Chat: "Review this function for correctness, security, performance, and test coverage. Give me the top issues first, then suggest the smallest safe improvement."

The Balance: Batch highly related work in a single conversation, but hit the "New Chat" button the moment you pivot to an entirely new topic.

8. Keep Custom Instructions Short

Custom instructions (.github/copilot-instructions.md) and AGENT.md files are powerful tools for enforcing team standards, but they are appended as base tokens to every single chat interaction. If they are bloated, they act as a hidden tax on every prompt.

✅ Keep it Concise & Rules-Based:	❌ Avoid Attaching to Every Prompt:
"Prefer small, testable changes."	Massive architecture blueprints
"Do not modify public APIs without calling it out."	Full enterprise coding standards documents
"Use our standard logging pattern."	Entire infrastructure runbooks
"When writing tests, follow the existing test style."	Pages of multi-shot coding examples

The Strategy: Use global/repository instructions only for non-negotiable, high-level rules. For specific workflows, use localized prompt files, specialized skills, or custom agents.

9. Use Skills and Specialized Agents for Repeatable Work

If your team frequently asks Copilot to perform identical workflows (e.g., API reviews, PR summarization, Accessibility audits, Terraform checks), turn them into reusable skills, prompt files, or custom agents.

This ensures prompt formatting remains minimal, highly consistent, and optimized for low token usage. Furthermore, if you discover an excellent workflow during an active chat session, ask Copilot to turn that knowledge into a SKILL file. This prevents future sessions from having to waste tokens "re-learning" the process.

10. Avoid Unnecessary Agentic Mode

Agent mode is incredibly capable, but it shouldn't be the default default mode for standard queries.

🛠️ Use Agent Mode For: Multi-file bug fixes, generating and validating complex test suites, cross-file refactoring, reproducing test failures, or implementing full PR tasks.
💬 Stick to Normal Chat/Completions For: Simple syntax lookups, single-file explanations, writing minor snippets, basic documentation rewrites, or simple formatting tweaks.

The Rule: If a task doesn't explicitly require autonomous file editing, tool execution, or terminal commands, stick to standard chat or inline completions.

11. Monitor Usage Weekly

Optimization without measurement is just guesswork. During your organization's transition into usage-based billing, establish a weekly review cadence to look at your telemetry data.

What to Look For:

Which models are drawing the most volume?
Are premium frontier models being used for simple tasks?
Which teams or workflows are spiking above their allocated credit budgets?
Is tool usage or long context history driving up the average cost per session?

Use these insights to refine default model guidance, create user-level budget overrides for true power users, update prompt templates, and host short, continuous training sessions. The goal is never to shame high usage—high usage is fantastic if it yields high-value code. The goal is to eliminate systemic token waste.

12. Create a Simple Team Policy

To make this stick, give your developers a lightweight checklist they can actually memorize. Here is a great blueprint for an internal team policy:

Completions First: Use inline completions for normal code acceleration.
Cheaper Models First: Default to lightweight models; escalate to frontier models only for deep reasoning.
Scope Wisely: Mention specific files/folders; avoid blind repository scans.
Plan First: Ask for an architectural plan before allowing agents to execute large edits.
Tool Hygiene: Keep tools and custom instructions tightly scoped and concise.
Review Cadence: Check usage metrics weekly and adjust budgets monthly.

Reusable, Cost-Aware Prompt Templates

Copy and paste these templates into your daily workflows to keep token counts down:

For Debugging: "Analyze this error using only the files I mention. Do not scan the whole repository. First explain the likely root cause, then suggest the smallest fix."
For Agent Mode: "Use tools only if needed. Before editing, create a short plan and list the files you need to inspect. Run only the narrowest relevant test."
For Code Review: "Review this pull request for correctness, security, and maintainability. Focus on high-impact issues only. Do not rewrite the code unless necessary."
For Testing: "Generate unit tests for this function using the existing test style. Do not modify production code. Keep the test scope narrow."
For Refactoring: "Refactor this file only. Preserve behavior. Do not change public APIs. Explain the risk before making edits."
For Large Repositories: "Do not analyze the entire repository. Start with /src/payment and /tests/payment. Ask before expanding scope."

Summary: What to Avoid

To keep budgets predictable, coach your teams away from these common anti-patterns:

Setting an expensive frontier model as the default for every single interaction.
Running full agent loops for questions that a simple chat could answer.
Allowing agents to freely scan entire codebases without folder constraints.
Enabling every single available MCP tool by default.
Dumping massive, raw terminal outputs or log dumps directly into the prompt.
Carrying massive, multi-hour chat histories instead of opening fresh sessions.
Disregarding your usage dashboards until the invoice arrives.

The Bigger Picture: Cost Optimization is Workflow Optimization

Optimizing Copilot costs isn't about using AI less—it’s about using AI better. Smaller context windows, tighter tool scoping, cleaner prompts, and proper model selection don't just lower the bill; they make the model significantly more accurate. When you overwhelm a model with irrelevant files and messy logs, you introduce noise that degrades the quality of the output.

Cost optimization is an engineering maturity practice, not a restriction. When managed through a smart operating model, GitHub Copilot will continue to securely accelerate our delivery, eliminate boilerplate toil, and maximize our focus on the work that truly matters.

References:

Token Economics: The New FinOps for Agentic AI | Microsoft Community Hub

Contributors:

This article is maintained by Microsoft. It was originally written by the following contributors.

Gaurav Bhardwaj | Senior Cloud Solution Architect
Dustin Ellis | Senior Cloud Solution Architect

Revolutionizing Document Intelligence: Scaling Construction Industries with AI-Driven Extraction

gauravbhardwaj — Thu, 18 Jun 2026 20:52:34 GMT

Introduction

Generative AI (GenAI) is poised to transform the construction industry by addressing chronic challenges such as low productivity, cost overruns, schedule delays, and labor shortages. By automating the analysis of drawings, specifications, contracts, and project documentation, GenAI can reduce manual effort, accelerate decision-making, and improve coordination across architects, engineers, contractors, and suppliers. Industry studies indicate that AI-powered workflows can increase productivity by 20–40% in planning, engineering, and administrative functions while reducing costly rework and errors. The result is faster project delivery, improved resource utilization, lower costs, and more predictable project outcomes.

A major opportunity for GenAI in construction lies in its ability to unlock the vast amount of information trapped within AutoCAD drawings, architectural plans, BIM models, specifications, and engineering documents. Today, project teams spend countless hours manually reviewing drawings, performing quantity takeoffs, identifying dependencies, and translating design intent into actionable work packages for downstream trades. GenAI can automate this process by extracting and interpreting dimensions, materials, quantities, assemblies, and building components directly from design artifacts, then intelligently distributing that information to foundation, framing, roofing, insulation, MEP, and finish teams. This creates a digital thread from design through execution, eliminating manual handoffs, reducing human error, and ensuring every stakeholder works from a single source of truth. The impact extends beyond productivity gains—GenAI enables more accurate material forecasting, streamlined procurement, reduced waste, faster response to design changes, fewer change orders, and greater confidence that the architect's vision is executed precisely in the field. In an industry where margins are tight and inefficiencies are costly, GenAI has the potential to fundamentally redefine how construction projects are planned, coordinated, and delivered.

This article specifically demonstrates how organizations can leverage Azure AI services—including Azure Content Understanding, Azure foundry, Azure Blob Storage, Azure Open AI—to extract, understand, and operationalize information from construction drawings and project documentation. The solution illustrates how Azure's AI platform can transform unstructured design artifacts into actionable intelligence that improves productivity, reduces risk, accelerates procurement, and enables more efficient execution across the entire construction lifecycle.

This transformation is now achievable through a hybrid AI architecture. By combining structured layout understanding models with Generative AI reasoning capabilities, organizations can build highly scalable, intelligent extraction systems that meet the rigorous safety and compliance standards of the construction sector.

The Evolution from GenAI Approach to Deterministic Precision

Starting with a Generative AI–driven approach to extract structured fields from documents is a fundamentally more effective initial strategy. It accelerates early-stage extraction without requiring large, labeled datasets, while simultaneously enabling structured data collection needed to train deterministic models—which typically require thousands of annotated samples.

This approach delivers immediate value by rapidly identifying relevant data patterns in documents and uncovering key factors that influence extraction accuracy, such as document quality, layout complexity, and multi-section ambiguity. At the same time, it naturally builds the dataset necessary to transition toward a more scalable and repeatable solution.

However, while powerful for contextual reasoning across document sections, Generative AI is inherently probabilistic and sensitive to input variability. For enterprise-grade reliability, precision, and repeatable structured document extraction, a complementary approach is required.

The optimal solution is a hybrid model that combines the strengths of both:

Azure Content Understanding provides precise, consistent field extraction with per-field confidence scores at scale.

Azure OpenAI GPT-5.2 (generative) adds contextual reasoning, validates ambiguous fields, fills extraction gaps, and interprets complex multi-section relationships.

AI Agent (bounded triage) handles exception cases with structured CORRECT/ACCEPT/ESCALATE decisions before human escalation. Together, they form a superior system—delivering higher accuracy, reduced ambiguity, bounded AI cost, and stronger auditability in complex real-world conditions.

Note : AI cannot compensate for inconsistent input data. Standardized document schemas and operational discipline remain prerequisites for reliable automation.

Solution Components and Architecture

The solution follows a modular, event-driven architecture that combines deterministic document understanding and Generative AI to enable scalable, intelligent extraction workflows. At a high level, documents are ingested, deduplicated, processed through Azure Content Understanding for primary extraction, enhanced with GPT-5.2 for gap-fill verification, validated against business rules, and routed through a confidence-based decision system before persistence. The code repository for the solution can be found here

Conceptual Architecture

Azure Architecture: -

The pipeline execution follows this flow: a document is uploaded to Azure Blob Storage, triggering the orchestrator. The pipeline checks for duplicates via SHA-256 hash against Cosmos DB. New documents are submitted to Azure Content Understanding, which returns structured fields with per-field confidence scores. The AI Schema Mapper then identifies gaps—fields that are missing or have confidence below 0.70—and sends only those to GPT-4.1 for verification. Results are normalized, validated against cross-field business rules, and routed based on aggregate confidence.

Throughout the pipeline, built-in feedback loops—quality filtering, validation checks, and confidence gates—ensure that only high-confidence results are persisted automatically, enabling a reliable and production-ready extraction system.

Azure Blob Storage — Primary storage for source PDFs and extraction artifacts. Standard_LRS, Hot tier, HTTPS-only with SAS-secured access for Content Understanding.
Azure Content Understanding — Primary deterministic extractor with custom analyzer supporting 100+ configurable fields. Returns per-field confidence scores (0.0–1.0) plus raw markdown text. Non-LLM, repeatable, and auditable.
Azure AI Foundry / OpenAI (GPT-5.2) — Bounded gap-fill verifier invoked only for missing or low-confidence fields (typically 10–20% of total). Temperature 0.0, JSON response format enforced, schema-aware prompting with domain rules.
Azure Cosmos DB (Serverless)— Document persistence with SHA-256 deduplication, version increment on re-processing, and partition-by-document-type for efficient querying. Pay-per-request scales from zero.
Azure Service Bus (Basic) — Event-driven queue integration with `document-processing` and `human-review` queues for processing triggers and escalation routing.
Application Insights + OpenTelemetry — End-to-end observability with per-stage telemetry events, custom metrics (fill_rate, record_confidence, extraction_duration_ms), and distributed tracing

Cost Impact of Hybrid Approach

Metric	CU-Only	GPT-Only	Hybrid (This Architecture)
Cost per document	~$0.01	$0.15–0.30	$0.03–0.05
Determinism	100%	Variable	95%+
Accuracy	75-80%	80–90%	90-95%
Auditability	Full	Limited	Per-field source attribution

Cost savings: 60–80% reduction compared to GPT-only by limiting LLM to gap fields.

Security and Enterprise Considerations

Azure Blob Storage:

Storage accounts can be secured by minimizing public exposure, enforcing strong identity‑based access, protecting data, and continuously monitoring for threats. Organizations should use Private Endpoints and disable public network access wherever possible, authenticate users and applications with Microsoft Entra ID instead of shared keys, and apply least‑privilege Azure RBAC with managed identities. Data should be encrypted in transit (TLS 1.2+) and at rest using Microsoft‑managed or customer‑managed keys stored in Azure Key Vault, while Microsoft Defender for Storage, logging, soft delete, backups, and Azure Policy should be enabled to detect threats, support recovery, and enforce compliance at scale. Content Safety can be called from the application layer to block uploads based on image content. Staging containers can be used to isolate untrusted uploads. Content Safety provides signals; your app enforces policy.

Azure Content Understanding / AI Vision:

Azure AI services support enterprise-grade security through Microsoft Entra ID–based authentication and Azure RBAC, ensuring only authorized applications can access extraction models. Network isolation can be enforced using Virtual Network (VNet) integration and Private Link to restrict public internet exposure. All data transmitted is encrypted in transit and at rest. Microsoft Defender for Cloud provides continuous security posture visibility across these AI workloads.

Azure OpenAI

Govern which models are approved for use and protect model artifacts and training data from unauthorized access through strong identity, network, encryption, and logging controls. AI applications should be designed with layered defenses, including multi‑stage content filtering, safety meta‑prompts, and least‑privilege permissions for agents and plugins to reduce the risk of prompt injection, data leakage, and unintended actions. High‑risk AI operations should include human‑in‑the‑loop review to prevent autonomous execution of harmful or incorrect outcomes. Organizations must continuously monitor AI systems for misuse, anomalous behavior, and data exfiltration, and they should perform ongoing AI red teaming to identify vulnerabilities such as jailbreaking, adversarial inputs, and model manipulation before they can be exploited.

Azure Cosmos DB

Azure Cosmos enhances network security by supporting access restrictions via Virtual Network (VNet) integrationand secure access through Private Link. Data protection is reinforced by integration with Microsoft Purview, which helps classify and label sensitive data, and Defender for Cosmos DBto detect threats and exfiltration attempts. Cosmos DB ensures all data is encrypted in transit using TLS 1.2+ (mandatory) and at rest using Microsoft-managed or customer-managed keys (CMKs).

Azure Functions / Compute

Secured with Entra ID authentication and managed identities, least-privilege RBAC, HTTPS-only access, private endpoints, VNet integration, and Key Vault for secrets. Hardened with Azure Policy, Defender for Cloud, and centralized logging.

Microsoft Foundry

Microsoft Foundry supports robust identity management using Azure Role-Based Access Control (RBAC) to assign roles within Microsoft Entra ID, and it supports Managed Identities for secure resource access. Conditional Access policies allow organizations to enforce access based on location, device, and risk level. For network security, Azure AI Foundry supports Private Link, Managed Network Isolation, and Network Security Groups (NSGs) to restrict resource access. Data is encrypted in transit and at rest using Microsoft-managed keys or optional Customer-Managed Keys (CMKs). Azure Policy enables auditing and enforcing configurations for all resources deployed in the environment. Additionally, Microsoft Entra Agent ID, which extends identity management and access capabilities to AI agents. AI agents created within Microsoft Foundry are automatically assigned identities in a Microsoft Entra directory centralizing agent and user management in one solution. AI Security Posture Management can be used to assess the security posture of AI workloads. Defender for AI Services provides threat protection and insights for you AI resources. Purview APIs enable Azure AI Foundry and developers to integrate data security and compliance controls into custom AI apps and agents. This includes enforcing policies based on how users interact with sensitive information in AI applications. Purview Sensitive Information Types can be used to detect sensitive data in user prompts and responses when interacting with AI applications.

DevOps Security

Security is further “shifted left” by integrating automated controls directly into CI/CD pipelines. GitHub Advanced Security for Azure DevOps, which provides dependency scanning, CodeQL-based static application security testing (SAST), and secret scanning to identify vulnerabilities and exposed credentials in code and third-party libraries. Infrastructure-as-code templates can be validated with Azure Policy and Microsoft Defender for Cloud, while pipeline protections such as protected branches and approvals reduce the risk of unauthorized changes. DevOps environments can be hardened using Azure Key Vault for secrets management, Managed Identities and Microsoft Entra ID for least-privilege access, and monitoring through Azure Monitor . Microsoft Defender for Cloud DevOps Security provides centralized code‑to‑cloud visibility across Azure DevOps, GitHub, and GitLab, identifying risks in code, secrets, dependencies, and IaC and helping teams prioritize fixes early in CI/CD pipelines

Related and Future Scenarios

Although document extraction serves as the initial use case, this architecture establishes a scalable pattern for many applications:

Insurance Claims Processing: Swap schema to claim fields; update CU analyzer for claim forms
Legal Contract Analysis: Schema for clauses, parties, dates; add NER in normalization
Healthcare Medical Records: HIPAA-compliant Cosmos; schema for diagnoses, medications, vitals
Financial Document Processing: Schema for transactions, accounts; add currency normalization
Engineering/Construction Plans: Schema for dimensions, materials, specifications
Digital Twin Integration: Feed extracted data into asset models for real-time facility visualization
Predictive Analytics: Track extracted values over time for trend detection and forecasting

Conclusion

Modernizing document extraction is not simply about applying AI—it requires aligning technology, operational discipline, and data quality. Early exploration using Generative AI enabled rapid learning and feasibility validation. However, a production-grade solution must be built on structured layout understanding models supported by standardized schema definitions and operational controls.

By combining primary structured extraction with Generative AI reasoning for bounded gap-fill verification, organizations can achieve scalable, repeatable, and auditable extraction processes. This hybrid approach enables reduced manual effort, lower error rates, and the transition from batch manual processing to intelligent, automated workflows.

The result is not just an automated extraction tool, but a scalable AI architecture for modern document intelligence—adaptable to any industry, any document type, and any structured data need.

Contributors:

This article is maintained by Microsoft. It was originally written by the following contributors.

Gaurav Bhardwaj | Senior Cloud Solution Architect – US Customer Success 
Manasa Ramalinga | Senior Principal Cloud Solution Architect – US Customer Success 
Abed Sau | Principal Cloud Solution Architect – US Customer Success

Announcing the Path to Production for Agents Webinar Series

brauerblogs — Tue, 07 Jul 2026 14:54:02 GMT

Many organizations have made significant progress exploring AI—building pilots, prototypes, and proofs of concept. Yet a common challenge remains: how do you move from promising experiments to production-ready systems that are secure, scalable, and trusted? Join us for the Path to Production Webinar Series on July 27-28, a two-day deep dive designed to help technical teams operationalize AI and agent-based solutions using proven architecture patterns, governance models, and engineering practices.

This simulive event will be available in two time zones, making it easier to participate.

Use this link to register: https://aka.ms/AccelerateThePathToProduction

Why this series matters

A large percentage of AI initiatives never make it to production—not because of lack of ambition, but because organizations struggle to establish trustworthy, governed AI systems; build scalable architectural foundations; manage risk, cost, and operational complexity; and ensure reliability in non-deterministic systems. This webinar series addresses those challenges head-on with concrete, actionable guidance spanning the full lifecycle of production AI systems.

What attendees will learn

This series delivers an implementation-focused roadmap for building, deploying, and operating AI agents at enterprise scale. Each session dives deep into governance, architecture patterns, orchestration, security, evaluation, and observability - with reference architectures and real-world engineering examples. Learn how to design scalable agent systems, integrate with enterprise data and services, and apply best practices for reliability and performance.

Attendees will leave with practical techniques and proven patterns to confidently ship production-grade agent solutions. After the workshop, customers who have Unified Contracts are eligible for a packaged set of engagements that will implement this guidance with your Microsoft cloud solution architects. Otherwise, contact your partner to learn more about taking advantage of the Frontier Transformation Offer through the Frontier Accelerate program.

Session overview

Day	Session	Focus	Speaker
July 27	Frontier Center of Excellence (CoE) & Governance	Create a governance framework with quality gates that helps organizations deliver secure, responsible, trustworthy AI at scale.	Akriti Mehta Divye Sheth
July 27	AI Landing Zones	Build a production-ready reference architecture for AI applications and agents with guardrails for networking, identity, security, and cost governance.	Nadeem Ishqair Bilal Amjad
July 27	Agentic Architecture	Adopt a governance-first, multi-agent architecture blueprint that embeds controls from user channels and orchestration through integration layers, data, and models.	Yeliz Kilinc Nour Shaker
July 28	AgentOps	Apply DevOps principles to production AI, including evaluation, CI/CD quality gates, observability, monitoring, red teaming, and incident response.	Paulo Lacerda Richard Healy
July 28	AI Security, Trust & Observability	Address prompt injection, data leakage, autonomous tool misuse, and AI-specific observability requirements for traceability, safety, and auditability.	Yuening Chen Raaid Mahbub
July 28	Solution Optimization	Reduce token cost, cut latency, tune RAG, optimize multi-agent coordination, and apply FinOps practices for sustainable scale.	Tanuja Bhamidipati Fatos Ismali

Day 1: Establishing the foundation for production AI

July 27

AI Center of Excellence (CoE) & Governance

Why do so many AI initiatives die in the PoC graveyard? Because organizations cannot trust the AI. This session shows how an AI CoE plus governance framework creates a uniform quality gate at every layer of your AI application in an organization for delivering a single, organization-wide view of secure, responsible, trustworthy AI that is ready to scale.

AI Landing Zones

Scaling AI from experimentation to production demands a secure, governed, and scalable foundation. This session explores how AI Landing Zones provide a production-ready reference architecture for deploying AI applications and agents with the right guardrails for networking, identity, security, and cost governance, aligned with the Cloud Adoption Framework and Well-Architected best practices. Attendees will learn how to design AI platforms that balance innovation with compliance, accelerate time-to-production using validated architectures and infrastructure-as-code, and integrate AI services into enterprise environments.

Agentic Architecture

Many enterprise AI pilots stall not for lack of technology, but because they lack a trustworthy architecture. This session introduces a governance-first, multi-agent architecture blueprint that closes the trust gap by embedding uniform controls and quality checks at every level, from user channels and agent orchestration through integration layers to core data and models, under a common governance and security framework. Attendees will learn how this layered agentic architecture creates a reliable, enterprise-wide AI fabric that organizations can adopt with confidence, aligning AI initiatives with high standards of trust, interoperability, and scale.

Day 2: Operating and scaling AI in production

July 28

AgentOps

This session covers the full lifecycle of deploying and operating agentic AI solutions in production. We will explore how teams can move from successful prototypes to production-ready agents using evaluation, CI/CD quality gates, observability, continuous monitoring, scheduled red teaming, and incident response practices. We will also cover how to apply DevOps principles to the unique challenges of AI systems, including non-deterministic behavior, prompt regression, model drift, tool-calling risk, and changing user behavior. Attendees will learn a practical AgentOps operating model for improving release confidence, detecting regressions earlier, and connecting agent operations back to Microsoft Foundry and Azure Monitor.

AI Security, Trust & Observability

This session focuses on securing AI systems in production, addressing risks beyond traditional application security such as prompt injection, data leakage, and autonomous tool misuse. It applies a defense-in-depth approach across identity, data protection, orchestration, and runtime controls. It also introduces AI-specific observability for trust and compliance, including traceability, safety and security monitoring, and auditability, ensuring AI systems are secure, controllable, and compliant at scale.

Solution Optimization

Getting AI to production is only half the battle. Once agentic workloads are live, organizations face compounding challenges including rising token costs, latency that degrades user trust, RAG pipelines that return noise instead of signal, and orchestration overhead that multiplies with every agent added to the mesh. This session provides a practical engineering playbook for optimizing agentic AI across the full stack, from model selection and inference routing through prompt compression, RAG tuning, caching strategies, and multi-agent coordination. It also covers the FinOps discipline required to control cost at scale, including capacity sizing, batch processing, and intelligent model routing. Attendees will leave with actionable patterns for reducing inference cost, cutting latency, and scaling reliably across regions.

Who should attend

Cloud and solution architects
AI and ML engineers and developers
Platform engineering and infrastructure teams
Technical decision-makers driving AI transformation initiatives

If your team is working to move AI beyond prototypes into production-scale systems, this series will provide directly applicable guidance for architecture, governance, operations, and optimization.

Next steps after the webinar series:

We will conduct a personalized assessment of your organization’s readiness to adopt AI agents at scale.

Call to action

Join us on July 27-28 to accelerate your path from AI experimentation to trusted, enterprise-scale production systems. Registration details can be added to this announcement before publication.

Use this link to register: Path to Production for Agents

Cloud Native Platforms: Build

KishoreKumarPattabiraman — Thu, 21 May 2026 22:41:29 GMT

Audience: Cloud architects, platform engineers, engineering leaders making design decisions

Reading time: 8 minutes

Series: Cloud Native Platforms. Build, Run, Evolve. This is Part 1 of 3.

Most engineering teams can build systems.

Few can scale them without rebuilding them.

As platforms grow, complexity does not increase linearly. It multiplies across users, services, tenants, regions, and integrations. The systems that struggle and the systems that scale are rarely separated by which cloud they run on. They are separated by a handful of design choices made early and applied consistently.

This post is about those choices.

The differentiator is not the cloud

Scalable platforms are not built with the right tools. They are built with the right design choices.

Cloud services have closed the gap on infrastructure. The differentiator is no longer which managed service a team picks. It is whether the platform is designed to absorb change, tolerate failure, and support visibility from day one. Five engineering disciplines determine whether a platform scales gracefully or collects technical debt while it grows.

Figure 1. The five disciplines compound into platform scale. Any one neglected becomes the constraint that forces a rewrite later.

1. Flexibility is the foundation of scale

Hard-coded systems work until they do not. The first request to add a tenant, a region, a SKU (a sellable product variant), or a regulatory variant is the moment a rigid design starts to bend. Each subsequent request adds weight.

Scalable platforms move behavior out of code:

Configuration replaces conditional logic
Feature flags enable safer, tenant-scoped rollouts
APIs evolve through versioning, not breaking changes
Schemas evolve additively. Breaking changes go through versioned contracts with a deprecation window long enough that consumers can migrate without downtime.

In practice

The pattern that works: configuration in a managed store, feature flags with tenant scope, and APIs versioned per consumer contract. Cost is the discipline of treating configuration as code (versioned, reviewed, audited). The return is that releases stop being events and start being routine. A change that previously needed a coordinated deployment can be executed in minutes, gated to a single tenant for verification, and rolled out broadly only after the signal is clean. Most platforms reach this state by retrofit, not by design. Doing it earlier costs less than waiting.

If a change requires a redeploy, it should require a very good reason.

2. Failures are normal. Resilience is a choice.

Distributed systems will fail in unpredictable ways. The real question is not how to prevent failure. It is how the system responds when failure happens.

Resilience is engineered, not inherited from the platform. The patterns that move the needle are well known and consistently applied:

Idempotent operations (safe to call multiple times with the same result) that make retries safe
Reliable messaging patterns such as the transaction outbox (writing the message to the same database transaction as the business change, then publishing asynchronously) to avoid lost or duplicated events
Decoupled services that contain blast radius (the scope of damage when one component fails)
Timeouts, retries, and circuit breakers (a wrapper around a dependency that stops calling it for a cool-off window after repeated failures) tuned per dependency
Bulkheads (isolation pools, often a separate compute or queue lane per workload class) that keep noisy neighbours from starving critical paths of resources

In practice

The pattern that works: every write that can be retried carries an idempotency key, every queue consumer is safe to replay, every event published goes through an outbox in the same transactional unit as the business change. When peak load triggers retries, duplicates collapse cleanly instead of producing duplicate orders, double-charged customers, or split-brain state. The contract changes outwards: callers can retry without thinking, queues can be at-least-once instead of exactly-once, and recovery moves from a manual cleanup task to a property of the system. Most teams that adopt this pattern stop seeing certain classes of incident entirely.

Implementation note

An idempotent API is not just a design preference. It changes how the rest of the system can be built. Once writes are safe to repeat, retries become cheap, queues become trustworthy, and recovery becomes automatic.

The naive implementation (read the key, if absent process and save) has a race. Two concurrent requests with the same key both miss the lookup, both call the processor, and both attempt to save. That is the failure mode idempotency exists to prevent. The pattern that survives production is an atomic reserve-then-execute: insert a row keyed by the idempotency key with a unique constraint before doing any work. The first writer wins. Concurrent callers either wait for the original to complete and read its result, or they receive a conflict response.

// Contract for the idempotency store. The two key methods are TryReserveAsync // (atomic insert with unique-key constraint) and CompleteAsync (record the // result of the first writer). GetCompletedResultAsync polls until the first // writer commits or returns 409 Conflict if the in-flight window exceeds the // configured deadline. public interface IIdempotencyStore { Task<Reservation> TryReserveAsync( string idempotencyKey, string requestHash, CancellationToken ct); Task CompleteAsync( string idempotencyKey, OrderResult result, CancellationToken ct); Task<OrderResult> GetCompletedResultAsync( string idempotencyKey, CancellationToken ct, TimeSpan? maxWait = null); } public readonly record struct Reservation( bool IsFirstWriter, string RequestHash); // Idempotency via atomic reserve-then-execute. // First writer wins; replays return the original result; concurrent // duplicates lose the race and read the winner's outcome (or get 409). public async Task<OrderResult> CreateOrderAsync( Order order, string idempotencyKey, CancellationToken ct) { var requestHash = StableHash(order); // canonical content hash // Atomic insert: succeeds for the first caller, fails for the rest. var reserved = await _store.TryReserveAsync( idempotencyKey, requestHash, ct); if (!reserved.IsFirstWriter) { if (reserved.RequestHash != requestHash) throw new IdempotencyKeyReusedException(); // A previous run committed (return its result) or is in-flight // (poll with a bounded deadline; 409 if exceeded). return await _store.GetCompletedResultAsync( idempotencyKey, ct, maxWait: TimeSpan.FromSeconds(5)); } // We are the first writer. Execute, persist, mark complete. var result = await _processor.ProcessAsync(order, ct); await _store.CompleteAsync(idempotencyKey, result, ct); return result; }

Three production details matter:

TTL or compaction on the idempotency record. Without it, the store grows forever. Most teams retain records for the request retry window plus a safety margin (commonly 24 to 72 hours).
Stable content hash, not the default object hash code. The request hash detects key reuse with a different body, so a client that reuses an idempotency key with a different payload receives IdempotencyKeyReusedException rather than silently getting the wrong result. Canonicalise field ordering, locale, and null handling before hashing.
Bound the in-flight window explicitly. The genuinely hard case is when the processor succeeded but the store write failed. Production-grade implementations either run the side-effect and the store write in the same transaction (when the processor and store share a database) or use the transaction outbox pattern to bridge them. The poll-with-deadline in GetCompletedResultAsync handles the duplicate-arrives-mid-flight case; the transactional boundary handles everything else.

3. Observability is not optional

Without observability, teams operate blind. As systems grow, the price of guessing rises faster than the price of seeing.

At build time, observability is a design property. The decisions made before the system reaches production are what determine whether it can be operated at all. The dashboards, alerts, and incident practices covered in Part 2 of this series rely on instrumentation choices made here.

The build-time work that pays off in production:

Request identifiers propagated through every service hop, every queue, every async boundary, so a single user action can be traced end to end
Structured logging with a consistent schema (event name, correlation id, tenant, severity) rather than free-form strings
Metrics emitted at the boundaries that matter (every external call, every queue read or write, every database operation), not only at the entry point
Tracing libraries integrated at the framework or middleware layer so coverage is automatic, not opt-in
Schemas designed so business signals (orders, sessions, transactions) and system signals (CPU, latency, errors) share the same identifiers and can be correlated later

In practice

The pattern that works: a single request id flowing through every service hop, every queue, every async boundary, propagated automatically at the framework layer rather than per-call. Add one structured logging schema across services (event name, correlation id, tenant, severity), so that a single query joins business events with system events. The investment is hours of upfront framework wiring. The return is that production diagnosis stops being archaeology. Cross-service questions become single dashboards; postmortems shrink from days to hours; and the dashboards in Part 2 actually work because the data underneath is shaped to support them.

4. Delivery practices set the ceiling

Scaling teams requires scaling delivery. Small inefficiencies in pipelines, environments, and release coordination compound into measurable drag.

Delivery maturity that pays off at scale:

Pipelines as code, reviewed and versioned like application code
Parallel deployments across services and regions where dependencies allow
Infrastructure as code with shared modules, not hand-managed environments
Automated quality gates: tests, security scans, dependency checks
Trunk-based development (developers commit to a single shared branch many times a day) with short-lived feature branches and progressive delivery. Important caveat: trunk-based works only when test automation and feature flags are already in place. Adopting it before those foundations exist tends to amplify production incidents rather than reduce them.

In practice

The pattern that works: pipelines run in parallel where dependencies allow, infrastructure provisioning is templated rather than per-environment, and quality gates run automatically rather than as discretionary steps. Sequential deployment of a multi-service platform across three environments takes hours; parallelised deployment of the same change takes minutes. The payback is not only release speed. It is the compounding cost reduction of every wait state for every engineer on every release. Teams that treat pipelines as a product feature, not an afterthought, ship more confidently and recover from bad changes faster because the rollback path was exercised, not invented during an incident.

Slow pipelines are not a tooling problem. They are a design problem.

5. Cost discipline is engineering work

Cloud platforms can become expensive quickly when cost is treated as someone else's problem. Cost is a property of the design, not a quarterly review.

The teams that get this right treat cost the same way they treat performance:

Elastic compute and storage tiers chosen per workload pattern
Non-production environments with automated scale-down windows (the easiest savings to leave on the table)
Tagging discipline so cost can be attributed to a service, a feature, a tenant
Egress and data-tier choices, not compute, dominate cloud bills past a certain scale. Right-size storage tiers (hot vs cool vs archive), eliminate cross-region chatter, and watch egress on the data plane more closely than compute on the request path.
Budgets and usage alerts wired into the same channels as reliability alerts
Cost reviews built into design discussions, not deferred to FinOps (Financial Operations: the practice of managing cloud spend as an engineering concern)

In practice

The pattern that works: non-production environments scale down automatically outside business hours, storage tiers match access patterns (hot, cool, archive), and tagging is enforced so every dollar can be attributed to a service or feature. Cost reviews happen at design time, not after the bill arrives. The biggest savings come from data plane decisions, not compute: cross-region egress, oversized storage tiers, and forgotten test environments dominate cloud bills past a certain scale. Treat cost as a first-class non-functional requirement, alongside latency and availability, and the discipline compounds in every design discussion that follows.

A scenario that ties it together

Figure 2. A reference architecture that puts the disciplines into one shape. The request path is decoupled, the data layer is purpose-fit, identity is brokered by managed identity throughout, private endpoints isolate the data tier from public networks, and observability runs as a first-class lane.

Picture a multi-tenant platform at a growth inflection. Onboarding a new tenant takes weeks because tenant-specific behaviour is hard-coded across services. Every release carries risk because there is no way to roll out a change to one tenant without affecting the rest. Incidents linger because logs and metrics live in different tools and nobody can correlate them in production.

Do not start with a rewrite. Start with the smallest set of changes that unlocks the next year of growth: extract configuration out of code, introduce tenant-aware feature flags, wire a unified observability view into the existing services, and parallelise the pipelines. None of these are architectural revolutions. They are design choices applied with discipline, in the order the disciplines compound.

Eighteen months in, onboarding a tenant takes hours instead of weeks. Releases move from monthly events to weekly increments. Incidents are caught earlier and resolved faster. The platform did not get bigger. It got more capable. The five disciplines did the work; the team made the choice to apply them.

What teams get wrong

The common pattern is architecting for the system you have, not the system you are growing into. It looks like progress because the current sprint ships. Pillars get postponed because they feel like overhead.

The cost surfaces later. Each shortcut becomes a constraint. The constraints compound, and three releases later the team is debating a rewrite.

The fix is not premature abstraction. It is small, deliberate investments in flexibility, resilience, observability, delivery, and cost from day one. The discipline is to make these investments before they are urgent.

Where to start when you cannot do everything at once

Five disciplines is a wall, and real teams cannot fund all five at once. The right order depends on whether the platform is being built fresh or already running.

For a system already in production and already in pain, the SRE community's hierarchy of reliability needs gives the most defensible starting order: monitoring and observability first (you cannot fix what you cannot see), then incident response (close the bleeding cleanly), then resilience patterns (idempotency, retries, decoupling) so the bleeding has fewer reasons to start, then flexibility and delivery so safe change can travel at speed. Cost discipline runs alongside throughout, never as the headline.

For a system being built fresh, the order in this post (flexibility, resilience, observability, delivery, cost) reflects the Azure Well-Architected Framework's emphasis on designing for change, failure, and visibility before scaling teams or workloads. Both orders are defensible. What is not defensible is leaving any of the five for later.

The most concrete starter from this post: request id propagation. A single correlation identifier travelling through every service hop, every queue, every async boundary, costs hours up front and pays back every time someone has to debug production for the rest of the platform's life. It is the smallest unit of the observability discipline and the foundation that the dashboards, traces, and incident response in Part 2 all depend on.

The shift

The most important transformation in scaling a platform is not technical. It is mindset.

The shift is from project thinking to platform thinking:

Build reusable capabilities, not one-off solutions
Design systems for long-term evolution, not the next release
Enable other teams, not just deliver for one team

Tools change. Cloud services evolve. The architectural fashions of this year will not be the architectural fashions of the next. What persists is the discipline behind the choices. Scalable systems are not built by tools. They are built by teams that treat design as continuous work. The same discipline shows up again in Part 2 (operating these systems) and Part 3 (using AI to augment that work). The tools change. The disciplines do not.

Want to discuss? What single design choice has paid the most dividends in the platforms you run? Drop a comment with patterns you have seen in your environment. Every reply gets read.

Next in this series: Running Cloud Native Platforms: Why Day 2 Decides Everything. Building is half the journey. The next post looks at what it takes to operate these platforms once they are in production.

Cloud Native Platforms: Run

KishoreKumarPattabiraman — Thu, 21 May 2026 22:41:09 GMT

Audience: SREs (Site Reliability Engineers), platform engineers, engineering managers running production systems

Reading time: 8 minutes

Series: Cloud Native Platforms. Build, Run, Evolve. This is Part 2 of 3.

Most systems are designed thoughtfully.

Most operations are inherited reactively.

The systems that survive are not the ones built with the most care. They are the ones operated with the most discipline. Production has a way of revealing every shortcut taken during design and every assumption left unverified.

This post is about what it takes to operate a platform once the build is done.

How they are run, not how they are built

Systems are not defined by how they are built. They are defined by how they are run.

A well-designed system that is operated reactively will fail in production. A modestly designed system that is operated with discipline will outperform it. Five operational disciplines decide which side of that line a platform lives on. Each one is engineering work, not a checklist for someone else to handle.

Figure 1. The incident lifecycle as a state machine. The states are not optional steps. They are the contract between the team and the system.

1. Observability is the backbone of reliability

Without observability, every operation becomes a guess. As systems grow, the cost of guessing rises faster than the cost of seeing.

Part 1 of this series argued that observability is a design property: instrumentation contracts, request id propagation, structured logging schemas. Production is where those design choices either pay off or do not. Strong observability in production is a contract that lets any engineer answer three questions in minutes: what failed, why it failed, and what the impact was. The shape of that contract matters more than the tool that implements it. (This three-question framing is community-popularised through the SRE community and writers such as Charity Majors. See Honeycomb's What is Observability for the canonical articulation of the three-pillars and question framing; the substance is older than the framing.)

Dashboards organised around user journeys, not infrastructure components
Service level indicators (SLIs: the specific measurements you care about, e.g., success rate, p99 latency) chosen from the user's perspective, not the database's
Alerts that page only on burn-rate against an SLO (Service Level Objective: the target value of an SLI, e.g., 99.9% of requests complete in under 800ms over a rolling month) using a multi-window strategy. A short window catches fast burns; a long window catches slow drifts. This is what makes SLOs operational rather than decorative.
Sampling and retention tuned for cost, but never for blind spots
The distinction between MTTA (mean time to acknowledge: how fast someone notices) and MTTR (mean time to restore: how fast service returns) tracked separately. Conflating them hides whether the team's bottleneck is detection, response, or fix.

In practice

The pattern that works: rebuild the operational view around two or three user journeys (sign-in, place order, view history) rather than per-component charts. Tie alerts to error budget burn rather than raw threshold crossings. Track MTTA and MTTR separately so the team's actual bottleneck (detection, response, or fix) is visible. The investment is rethinking what to measure, not buying a new tool. The return is that incidents stop being discovered by customer complaints first. Teams that make this shift typically find their existing telemetry was sufficient; only the questions being asked of it were wrong.

If a dashboard cannot answer "what is the user experiencing right now", it is not an observability dashboard. It is decoration.

2. Alerts are signals, not notifications

More alerts do not mean better monitoring. In practice, the opposite is true. Once alerts outpace the team's ability to act, important signals start getting missed.

Effective alerting works to a small set of rules:

Severity that maps to action, not to technical category
Ownership baked in, never inferred at runtime
Thresholds tied to user impact, not raw metric values
Noise treated as a defect, with a regular review cadence
Suppression and grouping for known multi-alert patterns

In practice

The pattern that works: audit every alert against one test, "what action would I take in the next five minutes if this fires now?" Demote alerts with no answer to dashboards. Remove alerts where the answer is the same as another alert's. Group related alerts so one incident produces one page, not twelve. Most teams discover their alert volume drops by an order of magnitude after a thorough audit, and the alerts that remain start getting trusted again. Trust is the precondition for every other operational practice. Without it, on-call rotations decay into noise filtering and the real signals get missed.

Figure 2. From raw events to pages, in approximate orders of magnitude. The numbers vary by team and workload; what does not vary is that each stage needs to remove one to two orders of magnitude of noise. Teams that page on raw events end up with on-call rotations nobody trusts.

3. Incident response is a practiced muscle

Failures are inevitable. Unstructured response is not.

The teams that recover quickly do not improvise during incidents. They follow a structure that has been practiced when nothing was on fire. The structure is intentionally simple, because incident time is the worst time to negotiate roles.

Clear roles: incident lead, communications lead, scribe, subject matter expert (the RACI model, Responsible-Accountable-Consulted-Informed, adapted for incident response)
Defined escalation paths with clear handoff criteria. Escalation means re-paging to a higher tier or specialist, not returning to detection. The lifecycle diagram in Figure 1 makes the distinction explicit.
Runbooks for the top failure modes, kept short enough to actually be read
Status communication on a fixed cadence, even when there is nothing new to say. Customer comms and internal comms are tracked separately.
Blameless postmortems (focus on the system that allowed the failure, not the person who pushed the button) that produce action items the team actually completes
Game days: scheduled exercises that simulate failure modes (region outage, dependency unavailability, traffic spike) under controlled conditions, so gaps in runbooks are found before incidents do

In practice

The pattern that works: name the incident lead and the comms lead before the first message goes out. Write runbooks short enough to be scannable at 3 AM. Run blameless postmortems with action items that actually get tracked to completion. Schedule game days quarterly so the runbooks are exercised before real incidents. Teams that operate with this structure do not have more engineers; they have engineers who are not single points of failure during recovery. The deepest experts stay the deepest experts, but the platform stops depending on whether they happen to be online.

Implementation note

A short, well-structured runbook outperforms a long, exhaustive one. The goal during an incident is not to think. It is to act on a procedure that has been thought through in calmer times.

# Runbook header pattern (keep it scannable in incident time) title: High latency on order API slo_protected: # this runbook protects two SLOs - order-completion-success - order-completion-latency severity: # derived from burn rate, not declared fast_burn: P1 # 14.4x budget burn over 1 hour => page now slow_burn: P2 # 6x budget burn over 6 hours => investigate owner: payments-team indicators: # triggers for evaluation, not severity - p99 (99th-percentile) latency exceeds the SLO target for 5 min - error rate exceeds the SLO target for 3 min on order-completion first_actions: - Open the order-journey dashboard. Confirm impact in business terms. - Check Service Bus queue depth and dead-letter rate (the most common cause of API latency under load is downstream backpressure) - Verify Cosmos DB RU/s saturation and partition hotspots - Inspect the most recent deployment for behavioural changes escalate_if: - Latency does not recover in 15 min - Error rate exceeds 5% (fast burn against the SLO) - Customer reports arrive before our own signals do rollback_path: - Feature flag "new-order-pipeline" can be disabled per-tenant - Last known good deployment id is in the release tracker note_on_scaling: # CPU is rarely the cause of latency in this service. Scale only after # confirming the bottleneck is compute, not a downstream dependency or # queue depth. Adding capacity to a saturated downstream amplifies the # incident; it does not resolve it.

The general principle behind that last note travels beyond this runbook: scale-out is the right remediation for compute saturation, not for downstream saturation. When latency rises because a database, queue, or external dependency is saturated, adding capacity in front of the bottleneck moves more requests into the bottleneck and makes the incident worse. This is one of the most common operational mistakes when the dashboard shows red and the on-call instinct says "add more".

4. Release confidence is engineered

Releases get harder as systems grow. The platforms that ship confidently at scale have engineered the path, not learned to fear it.

The patterns that change the math:

Feature flags that allow change without deploy
Canary deployments (releasing the new version to a small slice of traffic first, watching error budget burn before continuing) that surface problems on a small slice
Gradual rollouts with automated rollback triggers
Database migrations split from application releases
Release coordination that scales with services, not with team size

In practice

The pattern that works: every change ships behind a feature flag, canary deployments take a small slice of traffic first, and rollback is a one-click step in the pipeline rather than a procedure to be invented during an incident. The cost is the discipline of building rollback paths and exercising them. The return is releases that stop being events. Issues that previously triggered full rollbacks get isolated to a slice and rolled back automatically before they reach most users. The willingness to ship smaller, more frequent changes follows directly from the confidence that bad changes can be undone fast.

Big releases feel safe because they are rare. They are actually risky because every change rides together.

5. Reliability is continuous, not a milestone

Reliability is not achieved through tools alone. It requires continuous refinement, feedback-driven improvement, and a budget that the team can spend on operational work without negotiating each time.

The disciplines that keep systems reliable over years are codified well in the SRE-book framing of service level objectives and error budgets (the canonical reference is the Google SRE Book chapter on Service Level Objectives, with the operational follow-up in the SRE Workbook chapter on alerting on SLOs). The names matter less than the practice they enable.

SLOs chosen from the user's perspective, with two or three per service rather than ten. More SLOs means none of them shape behaviour.
Error budgets: the inverse of the SLO, expressing how much unreliability the team is willing to spend in a window. Used up early in the month means slow down on releases. Healthy means feature work keeps moving.
Multi-window burn-rate alerting turns SLOs from dashboards into pages: short window catches catastrophic failures, long window catches slow drift. Without burn-rate alerting, SLOs are observation, not operation. (The pattern is documented in the SRE Workbook.)
Reliability work has its own backlog, prioritised against features. Not a wishlist after every incident.
Regular game days that exercise failure modes (region failover, dependency outage, traffic spike) before they happen for real
Capacity planning informed by data, not by anxiety

In practice

The pattern that works: define two or three SLOs per service, expressed from the user's perspective. Compute the error budget weekly. When the budget is healthy, ship feature work. When the budget is burning fast, slow down and fix the cause. The conversation about which incidents matter and which can wait becomes possible because there is a shared number to point at. Reliability becomes a quantified property of the platform, not an opinion debated at every retrospective. Teams that adopt this discipline stop having the recurring "how reliable do we need to be?" argument and start having data-grounded trade-off discussions instead.

A scenario that ties it together

A platform was launching a new region. The build had gone well. Day 1 was clean. Two weeks in, latency started creeping up during peak hours. Alerts fired on raw thresholds, but no one could tell which ones to trust. Incident calls turned into long debugging sessions because three different teams owned overlapping pieces of the request path.

The team did not start by buying a new tool. They started by treating operations as engineering work. The dashboard was redesigned around the user journey. Alerts were audited and most were demoted or removed. Roles for incident response were written down. A short runbook covered the top failure modes. Releases were broken into canary slices behind feature flags.

None of this was new. It was discipline applied consistently to work that was previously assumed to be someone else's. The next region launch took half the effort, and the team's mean time to restore on the failures that did happen was measurably lower.

What teams get wrong

The common pattern is treating Day 2 as the cost of Day 1. Teams design beautifully, ship fast, then quietly absorb the operational debt. Dashboards proliferate. Alerts grow louder. Postmortems pile up.

The fix is not more dashboards. It is treating operations as engineering work with the same rigour as feature delivery. Operability is a property the system either has or does not. It is not earned by adding monitoring. It is earned by designing for visibility and operating with discipline.

Where to start

The most concrete starter from this post: an alert audit. List every alert that fires in the next week and apply a single test to each one: "what action would I take in the next five minutes?" Demote the alerts that have no answer. Remove the alerts where the answer is the same as another alert's. The audit takes a morning. The result usually halves alert volume and lifts trust on what remains, which is the precondition for every other operational practice in this post.

The shift

The most important shift in maturity is not technical. It is in stance.

The shift is from shipping software to operating systems:

Operations is not a phase that follows engineering. It is engineering.
Reliability is not a milestone reached. It is a discipline practiced.
Incidents are not interruptions to the work. They are the work.

The teams that internalise this shift run platforms that are smaller, calmer, and more trusted. They do not have fewer incidents because their systems are more advanced. They have fewer incidents because their operational discipline is more consistent. Part 3 of this series argues that the same discipline applies again, in a different domain: the practices that make platforms operable are the practices that make AI useful in delivery.

Want to discuss? What is the one operational practice your team adopted that changed how you sleep at night? Drop a comment with patterns you have seen in your environment. Every reply gets read.

Previously in this series: Building Cloud Native Platforms That Scale: Patterns That Actually Work. The first post covered the design choices that make scale possible.

Next in this series: AI-First Platform Engineering: From Copilot to Agentic Delivery. Cloud helped us scale infrastructure. The next post looks at how AI is now changing how we build and run platforms.

Cloud Native Platforms: Evolve

KishoreKumarPattabiraman — Thu, 21 May 2026 22:40:33 GMT

Audience: Engineering leaders, platform architects, senior developers exploring how to operationalise AI in their teams

Reading time: 8 minutes

Series: Cloud Native Platforms. Build, Run, Evolve. This is Part 3 of 3.

Cloud helped us scale infrastructure.

AI is starting to do the same thing for the work around the code: the planning, the testing, the release communication, the incident triage, the writing that surrounds writing software.

The conversation about AI in software has narrowed too quickly to "Copilot in the editor". The bigger story is happening across the lifecycle. Planning, design, development, testing, release, and operations are all being augmented at once. The platforms that adopt AI well are not the ones with the most usage. They are the ones with the clearest discipline around how it is used.

This post is about that discipline.

AI is changing how we engineer, not how we type

AI is not changing how we write code. It is changing how we engineer software.

Code generation is the surface. Underneath it, AI is reshaping the unit of leverage. The question is no longer how fast a developer can type. It is how well a workflow can be expressed as a reusable engineering asset. Six disciplines determine whether AI moves the needle on outcomes or just adds another tool to the stack.

Figure 1. AI across the SDLC. Each phase has clear AI assist points and clear human-owned validations. The boundary is not negotiable. It is the design.

1. From assistance to augmentation

Early AI tools focused on assisting individual developers. Code suggestions. Autocomplete. Quick refactors. The value was real but bounded by the editor.

The shift now is into structured workflows that span the lifecycle. The unit of leverage is no longer a single suggestion. It is a sequence of actions executed reliably across phases. ("Agentic" later in this post means a system that makes its own next-step decisions inside guardrails. A workflow follows a fixed sequence; an agent chooses the path.)

Code generation has become baseline, not differentiator
Workflow generation is where the largest gains live
Multi-step assistance with explicit human checkpoints
Context that travels across tools, not just within one

In practice

The pattern that works: start with the single highest-volume writing task on the team (commit messages, code review comments, release notes, postmortem first drafts) and turn the AI assist for that task into a shared workflow rather than each individual's private trick. The cost is one engineer's afternoon documenting the workflow and the eval set. The return is that every engineer on the team inherits the work, and the task that used to consume an engineer's morning every two weeks becomes a background step in the release process. Workflow generation, not faster typing, is where the gains compound across a team.

Code suggestions help one developer. Reusable workflows help the next ten.

2. AI across the SDLC, with guardrails

AI now has a useful role at every phase of delivery. The role is different at each phase, and the guardrails are different too.

Phase	What AI helps with	What humans must validate
Plan	Breaking down requirements, drafting acceptance criteria	Domain context, business priorities, customer impact
Build	Code generation, refactoring, scaffolding	Architectural fit, security boundaries, performance
Test	Test case generation, edge case discovery	Coverage of business-critical paths, regulatory cases
Release	Release notes, changelog summaries, communication drafts	Accuracy, tone, customer-facing claims
Operate	Log triage, incident summaries, runbook drafts	Root cause attribution, action item ownership

The guardrails are not optional decoration. They are the design.

In practice

The pattern that works: stage AI assists for release communication (changelog drafting, customer-facing release notes, internal release announcements) and require a human review before anything goes out. The draft arrives consistently, faster than a human could produce, and easier to compare across releases. The reviewer is not eliminated; the reviewer is moved from author to editor, which is where their judgment actually matters. Teams that adopt this pattern stop missing release-note deadlines and stop publishing inconsistent communication across products.

3. From prompts to reusable assets

Many teams begin with prompt experimentation. Individuals find techniques that work for their tasks. The result is a patchwork of personal practices that do not survive a team change.

The compounding value comes when prompts mature into reusable engineering assets.

Figure 2. The maturity model from prompts to agents. The value compounds at the workflow stage and accelerates at the agent stage. The disciplines that make agents safe are the same ones that made workflows reliable.

The maturity stages, in order of leverage:

Prompts: ad-hoc, individual, hard to share
Templates: parameterised prompts versioned with the project
Workflows: multi-step sequences with clear inputs, outputs, checkpoints
Agents: autonomous task chains operating within explicit guardrails

The diagram is a maturity ladder, not a graduation. In practice teams operate at all four stages simultaneously for different tasks. A senior engineer may use a one-off prompt to explore a refactor, run a versioned template for commit messages, hand off to a workflow for release notes, and trigger an agent for routine PR triage, all in the same hour. The point of the ladder is not to leave earlier stages behind. It is to know which stage a given task belongs to and to invest accordingly.

In practice

The pattern that works: pick the three prompts your team uses every week, codify them as parameterised templates in the same repository as the application code, and treat them as engineering artefacts (reviewed, versioned, owned). New engineers inherit the team's accumulated practice instead of building their own from scratch. Quality becomes consistent because the variance between individuals shrinks. Investment pays back in weeks, not quarters, and the maturity ladder keeps producing returns as the team moves from templates to workflows to agents.

4. Agentic delivery, with guardrails that survive a security review

The next stage is agentic. AI executes sequences of tasks within a defined scope. The risk is not that the agent will fail. It is that the system around the agent will not catch the failure, and that the failure modes are different in kind from traditional automation. Agents are non-deterministic, they can be manipulated through their inputs, and their actions can have side effects in systems the team does not own.

Five guardrails make agentic delivery safe. The first four are necessary. The fifth is what carries the agent through a security review at a regulated enterprise.

Identity and scope: the agent runs as a managed identity (or scoped service principal) with the smallest set of permissions that lets it do its job. Permissions are expressed as allowlists, not denylists. Tools fetched at runtime are subject to the same identity boundary as the agent itself.
Input quarantine: anything the agent reads from a user-controlled source (work item bodies, PR descriptions, customer tickets) is treated as untrusted text. The agent does not execute instructions found in fetched content, and tool calls are validated against an output schema before execution. This is the prompt-injection mitigation, and it is the most common gap in agentic systems shipped today.
Cost and blast-radius caps: every run has a maximum token budget, a maximum number of tool calls, and a maximum spend. Exceeding any cap aborts the run cleanly. Without caps, scoped credentials are not enough to bound the damage.
Evaluations and traceability: agents are evaluated against a fixed test set before deployment, and on every prompt or model change. Every action is logged with inputs, outputs, the model and prompt versions used, and the reasoning trace where the model exposes one. Logs are redacted for secrets and personally identifiable information at write time.
Reversibility taxonomy: actions are categorised by reversibility, not asserted to be reversible in general. A draft write to a private store is reversible. A post to a customer-facing channel is not reversible (deletion does not unsend). A database update may be reversible by a compensating transaction or not at all. Irreversible actions require human approval at the boundary, before they happen, not after. The agent is allowed to draft and stage. The human is the only one who is allowed to make the move that cannot be undone.

In practice

The pattern that works: start with one low-risk agent (release-notes drafter, PR triage assistant) running on read-only inputs, write-only-to-drafts permissions, and a hard cost cap per run. Require explicit human approval at the irreversible step. Wire up an evaluation set on day one, and rerun it on every prompt or model change. Treat regressions as failures, not warnings. The first agent the team ships is rarely the most valuable; it is the rehearsal that establishes the controls every later agent inherits. Teams that skip this rehearsal end up with an agent in production that no one feels safe extending.

Implementation note

An agent without a reversibility taxonomy and a regression eval set is a liability. The discipline is the same one that made workflows reliable: scoped identity, idempotency, traceability, and a clear boundary between machine action and human decision. The YAML below is illustrative, not a runtime contract; it is meant to show the shape of the controls a real agent definition would carry, not the syntax of any specific platform.

# Agent run definition (illustrative; not a specific platform's syntax) name: release-notes-drafter trigger: pre-release identity: type: managed-identity scope: tenant=<tenant-id> resource=release-tools/<app-id> permissions: allow: - read: work-items in milestone (filter: state=Done) - read: pull-requests in milestone (filter: merged) - write: drafts/release-notes/${run-id} # Production channels are NOT in the allowlist. The agent cannot post. limits: max_tokens_per_run: 80000 max_tool_calls_per_run: 20 max_runtime_seconds: 300 max_cost_usd: 0.40 on_exceeded: abort_with_partial_artifact input_handling: treat_fetched_content_as: untrusted # Indirect prompt injection is mitigated by the layered discipline below, # not by a single feature flag. Each item is a separate control. enforce_instruction_hierarchy: true validate_tool_args_against_schema: true validate_outputs_against_schema: true steps: - fetch: completed work items in milestone - draft: release notes from items - validate: required fields present - request-review: from: release-manager idempotency_key: ${milestone-id}-${draft-hash} - on-approval: action: post-to-internal-channel reversibility: not-reversible requires: explicit-human-click # the agent does NOT click this audit: log_inputs: true log_outputs: true redact: - secrets # Pattern-based: handles structured PII like emails, phones, IDs. - pii_patterns: [email, phone, national-id, payment-card, ip-address] # Entity-based: required for unstructured PII like names. Pattern alone # cannot redact a customer name without an entity-recognition step. - pii_entities: ner-based # names, locations, organisations retain: 365_days # tune to your audit policy, not to the demo evaluation: test_set: tests/release-notes/eval-v3.jsonl on_prompt_change: rerun on_model_change: rerun fail_threshold: 5_percent_regression

5. Where AI still needs human judgment

AI has clear boundaries. The boundaries are not embarrassing. They are the design.

What must stay human-owned:

Architectural trade-offs and design decisions
Security validation and threat modelling
Correctness for business-critical and regulatory paths
Domain context that has not been written down
Accountability for outcomes, not just outputs

The goal is collaboration, not replacement. The teams that get the most value from AI are not the ones with the most automation. They are the ones with the clearest sense of where automation ends and judgment begins.

In practice

The pattern that works: name the human-owned items explicitly in the team's working agreement (architecture, security, regulatory correctness, accountability) and audit every AI workflow against that list. When a workflow asks the AI to make a decision in any of those categories, redesign it so the AI prepares the analysis and a human makes the call. Most teams over-trust AI for one of these areas in their first six months and learn the hard way. Naming the boundary up front prevents the lesson from being paid in production. The clarity is the value; the model behind the workflow is interchangeable.

6. Responsible AI is engineering work

The first five disciplines decide whether AI moves the needle. The sixth decides whether the platform can defend the choices it makes with AI. Responsible AI is the engineering practice of building systems whose AI behaviour is fair, transparent, accountable, and safe by design, not by audit after the fact. Treating it as a compliance checkbox at the end of the project is how teams end up shipping AI workflows that fail security review, embarrass the company, or harm users.

Six controls turn responsible AI from a policy into engineering work. These map directly onto the practices Microsoft and the broader industry have converged on, but the names matter less than the practice they enable.

Fairness in inputs and outputs. The training data, eval set, and prompts are reviewed for systematic bias against any group the system serves. The eval set covers under-represented cases by design, not by accident, and regressions on those cases fail the build.
Transparency to end users. When a user sees AI-generated content, they are told. When a decision is AI-assisted, the path from input to output is explainable in plain language, not just in a model card buried in documentation.
Content safety filters. Inputs and outputs pass through safety classifiers (prompt injection, prohibited content, jailbreak patterns) before reaching the model and before reaching the user. Filtering decisions are logged and reviewable.
Accountability ownership. Every AI workflow has a named owner who is accountable for its outcomes, not just its uptime. The owner has the authority to pause or roll back the workflow when harm is detected.
Data minimisation and residency. The AI sees only the data it needs to do the task. Personally identifiable information and customer data are scoped, redacted, and kept inside the boundary the customer agreed to. Cross-tenant leakage is treated as a P1 incident, not a feature request.
Harm evaluation alongside quality evaluation. The eval set measures harm potential (toxicity, hallucination on factual queries, leakage of confidential context) with the same rigour as it measures correctness. Both must pass for a release to ship.

Figure 3. Responsible AI as a set of engineering controls around the AI workflow. The six controls fall into four categories: data discipline (fairness, data minimisation), model discipline (content safety, harm evaluation), deployment discipline (transparency to users), and governance (accountability ownership). All six are necessary; none is sufficient on its own.

In practice

The pattern that works: write the responsible AI plan before the first agent ships, not after the first incident. Pick one workflow that touches user data or generates customer-facing content, and use it as the reference implementation: fairness review on the eval set, content safety filters wrapping the model call, transparency annotation in the UI, redaction of identifying details in logs, harm evals running alongside quality evals on every change, and a named owner with explicit pause authority. The first such workflow takes longer to ship than the unconstrained version. Every workflow after it inherits the controls and ships faster than it would have without them. Teams that defer responsible AI to a future quarter end up retrofitting it under pressure, which is the most expensive way to do it.

A scenario that ties it together

Picture a platform team several months into using Copilot. Adoption is high. Productivity dashboards show gains. But defect rates are not improving and lead time is flat. Leadership asks the obvious question: is AI actually helping, or just feeling like help?

The answer is not to stop using AI. It is to change how AI is measured. Move adoption metrics to the background. Move outcome metrics to the front: defect escape rate, lead time for change, change failure rate, mean time to recovery. In parallel, promote the individual prompts that have proved themselves to shared templates, and the templates to versioned workflows. Retrofit responsible AI controls onto the workflows that shipped first: content safety filters, harm evaluations alongside quality evaluations, transparency annotations on customer-facing output, and a named owner for each workflow.

Six months later, the picture is different. Defect rate improves on the parts of the codebase where reusable workflows were introduced. Onboarding for new engineers is visibly faster. Release notes are consistent across teams. The shift is from celebrating use to tracking outcomes, and once the team measures what matters, the tooling decisions start making themselves.

What teams get wrong

The common pattern is measuring AI by usage, not by outcome. Adoption metrics tell you who tried Copilot. They do not tell you whether defects dropped, lead time improved, or release notes got better.

The fix is not less AI. It is better measurement. The four metrics named in the scenario above (defect escape rate, lead time for change, change failure rate, mean time to recovery) come from the DORA research on software delivery performance and have become a useful default. Two warnings travel with them. First, attribution is hard: an AI workflow rolled out alongside a test refactor and a CI pipeline change cannot claim credit cleanly. Second, baselines matter more than headlines: a single quarter's improvement is not a trend, and a single team's gain is not the platform's gain. Outcome measurement done well needs a baseline window, an attribution discipline, and a kill criterion for workflows that are not paying back. Done poorly, it is just adoption metrics with better names.

There is also the question of cost. AI usage carries a per-run token bill, an evaluation bill on every change, and (for agents) a cost cap that limits damage when something goes wrong. None of these are large compared to the engineering time saved when the workflow works. All of them are visible enough that a finance-aware reader will ask. Track them.

Where to start

The most concrete starter from this post: promote one personal prompt to a shared template. Pick the prompt that gets used most often (commit messages, code reviews, release notes, debugging assist), move it from someone's notes into the repository where the team versions everything else, and watch what changes when the next person on the team runs it. That is the smallest unit of the workflow shift this post argues for, and it is the step where prompts stop being individual practice and start becoming engineering assets.

The shift

The shift is from building systems to building smarter systems:

AI does not replace engineers. It changes what an engineer's leverage looks like.
The unit of value is the workflow, not the suggestion.
The discipline that made platforms operable is the same discipline that makes AI useful.
Responsible AI is not a compliance step. It is the sixth engineering discipline that lets the other five compound safely.

The series ends here, but the arc is consistent across all three posts. The disciplines that make platforms scale are the same disciplines that make AI useful. Build with discipline. Run with discipline. Evolve with discipline. The tools change. The disciplines do not.

Want to discuss? Where has AI moved the needle most in your delivery, and where has it disappointed you? Drop a comment with patterns you have seen in your environment. Every reply gets read.

Previously in this series:

Building Cloud Native Platforms That Scale: Patterns That Actually Work. Part 1 covered the design choices that make scale possible.
Running Cloud Native Platforms: Why Day 2 Decides Everything. Part 2 covered the operational disciplines that decide production outcomes.

This is the third and final post in the series.

WAR, Azure Advisor, and Us (Azure Arch Diagram Builder): Three Ways to Score an Azure Architecture

arturoqu — Thu, 21 May 2026 18:06:15 GMT

Author: Arturo Quiroga, Azure AI services Engineer - Senior Partner Solutions Architect — Microsoft

A few days ago I published From Prompt to Production: Building Azure Architecture Diagrams with AI, introducing the open-source Azure Architecture Diagram Builder. One feature got more follow-up questions than any other: the Well-Architected Framework (WAF) validation. Architects from partners and customers — many of whom already use Azure Advisor and the Well-Architected Review — wanted to know exactly what scoring algorithm we use, how it compares to Microsoft's official tools, and whether they should be using all three.

This post is that answer. It's a deep dive into how design-time WAF validation works, how Microsoft's two official WAF assessment algorithms work, and where each fits in the architecture lifecycle.

TL;DR. Microsoft ships two WAF assessment vehicles — the Well-Architected Review (questionnaire, scored from human answers) and the Azure Advisor score (healthy-resources-÷-applicable-resources weighted per subcategory, with Defender Secure Score for Security and cost-weighted math for Cost). Both require either a human filling in a form or live Azure telemetry. Our app runs at design time on a diagram, before anything is deployed, using a hybrid pipeline: a deterministic rule pre-scan followed by an LLM refinement pass. Same five WAF pillars, different lifecycle stage. Complementary, not competitive.

Why design-time validation matters

Every cost overrun, reliability gap, and security incident I've ever debugged was cheaper to fix on a whiteboard than in production. Yet most WAF tooling assumes the architecture already exists — either because there are deployed resources to scan (Advisor) or because someone has built enough of it to answer 60 specific questions about it (WAR).

That leaves a gap. Between "rough sketch" and "deployed resource group" there is no algorithmic WAF feedback loop. That's the gap the Diagram Builder fills.

Microsoft's two official WAF assessment algorithms

Before describing our approach, it's worth being precise about what Microsoft already ships, because the term "WAF assessment algorithm" can mean either of two very different things.

1. Azure Well-Architected Review (WAR) — questionnaire-based

The Well-Architected Review is a free self-assessment hosted on Microsoft Learn.

Aspect	Detail
Input	Human answers to ~60 questions mapped to the WAF pillar checklists
Workload variants	Core WAR, plus AI/ML, IoT, SAP on Azure, Azure Stack Hub, SaaS, Mission Critical
Scoring	Derived from the answers — each "no" or unanswered question subtracts from the pillar score
Output	Per-pillar maturity score + prioritized recommendations + optional Advisor integration
Improvement tracking	"Milestones" (point-in-time snapshots)
When to use	Periodic deep reviews; greenfield design baselining; brownfield audits

WAR is human-driven. The algorithm is essentially "how many of the recommended practices have you confirmed you do?" — which is exactly the right algorithm when the assessor is the workload team itself.

2. Azure Advisor Score — telemetry-based

The Advisor score is the closest thing Microsoft ships to a real, deterministic WAF algorithm. It runs continuously over your deployed Azure resources.

The math:

Pillar-specific overrides:

Security uses Microsoft Defender for Cloud's Secure Score model.
Cost weights by retail $ cost of healthy resources, plus age-of-recommendation weighting; postponed/dismissed items are removed from the denominator.
Reliability / Performance / Operational Excellence use the healthy-resources ratio above.

Key terms:

Healthy resource — a deployed resource with no open Advisor recommendation against it for that pillar.
Total applicable — resources Advisor was able to evaluate (excludes dismissed/snoozed).

Advisor is the right tool once you're in production. It cannot help you before deployment, because there is nothing to count as "healthy" or "applicable."

The missing stage: design time

Here's the lifecycle, with each tool's domain shaded:



Design / Diagram — Diagram Builder validation runs here.

Operate / Observe — Azure Advisor runs here continuously.
Periodic Review — WAR runs here, typically quarterly or at major milestones.

These three stages are sequential and complementary. Our app does not replace Advisor or WAR — it adds a feedback loop earlier in the lifecycle, where corrections are cheapest.

How design-time validation works in the Azure Architecture Diagram Builder

The validator is a two-phase hybrid pipeline: deterministic local rules first, then LLM refinement. The full source lives in three files:

src/services/architectureValidator.ts — orchestrator and prompt
src/services/wafPatternDetector.ts — topology + service rule engine
src/data/wafRules.ts — the rule knowledge base

Phase 1 — Deterministic rule pre-scan (~1 ms, no LLM)

When you click Validate Architecture, the validator runs a fully client-side rule engine against the diagram's services, connections, and groups. There are two kinds of rules:

Architecture-pattern rules

These fire when a topology anti-pattern is detected:

Pattern	Detection trigger
`single-region`	No global LB (Traffic Manager / Front Door) with ≥3 services
`single-database`	Exactly one database service, no replication signal
`no-cache`	Compute + database present, no Redis/CDN
`no-monitoring`	No Azure Monitor / App Insights / Log Analytics
`no-identity`	No Microsoft Entra ID
`no-waf`	Public web tier without WAF / Front Door / App Gateway
`direct-db-access`	An edge from a frontend service directly into a database
`no-key-vault`	4+ services and no Key Vault
`no-backup`	Database present, no Azure Backup / Recovery Services
`no-api-gateway`	2+ compute services and no APIM / App Gateway / Front Door

Service-specific rules

Every service in the in the generated Azure Architecture diagram is matched against SERVICE_SPECIFIC_RULES by normalized type — App Service, Functions, AKS, Cosmos DB, SQL Database, Storage, Key Vault, and 22 more.

The knowledge base at a glance

Metric	Count
Total rules	73
Architecture-pattern rules	10
Service-specific rules	63
Distinct Azure services covered	29
Rules tagged Reliability	18
Rules tagged Security	34
Rules tagged Cost Optimization	5
Rules tagged Operational Excellence	7
Rules tagged Performance Efficiency	9

The preliminary score

Each finding has a severity, and severity drives a fixed point deduction from a starting score of 100:

Severity	Deduction
critical	−12
high	−7
medium	−3
low	−1

Result is floored at 10 (so even a deliberately bad architecture scores at least 10) and ceilinged at 95 (no findings ≠ perfect — there's always something the model might still catch). This is the deterministic baseline before the LLM ever sees the architecture, and it's what makes the pipeline reproducible.

Phase 2 — LLM contextual refinement

The pre-scan output, the topology, and the optional natural-language description are folded into a focused prompt sent to one of seven Azure OpenAI models (GPT-5.1 through 5.4, GPT-5.x Codex variants, DeepSeek V3.2 Speciale, Grok 4.1 Fast). The system prompt gives the model explicit scoring guardrails:

Score based on what IS present, not what COULD be added.

A well-connected architecture with appropriate services should score 60–80.

Score below 50 only for critical gaps (no auth, no monitoring, single points of failure).

Findings are improvement suggestions, not reasons to penalize the score severely.

The model returns strict JSON:

{
  "overallScore": 0-100,
  "summary": "2–3 sentence assessment",
  "pillars": [
    {
      "pillar": "Reliability | Security | Cost Optimization | Operational Excellence | Performance Efficiency",
      "score": 0-100,
      "findings": [
        {
          "severity": "critical | high | medium | low",
          "category": "...",
          "issue": "...",
          "recommendation": "...",
          "resources": ["service-name-1", "service-name-2"],
          "source": "rule-based | ai-analysis"
        }
      ]
    }
  ],
  "quickWins": [ /* same shape as findings */ ]
}

Two things to call out:

Every finding is tagged rule-based or ai-analysis. That tag is the credibility lever. You can always see what the deterministic engine produced versus what the model contributed on top. If you don't trust the AI layer, you can ignore it entirely — the rule layer still stands.
The LLM is given pattern hints, not the entire rule catalog. The prompt stays small and focused, which is roughly 3–5× faster and cheaper than asking the LLM to do everything from scratch.

What the user sees

On every run the modal reports:

Overall WAF score (0–100)
Per-pillar score × 5 (0–100 each)
Severity breakdown — counts of critical / high / medium / low across all findings
Quick wins — high-impact, low-effort items the model surfaces separately
Hybrid metadata — local findings count, patterns detected, KB rules used, preliminary score, local elapsed ms
AI metrics — model used, reasoning effort, prompt/completion/total tokens, elapsed time
App Insights telemetry — an Architecture_Validated event with model, overall score, finding count, elapsed time

Worked example

Take this prompt, which I've used in demos with partners:

"A multi-region web application: Azure Front Door in front of two App Service instances in West US 2 and East US 2, both reading from an Azure SQL Database with geo-replication, with Application Insights for telemetry. No Entra ID, no Key Vault."

After generation, Validate Architecture runs:

Phase 1 — pre-scan (deterministic), ~1 ms

Patterns detected: no-identity, no-key-vault
Findings produced: 8 (1 critical, 1 high, 3 medium, 3 low)
Preliminary score: 100 − 12 − 7 − (3×3) − (1×3) = 69

Phase 2 — LLM refinement, ~6–9 s depending on model

The model accepts the two pattern hints, validates them in context, and adds three more findings of its own:

Finding	Source	Pillar	Severity
No Microsoft Entra ID for authentication	rule-based	Security	critical
No Key Vault for secret management	rule-based	Security	high
App Service slots not used for safe deploys	ai-analysis	Operational Excellence	medium
SQL DB geo-replication present but RTO/RPO not documented	ai-analysis	Reliability	medium
No CDN for static assets behind Front Door	ai-analysis	Performance Efficiency	low

Final scores returned by the model:

Pillar	Score
Reliability	78
Security	52
Cost Optimization	80
Operational Excellence	70
Performance Efficiency	75
Overall	71

The Security score is the lowest because two of the highest-severity findings landed there — exactly what a human reviewer would flag first.

Multi-model comparison

Because the deterministic floor is identical across runs, the Validation Comparison view becomes a fair shootout of what each LLM adds on top of the same baseline. The same diagram is scored by all seven models, and the UI surfaces:

Overall score per model
Per-pillar score per model
Severity-count deltas
Number of ai-analysis findings each model contributed
Quick wins each model identified

This is genuinely useful for two reasons. First, it shows that LLM scores vary — typically by ±5–10 points on the same architecture — which is exactly why we publish the rule-based vs ai-analysis tag. Second, it lets architects pick the model whose review style matches their own.

How we align with Microsoft's algorithms

Alignment point	What it means
Same five pillars	Identical names and scope to the official WAF
Same source material	Rules derived from WAF docs and Azure Architecture Center service guides
Severity-graded findings	Map conceptually to Advisor's high/medium/low impact recommendations
Per-pillar + overall scoring	Mirrors WAR/Advisor output shape, so the results feel familiar

Where we deliberately differ — and why

Concern	Microsoft	Diagram Builder	Why we differ
Needs deployed resources	Advisor: yes	No — works on a diagram	We're a design-time tool; the architecture doesn't exist yet
Needs human Q&A	WAR: yes	No — derived from the diagram	One-click validation inside the design flow
Healthy/Applicable ratio	Advisor: yes	No	No resource-health signal exists pre-deployment
Subcategory fixed weights	Advisor: yes	No explicit weights	Severity is the de-facto weight (12/7/3/1)
Defender Secure Score for Security	Advisor: yes	No	Defender requires deployed resources
Cost-weighted scoring	Advisor: yes	No (separate Cost Estimation feature)	Cost is a separate pipeline in our app
AI/LLM refinement	Neither	Yes	Catches context-specific issues a static catalog misses, and explains findings in natural language
Multi-model comparison	Neither	Yes	Lets architects see scoring variance across models

Honest limitations

I'd rather you hear these from me than discover them in production:

LLM scores drift. ±5–10 points across models on the same diagram is normal. Treat the score as directional, the findings as actionable. The rule-based tag is your anchor.
No live telemetry. We can't know if your App Service is actually using availability zones — only that you have App Service in the diagram. Advisor will tell you the truth post-deployment.
Generic ruleset. No specialized workload branches yet (AI/ML, IoT, SAP, SaaS). WAR has those.
No milestone tracking. Each validation run is independent. Compare runs manually using the Validation Comparison view.
Rule coverage is finite. 29 services and 73 rules is a strong start but not exhaustive — the LLM layer exists in part to compensate for that gap.

How to use all three together

A lifecycle that actually works:

Design — Use the Diagram Builder to sketch the architecture and validate at design time. Iterate until the per-pillar scores look reasonable and the critical/high findings are addressed.
Deploy — Generate Bicep from the diagram, deploy, and let Azure Advisor start scoring real resources.
Operate — Use Azure Advisor continuously. Use Defender Secure Score for security posture.
Periodic review — Run a Core WAR every quarter or at major milestones to capture the things only humans know (business context, tradeoffs, planned debt).

None of these three replace the others. They cover different stages of the same loop.

What's next

A few things on the roadmap I'd love feedback on:

Milestone tracking so design-time scores can be compared over time the way WAR milestones work.
Workload-specific rulesets mirroring WAR's branches — starting with AI/ML.
Direct Advisor handoff — once a diagram is deployed, surface the corresponding Advisor recommendations in the same UI to close the loop.

Try it, fork it, tell me where it's wrong

Live app: https://aka.ms/diagram-builder
Source: github.com/Arturo-Quiroga-MSFT/azure-architecture-diagram-builder

Useful references:

If you're a partner or customer architect who's already living in Advisor and WAR, I'd genuinely value your reaction — does the design-time stage feel like a real gap to you, or are you already covering it some other way? Open an issue on the repo or reply on LinkedIn.

Posted on the Azure Architecture Blog · Comments and issues welcome on the repo.

From Prompt to Production: Building Azure Architecture Diagrams with AI

arturoqu — Fri, 22 May 2026 18:35:07 GMT

Author: Arturo Quiroga, Senior Partner Solutions Architect — Microsoft

Cloud architects spend significant time translating ideas into architecture diagrams. They toggle between Visio, draw.io, pricing calculators, and documentation. According to the 2024 Stack Overflow Developer Survey, 61% of developers spend more than 30 minutes a day searching for answers or solutions, time lost to context-switching rather than design. What if you could describe your architecture in plain English and get a diagram, cost estimate, and deployment guide in minutes?

The Challenge: Fragmented Architecture Workflows

Designing Azure architectures today typically involves multiple disconnected steps:

Sketch the architecture in a diagramming tool
Look up official Azure icons and drag them into place
Research pricing across regions using the Azure Pricing Calculator
Validate the design against the Well-Architected Framework (WAF)
Write deployment documentation and Infrastructure as Code templates
Compare alternative designs manually

Each step lives in a different tool, and keeping them in sync as designs evolve is costly. The Azure Architecture Diagram Builder brings these workflows together in a single browser-based experience.

How It Works

Describe your architecture in natural language, for example "A HIPAA-compliant healthcare platform with FHIR APIs, event-driven processing, and multi-region disaster recovery", and the AI generates a diagram with grouped services, data flow connections, and logical organization.

Figure 1. Enter a natural-language prompt describing your architecture. Curated example prompts help you get started, and you can optionally upload an existing diagram for the AI to analyze.

The tool uses Azure OpenAI to power generation across multiple models, enabling you to choose the model that best fits your scenario — from fast iterations to deeper reasoning.

Key Features

AI-Powered Architecture Generation

Describe what you need in plain English, and the AI creates an architecture diagram with:

714 official Azure service icons across 29 categories
Smart grouping: services are logically organized (Frontend, Backend, Data, Security)
Data flow connections: labeled edges showing how data moves through the system
13 curated example prompts: from simple web apps to complex enterprise scenarios like Zero Trust networks, Industrial IoT with 5,000+ sensors, and global multiplayer gaming backends

Figure 2. A generated industrial IoT architecture. Top: the clean diagram view as initially produced. Bottom: the same diagram with per-service monthly cost overlays toggled on, plus a running subscription total in the toolbar.

Architecture Image Import

Already have an architecture on a whiteboard or in a screenshot? Upload the image and let the AI analyze it, mapping services to official Azure icons and recreating the architecture as an editable, interactive diagram.

Figure 3. Upload a photo of a whiteboard sketch (top-right reference panel) and the AI recreates it as an editable diagram with official Azure service icons and labeled data flow connections.

ARM Template Import

Import existing ARM templates to visualize your current infrastructure. The AI parses resource definitions and dependencies, groups related resources into logical layers, and produces a meaningful diagram of what you actually have deployed — a fast way to document an inherited environment or sanity-check a template before deployment.

Figure 4. ARM template import in action. Top: the parser status banner while resources and dependencies are being analyzed. Bottom: the resulting diagram, with resources auto-grouped into logical layers (Web Tier, Data Layer, Container Platform, Observability & Logging) and a Generated from: ARM Template badge linking the diagram back to its source file.

Well-Architected Framework Validation

Validate your architecture against all five WAF pillars — Security, Reliability, Performance Efficiency, Cost Optimization, and Operational Excellence. The validator provides:

An overall WAF score with pillar-level breakdowns
Specific findings with severity levels
Actionable recommendations you can select and apply

Select the recommendations you agree with, and the AI regenerates an improved architecture incorporating those changes.

Figure 5. WAF validation results showing the overall score, per-pillar breakdowns, and individual findings with severity badges. Tick the recommendations you want and the AI rebuilds the diagram with those changes applied.

Multi-Model Comparison

Run the same architecture prompt through multiple AI models side-by-side and compare:

Architecture Comparison: service counts, connection counts, groups, token usage, and latency
Validation Comparison: WAF scores across models, severity breakdowns, and finding counts
Apply Winner: pick the best result and apply it to the canvas with one click
Present Critique: a talking avatar narrates the AI-generated ranking with live closed captions

Figure 6. Multi-model comparison. Top: select the models and reasoning effort, then enter the prompt. Bottom: side-by-side results across all selected models with service counts, latency, token usage, and Fastest / Cheapest / Most Thorough badges.

Multi-Region Cost Estimation

Get cost estimates from the Azure Retail Prices API across 8 Azure regions: East US 2, Australia East, Canada Central, Brazil South, Mexico Central, West Europe, Sweden Central, and Southeast Asia. Features include:

Color-coded cost legend (green / yellow / red thresholds)
SKU and tier information for each service
Export options: CSV, JSON, plain-text summary, and an analysis report with top cost drivers, Reserved Instance flags, and a ranked multi-region comparison table

Figure 7. The cost legend overlay shows per-service pricing with color-coded thresholds. The region selector in the toolbar lets you re-price the entire architecture in any of eight Azure regions.

Deployment Guide Generation with Bicep

Generate step-by-step deployment documentation including:

Prerequisites and Azure resource requirements
Step-by-step deployment instructions
Bicep templates for each service (Infrastructure as Code)
Post-deployment verification steps
Security configuration recommendations

Figure 8. Each generated Deployment Guide opens with the architecture name, an estimated deployment time, and a prerequisites checklist covering subscription roles, CLI versions, Microsoft Entra ID permissions, and region requirements, followed by numbered, copy-ready deployment steps.

Figure 9. The Infrastructure as Code section produces a main.bicep orchestrator plus a per-service module (Log Analytics, Key Vault, Cosmos DB, SQL Database, Event Hubs, Azure Functions, and more). The Download All Templates button packages everything into a ready-to-deploy folder.

Workflow Animation & Avatar Presenter

Visualize how data flows through your architecture with step-by-step animations that highlight services on the canvas as each step plays. When the Azure Speech Service is configured, a photorealistic talking avatar can narrate the workflow or present model comparison results, with live word-by-word closed captions in a draggable, resizable panel.

Figure 10. A workflow step is highlighted on the canvas as the Avatar Presenter narrates that step. Live word-by-word closed captions appear in a draggable, resizable panel, useful for accessibility and stakeholder demos.

Export Options

Figure 11. A single-slide PowerPoint export, available in dark or light theme, ready to drop straight into a stakeholder deck.

Format	Use Case
PNG	Documentation, presentations
SVG	Scalable vector graphics
PPTX	Single PowerPoint slide (dark or light theme)
Draw.io	Edit in diagrams.net
JSON	Backup, version control
CSV / ZIP	Cost analysis with multi-region comparison

Highlights

The Azure Architecture Diagram Builder unifies the architecture design lifecycle in a single tool:

End-to-end workflow: from natural-language description to deployable Bicep templates without tool switching
Official Azure icons: 714 icons across 29 categories, mapped directly from the Azure service catalog
Live pricing: queries the Azure Retail Prices API at design time rather than relying on static estimates
WAF-integrated validation: architectural best practices built into the design loop rather than applied after the fact
Multi-model flexibility: choose the AI model that best suits each task, with fast models for iteration and reasoning models for complex designs
Open source: the source code is available for customization and contribution

One-Command Deploy with Azure Developer CLI

The fastest way to get your own instance running is with azd:

# Install azd (once)
brew tap azure/azd && brew install azd   # macOS
winget install microsoft.azd             # Windows

# Clone, configure, and deploy
git clone https://github.com/Arturo-Quiroga-MSFT/azure-architecture-diagram-builder
cd azure-architecture-diagram-builder
azd auth login
azd env set AZURE_OPENAI_ENDPOINT "https://your-resource.openai.azure.com/"
azd env set AZURE_OPENAI_API_KEY  "your-key"
azd up   # Provisions infrastructure + builds + deploys (~8 min)

azd up provisions the following via Bicep:

Resource	Purpose
Azure Container Registry	Stores the Docker image
Azure Container Apps	Runs the app (nginx + token server)
Log Analytics + Application Insights	Monitoring and telemetry
Azure Speech (S0)	Avatar Presenter (optional, keyless auth via managed identity)

Try It Today

The Azure Architecture Diagram Builder is available now:

Live demo: https://aka.ms/diagram-builder
Source code: GitHub repository
Documentation: See the Getting Started Guide for detailed setup instructions

We welcome feedback and contributions. Use the GitHub Issues page to report bugs, suggest features, or share your experience.

Tags: artificial intelligence · application · apps & devops · well architected · infrastructure

Azure Architecture Blog articles

Choosing the Right Agent in Microsoft Foundry

First, what actually makes something an agent?

Where should orchestration logic live and who should own the runtime?

Prompt Agents

Why teams start here

Good examples

Hosted Agents

When Hosted agents make sense

Good examples

The trade-off

Choosing Prompt Agent/Hosted Agents

What I would choose today

From Features to Flow: How Real-World Adoption Reshaped the Azure Architecture Diagram Builder

Adoption created a product feedback loop

Stage 1: Create — make the starting choice explicit

Stage 2: Refine — preserve human work

Quality is structural, not only visual

Stage 3: Validate & Improve — treat validation as a lifecycle

Stage 4: Share or Build — design for the artifact’s destination

The same journey now extends to agents

Learning from adoption without identifying people

What changed

The next boundary: logical versus physical architecture

Try it, challenge it, help shape what comes next

Skill or Sub-Agent. Choosing AI Capabilities You Will Actually Reuse

The wrong first question

The insight

1. Decide the iteration model first

2. Voice fidelity decides craft work

Implementation note

3. Put the human gate where the risk is

4. The pattern that scales is both

The operational trade-offs

Two capabilities, one team

What teams get wrong

A quick way to decide

Where to start depends on what you have already built

The shift

Mastering GitHub Copilot Budgets: How to Prevent Surprise Overages Without Blocking Devs

1. The Core Mental Model: The Water Park Analogy

2. How GitHub Evaluates Requests (The 3-Step Decision Flow)

Step 1: The Personal Guardrail (User-Level Budgets)

Step 2: The Included Pool Check

Step 3: Group & Enterprise Overage Caps

3. Key Takeaways & Common Pitfalls

The "Lowest Headroom Wins" Rule

What Gets Blocked vs. What Keeps Working?

Conclusion: Best Practices for Admins

Token Economics in Practice

Introduction: The cheap-token trap

The unit that actually matters: Cost per accepted task

From a metric to a controller

Two halves of the loop: feed-forward and feedback

How the equation lands on Azure

From Concept to Implementation

References

Preserve a Legacy IP During Azure Migration with Private Link Service Direct Connect

Why this matters

Customer challenge

Customer requirements

Solution and lab overview

Why this pattern works

When this pattern fits

Architecture

Traffic flow

Prerequisites and assumptions

Deploy the lab

Validate the solution

Production considerations

Clean up

Conclusion

References

About the author

Reminder: Path to Production for Agents Webinar Series Starts Next Week

Hypervelocity Engineering: Accelerating Enterprise AI with Azure AI Landing Zones

From Traditional Engineering to Hypervelocity Engineering

What is HyperVelocity Engineering?

The Core Principles of Hypervelocity Engineering

Applying RPIR to Enterprise Architecture