The Clinical Friction Ledger: Should Every Healthcare AI Tool Remove More Work Than It Creates?
The Clinical Friction Ledger: Should Every AI Feature Remove More Work Than It Creates? One question I keep coming back to is this: How can a healthcare organization determine whether an AI tool is actually reducing work? I propose a simple working framework—the Clinical Friction Ledger. On one side, record the friction removed: documentation time, unnecessary clicks, repeated data entry, handoffs, and waiting. On the other side, record the friction added: verification time, new alerts, exception handling, training, and work quietly transferred to another person or shift. An AI model can look impressive in a demonstration while making the overall care process harder. Before an AI pilot is scaled, both sides of this ledger should be examined. If the friction added outweighs the friction removed—or if the burden is simply shifted to someone else—the productivity claim is incomplete. The real measure of success is not only what the AI can do. It is whether the people closest to care experience less friction because of it. What would you put on each side of the Clinical Friction Ledger?Model router updates: new regions, a refreshed model pool, and understanding the hill climb
Across Microsoft, "hill climbing" has become shorthand for how real AI progress happens: not in one dramatic leap, but through a disciplined loop. Microsoft AI defines the hill climb as an organization that continuously improves, cycle after cycle, through more compute, better data, and sharper evaluation. Reinforcement fine-tuning in Foundry defines it as improving the deployable model package one measured step at a time across quality, latency, and cost. Different altitudes, same premise: progress is not a one-shot decision. It's a loop. For most teams, the decision of what model to use when is made manually or with custom routing tools. A developer picks a model based on benchmarks, familiarity, or the last launch that made headlines, ships it, and revisits the choice only when something breaks. In an ecosystem where the frontier moves monthly, that decision goes stale fast. Model router in Foundry Models brings the hill climb to the selection layer. What's new: a bigger pool, in more places This release expands where teams can deploy model router, broaden the supported model pool, and delivers updates through a stable endpoint. Together, these changes help teams run production workloads in more locations, match a wider range of tasks to suitable models, and adopt supported updates without changing the application integration. A refreshed model pool. The supported model list now includes Anthropic Claude Opus 4.8 — a high-capability model built for complex reasoning and long-form generation, for scenarios that demand depth, structure, and quality — and the GPT-5.6 family. Just as importantly, the pool is pruned: gpt-5-chat, gpt-5.2-chat, gpt-5.3-chat, Deepseek-V3.1 have been removed from the model router as models reach the end of their lifecycle and are deprecated in Foundry. New region availability. The model router is now available in 28 regions for global standard and 21 data zone regions. For many organizations, inference requests must stay within specific geographic boundaries for regulatory, governance, or customer-trust reasons — and intelligent routing shouldn't force a compromise on that. Find the full list of regions here. The most important detail is what you don't have to do: these updates occur automatically*. The endpoint remains stable as the supported model pool is refreshed, so teams do not need to redeploy the model router to receive the update. Applications can continue using the same integration while the model router evaluates requests against the current supported pool. Teams should continue monitoring routing traces and application outcomes to confirm that quality, cost, latency, and governance requirements are met. *Models from Anthropic still need to be deployed separately before they can be routed to through the model router. Interested in hearing more about what's new to the model router? Tune in for the next episode of Model Mondays with Sanjeev Jagtap and Lee Stott, where they talk all things model router from evaluations to hill climbing. Sign up here to watch live or view the replay: Model Mondays - Spotlight On Model router in Microsoft Foundry | Microsoft Reactor The selection-layer hill climb At the selection layer, a step is a routing decision. Each one is a micro-optimization against your objective, and each one is instrumented: every response from the model router includes a model field showing which underlying model was selected, so the climb leaves a complete, auditable trail. Model router supports three parts of the optimization loop: A/B testing to compare two router configurations to understand quality, cost, and latency tradeoffs; model decomposition to use routing results to decompose a single-model application into a multi-model or multi-agent design, and continuous routing to keep the router in production for continuous per-request selection. Each pattern turns model choice into a measured, repeatable process rather than a fixed decision. 1. A/B Testing Question: Which model or routing strategy should I use in production? A/B testing helps teams compare candidate models, model families, or router configurations against the same workload. Representative traffic is sent to competing deployments, and teams compare quality, cost, latency, and governance outcomes. The goal is to understand tradeoffs and identify the model or routing strategy that best meets workload requirements before promoting it to production. 2. Model Decomposition Question: What work is my application actually doing? Model decomposition uses model router as a diagnostic tool. By deploying the model router against a representative workload and examining routing telemetry, teams can see how requests naturally separate into different task classes. Simple retrieval, classification, and summarization requests may route to smaller models, while reasoning, planning, and agentic workflows may require more capable models. The goal is not to choose a winner, but to understand the structure of the workload and uncover opportunities for optimization, specialization, or architectural improvements. 3. Route continuously Question: Why choose a single model at all? Route continuously is the pattern model router was designed for but is not limited to. Rather than treating model selection as a one-time decision, teams leave the model router in production and allow the best-fit model to be selected for each request. As the supported model pool, regional availability, and platform capabilities evolve, teams can continue using the same endpoint while evaluating whether updates improve workload outcomes. Model selection becomes an ongoing optimization process rather than a project that must be repeated every time the model landscape changes. Together, these patterns illustrate a broader shift: the model router is more than a model. It is a tool for the optimization loop itself, helping teams evaluate tradeoffs, understand workload behavior, test hypotheses, and continuously refine model selection as requirements evolve. Whether used to compare candidate models, decompose applications into specialized tasks, or automate per-request routing in production, model router turns model selection into an observable, measurable, and repeatable process. As the model landscape continues to change, that optimization loop becomes a durable advantage. Getting Started Ready to start your own hill climb? Whether you're exploring the model router for the first time, evaluating routing strategies against your workload, or building a long-term optimization practice, these resources can help you move from experimentation to production with Microsoft Foundry. What's new in model router? Sign up for the next Model Mondays episode for a deep dive into new features, optimization patterns, and the latest model router updates. How do I build agents with model router? Check out the Model Router Agents Lab and build agent experiences with routing, retrieval, web search, tool calling, and multi-agent patterns. How do I evaluate model router? Compare model router against baseline models using your own prompts, then review quality, cost, latency, and routing decisions with the Auto Evaluation Toolkit. How do I optimize model router for my workload? Start your hill-climbing journey with the Model Mastery workshop, where you'll test one optimization lever at a time and measure how each change impacts workload outcomes. How do I build a model router optimization playbook? Explore the Model Releases repository to track new capabilities, understand the optimization question behind each release, and try focused notebooks that demonstrate one optimization lever at a time.
Choosing the Right Agent in Microsoft Foundry
Many discussions about Microsoft Foundry Agent Service eventually arrive at the same question: should this workload be implemented as a Prompt Agent or a Hosted Agent? While the documentation explains both options well, the architectural decision usually comes down to something much simpler: where do you want the orchestration logic to live? First, what actually makes something an agent? A basic AI assistant generates an answer. An agent can also decide what to do next, call tools, access data, maintain context and complete work across multiple steps. At the center of most agents are three building blocks: Model: provides language understanding, generation, and reasoning. Instructions: define the job, boundaries, role, and expected behaviour. Tools: connect the agent to knowledge and actions such as search, APIs, databases, code execution, MCP servers or business systems. For enterprise use, that is only the starting point. You also need identity, authorization, network controls, content safety, session management, evaluation, tracing, versioning, rollback, and cost controls. Foundry Agent Service provides the surrounding platform capabilities, while letting you choose how much runtime logic your team owns. Where should orchestration logic live and who should own the runtime? Understanding the Runtime Boundary When evaluating Foundry Agent Service, many teams focus on models. In practice, models are rarely the architectural differentiator. Most architecture reviews eventually come down to three questions: Who owns orchestration? Who owns state? Who owns operations? Foundry Agent Service provides a managed platform for these concerns, but the amount of control retained by engineering teams depends on the selected agent type. For most teams, the architectural decision usually comes down to one of two operating models Prompt agents: declarative agents defined by a model, instructions, and tools, with a managed runtime. Hosted agents: code-based agents that you package and run in Foundry, while the service manages the endpoint, identity, scaling, sessions, and observability. Prompt Agents With a Prompt Agent, engineering teams focus primarily on defining the model, instructions, tools, knowledge sources and identity configuration, while Foundry takes responsibility for the surrounding runtime. Why teams start here Prompt agents are usually the fastest route from an idea to a working, governed agent. They are a good fit when the behaviour can be expressed clearly through instructions and supported tools. You need to deliver quickly. The agent follows a fairly straightforward reasoning and tool-use loop. Foundry-supported tools cover the required integrations. You do not need custom libraries, middleware, or orchestration code. You want Foundry to own compute, scaling, and patching. Reviewers need an agent definition that is easy to inspect. Good examples Enterprise knowledge assistant. Employees ask about policies, engineering standards, procedures, or product information. The agent retrieves approved content and cites its sources. Document review assistant. The agent checks a proposal or design against an approved rubric and returns structured findings, while a human keeps responsibility for the final decision. Employee self-service agent. The agent answers questions and performs a small number of tightly scoped actions, such as checking request status or creating a support case. A useful warning sign A Prompt agent is probably becoming the wrong fit when the prompt starts looking like application code. Large branching instructions, retry logic written in prose, state-machine behaviour, custom payload handling, framework middleware or real-time media are all signs that runtime logic belongs in code instead. Hosted Agents Hosted Agents move the responsibility boundary. Instead of defining behaviour through configuration alone, engineers deploy an actual application into Foundry Agent Service. Hosted Agents are framework-agnostic. Whether your team builds with Agent Framework, LangGraph, Semantic Kernel, OpenAI Agents SDK, or a custom runtime, Foundry can host the application while managing the surrounding operational services. When Hosted agents make sense You need a particular agent framework or custom orchestration engine. The flow includes branching, parallel work, fan-out and fan-in, or human approvals. Business rules require a deterministic state machine around model reasoning. You need custom packages, middleware, algorithms, retries, caching, or error handling. The client sends custom payloads or webhooks. The session needs persistent files or custom state. The design includes multi-agent orchestration or real-time voice. Good examples A bank onboarding workflow where uploaded documents must be validated, checked against multiple systems, and routed to a human when confidence drops below a threshold. A fraud investigation agent that gathers transaction history, enriches data from multiple internal systems, applies bank-specific risk rules, requests additional evidence when required, and generates a recommended outcome for an investigator. The process involves long-running workflows, branching logic and audit requirements that are better suited to code-based orchestration. A lending workflow that coordinates document collection, credit bureau checks, income verification, affordability assessments, policy exceptions, and approval routing. The process spans multiple systems and often requires deterministic decision paths that extend beyond prompt-driven orchestration. A security operations agent that aggregates alerts from SIEM platforms, enriches incidents with threat intelligence, executes automated containment actions, opens tickets, requests approvals for high-impact remediation steps, and maintains a complete audit trail of decisions and actions. The trade-off More control also means more ownership. Your team must secure and patch the code and dependencies, test the runtime, manage supply-chain risk, and think about compute sizing, cold starts, session lifecycle, and cost. Hosted agents reduce platform plumbing, but they do not remove application engineering. Choosing Prompt Agent/Hosted Agents 1) Runtime Control Is Usually the Real Requirement A pattern I see quite often is teams arriving at the solution before they've fully articulated the requirement. The conversation usually starts with "We need a Hosted Agent," but after digging into the workload, the real requirements turn out to be things like persistent state, webhook processing, custom orchestration, background execution, framework-specific capabilities, or human approval workflows. These are runtime concerns, not agent concerns and they're usually the factors that determine whether a Hosted Agent is necessary. Hosted Agents are valuable because they give engineering teams control over those aspects of execution while still offloading much of the operational infrastructure to Foundry. This is also where teams most commonly choose the wrong agent type. A frequent assumption is that existing investments in frameworks such as LangGraph or Semantic Kernel automatically imply a Hosted Agent architecture. In practice, many of these workloads are relatively simple orchestration scenarios that can be implemented effectively as Prompt Agents, with lower operational overhead and less infrastructure to manage. My advice is usually to start by identifying the runtime requirements rather than selecting an agent type. Once those requirements are clear, the right architecture often becomes obvious. 2) When Hosted Agents become mandatory The moment you need custom Python packages, long-running workflows, external SDKs, deterministic orchestration or framework-specific capabilities, the conversation shifts from Prompt Agents to Hosted Agents. What I would choose today If I were starting a new project today, I'd begin with a Prompt Agent unless there was a clear reason not to. In my experience, Prompt Agents cover far more enterprise use cases than many teams initially expect. The best projects tend to start simple, prove value, learn where the limitations are, and then introduce Hosted Agents only when runtime customization becomes a genuine requirement. That progression is usually far less risky than leading with a fully custom solution. Quickstart: Create a prompt agent - Microsoft Foundry | Microsoft Learn Hosted agents in Foundry Agent Service - Microsoft Foundry | Microsoft Learn
From Features to Flow: How Real-World Adoption Reshaped the Azure Architecture Diagram Builder
In May, I introduced the open-source Azure Architecture Diagram Builder as a way to move from a natural-language prompt to an Azure architecture diagram, cost estimate, Well-Architected assessment, and deployment guidance. In July, I shared how the project had become agent-ready through Model Context Protocol (MCP). Those posts described what the tool could do. The more interesting story came next: what happened when people actually used it. As adoption grew, the central product question changed. It was no longer simply, Can AI generate an Azure architecture? It became: How do we help an architect choose how to begin, improve a result without losing their work, validate it responsibly, and turn it into something another person can use? That question reshaped the Azure Architecture Diagram Builder from a collection of capabilities into a guided workflow: Create → Refine → Validate & Improve → Share or Build This post explains what we learned, what changed in the product, and why the hardest part of AI-assisted architecture is not the first diagram. It is everything that comes after it. TL;DR. Growing adoption created a feedback loop. Aggregate usage showed that people moved beyond generation into validation, recommendations, exports, and deployment guidance. Privacy-safe feedback revealed recurring problems with diagram integrity, preservation of human edits, cost credibility, export quality, and validation continuity. Those signals led to a four-stage architecture journey that keeps human judgment and professional review at the center. The same lesson now shapes agent access and the next product boundary: distinguish logical proposals from evidence-backed physical architecture. Adoption created a product feedback loop As of August 5, 2026, the first two Azure Architecture Blog articles had accumulated approximately 12,100 combined views. A refreshed view of deduplicated application telemetry through August 13 recorded: Activity Aggregate count Architecture generation and refinement events 5,023 Well-Architected validations 960 Recommendations applied 175 Diagram exports 2,020 Deployment guides generated 212 As of August 13, the public repository had reached 45 stars and 14 forks. In GitHub’s current rolling 14-day window, the repository recorded 277 unique visitors and 67 unique cloners. These numbers measure different things and should not be added together. Article views are not unique readers. Application activity uses anonymous telemetry identifiers, not verified people. GitHub traffic is a rolling aggregate window. The signals are useful because of the pattern they reveal, not because they can be combined into one headline user count. Activity also accelerated during the period following the second article. Compared with the May 19–July 9 baseline, daily activity from July 10 through August 13 was approximately 7.0 times higher for architecture generation and refinement, 5.6 times higher for Well-Architected validation, and 5.9 times higher for recommendation application. The timing coincided with publication; it does not prove that the article alone caused the growth. The important product lesson was simpler: people were not stopping after the first diagram. They were testing alternatives, validating designs, applying recommendations, exporting artifacts, and asking how to move toward implementation. Generation was the entry point, not the complete job. The first guided-journey signals reinforce the need for more than one starting path. Through August 13, the new journey instrumentation recorded 880 interactions from 174 anonymous identifiers across 241 sessions. At first start, structured brief/image generation and Guided Chat were selected at almost the same frequency (158 and 156 events), while template and live-Azure import added another 68 selections. These are interaction counts, not unique people or conversion rates, and the window is still too early to claim that the journey improves completion. They are enough to show that architecture work does not begin in one uniform way. In-product Start Here panel showing the four-stage Azure Architecture Diagram Builder journey: Create, Refine, Validate and Improve, and Share or Build. Figure 1. The in-product Start Here panel explains one complete architecture loop. The stages are recommendations, not gates, and direct access to every tool remains available. Stage 1: Create — make the starting choice explicit As capabilities accumulated, the first screen became harder to interpret. Architecture Chat and structured generation were both useful, but they competed for attention. Importing an existing architecture was available, yet easy to miss. The new starting experience makes three paths explicit: Starting path Best suited for Guided Chat Exploring requirements conversationally and refining them over multiple turns Generate Diagram Providing a structured brief or image and producing a first architecture quickly Import Existing Opening an existing architecture or infrastructure artifact for analysis and editing This is not a marketing landing page placed in front of the tool. It is a small decision point inside the authoring experience. Once a path is selected, the user lands on the real canvas. The distinction matters because different architecture tasks begin with different levels of certainty. Sometimes the architect knows the target services. Sometimes the problem needs discovery. Sometimes the architecture already exists and the work is to understand or improve it. The product should acknowledge those differences instead of pretending every design starts with a perfect prompt. Start chooser presenting Guided Chat, Generate Diagram, and Import Existing as three equal entry paths. Figure 2. Three starting paths reflect three different architecture situations: discovery, structured generation, and analysis of an existing design. Stage 2: Refine — preserve human work One of the clearest feedback themes was not about adding another AI capability. It was about preventing AI from casually undoing human effort. An architect might spend time arranging a one-page diagram for a review, resizing groups, moving labels, or emphasizing a specific boundary. A subsequent AI refinement could improve the service selection while disrupting that carefully prepared layout. The design principle that emerged was straightforward: AI acceleration should preserve deliberate human work by default. Refinement now retains existing node positions, group geometry, sizes, and viewport context whenever possible. The model can change the architecture without treating every turn as permission to redraw the entire document. The same principle applies beyond geometry: Preserve the prior validation result when recommendations change the architecture. Preserve the active light or dark theme in exported artifacts. Preserve the distinction between the authoring canvas and the presentation deliverable. Preserve user-configured pricing assumptions rather than replacing them with one fixed estimate. This is a broader lesson for AI-assisted tools. A generated result is not the only source of value. The edits, judgments, and communication choices a person adds afterward are part of the artifact too. Before-and-after AADB canvases showing an AI refinement that adds Azure Front Door and WAF while retaining the positions of eight existing services and the anchors of four existing groups. Figure 3. In this controlled synthetic refinement, all eight existing service positions and four group anchors remained unchanged. The containing Application group expanded to accommodate the new edge tier, so preservation does not imply that every group dimension stays fixed. Quality is structural, not only visual A diagram can look polished while still being architecturally confusing. Early feedback exposed cases where a generated service appeared disconnected because a model referenced a display name instead of the service identifier used by the canvas. The correction was not another prompt instruction alone. The application now resolves connection endpoints across identifiers, normalized service names, and service-type aliases. It repairs valid edges, drops invalid or self-referential edges, detects remaining orphan nodes, and records aggregate integrity signals. That creates a more useful definition of diagram quality: Are the services connected as intended? Were any generated edges repaired or dropped? Are there orphaned nodes? Did refinement preserve the existing layout? Did an architecture change receive a fresh validation? Visual polish still matters, especially when an artifact leaves the editor. But structural integrity gives the product something deterministic to test and monitor. Stage 3: Validate & Improve — treat validation as a lifecycle The Azure Well-Architected Framework is most useful when validation becomes iterative rather than ceremonial. The Diagram Builder can assess a proposed design across the five Well-Architected pillars, surface findings, and apply selected recommendations. But that workflow exposed an important state-management problem: when the architecture changed, the prior validation result disappeared along with the obvious route back to revalidation. The updated experience keeps the previous report, marks it Revalidate Needed, and makes clear that the score describes an earlier state of the architecture. A new validation replaces it only after the updated design has been assessed. This distinction prevents a stale score from looking current. It also clarifies what an architecture-level assessment can and cannot prove. A diagram may show that a WAF, cache, backup service, or secondary region exists. It usually cannot prove that purge protection, diagnostic routing, encryption settings, role assignments, health probes, or failover policies are configured correctly. That is why validation findings need to distinguish between: Pattern-level gaps — missing or misplaced architectural components Configuration-level gaps — required settings that must be verified in Infrastructure as Code or the deployed environment Generated scores and recommendations help architects review a design; they do not replace an Azure Well-Architected Review, security review, deployment validation, or professional judgment. Validation result retained after architecture recommendations are applied, with a Revalidate Needed status and action. Figure 4. Architecture changes make a previous validation historical, not useless. The result remains available while the interface clearly asks for a fresh validation. Stage 4: Share or Build — design for the artifact’s destination The editing canvas and the final deliverable serve different purposes. Canvas dots, handles, navigation controls, and selection states help during authoring. They can make an exported diagram feel unfinished. The Diagram Builder now separates those concerns with Plain, Dots, and Grid export backgrounds while preserving the active light or dark theme. The same AADB architecture shown first on the editing canvas with the export menu open and then as the resulting Plain PNG without authoring controls. Figure 5. Authoring and delivery are different contexts. The upper view shows the editable canvas and its real export controls; the lower view is the Plain PNG produced from that same canvas, without editing chrome. Cost language is deliberately qualified. Azure services often combine fixed, usage-based, and configuration-dependent charges. A baseline that includes six numerically priced services but excludes 20 usage-based items is not the total cost of the architecture. The output identifies those exclusions rather than treating missing values as zero. The final stage also includes deployment guides and Infrastructure as Code. Here, honesty about artifact coverage is essential. A generated Bicep file may be a useful starter while still omitting private endpoints, diagnostic settings, failover configuration, or service-specific resources. The artifact should state what it implements, what remains conceptual, and whether Azure Resource Manager validation passed. AI-generated diagrams, costs, validation results, deployment guides, and Infrastructure as Code should all be reviewed and validated before production use. The same journey now extends to agents The MCP server introduced in the previous article makes the Diagram Builder available to agent experiences such as Microsoft Scout. The four-stage journey provides a useful way to think about agent orchestration too: Import or create one canonical architecture. Refine it without silently changing the intended topology. Validate it, apply supported improvements, and revalidate. Render or generate artifacts with explicit coverage and limitations. The current MCP surface exposes 12 tools, three resources, and three reusable prompts. It can normalize an existing architecture, validate and harden it deterministically, estimate regional costs from a dated pricing snapshot, render presentation/technical/cost views, and generate Bicep, Terraform, and deployment guidance. The calling agent still owns orchestration and reasoning; the MCP server is intended to remain a deterministic architecture capability, not a second hidden agent. The native MCP renderer can project one canonical architecture into three communication profiles: Presentation emphasizes the primary request path, reduces supporting labels, and removes pricing. Technical preserves complete connection detail for engineering inspection. Cost retains the focused composition while adding service-level pricing assumptions, a fixed-priced baseline, and explicit exclusions. These are MCP-generated SVG views, not Blueprint diagrams or screenshots of the editable web canvas. The services, connections, and groups remain the same; only the information treatment changes. The AADB MCP renderer projecting the same canonical architecture into presentation, technical, and cost SVG profiles. Figure 6. Native AADB MCP output from one 8-service, 9-connection, 4-group architecture. Presentation prioritizes the story, Technical exposes connection detail, and Cost foregrounds pricing assumptions and exclusions. Recent work on the MCP renderer added purpose-built presentation, technical, and cost profiles. More importantly, testing agent-generated artifacts reinforced an accountability principle: a polished diagram and a compiled Bicep file do not prove deployability. An agent workflow should report whether topology changed, whether validation improved, which services are represented only conceptually, and whether the generated IaC passed Azure preflight. That is more useful than an unsupported claim that a design is production-ready. Trust also includes the tool boundary itself. The hosted MCP endpoints now require a bearer token for real session operations; missing or incorrect credentials are rejected. A shared token is appropriate for the current controlled integration, but it is not the end state for enterprise multi-user access. Entra ID/OAuth, per-client authorization, rotation, and revocation remain future hardening work. Microsoft Scout response after an authenticated Azure Architecture Diagram Builder MCP workflow, showing the tools used, initial and final validation scores, cost scope, Bicep classification, rendered architecture, artifact links, coverage gaps, and no-deployment warning. Figure 7. The guided lifecycle extends beyond the web application. In this synthetic Scout run with GPT-5.6 Sol, the agent used authenticated AADB MCP tools to validate, harden, cost, render, and generate starter artifacts while explicitly reporting coverage gaps and that nothing was deployed. Learning from adoption without identifying people Product learning does not require reconstructing individual identities. The findings behind this article use aggregate, deduplicated application telemetry, public article counters, public repository totals, and paraphrased feedback themes. They do not correlate Application Insights identifiers, feedback records, GitHub accounts, or email addresses. Written feedback remains submittable without contact information. When someone explicitly opts into follow-up, the email address is stored with the feedback record in Cosmos DB and is not sent to normal product telemetry. The current 180-day expiry field is a retention marker; automated deletion must be implemented and verified before describing that retention period as enforced. Those boundaries matter for both product design and public writing: Aggregate activity rather than profiling individuals. Paraphrase themes rather than publishing comments without permission. Keep optional contact consent separate from telemetry. Avoid presenting anonymous identifiers as confirmed people. Avoid claiming that publication timing proves acquisition causality. This is not a claim of legal compliance. It is a product discipline: collect less, preserve user agency, and make only the claims the evidence supports. What changed The guided journey is the visible result, but the deeper change is how the project now evaluates progress. Earlier question Better question Did the model generate a diagram? Did it generate a connected and understandable architecture? Did the user click Validate? Was the current architecture validated, and was it revalidated after changes? Did export start? Did a professional artifact finish generating successfully? Does the IaC compile? What does it actually implement, and does Azure preflight pass? How many features exist? Can an architect understand the next useful step? The model portfolio continued to evolve as well. The production selector now contains 15 configured entries, including MAI-Thinking-1 (Public Preview). But the more consequential changes in this article are deliberately model-independent: preserve human work, keep state and provenance explicit, qualify generated artifacts, and authenticate the tools agents can call. The goal is not to remove flexibility. Architects can still open any tool directly, rearrange the canvas, reject recommendations, change pricing assumptions, or export at any point. The goal is to make the workflow coherent without pretending architecture itself is linear. The next boundary: logical versus physical architecture Recent feedback points to a harder problem than adding another model or export format. Architects working with private Azure AI landing zones need to distinguish shared platform resources from project-owned resources, preserve VNet and subnet boundaries, and reason about CIDRs, NSGs, route tables, private endpoints, DNS, and managed identities. The current Topology mode can show services and relationships, but it should not imply exact physical fidelity when those facts are absent. A useful logical diagram answers what exists and how it interacts. A physical or low-level design must answer where it is deployed, how it is isolated, and which values came from evidence. That is the next technical direction I am exploring: an evidence-aware Physical Architecture view backed by deterministic reconstruction from Terraform plan/state, ARM, or a live Azure inventory. Exact fields would be labeled as observed or resolved; AI suggestions would remain explicitly proposed; unsupported or missing inputs would be reported instead of silently invented. This capability is not shipped today, and it will require its own schema, validation rules, layout, security review, and evaluation set. That distinction matters. The lesson from adoption is not to put every architecture concern into one crowded canvas. It is to make each artifact’s purpose and evidence boundary clear. Try it, challenge it, help shape what comes next The Azure Architecture Diagram Builder remains open source, and the live experience is available today: Live app: https://aka.ms/diagram-builder Source code: github.com/Arturo-Quiroga-MSFT/azure-architecture-diagram-builder Getting started: Documentation and deployment guidance The next phase is to measure whether the guided journey helps people complete the full loop, especially recommendation-to-revalidation and artifact-generation success. In parallel, I am beginning the narrower physical-architecture investigation described above. Both efforts will use aggregate signals, reviewed fixtures, and sufficiently large cohorts rather than individual journey reconstruction. Try the workflow with a real architecture problem. Tell me where the handoffs are unclear, where the diagram loses intent, or where an artifact claims more than it implements. Those are the gaps worth fixing next. Measurement note: Article views are rounded public counters observed August 5, 2026. Application figures use deduplicated retained telemetry through August 13 and anonymous identifiers. GitHub totals and rolling 14-day traffic were observed August 13. The comparison windows are May 19–July 9 and July 10–August 13. These signals have different populations and must not be added together. Timing comparisons show concurrent activity, not causal attribution.August 20 Federal Event: Accelerating Enterprise Modernization, AI & Cybersecurity
Overview Federal agencies are under increasing pressure to modernize mission systems, strengthen cybersecurity, and responsibly adopt artificial intelligence. Executive Order 14409, record federal technology investments, and Microsoft's OneGov initiative have created a unique opportunity to accelerate secure digital transformation. What Attendees Will Learn How agencies are translating AI strategy into operational outcomes. Microsoft's latest AI, cybersecurity, and modernization capabilities. How Microsoft OneGov is accelerating secure AI adoption across federal agencies. Real-world modernization strategies and success stories from CDC and USGS. Practical approaches to modernizing legacy systems while improving mission outcomes. Plus: Every attendee receives a complimentary 30-minute FY27 AI Readiness Consultation with a MERP Solutions Architect. Why Attend Join MERP Systems and Microsoft to discover the latest AI and cybersecurity innovations and hear directly from CDC and USGS as they share real-world enterprise modernization strategies and success stories. Attendees will leave with actionable insights, proven approaches, and a better understanding of how to accelerate secure AI adoption across their organizations. Event Link- https://events.teams.microsoft.com/event/b58ebc1e-e927-4e66-b73c-f25d7f43be7f@c33861f9-4a57-4bc4-bc88-d6e0c7b92c37?source=copyLinkOneEventsShareDialogSkill or Sub-Agent. Choosing AI Capabilities You Will Actually Reuse
Audience: Cloud architects, platform engineers, engineering leaders The wrong first question Most teams building AI capabilities start with the wrong question. They ask which model to use. The model matters less than the shape of the capability around it. The first real fork is this. Are you building a skill or a sub-agent? Get that wrong and no model choice will save you. A skill and a sub-agent are two different delivery shapes, and each one fails at the other one's job. The insight The choice between a skill and a sub-agent is not about model power. It comes down to four checks. How the work iterates, whether the output carries a voice, how far an early wrong turn spreads, and how often it recurs. Score each, count which way they lean, and the shape falls out. An even split means build both, and let the skill drive the sub-agent. The three sections below take the checks worth a pause. Frequency is the plain one: a one-off craft piece leans to a skill, a repeatable batch job to a sub-agent. A skill lives inside the conversation. It reads files, asks a question, refines with the author, and keeps a human in the loop mid-flight. A sub-agent takes one prompt, runs to completion, and returns one report. Both are useful, for different work. Dimension Skill Sub-agent Iteration Conversation, many turns One hand-off, one pass Voice Holds a style profile and applies it Drifts toward generic by design Human gate Every turn Once, at the end Best for Craft, subjective output Bounded, structured output Table 1. The same three dimensions decide the shape every time. 1. Decide the iteration model first Before anything else, architects should ask how the work actually happens. Is it a conversation or a hand-off? That single answer removes most of the ambiguity. Craft work needs back and forth Batch work needs one clean pass Conversations need memory of the thread Hand-offs need a bounded input and a clear output A skill is right when the value comes from iteration. A blog post, a design review, a tricky refactor. A sub-agent is right when the work is well defined and the output is the deliverable. In practice The pattern that works: use a skill when the team expects three or four rounds of "close, but change this". The trade-off: a skill costs more attention per run because a human stays involved. The trap to avoid: forcing iterative craft into a one-shot agent and then editing the output by hand every time. 2. Voice fidelity decides craft work Some outputs have a voice. An article, a customer email, an architecture narrative. Others do not. A query result, a data export, a status summary. The line between them is not cosmetic. It decides which shape survives review. Voice-heavy work favours a skill Voice-neutral work favours a sub-agent Skills can hold a style profile and apply it Sub-agents drift toward generic by design When the output carries a name, fidelity is the whole game. A capable model with no voice anchor produces text that reads like it came from a committee. In practice The pattern that works: give a skill an explicit voice profile with banned phrases and cadence rules. Let it self-check before it shows anyone anything. The trade-off: the profile takes real effort to write once. The trap to avoid: expecting a stateless agent to match a personal style from a single prompt. Implementation note A voice profile is not documentation. It lives in the skill definition, an executable contract the skill checks itself against before a draft is ever shown. A small profile goes a long way. # voice-profile (excerpt) banned_phrases: [seamless, robust, game-changing, leverage the power] forbid: [em-dash, semicolon, exclamation in body] max_avg_sentence_words: 20 require: - one "In practice" block per section - a closing discussion question self_check: run before any draft is shown to a human 3. Put the human gate where the risk is Every AI capability needs a human review gate. The design question is where that gate sits. Placement is the difference between catching a problem early and unpicking it later. Skills gate continuously, turn by turn Sub-agents gate once, at the end Continuous gates catch drift early End gates are cheaper but riskier for craft If a wrong turn early corrupts everything after it, the team wants a skill. If the work is bounded and a bad output is easy to spot and discard, an end gate is fine. In practice The pattern that works: match the gate to the blast radius. High blast radius and subjective quality point to a skill. Low blast radius and objective output point to a sub-agent. The trap to avoid: a one-shot agent doing forty minutes of unattended work that a human then has to unpick. 4. The pattern that scales is both The mature answer is not one or the other. It is a skill on top of a sub-agent. The two shapes compose cleanly when each one keeps to its own job. The skill orchestrates and holds the voice The sub-agent executes bounded sub-tasks The human reviews at the skill layer Each layer does what it is good at Figure 1. In the combined pattern the human reviews at the skill layer, and the sub-agent only touches the bounded task. The skill runs the conversation and keeps quality. When it needs a bounded, repeatable job done, it delegates to a sub-agent. The result is iteration where craft lives and automation where the work is mechanical. In practice The pattern that works: a skill drafts and refines an article with the author, and calls a sub-agent to fetch and summarise reference material. The trade-off: two layers are more to build than one. The trap to avoid: collapsing both into a single agent and losing either the voice or the automation. The operational trade-offs Shape is not only a design choice. It shows up in cost, latency, and how you debug a bad run. Architects should price these in before committing to a pattern. A skill spends more tokens and more human minutes per run A sub-agent spends compute once and returns fast A skill fails in small, visible steps you can correct A sub-agent fails as one block you inspect after the fact The cost of a skill is attention. Someone stays in the loop and that time is real. The cost of a sub-agent is rework. When a one-shot run goes wrong, the whole output is suspect and someone redoes it. Observability follows the same split. A skill leaves a turn-by-turn trail you can read. A sub-agent leaves one input and one output. You instrument the boundary and log the prompt and the result. Pick the shape whose failure mode your team can afford. The wrong shape does not announce itself. It shows up later as a cost line or a rewrite. Two capabilities, one team Consider a team standardising its engineering work with AI. Two capabilities land on the backlog in the same week. The first is recurring status queries. Well defined input, structured output, no voice. A stateless sub-agent fits. One prompt in, one report out, gate at the end. It works on day one and keeps working. The second is authored technical content. Subjective, voice-heavy, many rounds of refinement. The reflex is to reuse the sub-agent that just shipped. That reflex is the mistake. The queries stay clean. The content reads flat and generic, and every draft needs a heavy human rewrite. Rebuilt as a skill with a voice profile and a turn-by-turn gate, the same work compounds instead of fighting back. Same team, same models, two different shapes of work, and only one right tool for each. What teams get wrong The common pattern is defaulting to whichever shape the team built first. A team ships one sub-agent, likes it, and forces every new problem into a sub-agent. Or it builds one skill and runs everything as a conversation, including batch work that should be automated. It looks like consistency. It feels like reuse. But it leads to craft work that reads generic and batch work that needs babysitting. The fix is not a better model. It is naming the shape of the work before picking the tool. The three shapes to watch for in your own stack: A voiced deliverable coming out of a one-shot agent, rewritten by hand every run. A skill wearing a sub-agent costume. A batch job run as a conversation and babysat turn by turn. A sub-agent wearing a skill costume. A large workflow forced into one agent that holds neither the voice nor the automation. Two shapes collapsed into one. Name which one you are looking at, and the fix picks itself. A quick way to decide When a new capability lands on the backlog, run four checks before picking a tool. Iteration: conversation or one hand-off Output: subjective and voiced, or structured and neutral Blast radius: does an early wrong turn corrupt the rest Frequency: a one-off craft piece, or a repeatable batch job Three or more answers leaning subjective and iterative point to a skill. Three or more leaning structured and repeatable point to a sub-agent. A split answer usually means a skill orchestrating a sub-agent underneath. Figure 2. Score the four checks and count the leanings. Three or four one way pick the shape. An even split means a skill orchestrating a sub-agent. Where to start depends on what you have already built The framework is the destination. Where you start depends on what your team has shipped so far. Find your stage and take the one first move for it this week. Stage First move, this week Watch out for Just starting, nothing built yet Pick the single task you repeat most and write a one-paragraph capability brief for it, iteration, output, blast radius, and frequency, before you build. The brief names the shape, and the shape names the tool. Building a general assistant before you have named one concrete job. One capability, reused for everything List every job you push through the one tool, find the one whose shape does not match, and rebuild just that one in the right shape. You do not need to replace what works. Forcing new work into the tool you already have. A small fleet, a handful of capabilities Take your largest layered workflow and split it, a skill that holds the voice and the human gate on top, a sub-agent that does the bounded work underneath. Capabilities that duplicate each other with no composition between them. Table 2. Same framework, different first move. What you have already built decides where the leverage is this week. Your setup also shapes the answer. A solo builder should optimise for their own voice and iteration speed, where one strong skill beats three thin ones. A platform team should standardise the capability brief and a shared voice profile, so the fleet stays consistent as more people add to it, and a new capability inherits the house style instead of drifting from it. Figure 3. Whatever you have built so far, the first move has the same shape. Name the work before the tool, then match the shape to the tool. The shift The shift is from "what can the model do" to "what shape is the work". Model capability is table stakes now. The advantage is in matching the capability to the work. Our own capability fleet is built this way, interactive skills and autonomous sub-agents in separate places with an orchestrator on top, and that split is what keeps it maintainable as it grows. Iterative and voice-heavy points to a skill. Bounded and mechanical points to a sub-agent. Large and layered points to a skill orchestrating sub-agents. Decide that first, and the model becomes a detail the team can change later without rebuilding anything. Most teams collapse both ideas into "automation" and end up with neither. The teams that separate them build capabilities they actually reuse. Want to discuss? Drop a comment with patterns you have seen in your environment. I read every reply.Resource Guide: Making Physical AI Practical for Real‑World Industrial Operations
Microsoft’s adaptive cloud approach enables organizations to turn operational technology (OT) data into intelligent actions, autonomously, without requiring everything to live in the cloud by unifying cloud-to-edge management plane, data plane, and intelligence platform. At the center of this approach are key foundational technologies: Key Purpose Offering Direct-to-cloud device management + telemetry ingestion Azure IoT Hub Industrial connectivity + edge data plane Azure IoT Operations Unified analytics + real-time intelligence Microsoft Fabric On-device AI inferencing runtime Foundry Local Microsoft Azure IoT Gartner winner: Microsoft named a Leader in the 2025 Gartner® Magic Quadrant™ for Global Industrial IoT Platforms See it all come together Before diving into each component, watch this end-to-end demo showing how Azure IoT Operations, Azure IoT Hub, Microsoft Fabric, and Foundry Local work as one stack across the edge-to-cloud lifecycle - Making industrial AI practical for real-world operations with adaptive cloud. How these components work together Azure IoT Operations and Azure IoT Hub collect real-time data from operational assets and send semantically-ready, modeled data to Microsoft Fabric, where it's contextualized with enterprise data for downstream analytics. Microsoft Foundry extends to the edge through Foundry Local, so the same tooling used to deploy and manage AI models in the cloud applies to edge use cases. All of it integrates into Azure Resource Manager, bringing OT devices, assets, and edge AI models into the same management and security paradigm as every other Azure-managed resource. This blog walks through where to get started with each product capability: 1. Manage Cloud-Connected Devices and Telemetry with Azure IoT Hub Azure IoT Hub is a fully managed cloud service that enables secure bidirectional communication, device-to-cloud telemetry ingestion, cloud-to-device command execution, per-device authentication, remote management and more. Telemetry from IoT Hub can also be routed downstream into analytics platforms like Microsoft Fabric for visualization or AI modeling. Recommended Usage: Devices that utilize IoT Hub are distributed, stand-alone devices with fixed-functions. These devices typically do not require cloud-managed containerized workloads or cloud-managed proximal industrial protocol connectivity. Examples of appropriate device-to-cloud IoT Hub endpoint devices include water monitoring stations, vehicle telematics, distributed fluid level sensors, etc. Resources Current in-market services overview: IoT Hub: What is Azure IoT Hub? - Azure IoT Hub DPS: Overview of Azure IoT Hub Device Provisioning Service - Azure IoT Hub Device Provisioning Service ADU: Introduction to Device Update for Azure IoT Hub Building scalable solutions with Azure IoT platform: Best practices for large-scale IoT deployments - Azure IoT Hub Device Provisioning Service Scale Out an Azure IoT Hub-based Solution to Support Millions of Devices - Azure Architecture Center Azure IoT Hub scaling Try out our preview of new IoT Hub capabilities (integration with Azure Device Registry and Certificate Management) Learn more about these capabilities on our blog post: Azure IoT Hub + Azure Device Registry (Preview Refresh): Device Trust and Management at Fleet Scale… Integration with Azure Device Registry (preview): Integration with Azure Device Registry (preview) - Azure IoT Hub Microsoft-backed X.509 certificate management (preview): What is Microsoft-backed X.509 Certificate Management (Preview)? - Azure IoT Hub How to start with the preview: Deploy IoT Hub with ADR integration and certificate management (Preview) - Azure IoT Hub 2. Connect Industrial Assets with Azure IoT Operations Azure IoT Operations provides a unified data plane for the edge that runs on Azure Arc–enabled Kubernetes clusters and supports open industrial standards. It allows organizations to connect and capture equipment telemetry, normalize OT data locally, route hot-path signals to real-time analytics, securely manage layered industrial networks, and more. Edge‑processed data can then be sent upstream to Microsoft Fabric for AI‑driven analysis. Recommended Usage: Azure IoT Operations is intended to be the data plane for an adaptive cloud deployment extending the management, data, and AI capabilities of the Microsoft cloud to an on-prem device. This device binds to these cloud planes providing a platform for local data processing and intermittent connectivity. The target for these devices range from a small-gateway-style PC to a full data center. Azure IoT Operations endpoints enable cloud-managed containerized workloads and cloud-managed proximal industrial protocol connectivity. Examples of appropriate adaptive cloud and Azure IoT Operations endpoints include, on-robot computers, industrial machine controllers, retail store sensor/vision processing, and top-of-factory site infrastructure for line of business applications. Resources Azure IoT Operations Overview Azure IoT Operations Documentation Hub Releases · Azure/azure-iot-operations Quickstart: explore-iot-operations/quickstart at main · Azure-Samples/explore-iot-operations Latest release update: Open-source framework for scaling robotics from simulation to production on Azure + NVIDIA: microsoft/physical-ai-toolchain Demo video showcasing this in action: Making industrial AI practical for real-world operations with adaptive cloud How we built the demo: explore-iot-operations/quickstart at main · Azure-Samples/explore-iot-operations Edge-AI: microsoft/edge-ai: Production-ready Infrastructure as Code, applications, pluggable components, and… Latest Announcements & Blogs Making Physical AI Practical for Real-World Industrial Operations: Part 1 | Microsoft Community Hub Making Physical AI Practical for Real-World Industrial Operations: Part 2 | Microsoft Community Hub Introducing small form factor infrastructure: embed intelligence into physical systems Unlock Industrial Intelligence | Microsoft Hannover Messe 2026 From pilots to production: How Microsoft and partners are accelerating intelligent operations Partner Solutions How Mesh Systems Builds on Azure IoT Hub and Azure IoT Operations to Accelerate Industrial AI | Microsoft Community Hub Unlocking the Human Telemetry Layer for Safer Industrial Operations | Microsoft Community Hub Unlocking Smart Manufacturing: Siemens Industrial Edge Meets Azure IoT Operations Solving the Data Challenge for Manufacturers with Sight Machine & Azure IoT Operations | Microsoft Community Hub Microsoft and Rockwell Automation: Transforming Industrial AI Together | Microsoft Community Hub 3. Advanced Analytics with Microsoft Fabric Microsoft Fabric delivers a unified, end‑to‑end analytics platform that transforms streaming OT telemetry into real‑time insights and live dashboards. Fabric Operations Agents monitor industrial signals to recommend targeted actions, while Fabric IQ provides a shared semantic foundation that enables AI agents to reason over enterprise data with business context. Together, Fabric turns live industrial data into AI‑powered operational intelligence. Resources Get Started with Microsoft Fabric Learning Path Fabric Real-Time Intelligence documentation - Microsoft Fabric | Microsoft Learn Create and Configure Operations Agents - Microsoft Fabric | Microsoft Learn Fabric IQ documentation - Microsoft Fabric | Microsoft Learn 4.Run AI Models On‑Device with Foundry Local Foundry Local extends on‑device AI to Arc‑enabled Kubernetes edge clusters, providing a Microsoft‑validated inferencing layer for running AI models in industrial, disconnected or sovereign environments. Resources Foundry Local on Azure Local Documentation Participate in Foundry Local on Azure Local preview form Foundry Local on Azure Local: HELM deployment Demo Customer Stories Chevron: Chevron plans facilities of the future with Azure IoT Operations Husqvarna: Husqvarna Group Boosts Operational Efficiency with Azure Adaptive Cloud Ecopetrol: Azure IoT Operations and Azure IoT for energy help Ecopetrol optimize energy distribution while lowering operational costs P&G: Procter & Gamble cuts model deployment time up to 90% with Azure IoT Operations Toyota: Toyota Industries innovates its paint shop processes with Azure industrial AI and Azure IoT HubTrain a simple Recommendation Engine using the new Azure AI Studio
The AI Studio Odyssey: Embark on a journey to the heart of personalization with our latest guide, “Train a Simple Recommendation Engine using the new Azure AI Studio.” Unlock the secrets of the all-new Azure AI Studio intuitive tools to craft a recommendation system that feels like magic, yet is grounded in data and user preferences. Ready to enchant your audience? Grab some popcorn and read on!Introducing GPT-transcribe and GPT-live-transcribe in Microsoft Foundry
A transcription model hears “account number 8-4-7-2” but returns “account number eighty-four seventy-two.” A single error can break a downstream automation workflow. Developers building voice applications need transcription models that can handle real-world audio conditions, natural speech patterns, and business-critical details, including codes, dates, addresses, account numbers, mixed-language conversations, specialized terminology, and quiet or low-volume speech. GPT-transcribe and GPT-live-transcribe do just that and are available in Microsoft Foundry today. Two updates to the audio model family designed to improve automatic speech recognition across asynchronous transcription and live streaming scenarios. Built for More Accurate Transcription in Real-World Audio GPT-transcribe is the highest accuracy ASR model from Open AI, designed for asynchronous speech-to-text transcription of completed audio files and batch workloads. It accepts audio input and returns text output, making it a strong fit for workflows that process recorded, uploaded, or submitted audio, including meeting recordings, voicemails, and media files. GPT-live-transcribe is designed for low-latency streaming transcription through the Realtime API. It supports real-time audio input and text output, helping developers build live experiences where speech needs to be transcribed continuously as audio arrives. This model also introduces “tunable latency” where developers can adjust the latency/accuracy trade-off for streaming. It is a strong fit for live captions, voice assistants, contact center workflows, accessibility experiences, field service applications, real-time intake, and monitoring systems. Together, these models give developers transcription options in Microsoft Foundry for stored audio and live voice interactions. Their text output can support downstream workflows such as search, summarization, routing, analytics, automation, and quality review. What’s New in Both Models The features of the new transcription models focus on improving transcription quality in real-world audio environments where speech can be brief, noisy, accented, quiet, domain-specific, or mixed across languages. Key capabilities include: Background noise: Helps isolate speech in noisy environments so transcription quality can remain more reliable when audio conditions are not controlled. Short utterances: Improves recognition of brief commands, confirmations, interruptions, and clipped speech that can be difficult to capture accurately. Alphanumeric perception: Strengthens transcription of IDs, codes, phone numbers, dates, addresses, account numbers, and mixed letter-number sequences. Domain terminology understanding: Improves recognition of specialized vocabulary used in product, workflow, industry, and business-process contexts. Codemix: Improves understanding when speakers switch between languages within a conversation or utterance. Context awareness: Uses topic hints and past conversation context to improve transcription accuracy and help maintain consistency. Accent robustness: Improves handling of regional accents, non-native accents, dialects, and varied speaking styles. Whispering: Improves recognition of quiet or low-volume speech, including whispered commands and private dictation. Live captioning and accessibility experiences: Generate real-time captions for meetings, events, media experiences, and assistive applications. Contact center and voice workflows: Capture spoken details as conversations happen, supporting routing, quality review, summarization, and downstream automation. Monitoring, analytics, and compliance workflows: Provide text visibility into ongoing spoken input so teams can analyze, review, and act on conversation data. Also Available: GPT-realtime-2.1 and GPT-realtime-mini-2.1 gpt-realtime-2.1 and gpt-realtime-mini-2.1 are also available in Microsoft Foundry for developers building speech-to-speech applications. Unlike GPT-transcribe and GPT-live-transcribe, which return text, these models accept audio and generate audio for low-latency conversational experiences over the Realtime API. gpt-realtime-2.1 focuses on interaction quality and robustness, while gpt-realtime-mini-2.1 provides a smaller, faster, and more cost-efficient option for high-volume deployments. Together with GPT-transcribe and GPT-live-transcribe, these realtime audio updates give developers more flexibility to build voice applications that need both accurate transcription and responsive spoken interaction, whether the experience is centered on capturing speech as text, responding with audio, or combining both patterns in a single workflow. Use Cases by Model GPT-transcribe Use GPT-transcribe when the application needs accurate text transcripts from recorded, uploaded, or submitted audio. It is a strong fit for meeting and call transcription, media transcription, customer support intake, voicemail and message processing, quality review, compliance workflows, and domain-specific transcription where short utterances, structured alphanumeric details, specialized terminology, accents, background noise, code-mixed speech, or quiet audio can affect downstream accuracy. GPT-live-transcribe Use GPT-live-transcribe when the application needs live streaming transcription with low latency. It is designed for real-time captions, accessibility experiences, contact center transcription, voice-enabled workflows, live monitoring, operational dashboards, and agent-assist scenarios where spoken input needs to become text continuously as the interaction unfolds. Pricing The following pricing example shows Global Standard rates by model and modality. Rates for GPT-realtime-2.1 and GPT-realtime-mini-2.1 are listed per 1 million tokens. GPT-transcribe and GPT-live-transcribe are listed per audio hour. Model Deployment Modality Input Cached Input Output GPT-realtime-2.1 Global Standard Audio $32.00 $0.40 $64.00 Text $4.00 $0.40 $24.00 Image $5.00 $0.50 -- GPT-realtime-mini-2.1 Global Standard Audio $10.00 $0.30 $20.00 Text $0.60 $0.06 $2.40 Image $0.80 $0.08 -- GPT-live-transcribe Global Standard Audio -- -- $1.02/hour GPT-transcribe Global Standard Audio -- -- $0.27/hour Getting Started Choose GPT-transcribe when your application processes complete audio files asynchronously, or GPT-live-transcribe when it needs text continuously as speech arrives. Try the models in Microsoft Foundry, then use the resources below to explore the Realtime API, follow the audio quickstart, compare available models, and review Azure OpenAI in Foundry Models documentation. For asynchronous transcription, submit a complete audio file to GPT-transcribe and process the returned transcript after the request completes. This pattern works well for recordings, voicemails, and uploaded media. For streaming transcription, open a Realtime API session with GPT-live-transcribe, send audio as it is captured, and handle incremental transcript events. This pattern supports live captioning and agent-assist experiences that need text during an active interaction. Refer to the linked quickstart and Realtime API documentation for current SDK setup, authentication, request schemas, and supported audio formats. Explore Microsoft Learn documentation to learn more: Use GPT Realtime API for speech and audio with Azure OpenAI in Foundry Models GPT Realtime audio quickstart Azure OpenAI in Foundry Models overview
Token Economics in Practice
Introduction: The cheap-token trap Token prices alone are a poor economic model for agents. The price of reaching a fixed capability has fallen sharply — In a 2025 Report Stanford's AI Index reported a roughly 280-fold drop in the cost of GPT-3.5-level inference between late 2022 and late 2024, and Epoch AI tracks steep (if uneven) per-benchmark price declines. The intuitive conclusion is that agents are getting cheaper to run. The operational reality is the opposite. Agents turn cheaper inference into longer, stochastic trajectories: growing context windows, repeated tool schemas, retries, reflection loops, and sub-agent fan-out. In one study of agentic coding, repeated runs of the same agent on the same task varied in token cost by as much as 30× for coding agents. When a single logical task can cost you thirty times more depending on the path the agent takes, optimizing average cost per token will happily make the wrong system look efficient. Similar argument can be made for other agentic systems where we may need more than one tries, more than one MCP Calls, Reasoning or use of multiple skills, hooks or tool calls to arrive at a completed task. So, the leading question of token economics isn't "what's the token price?" It's "what does it cost to get one accepted unit of useful work — and how confident can we be in that number before the agent runs?" The unit that actually matters: Cost per accepted task I use token economics to mean managing the unit economics of useful AI work under uncertainty. The meaningful unit is cost per accepted task, not cost per token. Let A = 1 mean a task passed its acceptance rubric. The long-run unit cost of a policy π is approximately: The numerator is expected task cost; the denominator is the probability the output is actually acceptable. This follows the FinOps distinction between successful and unsuccessful AI outputs and the recommendation to connect cost with workload value. It is a working definition for this project, not a quoted standard — but it reframes the engineering problem immediately. A "cheaper" policy that halves cost while dropping acceptance from 95% to 70% is more expensive per accepted task, and only this ratio makes that visible. That reframing turns "pick the cheapest model" into a five-step discipline: Forecast a distribution, not a single token estimate. Select a cost policy that is plausible for the task and its risk. Enforce routing, context, cache, and budget controls during execution. Evaluate whether the output still clears a workload-specific quality floor. Revert unsafe savings, reconcile predicted vs. actual usage, and calibrate the next forecast. From a metric to a controller If cost is a random variable, the objective is a stochastic one. Minimize expected task cost subject to two constraints — a quality floor on every workload segment, and a bound on how often you blow the budget subject to a per-segment quality floor: and a chance constraint on budget breach: Here π is the policy; C_task is total task cost; Q_s is quality for a supported segment s with floor Q_min; B is the budget; and ε is the tolerated breach probability. The pieces are all borrowed — stochastic optimization for the expected-cost objective; FrugalGPT and Confident Adaptive Language Modeling for the LLM precedent of cutting cost while preserving performance; SRE service-level objectives for treating "acceptable service" as an action-driving threshold and Group DRO for the insight that averages hide group failures; and Charnes–Cooper chance-constrained programming for the probabilistic budget limit. The synthesis — wiring them into one agent controller — is the contribution. Two honest caveats travel with this controller: Q_s needs a confidence-adjusted lower bound (sparse segments shouldn't trigger changes on two samples), and the chance constraint is not a guarantee until your forecast's percentile coverage is calibrated against real traces. A modeled P95 is a planning estimate, not a promised 5% breach bound. Two halves of the loop: feed-forward and feedback The current work is result of two self-prototypes — FutureTokenPredictor and TokenGov — built to make agent unit economics operable on Azure. These are reusable implementation patterns and experiments. The controller splits cleanly into a planning half and a runtime half. FutureTokenPredictor is the feed-forward side. It models workflow archetypes and uncertain iteration counts to produce P50/P95-style planning estimates before execution and recommends a policy. It stays outside the request path. TokenGov is the feedback side. Its request path applies the admitted cost policy; an out-of-band control plane evaluates outcomes and changes externalized policy when quality regresses. Runtime telemetry then flows back to the predictor as calibration data for the next forecast. Neither half is sufficient alone. Prediction without control is a spreadsheet. Control without quality feedback silently degrades your hardest segments. The value is the wire between them: a forecast that becomes an enforceable policy, an eval verdict that can reverse a cost action, and actuals that sharpen the next forecast. How the equation lands on Azure This is where token economics stops being a metric and becomes architecture. Each term in the controller maps to a concrete Azure control: Controller term Azure control in practice π (policy) Externalized in Azure App Configuration; enforced by API Management GenAI gateway (routing, context, cache, token policies) E[C_task | π] (expected cost) Reconstructed from APIM gateway, model, and Application Insights telemetry Q_s (segment quality) Azure AI Foundry evaluation over golden sets and sampled production traces B, ε (budget, breach tolerance) Forecast-informed limits and Azure Monitor alerts; Cost Management for allocation Reversion A Monitor-triggered Azure Function tightens or reverts policy in App Configuration — closing the eval-to-enforcement loop without a code deployment Most of these primitives already exist and are individually documented: APIM provides token quotas, semantic caching, and token metrics; Foundry Model Router offers cost/balanced/quality routing modes; Foundry cloud evaluation scores datasets and sampled traces. The interesting gap they don't close on their own is the connected mechanism — an evaluation verdict that can constrain or reverse a cost-saving action, and actual usage that improves the next forecast. Here is the full two-plane view. FutureTokenPredictor forecasts and recommends before execution; TokenGov owns runtime enforcement and quality-triggered reversion; prediction IDs join forecasts to actual telemetry so calibration can improve the next estimate. From Concept to Implementation Version 1 release the Token Prediction and forecast ability using a local mcp server called FutureTokenPredictor using a local MCP server modeled behind a simple UI, where you can create an assessment for your UI Workload. It lets you simple describe the AI / Agentic Solution you want to build and suggested a topology for it. From there , depending on your model selection, the studio, helps you predict the range of token usage and its estimated costs. In full version, this forecast is used to build a policy and govern your AI Spend accordingly. If you want to read more about the FutureTokenPredictor and how it works, check out my earlier blog Agentic Currency – Tokens and AI Infra: Full-Stack Cost Prediction for Autonomous Agents Version 2 with full governance and control will be released soon. TokenEconomics is available in the GitHub Repo TokenEconomics Clone it, experiment and test it out. Please provide feedback via a pull request on the repo or directly here via comments Happy Reading! References The 2025 AI Index Report | Stanford HAI Chance-Constrained Programming | JSTOR How are AI agents spending your tokens? - Stanford Digital Economy Lab FinOps for AI Overview AI gateway capabilities in Azure API Management | Microsoft Learn Model router for Microsoft Foundry concepts - Microsoft Foundry | Microsoft Learn Also Read Optimizing GitHub Copilot Cost in the Usage-Based Billing Era | Microsoft Community Hub Token Economics: The New FinOps for Agentic AI | Microsoft Community Hub
