The Model Isn’t the Boss: An Azure Architecture for Accountable AI Agents

Imagine a housing organization that maintains several thousand rented homes. Tenants report repairs through a website, an app, a phone line or a support worker who completes the form with them, and many reports include a photo. Asset history, approved repair guidance, contractor availability and work orders sit in separate systems, which makes coordination hard. The organization wants better diagnosis support, bookings that suit both the job and the household, trustworthy completion evidence and coordination with planned renovation. Some tenants need a person rather than a form, and automated vulnerability decisions are outside what this example allows.

What informed this pattern

This architecture synthesizes recurring control and integration concerns encountered in enterprise AI architecture discussions and design reviews. These include models being given direct access to transactional APIs, model confidence being treated as business authorization, uncertain partner outcomes being retried without reconciliation, and observability capturing more personal or business content than operations require. Customer-specific details have been generalized or removed.

The housing-maintenance journey is an illustrative scenario rather than a description of a specific customer implementation. It provides a concrete way to examine those recurring concerns and turn them into reusable architecture guidance. Here, a tenant is a person who rents a home, not a Microsoft Entra tenant.

The resulting three-plane model is my synthesis of those observations: separate nondeterministic reasoning from deterministic authorization and durable execution, then connect both to governed operational evidence. The purpose is not to present a verified production reference architecture, but to provide a pattern that teams can challenge, adapt and validate through a bounded pilot.

The repair journey exposes the real control problem

A capable model can describe a leaking trap from a photo. The consequential steps are actions: booking a contractor, issuing a payment instruction or alerting a caseworker. Each needs answers a model cannot supply on its own authority: who asked, which work item, which policy, who confirmed and what actually happened.

The Azure Well-Architected Framework’s AI workload guidance notes that AI replaces deterministic functionality in parts of a workload with nondeterministic behavior. Connect a chatbot straight to a work-order API and that nondeterminism reaches records, because the component interpreting untrusted text also changes them. This article proposes a different organizing principle:

AI reasons; authorized services act; governed data measures what happened.

Three planes, one accountable journey

The principle becomes architecture through three logical planes, which are this article’s explanatory model, not Microsoft product boundaries.

• Experience and reasoning: digital and assisted intake, grounded interpretation of text and photos, specialist recommendations and per-request coordination.
• Trusted action: deterministic policy checks, explicit authorization, durable commands, partner adapters, authoritative state and human handoffs.
• Operational intelligence: curated events and data, outcome analysis, evaluation evidence and governed improvement of knowledge and operating policy.

Identity, privacy, governance, observability and evaluation cross all three. Well-Architected application design guidance separates client, intelligence, inferencing, knowledge and tools layers, each with its own policies and identities, and routes data access through an API that enforces authorization. The planes group those layers by accountability and extend the rule to actions: reasoning proposes; only trusted action changes a record, after checking policy and authoritative state. The result is one decision and evidence chain:

request → recommendation and sources → authorization → command → execution outcome → completion evidence → operational measurement

Permission is not execution: an authorized booking can still time out, be rejected or succeed silently. Evidence is not exposure: records link through work-item references and correlation IDs rather than copies of prompts, photos, addresses or narratives, as Azure Monitor’s personal data guidance recommends. The chain makes the design methodology’s call for traceable AI results concrete.

Walk one request from intake to completion

Figure 1. The illustrative journey from report to completion evidence. Each step has an accountable owner, and emergencies, unclear diagnoses, wellbeing signals and disputes go to people. Original conceptual diagram.

Take a bounded, non-emergency report: water marks under a kitchen sink, submitted in the app with a photo. A per-request supervisor, ordinary application code that owns a durable work-item record, coordinates five logical roles. They are responsibilities, not necessarily five models or open-ended agent conversations.

Intake records whether the photo may be used for diagnosis. The diagnosis role retrieves approved guidance and asset history through scoped services and returns a structured recommendation with sources; a rule, not the model, decides whether it is specific enough to book. Scheduling offers slots that fit the confirmed skill, parts, urgency and access needs. Afterward, contractor evidence is checked against the job type’s requirements, and disputes go to a finance reviewer. Repeated reports of one fault can send a restricted signal to a caseworker, who decides what to do.

Planned renovation is a governed extension: the supervisor may suggest combining a compatible visit, but efficiency never overrides urgent-repair policy or the human and budget approvals that renovation requires.

This shape follows the same application design guidance: orchestration for predictable, compliance-sensitive workflows, agents where dynamic reasoning justifies its variability, and hybrids that start deterministic. Microsoft Agent Framework draws a similar line between agents and explicit workflows.

Map responsibilities to Azure, not a shopping list

Figure 2. The three planes mapped to candidate Azure and Fabric services. Only proposals cross into the trusted-action plane, and only reference-only events flow into analytics. Original conceptual diagram, not a deployment or a required bill of materials.

Verify each candidate against your estate, region and policies; the boundary column matters more than the product name.

Three tradeoffs deserve an explicit decision:

• Managed or custom orchestration. Foundry prompt or hosted agents reduce hosting work; custom code gives full control over state transitions and tests. Either way, keep business state in a store you own.
• Asynchronous or immediate confirmation. “Request received, booking pending,” confirmed later as in the Asynchronous Request-Reply pattern, tolerates slow partners. An instant answer must never claim a booking the contractor has not accepted.
• Freshness or data movement. Querying a source directly simplifies security; indexes and analytical copies add reach but need security trimming, freshness targets and deletion handling, as the grounding data guidance explains.

A minimal pilot core is one model deployment, one search index, the supervisor, a gateway, a queue and the existing systems. Fabric, broader Purview coverage, real-time analytics and multi-region designs can follow when measured needs justify them.

Make tools safe before making them autonomous

Figure 3. The control point. Authorization can deny a proposal, an authorized command can still end in an unknown outcome, and reconciliation comes before any retry. Original conceptual diagram.

The threat model is short: any tenant text, photo, retrieved document or contractor message can carry instructions, errors or someone else’s data, and a model reading it may propose an action it should not. The answer is ordinary security at the right boundary, applying the security principles of verifying explicitly, least privilege and assuming breach.

• Treat inputs as untrusted. Validate size and schema. Scan text with injection detection such as Prompt Shields, which detects manipulation attempts, not harmful, ungrounded or noncompliant content, and may perform differently on non-English text. Inspect images too.
• Allowlist narrow tools. Expose “propose slots for this work item,” never generic database or HTTP access, and validate tool usage before invocation.
• Authorize on the server. A command is valid only when bound to the intended work item, allowed action, trusted confirmation, current policy version and required evidence. A model writing “approved” has produced text, not approval.
• Separate identities. Tenants use a customer identity service such as Microsoft Entra External ID; staff, caseworkers and contractor firms get separate accounts or scoped credentials; workloads use least-privilege managed identities. Enforce access at every storage, retrieval, tool and analytics boundary.
• Minimize by purpose. Photo permission covers diagnosis, not unrelated reuse. Privacy and legal owners decide retention, deletion and the lawful basis for each processing step, which is not always consent. Wellbeing signals stay restricted.
• Fail closed. If identity, policy or partner configuration cannot be evaluated, nothing is written and the failure is logged as a failure, never as success. A per-action kill switch stops automated writes; staff keep manual access.

This matches the Well-Architected responsible AI guidance for agents that write: auditable activity, role-based access, circuit breakers, independent oversight and multimodal input inspection. Foundry guardrails can also scan proposed tool calls and responses, in preview, but do not replace server-side authorization.

Governance has three layers that do not substitute for one another: Azure Policy for resource configuration such as allowed regions; Fabric and Microsoft Purview for classifying, protecting and auditing supported data; and application authorization for each booking or payment. None establishes legal compliance, and labels do not follow data through every connector and index; in Fabric, mandatory labeling currently covers only Power BI items.

Use Fabric to understand and improve the service

The reasoning and action planes decide what should happen now. Microsoft Fabric earns its place by showing what has been happening and whether it works.

• Curate permitted data. Bring work-order, asset, contractor and completion data in with Data Factory, or OneLake shortcuts and mirroring where sources are supported. Document freshness, ownership and copies: shortcuts point to data in place; mirroring keeps a replica.
• Model for reuse. A lakehouse or warehouse holds conformed facts, such as visits per work item, under data contracts with source owners and with only lawful, necessary fields.
• Govern discovery and access. The OneLake catalog supports discovery, governance status and security roles; OneLake security can restrict tables, rows and columns; Purview adds labels, data loss prevention and audit for supported items. None authorizes an agent’s tool call.
• Choose timeliness deliberately. Real-Time Intelligence suits streams where minutes matter, such as safety signals in agent telemetry, but analytical real time is not a transactional guarantee. A Power BI semantic model keeps measures such as repeat-visit rate consistently defined.
• Close the loop through people. A finding such as “this fault type is often misread from photos” becomes a proposed change to guidance, evaluation data or policy, released through change control, never an automatic rewrite.

Keep raw wellbeing narratives and unnecessary tenant identifiers out of the analytical estate. Fabric does include transactional databases; excluding it from booking, payment and command execution is a design boundary, not a product limitation. A first pilot may need only existing reporting; Fabric pays off when several sources, shared definitions and governed reuse justify its capacity, skills and governance costs.

Evaluate the model; test the whole workflow

Figure 4. The evidence loop. Model evaluation, whole-workflow testing and operational intelligence feed human-approved change, never automatic production rewrites. Original conceptual diagram.

Well-Architected testing guidance treats model evaluation and whole-system testing as separate processes with distinct datasets. This design uses three evidence layers:

1. Model and retrieval quality. Diagnosis accuracy against human-labeled cases, groundedness of cited guidance, structured-output validity and behavior under uncertainty, sliced by language and accessibility where lawful. Compare candidate models on quality, latency and cost with the same data, using Foundry evaluators, including agent evaluators, some in preview.
2. Workflow and control tests. Deterministic assertions, not judge scores: no photo permission, no photo processing; unconfirmed diagnosis, no booking; malformed output, rejection and handoff; injected instruction in a caption or contractor note, no unauthorized tool call; duplicate or unknown booking, reconciliation; disputed completion, finance review; wellbeing signal, restricted human queue only. Mock partner failures and timeouts, as the guidance recommends.
3. Operational outcomes. First-time fix where defined, repeat visits, comparable cost per job, urgent-repair target adherence and tenant experience, each with a period, denominator, attribution method and limitations.

Build datasets from human-reviewed, lawfully de-identified examples where approved, and label synthetic cases. Release gates cover approved model, prompt and policy versions, regression runs, staged rollout and rehearsed rollback, as the operations guidance advises. Evaluation and monitoring data needs its own access and retention rules. Thresholds such as zero unauthorized writes are your proposals, not Microsoft-mandated numbers, and passing a test set does not prove production safety, fairness or savings.

Turn the five pillars into evidence

The Well-Architected design principles for AI workloads are most useful when each pillar becomes a decision with evidence attached.

Start small and prove the boundaries

A credible pilot proves its boundaries before widening autonomy:

1. Shadow mode. Reasoning runs on permitted reports while staff decide; compare recommendations with staff outcomes.
2. Assisted booking. The system proposes, staff confirm, and every write passes through the trusted-action plane with a decision record.
3. Bounded autonomy. Low-risk, well-tested job types book automatically within policy; payments and wellbeing follow-up stay human-approved.

Monday-morning checklist

• Name the human owner of each action type and of work-item state.
• List every tool with its identity, scope and denial test.
• Define decision-record fields and the correlation reference.
• Write the control-test suite before the first write-enabled release.
• Agree on measures, baselines and denominators with operations and finance.
• Decide what never enters logs or analytics.
• Rehearse the kill switch and the manual path.

What transfers to other field-service work: work-item state, grounded recommendations, authorized tools, partner integration, completion evidence and outcome measurement. What must be adapted: urgency rules, payment limits, privacy purpose, human ownership, retention and local regulation. Skip the pattern where a form and rules engine already work, and never carry this autonomy policy unchanged into healthcare or other consequential domains.

Sources

Facts checked on November 7, 2026. Microsoft guidance supplies design principles; Features marked preview can change before general availability.

• AI workloads on Azure. Microsoft Learn, last updated May 7, 2026.
• Application design for AI workloads on Azure. Microsoft Learn, last updated October 8, 2025.
• Manage personal data in Azure Monitor Logs. Microsoft Learn, last updated August 10, 2026.
• Design methodology for AI workloads on Azure. Microsoft Learn, last updated January 9, 2025.
• Microsoft Agent Framework overview. Microsoft Learn, last updated October 28, 2026.
• How to use vision-enabled chat models. Microsoft Learn, last updated July 29, 2026.
• How to use structured outputs with Azure OpenAI in Microsoft Foundry Models. Microsoft Learn, last updated August 24, 2026.
• Data, privacy, and security for Models sold by Azure in Microsoft Foundry. Microsoft Learn, last updated May 18, 2026.
• Document-level access control in Azure AI Search. Microsoft Learn, last updated September 17, 2026.
• Azure Container Apps overview. Microsoft Learn, last updated July 7, 2026.
• Durable Functions overview. Microsoft Learn, last updated June 2, 2026.
• What is Microsoft Foundry Agent Service? Microsoft Learn, last updated October 26, 2026.
• What is Azure API Management? Microsoft Learn, last updated October 13, 2025.
• Message transfers, locks, and settlement (Azure Service Bus). Microsoft Learn, last updated October 6, 2026.
• Azure Service Bus duplicate message detection. Microsoft Learn, last updated October 13, 2026.
• Service Bus dead-letter queues. Microsoft Learn, last updated July 22, 2026.
• About Azure Key Vault. Microsoft Learn, last updated September 22, 2026.
• Introduction to Application Insights: OpenTelemetry observability. Microsoft Learn, last updated October 9, 2026.
• Asynchronous Request-Reply pattern. Microsoft Learn, last updated March 30, 2026.
• Grounding data design for AI workloads on Azure. Microsoft Learn, last updated May 28, 2025.
• Security design principles. Microsoft Learn, last updated November 15, 2023.
• Prompt Shields in Azure AI Content Safety. Microsoft Learn, last updated October 14, 2026.
• Introduction to Microsoft Entra External ID. Microsoft Learn, last updated September 2, 2026.
• Managed identities for Azure resources. Microsoft Learn, last updated August 19, 2025.
• Responsible AI in Azure workloads. Microsoft Learn, last updated July 14, 2025.
• Guardrails and controls overview in Microsoft Foundry. Microsoft Learn, last updated October 20, 2026.
• Overview of Azure Policy. Microsoft Learn, last updated July 7, 2026.
• Use Microsoft Purview with Microsoft Fabric. Microsoft Learn, last updated September 28, 2026.
• Information protection in Microsoft Fabric. Microsoft Learn, last updated July 16, 2026.
• What is Microsoft Fabric? Microsoft Learn, last updated March 31, 2026.
• What is Data Factory in Microsoft Fabric? Microsoft Learn, last updated August 5, 2026.
• OneLake shortcuts. Microsoft Learn, last updated July 13, 2026.
• What is Mirroring in Fabric? Microsoft Learn, last updated August 28, 2026.
• OneLake catalog overview. Microsoft Learn, last updated September 28, 2026.
• Data security in OneLake. Microsoft Learn, last updated September 1, 2026.
• What is Real-Time Intelligence in Microsoft Fabric? Microsoft Learn, last updated May 20, 2026.
• Semantic models in the Power BI service. Microsoft Learn, last updated October 1, 2025.
• Understand Microsoft Fabric licenses and capacity. Microsoft Learn, last updated August 5, 2026.
• Test and evaluate AI workloads on Azure. Microsoft Learn, last updated September 5, 2025.
• Observability in generative AI (Microsoft Foundry). Microsoft Learn, last updated October 16, 2026.
• Agent evaluators (Microsoft Foundry). Microsoft Learn, last updated September 30, 2026.
• AI workload operations on Azure. Microsoft Learn, last updated November 18, 2024.
• Design principles for AI workloads on Azure. Microsoft Learn, last updated January 9, 2025.