Thursday Architecture Lesson #2
Azure Storage: Blob, Files, Disks or Azure SQL? Don't memorize the services. Understand the workload. Remember this: B → Blob Storage Need object storage for images, documents, backups, logs or large unstructured data. F → Azure Files Need shared file storage that multiple applications or servers can access using standard file shares. D → Managed Disks Need persistent block storage attached to Azure VMs. S → Azure SQL Need relational data with structured schemas, transactions, relationships and SQL capabilities. 🧠 The Architect's Rule: "Don't choose storage because it is available. Choose it because the workload needs it." The right storage architecture isn't the one with the most features. It's the one that gives the business the right balance of: Performance • Cost • Availability • Security • Scalability • Data access requirements • Operational complexity For example, storing application documents in Azure SQL just because the application already uses SQL may not always be the best architecture. Likewise, using Blob Storage for transactional relational data creates a completely different set of challenges. The question an architect should ask is not: "Which Azure storage service is the most powerful?" Instead ask: "What type of data do I have, and how does the application need to use it?" Architecture Lesson #2 takeaway: Start with the data characteristics and access pattern. Then choose the storage service. What is the first thing you consider when choosing an Azure storage service?Thursday Architecture Lesson #1
Azure Compute: VM, App Service, Container Apps or AKS? Don't memorize the services. Remember this: V → VM Need maximum control or have a legacy workload. A → App Service Want managed hosting for web apps or APIs. C → Container Apps Want containers without taking on Kubernetes complexity. K → AKS You genuinely need Kubernetes capabilities. 🧠 The Architect's Rule: “Don't choose AKS because you can. Choose AKS because you need it.” The right architecture isn't the one with the most powerful technology. It's the one that gives the business the right balance of: Capability • Cost • Security • Scalability • Operational complexity For example, if a simple web application can run effectively on App Service, introducing Kubernetes may add complexity without adding meaningful business value. The best architects don't ask: “What is the most advanced technology we can use?” They ask: “What is the simplest architecture that meets the requirements?” 💬 Your turn: If you were modernizing a traditional 3-tier application today, which would you choose first — VM, App Service, Container Apps or AKS? And most importantly, why?From AI PoC to Production: 7 Architecture Decisions Every Enterprise Must Get Right
Architecture Deep Dive Moving from an AI Proof of Concept to an enterprise-ready production workload requires much more than selecting the right model. AI is no longer just an experimentation topic. Across enterprises, teams are building copilots, RAG applications, AI agents, intelligent automation and domain-specific AI solutions. But there is a significant difference between making an AI Proof of Concept work and making an AI solution production-ready. A PoC asks: “Can we make AI do this?” Production asks much harder questions: “Can we make it secure, reliable, scalable, observable, governed and financially sustainable?” That is where architecture becomes critical. Microsoft’s Azure Well-Architected guidance for AI workloads highlights that AI systems introduce architectural considerations beyond traditional applications, including nondeterministic behavior, grounding data, model operations, testing, responsible AI and continuous evaluation. Here are seven architecture decisions I believe every enterprise should consider before moving an AI workload from PoC to production. 1. Start With the Business Outcome — Not the Model One of the most common mistakes is starting with: “Which AI model should we use?” The better question is: “What business problem are we solving?” Before selecting a model or Azure service, define: • The business outcome • The users • The expected experience • The measurable success criteria • Regulatory and compliance requirements • Data sensitivity • Expected scale For example: Instead of saying: “We want to build an enterprise chatbot.” Define the outcome: “We want employees to find accurate information from 500,000 internal documents in less than 5 seconds while respecting existing access permissions.” That single statement changes the architecture conversation completely. 2. Design the Data and Grounding Architecture First Enterprise AI is only as useful as the information it can access and trust. For many enterprise scenarios, the challenge isn't simply selecting a powerful model. The challenge is providing the model with the right context. This is where grounding and RAG architectures become important. A typical flow looks like: User → Application → Orchestration → Knowledge/Retrieval → Model → Response But an enterprise implementation also needs to consider: • Data ingestion • Chunking and enrichment • Metadata • Indexing • Access control • Data freshness • Source attribution • Retrieval quality • Auditability Microsoft's current AI architecture guidance explicitly treats the knowledge layer as a core architectural component and emphasizes enforcing data access policies and authorization within that layer. The key architectural question is therefore not: “Can the model answer the question?” It is: “Can the model answer the question using authorized, relevant and trustworthy enterprise data?” 3. Separate Intelligence, Inference, Knowledge and Tools AI applications are becoming more sophisticated. Modern architectures may involve models, agents, orchestration, enterprise data and external tools. Putting everything into one application layer quickly becomes difficult to secure, scale and operate. A better approach is to establish clear architectural boundaries. A useful conceptual model is: Client Layer ↓ Intelligence / Orchestration Layer ↓ Inference Layer ↓ Knowledge Layer ↓ Tools / Business APIs Each layer can have its own: • Identity • Security policies • Scaling strategy • Monitoring • Caching • Failure handling This separation becomes particularly important when moving from a simple chatbot to agentic AI applications. Microsoft's AI application design guidance recommends distinct client, intelligence, inference, knowledge and tools layers for intelligent applications. 4. Treat Security as an Architecture Principle — Not a Checklist AI introduces new security considerations. You need to think beyond traditional application security. Ask: • Who can access the AI application? • What data can the user retrieve? • Can the model access information the user cannot? • How are identities propagated across components? • How are prompts and responses protected? • How are AI tools authorized? • How are sensitive outputs detected? • How are activities audited? One particularly important principle is: The AI system should not become an alternative path around existing enterprise authorization. If an employee cannot access a document directly, the AI assistant should not expose that document through a generated response. Security therefore needs to exist across the entire AI architecture: Identity → Data → Retrieval → Model → Tools → Output 5. Design for Scale and Reliability Before You Need It A PoC might have: 10 users 100 documents 1 model 1 environment Production might have: 100,000 users Millions of documents Multiple models Multiple business applications Continuous availability requirements The architecture must therefore consider: • Horizontal scaling • Availability Zones • Regional resiliency • Load balancing • Model availability • Rate limiting • Failover • Caching • Capacity planning AI workloads also have unique infrastructure considerations. Inference capacity can become a bottleneck, and GPU-based workloads can introduce significant infrastructure costs. Microsoft's current Azure AI architecture guidance recommends designing for scalability and availability across the intelligence, orchestration, inference and knowledge layers. 6. Cost Must Be Designed Into the Architecture AI can create unexpected cost growth. A solution may work perfectly from a technical perspective and still fail the business case because of: • Token consumption • Model selection • GPU utilization • Storage • Data processing • Retrieval infrastructure • Logging • Network traffic • High-frequency inference Therefore, ask: “What is the expected cost per transaction?” Then model: Users × Requests × Tokens × Model Cost But don't stop there. Also evaluate: • Caching opportunities • Model routing • Smaller models for simpler tasks • Batch processing • GPU utilization • Resource scaling • Storage optimization Microsoft's Well-Architected guidance specifically highlights monitoring utilization and avoiding unnecessary AI infrastructure costs. The cheapest architecture isn't necessarily the best architecture. The goal is: Maximum business value per unit of AI spend. 7. Production Requires Continuous Evaluation and Observability Traditional applications usually monitor: CPU Memory Latency Errors Availability AI applications need more. You also need to understand: • Response quality • Grounding accuracy • Retrieval relevance • Hallucination rate • Model performance • Prompt effectiveness • Safety violations • User feedback • Token consumption • Cost per interaction AI is nondeterministic. The same input may not always produce exactly the same output. That means testing cannot simply end when the application goes live. Production evaluation becomes part of the architecture. Microsoft's guidance recommends extending observability to AI-specific quality metrics and supporting testing and evaluation with real production inputs. The Architecture Mindset Shift The biggest transition from PoC to production is not necessarily choosing a better model. It is changing the questions we ask. PoC thinking: “Can AI do it?” Production thinking: “Can the enterprise operate it safely and economically at scale?” That leads to a different architecture conversation: Business Outcome ↓ Data & Grounding ↓ Security & Identity ↓ AI Application Architecture ↓ Infrastructure & Scalability ↓ Observability & Governance ↓ Cost Optimization ↓ Continuous Evaluation My 7-Question Production Readiness Test Before approving an enterprise AI workload for production, I would ask: 1. What measurable business outcome are we delivering? 2. Can we trust and govern the data being used? 3. Can the AI respect existing identity and authorization boundaries? 4. Can every major component scale and recover from failure? 5. Can we measure AI quality—not just infrastructure health? 6. Do we understand the cost at production scale? 7. Can we continuously evaluate, improve and govern the solution? If the answer to several of these is “not yet”, the solution may still be a PoC. And that's perfectly fine. The objective isn't to rush an AI PoC into production. The objective is to build the architecture that makes production possible. Final Thought AI architecture is becoming less about: “Which model should we use?” And increasingly about: “How do we build an AI system that the enterprise can trust?” That is the real journey: PoC → Architecture → Production → Scale → Business Value The organizations that get this architecture right will be in a much stronger position to move from AI experimentation to sustainable enterprise AI adoption. What do you think is the biggest challenge when moving an enterprise AI solution from PoC to production — security, data, scalability, cost, or something else?The Next Generation of Agents with Azure and Microsoft Foundry
Every company has a help desk, and every help desk answers the same twenty questions over and over: my VPN keeps dropping, I lost my MFA device, I need access to the Finance share. Sound familiar? That is exactly what makes it the perfect proving ground for an AI agent — not another chat demo that just talks, but a system that answers from real documentation, knows when it is not allowed to answer, and hands off to humans through a real channel. So that is what we are building today: HelpDesk Copilot, a Contoso IT service desk where a Microsoft Foundry agent triages employee questions, answers them grounded on an IT knowledge base with citations, and — when policy demands a human — creates a ticket that flows asynchronously through Dapr and Azure Service Bus into Table Storage and an Adaptive Card in a Microsoft Teams channel. The whole thing runs on Azure Container Apps, is provisioned entirely with Terraform, and — my favorite part — contains zero API keys. Every service-to-service call, from pulling container images to invoking the Foundry agent, uses Microsoft Entra ID and managed identities. The Foundry account has local key authentication disabled outright. Here is what we will cover: The architecture: three ACA apps, one Foundry Prompt Agent, and an event-driven ticket pipeline Why I chose one agent instead of a multi-agent orchestra — and why that was the honest choice Grounded, streaming answers with Foundry File Search and citations The escalation path: Dapr pub/sub, a Service Bus topic, deterministic ticket IDs, and idempotency The identity model: five managed identities, zero connection strings Terraform notes: the Foundry provider landscape is not what you expect Observability with OpenTelemetry and Foundry's cloud evaluation API Grab a coffee — let's build! The Architecture Three container apps live inside one ACA environment, and each one has a deliberately different network posture: Frontend — React 18 + Vite served by nginx, with external ingress. This is the only public URL: the chat UI and a live ticket panel. API — FastAPI with a Dapr sidecar, internal ingress only. It runs the agent conversation loop, streams Server-Sent Events, executes tools, and publishes ticket events. Ticket worker — FastAPI with a Dapr sidecar and no ingress at all. It exists only to consume Service Bus messages via Dapr, and KEDA wakes it from zero replicas based on subscription backlog. The frontend's nginx proxies browser calls to the API over the ACA environment's internal DNS — the API is never exposed to the internet. Around the environment sit Microsoft Foundry (a Prompt Agent plus a File Search vector store), a Service Bus topic, Table Storage as the ticket read model, Key Vault holding exactly one secret, Azure Container Registry, and Application Insights on a Log Analytics workspace. One Agent, Not an Orchestra My original design called for an orchestrator agent routing to specialist agents. Reality intervened: the Connected Agents pattern I planned to use is deprecated, and the workflow orchestration alternatives are still in preview. I could have demoware'd my way around that — instead, I redesigned around one Foundry Prompt Agent with three capabilities: File Search over the Contoso IT knowledge base, for grounded answers with citations A local create_ticket function tool, for escalation A local get_ticket_status function tool, for lookup Its instructions enforce the policy: search first, cite your source, and only create a ticket when no procedure covers the problem — or when the procedure explicitly requires human intervention (Finance-share access, a lost device, all MFA methods gone). Here is the takeaway I want you to keep: a single well-instructed agent with sharp tools beats a fragile multi-agent mesh for this problem size. Multi-agent is a topology, not a virtue. When the platform's orchestration story stabilizes, this design has an obvious seam to split along — until then, one agent is simpler to reason about, cheaper to run, and easier to evaluate. Similar honesty applies to retrieval: with ten markdown documents, Azure AI Search would be architectural cosplay. Foundry's built-in File Search vector store is the right-sized tool. When the corpus grows into thousands of documents needing hybrid or semantic ranking, that is the upgrade path. The Knowledge Path: Streaming Grounded Answers An employee asks: "My VPN keeps dropping every hour." The flow: The frontend POSTs to /chat with the message and an optional conversation_id The API creates (or continues) a Foundry conversation and requests a streamed response The agent runs File Search over the IT docs and gets relevant chunks back Text deltas and file citation annotations stream back through the API as Server-Sent Events The employee watches the answer type itself out, with the source document cited beneath it Citations are not decoration. In an IT support context, "the answer came from the official VPN procedure" is the difference between a trustworthy assistant and a liability. The frontend de-duplicates cited filenames per answer and shows them inline. The heart of the API is the tool-call loop. When the agent requests a local function, the API executes it and feeds the result back into the same Foundry conversation, so the agent composes the final employee-facing message. The agent stays the author of the conversation; the API stays the executor of side effects. Here is the loop, from agent_service.py: while True: stream = openai.responses.create( input=pending_input, conversation=conversation_id, stream=True, extra_body={"agent_reference": agent_reference}, ) function_outputs = [] for chunk in stream: if chunk.type == "response.output_text.delta": yield {"event": "delta", "data": {"text": chunk.delta}} elif chunk.type == "response.output_item.done": item = chunk.item if item.type == "function_call": yield {"event": "tool_call", "data": {"name": item.name}} output = self._execute_tool(item.name, item.arguments, conversation_id) function_outputs.append({ "type": "function_call_output", "call_id": item.call_id, "output": json.dumps(output), }) if function_outputs: pending_input = function_outputs continue # submit tool outputs and let the agent finish its answer break The Escalation Path: Events, Not Awaits Now the interesting request: "I need Finance-share access for an audit." The knowledge base says restricted Finance access always requires a ticket. The agent emits create_ticket, and this is where the architecture earns its keep: The API validates the tool input and computes a deterministic ticket ID It publishes a ticket.created event via its Dapr sidecar to the Service Bus topic ticket-events — and immediately streams the confirmation with the ticket ID back to the employee Dapr delivers the event to the worker through the ticket-worker subscription The worker upserts the ticket into Table Storage, reads the optional Teams webhook URL from Key Vault, and posts the payload to a Power Automate HTTP flow The IT team gets an Adaptive Card in their Teams channel. A human is now in the loop — a real one. Chat acknowledgement never waits for persistence or Teams delivery. Three deliberate consequences follow. Idempotency end-to-end. The ticket ID is derived, not generated: def compute_ticket_id(conversation_id: str, subject: str) -> str: """Deterministic ticket ID from (conversation, subject) so a repeated create_ticket tool call for the same issue in the same conversation collapses to the same ID instead of creating a duplicate. """ key = f"{conversation_id}:{subject.strip().lower()}" return hashlib.sha256(key.encode("utf-8")).hexdigest()[:16] If the model retries the tool call, or Service Bus redelivers the event (the subscription allows up to 10 deliveries), the worker upserts the same row instead of minting duplicate tickets. Idempotency is designed in at the ID level, not bolted on with dedup logic afterwards. Eventual consistency, explained honestly. The ticket row may not exist for a few seconds while KEDA wakes the worker. Both the agent's status tool and the ticket endpoints treat "not visible yet" as a normal state and say so, and the UI polls every five seconds. Distributed systems do not hide their nature here — they narrate it. A topic, not a queue. Today there is one subscription, so operationally it behaves like a work queue. But "a ticket was created" is an event, and tomorrow an ITSM connector, an audit log, or an analytics pipeline can each get their own subscription without the API changing a single line. Publishers describe facts; subscribers decide what facts mean. This is the payload that travels unchanged from tool call, through Dapr and Service Bus, into the worker, Table Storage, and the Teams flow: { "type": "ticket.created", "ticket_id": "9a549ad5d5f723d4", "conversation_id": "conversation-id", "subject": "Request for access to finance shared drive", "description": "I need access to the finance shared drive for an audit.", "category": "shared-drive-access", "urgency": "high", "requester_email": "email address removed for privacy reasons", "status": "New", "created_at": "2026-07-17T16:30:28.396538+00:00", "updated_at": "2026-07-17T16:30:28.396538+00:00" } And the failure mode is designed too: if the Teams webhook is unset or down, the worker logs a warning and keeps the persisted ticket. Persistence happens first and returns success or retry to Dapr based only on the table write — so a Teams outage cannot cause repeated ticket writes. Zero Keys: The Identity Model This is the part I am proudest of. Every hop authenticates with Entra ID via DefaultAzureCredential — in ACA, AZURE_CLIENT_ID selects each app's user-assigned managed identity; locally, the same code rides on az login. API identity — AcrPull, Foundry agent access, Storage Table Data Reader, Key Vault Secrets User, Service Bus Sender. It invokes the agent, reads tickets, publishes events. Worker identity — AcrPull, Storage Table Data Contributor, Key Vault Secrets User, Service Bus Receiver. The sole writer of tickets. Frontend identity — AcrPull. It pulls its image, nothing more. Shared Dapr identity — Service Bus Data Owner, scoped to authenticating the Dapr component and the KEDA scaler. Notice the reader/writer split: the API physically cannot modify a ticket, and the worker is the only writer. Least privilege is not a slide bullet here; it is enforced by RBAC per identity. The single unavoidable secret — the Power Automate webhook URL, which is bearer-style by nature — lives in Key Vault, and nowhere else. Terraform Notes: The Provider Landscape Is Not What You Expect Terraform is the source of truth for all Azure resources, split into four modules: platform, foundry, observability, and aca. Two lessons here were worth the price of admission. The obvious-looking resources are the wrong ones.When I started, I assumed I would needazapi for the Foundry pieces. The real surprise was different: azurerm 4.x does ship azurerm_ai_foundry and azurerm_ai_foundry_project — but those provision the classic, hub-based Foundry model, not the GA project-based Foundry Agent Service. The current model is provisioned directly on a Cognitive Services account, fully covered by azurerm, no azapi required: resource "azurerm_cognitive_account" "this" { name = "${var.prefix}-${var.environment}-foundry-${var.random_suffix}" resource_group_name = var.resource_group_name location = var.location kind = "AIServices" sku_name = "S0" # Required for the account to work as a Foundry resource # (agents, projects) rather than plain Cognitive Services. custom_subdomain_name = "${var.prefix}-${var.environment}-foundry-${var.random_suffix}" project_management_enabled = true # Enforces "no API keys anywhere" at the account level: only # Entra ID auth is accepted, key-based auth is rejected outright. local_auth_enabled = false identity { type = "SystemAssigned" } } There was one genuine gotcha, though: the built-in Foundry Agent Consumer role grants enough to call the Responses API against an existing thread, but conversations.create() — which the API calls on every new chat — needs the agents/write data action too. I confirmed that live, with a 403 to show for it. The broader Foundry User role would work, but it also grants key-listing and the whole Cognitive Services surface. The fix is a small custom role definition granting exactly the three data actions the chat runtime exercises: interact, agents read, agents write. Least privilege, again. 2. Terraform provisions infrastructure — it does not configure agents. The vector store, document upload, and agent version are deliberately not Terraform resources. A seed_knowledge.py bootstrap runs after terraform apply, authenticated as a principal Terraform granted the Foundry Project Manager role. Agent instructions and knowledge content change on an application cadence, not an infrastructure cadence — mixing the two lifecycles is how you end up re-uploading your knowledge base because you resized a container app. Dapr's entity management is disabled for the same reason: Terraform explicitly owns the topic and subscription. Observability and Evaluation The API initializes Azure Monitor OpenTelemetry, and every Foundry invocation gets a custom agent.invoke span carrying the agent name, conversation ID, selected tools, and input/output token counts when the response exposes them. Platform logs and metrics from all three apps flow to the same Log Analytics workspace. Ask "what did that conversation cost and which tool did it pick?" and App Insights answers. There is also an evaluation script that pushes ten fixed questions through Foundry's cloud evaluation API, scoring intent resolution, coherence, and task adherence. File-search questions evaluate end-to-end; locally executed function tools need a response-capture approach — a limitation worth knowing before you promise your boss automated agent QA. If this section feels short, good — agent observability and governance in production deserves its own post, and it is getting one. Consider this the trailer. What This Deliberately Is Not Honesty section. HelpDesk Copilot is production-shaped, not production-finished: No browser authentication yet. Conversation IDs partition the ticket panel but are not an authorization boundary. A real rollout adds Entra ID sign-in and server-side authorization before exposing ticket data. Tickets stay New. The human handoff is the real Teams notification, not a simulated ITSM lifecycle. Wiring status updates back from an ITSM tool is exactly what the topic's future subscriptions are for. Global ticket lookup scans partitions. Fine at demo volume; a high-volume system adds an index. I would rather ship a clear boundary than a hidden one. Try It The full source — Terraform modules, all three services, the knowledge base, seed and evaluation scripts — is on GitHub: passadis/foundry-ticketing Quickstart: terraform apply, run seed_knowledge.py, build and push the three images, and ask the public URL why your VPN keeps dropping. With scale-to-zero on all three apps and a small model deployment, idle cost is close to nothing — the architecture only spends money when someone needs help. Conclusion The AI part of this solution is maybe twenty percent of the code. The rest is the unglamorous engineering that makes an agent deployable: identity, ingress boundaries, idempotent events, honest eventual consistency, IaC lifecycle boundaries, and telemetry. That ratio is the real lesson — and it is exactly why Azure Container Apps plus Microsoft Foundry is such a productive pairing: the platform absorbs the undifferentiated heavy lifting so the interesting decisions remain yours. Next up in this series: taking the agent.invoke spans further — tracing, token economics, drift, and governance for agents in production with Foundry's control plane and Azure Monitor. Until then — happy building! 🚀Governance Is the New Bottleneck: What Agent 365 Means for Admins ?
Hi all , following up on my last post about token limits, I wanted to write about something that's been on my mind a lot lately: the sheer number of agents quietly showing up across our tenant. Not just the ones we built deliberately in Copilot Studio, but ones people spun up in Power Platform, ones connected through Teams, and a few I genuinely couldn't trace back to an owner when I went looking. That's the moment this topic stopped being theoretical for me. For the last couple of years, the Microsoft AI conversation was mostly about capability -what can Copilot do, which model is better, how do I write a good prompt. That conversation hasn't gone away, but a second one has caught up to it fast: who's actually watching all of this. Microsoft's own 2026 Work Trend Index makes the shift explicit this isn't about saving a few minutes in Outlook anymore, it's about organizations redesigning how work gets divided between people and agents. And the moment agents start acting semi-independently across your tenant, "how many do we have, and what are they allowed to touch" becomes a real operational question, not a hypothetical one. That's exactly the gap Microsoft Agent 365 is built to close. It went generally available on May 1, 2026, alongside Microsoft 365 E7, and I think it's worth understanding properly especially if you're the one who ends up fielding the "wait, there's an agent doing what?" conversation. What Agent 365 Actually Is (and Isn't) The first thing worth clearing up: Agent 365 doesn't build agents. That's still Copilot Studio's job, or Foundry, or whatever platform your team is using. Agent 365 is the layer that sits on top of all of that it's a control plane, not a construction tool. Think of it less like "another AI product" and more like the admin and security backbone that was honestly missing from the picture until now. Microsoft frames it around three pillars: observe, govern, and secure. In practice, that means every agent in your tenant whether it was built in Copilot Studio, imported from AWS or Google Cloud, or even running locally on someone's Windows machine gets registered, gets its own identity through Microsoft Entra, and becomes something you can actually see and act on instead of just hoping it's behaving. That identity piece is the part I think gets underrated. Each agent gets its own Entra Agent ID, the same way a human user would. That's a meaningful shift it means conditional access policies, auditing, and compliance tooling that already exist for people can now extend to agents instead of treating them as some invisible background process. Why This Matters Right Now Here's the honest version of what's been happening across a lot of organizations, including bits of what I've seen firsthand: agent creation has gotten easy. Almost too easy. Between Copilot Studio, Power Platform, and now agentic mode built directly into Word, Excel, and PowerPoint, it doesn't take much for someone in a business unit to spin up something that's technically an AI agent with access to real data without IT or security ever being looped in. Microsoft has been fairly direct about this risk themselves, which I appreciated seeing in writing rather than just implied: the speed of agent development shows real value, but without guardrails, that pace turns into blind spots, lower ROI, and genuine security exposure. That's not vendor fear-mongering, that's just what happens when adoption outpaces oversight in any technology, and agents are no exception. What makes this particular moment different from past "shadow IT" waves is that agents don't just store or move data they can act on it. An agent with the wrong scope isn't just a compliance footnote, it's something that could send an email, modify a file, or trigger a workflow on its own. That's a different risk category than an unsanctioned spreadsheet sitting in someone's OneDrive. What You Actually Get With Agent 365 A few capabilities stood out to me as genuinely useful rather than just checkbox governance: The overview dashboard gives you a real-time view of your entire agent fleet total registered agents, active users, connected platforms, runtime hours, and risk signals, all in one place. Before this, getting even a rough headcount of "how many agents exist in our tenant" was a manual, frustrating exercise. Registry sync extends that visibility beyond Microsoft's own tools. It can pull in agents built on AWS Bedrock and Google Cloud, so you're not stuck with three different governance stories depending on where an agent happens to live. For organizations that are realistically never going to be 100% single-vendor, that matters. Lifecycle actions install, publish, block, unblock, delete, reassign ownership are now available directly from the registry. That's a big deal operationally. Before, tracking down who owned a rogue or abandoned agent could turn into an actual investigation. Now it's a few clicks. Local agent controls through Defender and Intune are rolling in too, extending management down to agents running on individual Windows endpoints, not just cloud-hosted ones. Given how much agent activity is starting to happen at the device level, this closes a gap that would've otherwise been a blind spot. Conditional access for agents, through Entra, means you can apply the same kind of dynamic, granular access policies to agents that you'd apply to a human user which is really the whole philosophical shift Agent 365 represents: agents aren't a separate, ungoverned category anymore, they're first-class identities in your tenant. What This Means for Admins, Practically If you're managing a tenant with any real Copilot or agent activity, here's where I'd actually start: Don't wait for "full autonomy" to engage. It's tempting to think governance can wait until agents are doing something more dramatic than they are today. Microsoft's own guidance pushes against that the advice is to establish visibility and guardrails early, while adoption is still accelerating, not after. Get a real inventory first. Before writing new policies, it's worth just knowing what already exists. I'd genuinely bet most tenants have more agents running than the admin team could name off the top of their head. The overview dashboard is the fastest way to close that gap. Loop in more than just IT. Agent 365 licensing and controls touch the M365 admin center, Entra, Defender, Purview, and Intune which means this isn't a single-team rollout. Security, compliance, and helpdesk all need to understand what's changing, especially the distinction between Frontier (preview, no production SLA) and GA (production-ready, supported). Understand the licensing model before you scope a rollout. Agent 365 is licensed per human user — the person who manages, sponsors, or is served by an agent rather than per agent. It's available standalone at $15 per user per month, or bundled into Microsoft 365 E7. Worth mapping that against your actual agent-using population rather than assuming it's a flat cost per bot. Treat this as incremental, not a one-time setup. Microsoft has said plainly that Agent 365's capabilities will keep evolving as adoption patterns and governance models mature. This isn't a project you finish and close out it's closer to how you'd think about identity and access management generally: ongoing, not a one-time rollout. The Bigger Shift Underneath All This What I find genuinely interesting about Agent 365 is what it signals about where Microsoft thinks this is all heading. They're not just selling a better Copilot anymore they're positioning Microsoft 365 as the place where AI-driven work gets governed, regardless of which vendor's model or platform an agent actually runs on. Whether that's the right long-term answer for every organization is a fair thing to debate. But the underlying problem it's solving that agents were multiplying faster than anyone's ability to see or control them is real, and I don't think it's specific to Microsoft shops. If your organization is building agents in Copilot Studio, experimenting with Foundry, or even just watching Copilot's agentic mode quietly take on more autonomous work in Word and Excel, this is worth getting ahead of. The teams that treat agent governance as a foundational layer now are going to have a much easier time scaling adoption later than the ones who bolt it on after something goes wrong. Curious whether others are already rolling out Agent 365, or still in the "let's figure out how many agents we actually have" phase I suspect a lot of us are somewhere in between. Would love to hear how your organization is approaching this. Cheers, and happy reading. Surya Vennapusa-MCTToken Limit Exceeded? What's Actually Going On and What to Do About It ?
Hi All, Based on some recent experience across the organisation with token limit issues, I wanted to put my thoughts down and actually dig into what's happening under the hood, rather than just chalking it up to "we need a bigger plan." If you work anywhere near the Microsoft ecosystem these days, you're probably touching more AI tools than you realize. Copilot in Word and Excel, GitHub Copilot while you code, Copilot Studio if you're building agents, maybe Security Copilot or Copilot for Sales depending on your role, and increasingly Azure AI Foundry if your team is building anything custom. I work across a good chunk of this stack day to day, and at some point, almost everyone runs into the same wall: "Token limit exceeded." "You've reached your usage limit." "Upgrade to continue." The first instinct is usually to assume you did something wrong wrote too much, uploaded too big a file, or just need a fatter subscription. Sometimes that's the actual story. But honestly, often, that error message is standing in for three completely different problems that all happen to look identical from the outside. One is about how much text a model can physically process at once. One is about your license or credits running dry. And one has nothing to do with size at all it's just about how fast you're sending requests. Once you know which of these three, you're dealing with, the fix becomes obvious. Until then, "upgrade your plan" feels like the only lever you've got even when it isn't. This post walks through what a token is, why Microsoft's various Copilots each handle this differently, and what habits genuinely cut down on these interruptions instead of just throwing money at the problem. Part 1: So What Is a Token, Really? A token isn't a word, and it isn't a character it's somewhere in between. It's the small chunk of text a model's tokenizer breaks your input into before it can do anything with it. Take a word like "unbelievable." A tokenizer might split it into three pieces something like "un," "believ," and "able." Short, everyday words usually come out as a single token. But code, technical jargon, acronyms, and non-English text tend to fragment into a lot more tokens than you'd guess just by looking at the word count. This is why every AI tool has a ceiling on how much it can handle in one go, and that ceiling isn't measured in words or characters it's measured in tokens. Your prompt, any documents or emails it pulls in as context, the back-and-forth history of your conversation, and the response itself all draw from the same pool. Once that pool runs dry, something has to give: the tool truncates, rejects the request outright, or quietly summarizes older context to make room. The part that trips people up: token count doesn't map cleanly to word count. A short, dense paragraph full of code or acronyms can eat up more tokens than a much longer plain-English message. Part 2: Three Different Limits, One Confusing Error Message This isn't always obvious upfront, even to a lot of admins managing these tools: "token limit exceeded" is really a stand-in phrase for three separate limits, and they don't behave the same way. This isn't unique to Microsoft either every major AI platform bundles these same three things behind similarly vague error messages. Microsoft's stack just makes a good case study because so many of us touch multiple pieces of it in the same week. The context window is the ceiling on how much text a specific model can process in a single request everything from your prompt to retrieved documents to chat history. This is tied to the model itself, not your subscription. Swap from one model to another inside the same tool, and this ceiling can move without you doing anything differently. Your license, credits, or feature allowance is a completely separate thing. This is what Microsoft 365 Copilot plans track through AI credits and feature limits, and it's what Copilot Studio measures through Copilot credits at the environment level. A single action summarizing an inbox, generating an agent response, running an analysis deducts from this pool regardless of how small your actual prompt felt. Run out, and you get blocked, even if you're nowhere near any context window limit. The rate limit is about speed, not size. Copilot Studio, for instance, enforces quotas measured in requests per minute or per hour to keep the system stable under load. Send messages too quickly, which happens easily with automations, flows, or bots, and you can get throttled even with a tiny prompt and plenty of credits left. The reason this matters: a plan upgrade only ever fixes the second one. If you're actually running into the model's context window or getting rate-limited, paying for a bigger license won't change anything, and that mismatch is exactly where most of the frustration comes from. Part 3: How This Plays Out Across the Microsoft AI Stack The Microsoft ecosystem isn't one AI tool wearing different outfits it's genuinely several different systems, each handling tokens and limits in its own way. Here's a tour of the ones people run into most. Microsoft 365 Copilot (the one living inside Word, Excel, Outlook, Teams) doesn't work off a single published token number the way a developer tool would. Instead, it dynamically pulls together your prompt, recent chat history, and relevant snippets retrieved from Microsoft Graph your files, emails, and messages and quietly summarizes or drops older material to stay within bounds. Where this usually breaks isn't the context window at all; it's the AI credit and feature-limit system running out, often without much warning until you're mid-task. GitHub Copilot Chat is more like a traditional developer tool. It has a fixed, published token window tied to whichever model you've selected, and that limit applies consistently whether you're in the browser, VS Code, or the CLI. The failure mode here is usually a long conversation or a big multi-file context quietly creeping past that ceiling. Copilot Studio, where a lot of custom agent-building happens, runs on Copilot credits per interaction, plus its own requests-per-minute and requests-per-hour quotas at the environment level. If you're grounding an agent in SharePoint content, there's also a separate file-size ceiling to watch content over a certain size can get silently excluded from generative answers depending on your tenant's licensing. Azure AI Foundry (recently renamed to Microsoft Foundry, in case you've seen both names floating around) is where this gets more directly in your control. If your team is building custom applications on top of Azure OpenAI or other models in the Foundry catalog, which now includes everything from GPT to Phi to Claude to Llama, you're working with explicit, published context windows per model, and you're billed per token rather than per credit. It's a different mental model entirely: less "you hit a wall," more "you're paying by the word, so design accordingly." Security Copilot, if your org uses it for threat analysis and incident response, runs on its own capacity model pooled compute units at the tenant level rather than a simple per-user cap. It's easy to assume this behaves like M365 Copilot license limits; it doesn't. Copilot for Sales, embedded in Outlook and Teams for CRM-connected work, and Copilot in Power BI, which now goes beyond generating summaries to actually helping build and refine semantic models, both draw from their own feature-specific allowances layered on top of whatever base Microsoft 365 or Power Platform license you're on. And then there's the multi-model wrinkle that trips up teams the most: because tools like Copilot Studio and GitHub Copilot let you choose between GPT-based models, Claude, and others, the exact same prompt can have a different effective context window and a different token cost purely based on which model handled it that day. This is a big, underrated reason behind the "it worked fine yesterday, why not now" complaint. Part 4: What Actually Helps ? Some of this is genuinely outside your control, but a fair amount isn't. If you're just using these tools day to day, the single biggest habit shift is not letting conversations run forever. Long threads in Copilot Chat or Copilot Studio keep accumulating history, and that history eats into the same budget as whatever you're asking right now. Starting fresh periodically costs you nothing and buys back a lot of headroom. Large documents are worth splitting up before you feed them in, especially for SharePoint-grounded agents, where oversized files can get quietly excluded rather than cleanly rejected you won't necessarily know it happened unless you're looking for it. And it's worth resisting the urge to default to the heaviest, most capable model for every single task. Lighter models are usually faster, cheaper, and often sit under a more generous limit than the flagship ones, and most everyday tasks genuinely don't need the biggest model available. Before you go asking IT for a license upgrade, it's worth a quick sanity check on which limit you actually hit. If it's a rate limit, waiting a minute and retrying usually solves it outright. If it's a context window problem, trimming your prompt or starting a new session fixes it. An upgrade only helps if you've genuinely run out of credits or feature allowance, and that's worth confirming before you file the request. If you're on the building side Copilot Studio agents, Foundry applications, anything with RAG-style grounding a couple of things pay off quickly. Keep an eye on credit or token consumption proactively rather than discovering it's gone when the agent goes down mid-conversation. Be deliberate about what goes into system prompts and orchestration instructions, since those draw from the same budget as the end user's actual message, often invisibly to whoever's chatting with the agent. And spend real time getting chunk size right for knowledge sources too large and you're burning budget on irrelevant context, too small and the agent loses the thread. Part 5: Quick Checklist Before You Escalate Is this actually a context window problem -prompt, history, and attachments too big for the model in use? Have you genuinely run out of credits or feature allowance on your plan? Could this be a rate limit -too many requests too fast, especially from a flow or automation? Did the underlying model change since last time, quietly shifting the effective window? For Studio or Foundry work, is this a tenant or environment-level limit rather than something tied to you personally? Closing Thoughts Tokenization is one of those things that stays completely invisible right up until it isn't. Across a stack as sprawling as Microsoft's M365 Copilot, GitHub Copilot, Copilot Studio, Foundry, Security Copilot, and everything layered on top "token limit exceeded" almost never means one single thing. It means you've hit one of three very different walls, and each one needs a different response. If your team builds or maintains any of these tools, this is genuinely worth putting in front of people early. Most of the "why did this break" tickets in this space aren't about tokens at all. They're about nobody knowing which limit actually got hit, or where in this increasingly large ecosystem it happened. I'm curious how this shows up for others has your team standardized on one model across these tools, or are you juggling several depending on the task? I'd love to hear what patterns you've run into. Cheers, and happy reading. - By Surya Vennapusa, MCTHow can you stay competitive and relevant in an AI-Driven World?
In a world where AI tools evolve weekly and yesterday's skills can feel obsolete overnight, this blog offers a grounded, human-first guide for cloud and technology professionals who want to stay ahead not by chasing every trend, but by building the right foundations. Across six core themes, the post walks readers through understanding what AI truly changes in the workplace, committing to deliberate and structured learning through platforms like Microsoft Learn, getting hands-on with real Azure AI projects beyond just certifications, and doubling down on the human skills critical thinking, communication, and ethical judgment that AI simply cannot replicate. The blog also makes the case for community and network as a long-term career asset, and closes with a call to develop an AI mindset rooted in curiosity, adaptability, and a willingness to experiment and share openly. Whether you're a cloud architect, a security professional preparing for AZ-500 or SC-200, or simply someone navigating what this AI shift means for your career this post is written for you. Key Takeaways for Readers: Understand AI's real impact · Build a deliberate learning habit · Go hands-on with Azure AI tools · Strengthen human skills · Invest in community · Cultivate an AI-first mindsetBuilding an Agentic, AI-Powered Helpdesk with Agents Framework, Azure, and Microsoft 365
The article describes how to build an agentic, AI-powered helpdesk using Azure, Microsoft 365, and the Microsoft Agent Framework. The goal is to automate ticket handling, enrich requests with AI, and integrate seamlessly with M365 tools like Teams, Planner, and Power Automate.Azure Innovators Hub: Data & Intelligence MVP Live 2026
Where leading Data MVPs shape the future of Cloud, AI, and Intelligent Systems The Azure Innovators Hub presents a premier online gathering featuring five distinguished Microsoft MVPs from the Data Platform ecosystem, each offering expertise in Cloud Architecture, AI/ML, Governance, and Analytics. This summit examines how modern data engineering, responsible AI, and scalable cloud practices intersect to drive the next wave of intelligent solutions. Attendees can expect expert insights, proven patterns, and practical experience from professionals immersed in data at scale. No Registration required Azure Innovators Hub https://www.meetup.com/athens-azure-tech-group/
