Apps on Azure Blog articles

AKS App Routing's Next Chapter: Gateway API with Istio

samcogan — Sun, 19 Apr 2026 18:25:14 GMT

If you've been following my previous posts on the Ingress NGINX retirement, you'll know the story so far. The community Ingress NGINX project was retired in March 2026, and Microsoft's extended support for the NGINX-based App Routing add-on runs until November 2026. I've covered migrating from standalone NGINX to the App Routing add-on to buy time, and migrating to Application Gateway for Containers as a long-term option. In both of those posts I mentioned that Microsoft was working on a new version of the App Routing add-on based on Istio and the Gateway API. Well, it's here, in preview at least.

The App Routing Gateway API implementation is Microsoft's recommended migration path for anyone currently using the NGINX-based App Routing add-on. It moves you off NGINX entirely and onto the Kubernetes Gateway API, with a lightweight Istio control plane handling the gateway infrastructure under the hood. Let's look at what this actually is, how it differs from other options, and how to migrate from both standalone NGINX and the existing App Routing add-on.

What Is It?

The new App Routing mode uses the Kubernetes Gateway API instead of the Ingress API. When you enable the add-on, AKS deploys an Istio control plane (istiod) to manage Envoy-based gateway proxies. The important thing to understand here is that this is not the full Istio service mesh. There's no sidecar injection, no Istio CRDs installed for your workloads. It's Istio doing one specific job: managing gateway proxies for ingress traffic.

When you create a Gateway resource, AKS provisions an Envoy Deployment, a LoadBalancer Service, a HorizontalPodAutoscaler (defaulting to 2-5 replicas at 80% CPU), and a PodDisruptionBudget. All managed. You write Gateway and HTTPRoute resources, and AKS handles everything else.

This is a fundamentally different API from what you're used to with Ingress. Instead of a single Ingress resource that combines the entry point and routing rules, Gateway API splits things into layers:

GatewayClass defines the type of gateway infrastructure (provided by AKS in this case)
Gateway creates the actual gateway with its listeners
HTTPRoute defines the routing rules and attaches to a Gateway

This separation is one of Gateway API's main selling points. Platform teams can own the Gateway resources while application teams manage their own HTTPRoutes independently, without needing to modify shared infrastructure. If you've ever had a team accidentally break routing for everyone by editing a shared Ingress, you'll appreciate why this matters.

How It Differs From the Istio Service Mesh Add-On

If you're already running or considering the Istio service mesh add-on for AKS, this is a different thing.

The App Routing Gateway API mode uses the approuting-istio GatewayClass, doesn't install Istio CRDs, doesn't enable sidecar injection, and handles upgrades in-place. The full Istio service mesh add-on uses the istio GatewayClass, installs Istio CRDs cluster-wide, enables sidecar injection, and uses canary upgrades for minor versions.

The two cannot run at the same time. If you have the Istio service mesh add-on enabled, you need to disable it before enabling App Routing Gateway API (and vice versa). If you need full mesh capabilities like mTLS between services, traffic policies, and telemetry, stick with the Istio service mesh add-on. If you just need managed ingress via Gateway API without the mesh overhead, this is the right choice.

Current Limitations

The new App Routing solution is in preview, so should not be run in production yet. There are also some gaps compared to the existing add-on, which you need to be aware of before planning a production migration.

The biggest one: DNS and TLS certificate management via the add-on isn't supported yet for Gateway API. If you're currently using az aks approuting update and az aks approuting zone add to automate Key Vault and Azure DNS integration with the NGINX-based add-on, that workflow doesn't carry over. TLS termination is still possible, but you'll need to set it up manually. The AKS docs cover the steps, but it's more hands-on than what the NGINX add-on gives you today. This is expected to be addressed when the feature reaches GA.

SNI passthrough (TLSRoute) and egress traffic management aren't supported either. And as mentioned, it's mutually exclusive with the Istio service mesh add-on.

For production workloads that depend heavily on automated DNS and TLS management, you may want to wait until GA, or look at Application Gateway for Containers as an alternative. But for teams that can handle TLS setup manually, for non-production environments, there's no reason not to start testing this now.

Getting Started

Before you can enable the feature, you need the aks-preview CLI extension (version 19.0.0b24 or later), the Managed Gateway API CRDs enabled, and the App Routing Gateway API preview feature flag registered:

az extension add --name aks-preview az extension update --name aks-preview # Managed Gateway API CRDs (required dependency) az feature register --namespace "Microsoft.ContainerService" --name "ManagedGatewayAPIPreview" # App Routing Gateway API implementation az feature register --namespace "Microsoft.ContainerService" --name "AppRoutingIstioGatewayAPIPreview"

Feature flag registration can take a few minutes. Once they're registered, enable the add-on on a new or existing cluster. You need both --enable-gateway-api (for the managed Gateway API CRD installation) and --enable-app-routing-istio (for the Istio-based implementation):

# New cluster az aks create \ --resource-group ${RESOURCE_GROUP} \ --name ${CLUSTER} \ --location swedencentral \ --enable-gateway-api \ --enable-app-routing-istio # Existing cluster az aks update \ --resource-group ${RESOURCE_GROUP} \ --name ${CLUSTER} \ --enable-gateway-api \ --enable-app-routing-istio

Verify istiod is running:

kubectl get pods -n aks-istio-system

You should see two istiod pods in a Running state.

From here, you can create a Gateway and HTTPRoute to test traffic flow. The AKS quickstart walks through this with the httpbin sample app if you want a quick validation.

Migrating From NGINX Ingress

Whether you're running standalone NGINX (self-installed via Helm) or the NGINX-based App Routing add-on, the migration process is essentially the same. You're moving from Ingress API resources to Gateway API resources, and the new controller runs alongside your existing one during the transition. The only real differences are what you're cleaning up at the end and, if you're on the App Routing add-on, whether you were relying on its built-in DNS and TLS automation.

Inventory Your Ingress Resources

Before anything else, understand what you have:

kubectl get ingress --all-namespaces \ -o custom-columns='NAMESPACE:.metadata.namespace,NAME:.metadata.name,CLASS:.spec.ingressClassName'

Look specifically for custom snippets, lua configurations, or anything that relies heavily on NGINX-specific behaviour. These won't have direct equivalents in Gateway API and will need manual attention.

Convert Ingress Resources to Gateway API

The ingress2gateway tool (v1.0.0) handles conversion of Ingress resources to Gateway API equivalents. It supports over 30 common NGINX annotations and generates Gateway and HTTPRoute YAML. It works regardless of whether your Ingress resources use the nginx or webapprouting.kubernetes.azure.com IngressClass:

# Install go install github.com/kubernetes-sigs/ingress2gateway@v1.0.0 # Convert from live cluster ingress2gateway print --providers=ingress-nginx -A > gateway-resources.yaml # Or convert from a local file ingress2gateway print --providers=ingress-nginx --input-file=./manifests/ingress.yaml > gateway-resources.yaml

Review the output carefully. The tool flags annotations it can't convert as comments in the generated YAML, so you'll know exactly what needs manual work. Common gaps include custom configuration snippets and regex-based rewrites that don't map cleanly to Gateway API's routing model.

Make sure you update the gatewayClassName in the generated Gateway resources to approuting-istio. The tool may generate a generic GatewayClass name that you'll need to change.

Handle DNS and TLS

If you're coming from standalone NGINX, you're likely managing DNS and TLS yourself already, so nothing changes here: just make sure your certificate Secrets and DNS records are ready for the new Gateway IP.

If you're coming from the App Routing add-on and relying on its built-in DNS and TLS management (via az aks approuting zone add and Key Vault integration), this is the part that needs extra thought. That automation doesn't carry over to the Gateway API implementation yet, so you'll need to handle it differently until GA.

For TLS, you can either create Kubernetes Secrets with your certificates manually or set up a workflow to sync them from Key Vault. The AKS docs on securing Gateway API traffic cover the manual approach. For DNS, you'll need to manage records yourself or use ExternalDNS to automate it. ExternalDNS supports Gateway API resources, so this is a viable path if you want automation.

Deploy and Validate

With the add-on enabled, apply your converted resources:

kubectl apply -f gateway-resources.yaml

Wait for the Gateway to be programmed and get the external IP:

kubectl wait --for=condition=programmed gateways.gateway.networking.k8s.io <gateway-name> export GATEWAY_IP=$(kubectl get gateways.gateway.networking.k8s.io <gateway-name> -ojsonpath='{.status.addresses[0].value}')

The key thing here is that your existing NGINX controller (whether standalone or add-on managed) is still running and serving production traffic. The Gateway API resources are handled separately by the Istio-based controller in aks-istio-system. This parallel running is what makes the migration safe.

Test your routes against the new Gateway IP, you'll need to provide the appropriate URL as a host header, as your DNS will still be pointing at the NGINX Add-On at this point.

curl -H "Host: myapp.example.com" http://$GATEWAY_IP

Run your full validation suite. Check TLS, path routing, headers, authentication, anything your applications depend on. Take your time here; nothing changes for production until you update DNS.

Cut Over DNS and Clean Up

Once you're confident, lower your DNS TTL to 60 seconds (do this well in advance), then update your DNS records to point to the new Gateway IP. Keep the old NGINX controller running for 24-48 hours as a rollback option.

After traffic has been flowing cleanly through the Gateway API path, clean up the old setup. What this looks like depends on where you started:

If you were on standalone NGINX:

helm uninstall ingress-nginx -n ingress-nginx kubectl delete namespace ingress-nginx

If you were on the App Routing add-on with NGINX:

Verify nothing is still using the old IngressClass:

kubectl get ingress --all-namespaces \ -o custom-columns='NAMESPACE:.metadata.namespace,NAME:.metadata.name,CLASS:.spec.ingressClassName' \ | grep "webapprouting"

Delete any remaining Ingress resources that reference the old class, then disable the NGINX-based App Routing add-on:

az aks approuting disable --resource-group ${RESOURCE_GROUP} --name ${CLUSTER}

Some resources (configMaps, secrets, and the controller deployment) will remain in the app-routing-system namespace after disabling. You can clean these up by deleting the namespace once you're satisfied everything is running through the Gateway API path:

kubectl delete ns app-routing-system

In both cases, clean up any old Ingress resources that are no longer being used.

Upgrades and Lifecycle

The Istio control plane version is tied to your AKS cluster's Kubernetes version. AKS automatically handles patch upgrades as part of its release cycle, and minor version upgrades happen in-place when you upgrade your cluster's Kubernetes version or when a new Istio minor version is released for your AKS version.

One thing to be aware of - unlike the Istio service mesh add-on, upgrades here are in-place, not canary-based. The HPA and PDB on each Gateway help minimise disruption, but plan accordingly for production. If you have maintenance windows configured, the istiod upgrades will respect them.

What Should You Do Now?

The timeline hasn't changed. The standalone NGINX Ingress project was retired in March 2026, so if you're still running that, you're already on unsupported software. The NGINX App Routing add-on is supported until November 2026, which gives you a window, but it's not a long one.

If you're on standalone NGINX you could get onto the App Routing add-on now to buy time (I covered this in my earlier post), then plan your migration to either the Gateway API mode or AGC.

If you're on the NGINX App Routing add-on: start testing the Gateway API mode in non-production now. Get familiar with the Gateway API resource model, understand the TLS and DNS gaps in the preview, and be ready to migrate when the feature reaches GA or when November gets close, whichever comes first.

If you need production-ready TLS and DNS automation today and can't wait for GA, App Gateway for Containers is your best option right now.

Whatever path you choose, make sure you have a plan in place before November. Running unsupported ingress software on production infrastructure isn't where you want to be.

Autonomous AKS Incident Response with Azure SRE Agent: From Alert to Verified Recovery in Minutes

hailukassa — Mon, 20 Apr 2026 18:07:34 GMT

When a Sev1 alert fires on an AKS cluster, detection is rarely the hard part. The hard part is what comes next: proving what broke, why it broke, and fixing it without widening the blast radius, all under time pressure, often at 2 a.m.

Azure SRE Agent is designed to close that gap. It connects Azure-native observability, AKS diagnostics, and engineering workflows into a single incident-response loop that can investigate, remediate, verify, and follow up, without waiting for a human to page through dashboards and run ad-hoc kubectl commands.

This post walks through that loop in two real AKS failure scenarios. In both cases, the agent received an incident, investigated Azure Monitor and AKS signals, applied targeted remediation, verified recovery, and created follow-up in GitHub, all while keeping the team informed in Microsoft Teams.

Core concepts

Azure SRE Agent is a governed incident-response system, not a conversational assistant with infrastructure access. Five concepts matter most in an AKS incident workflow:

Incident platform. Where incidents originate. In this demo, that is Azure Monitor.
Built-in Azure capabilities. The agent uses Azure Monitor, Log Analytics, Azure Resource Graph, Azure CLI/ARM, and AKS diagnostics without requiring external connectors.
Connectors. Extend the workflow to systems such as GitHub, Teams, Kusto, and MCP servers.
Permission levels. Reader for investigation and read oriented access, privileged for operational changes when allowed.
Run modes. Review for approval-gated execution and Autonomous for direct execution.

The most important production controls are permission level and run mode, not prompt quality. Custom instructions can shape workflow behavior, but they do not replace RBAC, telemetry quality, or tool availability.

The safest production rollout path:

Start: Reader + Review Then: Privileged + Review Finally: Privileged + Autonomous. Only for narrow, trusted incident paths.

Demo environment

The full scripts and manifests are available if you want to reproduce this:

Demo repository: github.com/hailugebru/azure-sre-agents-aks. The README includes setup and configuration details.

The environment uses an AKS cluster with node auto-provisioning (NAP), Azure CNI Overlay powered by Cilium, managed Prometheus metrics, the AKS Store sample microservices application, and Azure SRE Agent configured for incident-triggered investigation and remediation. This setup is intentionally realistic but minimal. It provides enough surface area to exercise real AKS failure modes without distracting from the incident workflow itself.

Azure Monitor → Action Group → Azure SRE Agent → AKS Cluster (Alert) (Webhook) (Investigate / Fix) (Recover) ↓ Teams notification + GitHub issue → GitHub Agent → PR for review

How the agent was configured

Configuration came down to four things: scope, permissions, incident intake, and response mode. I scoped the agent to the demo resource group and used its user-assigned managed identity (UAMI) for Azure access. That scope defined what the agent could investigate, while RBAC determined what actions it could take.

I used broader AKS permissions than I would recommend as a default production baseline so the agent could complete remediation end to end in the lab. That is an important distinction: permissions control what the agent can access, while run mode controls whether it asks for approval or acts directly. For this scenario, Azure Monitor served as the incident platform, and I set the response plan to Autonomous for a narrow, trusted path so the workflow could run without manual approval gates.

I also added Teams and GitHub integrations so the workflow could extend beyond Azure. Teams provided milestone updates during the incident, and GitHub provided durable follow up after remediation. For the complete setup, see the README.

A note on context. The more context you can provide the agent about your environment, resources, runbooks, and conventions, the better it performs. Scope boundaries, known workloads, common failure patterns, and links to relevant documentation all sharpen its investigations and reduce the time it spends exploring. Treat custom instructions and connector content as first-class inputs, not afterthoughts.

Two incidents, two response modes

These incidents occurred on the same cluster in one session and illustrate two realistic operating modes:

Alert triggered automation. The agent acts when Azure Monitor fires.
Ad hoc chat investigation. An engineer sees a symptom first and asks the agent to investigate.

Both matter in real environments. The first is your scale path. The second is your operator assist path.

Incident 1. CPU starvation (alert driven, ~8 min MTTR)

The makeline-service deployment manifest contained a CPU and memory configuration that was not viable for startup:

resources: requests: cpu: 1m memory: 6Mi limits: cpu: 5m memory: 20Mi

Within five minutes, Azure Monitor fired the pod-not-healthy Sev1 alert. The agent picked it up immediately.

Here is the key diagnostic conclusion the agent reached from the pod state, probe behavior, and exit code:

"Exit code 1 (not 137) rules out OOMKill. The pod failed at startup, not at runtime memory pressure. CPU limit of 5m is insufficient for the process to bind its port before the startup probe times out. This is a configuration error, not a resource exhaustion scenario."

That is the kind of distinction that often takes an on call engineer several minutes to prove under pressure: startup failure from CPU starvation vs. runtime termination from memory pressure.

The agent then:

Identified three additional CPU-throttled pods at 112 to 200% of configured limit using kubectl top.
Patched four workloads: makeline-service, virtual-customer, virtual-worker, and mongodb.
Verified that all affected pods returned to healthy running state with 0 restarts cluster wide.

Azure SRE Agent's Incident History blade confirming full cluster recovery: 4 patches applied, 0 unhealthy pods — no human intervention required.

Outcome. Full cluster recovery in ~8 minutes, 0 human interventions.

Incident 2. OOMKilled (chat driven, ~4 min MTTR)

For the second case, I deployed a deliberately undersized version of order-service:

kubectl apply -f .\manifests\aks-store\order-service-changed.yaml -n pets

I started this case from chat before the pod-phase alert fired to demonstrate the interactive troubleshooting flow. That was a demo choice, not an alerting gap. CrashLoopBackOff is a container waiting reason, not a pod phase, so production coverage should come from Prometheus based crash-loop signals rather than pod phase alone. Here is the PromQL query I use in Azure Monitor to catch this class of failure:

sum by (namespace, pod) ( ( max_over_time( kube_pod_container_status_waiting_reason{ namespace="pets", reason="CrashLoopBackOff" }[5m] ) == 1 ) and on (namespace, pod, container) ( increase( kube_pod_container_status_restarts_total{ namespace="pets" }[15m] ) > 0 ) ) > 0

This query fires when a container has been in CrashLoopBackOff within the last 5 minutes and its restart count has increased in the last 15 minutes. In production, replace the hardcoded namespace with a regex matcher or remove it entirely to cover all namespaces.

The order-service pod in the pets namespace is not healthy. Please investigate, identify the root cause, and fix it.

The agent's reasoning:

"Container logs are empty. The process was killed before it could write its first log line. Exit code 137 confirms OOMKill. No NODE_OPTIONS in the ConfigMap rules out a V8 heap misconfiguration. The 20Mi limit is 12.8x below the pod's observed 50Mi runtime baseline. This limit was never viable for this workload."

The agent increased the memory limit (20Mi to 128Mi) and request (10Mi to 50Mi), then verified the new pod stabilized at 74Mi/128Mi (58% utilization) with 0 restarts.

Outcome. Service recovered in ~4 minutes without any manual cluster interaction.

Side by side comparison

Dimension	Incident 1: CPU starvation	Incident 2: OOMKilled
Trigger	Azure Monitor alert (automated)	Engineer chat prompt (ad hoc)
Failure mode	CPU too low for startup probe to pass	Memory limit too low for process to start
Key signal	Exit code 1, probe timeout	Exit code 137, empty container logs
Blast radius	4 workloads affected cluster wide	1 workload in target namespace
Remediation	CPU request/limit patches across 4 deployments	Memory request/limit patch on 1 deployment
MTTR	~8 min	~4 min
Human interventions	0	0

Why this matters

Most AKS environments already emit rich telemetry through Azure Monitor and managed Prometheus. What is still manual is the response: engineers paging through dashboards, running ad-hoc kubectl commands, and applying hotfixes under time pressure. Azure SRE Agent changes that by turning repeatable investigation and remediation paths into an automated workflow.

The value isn't just that the agent patched a CPU limit. It's that the investigation, remediation, and verification loop is the same regardless of failure mode, and it runs while your team sleeps.

In this lab, the impact was measurable:

Metric	This demo with Azure SRE Agent
Alert to recovery	~4 to 8 min
Human interventions	0
Scope of investigation	Cluster wide, automated
Correlate evidence and diagnose	~2 min
Apply fix and verify	~4 min
Post incident follow-up	GitHub issue + draft PR

These results came from a controlled run on April 10, 2026. Real world outcomes depend on alert quality, cluster size, and how much automation you enable. For reference, industry reports from PagerDuty and Datadog typically place manual Sev1 MTTR in the 30 to 120 minute range for Kubernetes environments.

Teams + GitHub follow-up

Runtime remediation is only half the story. If the workflow ends when the pod becomes healthy again, the same issue returns on the next deployment. That is why the post incident path matters.

After Incident 1 resolved, Azure SRE Agent used the GitHub connector to file an issue with the incident summary, root cause, and runtime changes. In the demo, I assigned that issue to GitHub Copilot agent, which opened a draft pull request to align the source manifests with the hotfix. The agent can also be configured to submit the PR directly in the same workflow, not just open the issue, so the fix is in your review queue by the time anyone sees the notification. Human review still remains the final control point before merge. Setup details for the GitHub connector are in the demo repo README, and the official reference is in the Azure SRE Agent docs.

Azure SRE Agent fixes the live issue, and the GitHub follow-up prepares the durable source change so future deployments do not reintroduce the same configuration problem.

The operations to engineering handoff: Azure SRE Agent fixed the live cluster; GitHub Copilot agent prepares the durable source change so the same misconfiguration can't ship again.

In parallel, the Teams connector posted milestone updates during the incident:

Investigation started.
Root cause and remediation identified.
Incident resolved.

Teams handled real time situational awareness. GitHub handled durable engineering follow-up. Together, they closed the gap between operations and software delivery.

Key takeaways

Three things to carry forward

Treat Azure SRE Agent as a governed incident response system, not a chatbot with infrastructure access. The most important controls are permission levels and run modes, not prompt quality.
Anchor detection in your existing incident platforms. For this demo, we used Prometheus and Azure Monitor, but the pattern applies regardless of where your signals live. Use connectors to extend the workflow outward. Teams for real time coordination, GitHub for durable engineering follow-up.
Start where you're comfortable. If you are just getting your feet wet, begin with one resource group, one incident type, and Review mode. Validate that telemetry flows, RBAC is scoped correctly, and your alert rules cover the failure modes you actually care about before enabling Autonomous. Expand only once each layer is trusted.

Next steps

Add Prometheus based alert coverage for ImagePullBackOff and node resource pressure to complement the pod phase rule.
Expand to multi cluster managed scopes once the single cluster path is trusted and validated.
Explore how NAP and Azure SRE Agent complement each other — NAP manages infrastructure capacity, while the agent investigates and remediates incidents.

I'd like to thank Cary Chai, Senior Product Manager for Azure SRE Agent, for his early technical guidance and thorough review — his feedback sharpened both the accuracy and quality of this post.

New in Azure SRE Agent: Log Analytics and Application Insights Connectors

Dalibor_Kovacevic — Fri, 17 Apr 2026 02:14:28 GMT

Azure SRE Agent now supports Log Analytics and Application Insights as log providers, backed by the Azure MCP Server. Connect your workspaces and App Insights resources, and the agent can query them directly during investigations.

Why This Matters

Log Analytics and Application Insights are common destinations for Azure operational data - container logs, application traces, dependency failures, security events. The agent could already access this data through az monitor CLI commands if you granted RBAC roles to its managed identity, and that approach still works. But it required manual RBAC setup and the agent had to shell out to CLI for every query.

With these connectors, setup is simpler and querying is faster. You pick a workspace, we handle the RBAC grants, and the agent gets native MCP-backed query tools instead of going through CLI.

What You Get

Two new connector types in Builder > Connectors (or through the onboarding flow under Logs):

Log Analytics - connect a workspace. The agent can query ContainerLog, Syslog, AzureDiagnostics, KubeEvents, SecurityEvent, custom tables, anything in that workspace.
Application Insights - connect an App Insights resource. The agent gets access to requests, dependencies, exceptions, traces, and custom telemetry.

New Connectors during onboarding.

You can connect multiple workspaces and App Insights resources. The agent knows which ones are available and targets the right one based on the investigation.

Setup

If you want early access, please enable: Early access to features under Settings > Basics.

Early access to features

From there you can add connectors in two ways:

Through onboarding: Click Logs in the onboarding flow, then select Log Analytics Workspace or Application Insights under Additional connectors.

Through Builder: Go to Builder > Connectors in the sidebar and add a Log Analytics or Application Insights connector.

Pick your resource from the dropdown and save. If discovery doesn't find your resource, both connector types have a manual entry fallback.

On save, we grant the agent's managed identity Log Analytics Reader and Monitoring Reader on the target resource group. If your account can't assign roles, you can grant them separately.

Backed by Azure MCP

Under the hood, this uses the Azure MCP Server with the monitor namespace. When you save your first connector, we spin up an MCP server instance automatically. The agent gets access to tools like:

monitor_workspace_log_query - KQL against a workspace
monitor_resource_log_query - KQL against a specific resource
monitor_workspace_list - discover workspaces
monitor_table_list - list tables in a workspace

Everything is read-only. The agent can query but never modify your monitoring configuration.

If different connectors use different managed identities, the system handles per-call identity routing automatically.

What It Looks Like

An alert fires on your AKS cluster. The agent starts investigating and queries your connected workspace:

The agent also ships with built-in skills for common Log Analytics and App Insights query patterns, so it knows which tables to look at and how to structure queries for typical failure scenarios.

Things to Know

Read-only - the agent can query data but cannot modify alerts, retention, or workspace config
Resource discovery needs Reader - the dropdown uses Azure Resource Graph. If your resources don't show up, use the manual entry fallback
One identity per connector - if workspaces need different managed identities, create separate connectors

Learn More

We'd love feedback. Try it out and let us know what works and what doesn't.

Azure SRE Agent is generally available. Learn more at sre.azure.com/docs.

Event-Driven IaC Operations with Azure SRE Agent: Terraform Drift Detection via HTTP Triggers

Vineela-Suri — Fri, 17 Apr 2026 02:13:18 GMT

What Happens After terraform plan Finds Drift?

If your team is like most, the answer looks something like this:

A nightly terraform plan runs and finds 3 drifted resources
A notification lands in Slack or Teams
Someone files a ticket
During the next sprint, an engineer opens 4 browser tabs — Terraform state, Azure Portal, Activity Log, Application Insights — and spends 30 minutes piecing together what happened
They discover the drift was caused by an on-call engineer who scaled up the App Service during a latency incident at 2 AM
They revert the drift with terraform apply
The app goes down because they just scaled it back down while the bug that caused the incident is still deployed

Step 7 is the one nobody talks about. Drift detection tooling has gotten remarkably good — scheduled plans, speculative runs, drift alerts — but the output is always the same: a list of differences. What changed. Not why. Not whether it's safe to fix.

The gap isn't detection. It's everything that happens after detection.

HTTP Triggers in Azure SRE Agent close that gap. They turn the structured output that drift detection already produces — webhook payloads, plan summaries, run notifications — into the starting point of an autonomous investigation. Detection feeds the agent. The agent does the rest: correlates with incidents, reads source code, classifies severity, recommends context-aware remediation, notifies the team, and even ships a fix.

Here's what that looks like end to end.

What you'll see in this blog:

An agent that classifies drift as Benign, Risky, or Critical — not just "changed"

Incident correlation that links a SKU change to a latency spike in Application Insights

A remediation recommendation that says "Do NOT revert" — and why reverting would cause an outage

A Teams notification with the full investigation summary

An agent that reviews its own performance, finds gaps, and improves its own skill file

A pull request the agent created on its own to fix the root cause

The Pipeline: Detection to Resolution in One Webhook

The architecture is straightforward. Terraform Cloud (or any drift detection tool) sends a webhook when it finds drift. An Azure Logic App adds authentication. The SRE Agent's HTTP Trigger receives it and starts an autonomous investigation.

Terraform Drift Detection Pipeline Architecture — Terraform Cloud sends a webhook to the Azure Logic App auth bridge, which forwards an authenticated request to the SRE Agent HTTP Trigger. The agent then performs drift detection, incident correlation, source code analysis, smart remediation, Teams notification, self-improving skill updates, and auto PR creation.The end-to-end pipeline: Terraform Cloud detects drift and sends a webhook. The Logic App adds Azure AD authentication via Managed Identity. The SRE Agent's HTTP Trigger fires and the agent autonomously investigates across 7 dimensions.

Setting Up the Pipeline

Step 1: Deploy the Infrastructure with Terraform

We start with a simple Azure App Service running a Node.js application, deployed via Terraform. The Terraform configuration defines the desired state:

App Service Plan: B1 (Basic) — single vCPU, ~$13/mo
App Service: Node 20-lts with TLS 1.2
Tags: environment: demo, managed_by: terraform, project: sre-agent-iac-blog

resource "azurerm_service_plan" "demo" { name = "iacdemo-plan" resource_group_name = azurerm_resource_group.demo.name location = azurerm_resource_group.demo.location os_type = "Linux" sku_name = "B1" }

A Logic App is also deployed to act as the authentication bridge between Terraform Cloud webhooks and the SRE Agent's HTTP Trigger endpoint, using Managed Identity to acquire Azure AD tokens. Learn more about HTTP Triggers here.

Step 2: Create the Drift Analysis Skill

Skills are domain knowledge files that teach the agent how to approach a problem. We create a terraform-drift-analysis skill with an 8-step workflow:

Identify Scope — Which resource group and resources to check
Detect Drift — Compare Terraform config against Azure reality
Correlate with Incidents — Check Activity Log and App Insights
Classify Severity — Benign, Risky, or Critical
Investigate Root Cause — Read source code from the connected repository
Generate Drift Report — Structured summary with severity-coded table
Recommend Smart Remediation — Context-aware: don't blindly revert
Notify Team — Post findings to Microsoft Teams

The key insight in the skill: "NEVER revert critical drift that is actively mitigating an incident." This teaches the agent to think like an experienced SRE, not just a diff tool.

Step 3: Create the HTTP Trigger

In the SRE Agent UI, we create an HTTP Trigger named tfc-drift-handler with a 7-step agent prompt:

A Terraform Cloud run has completed and detected infrastructure drift. Workspace: {payload.workspace_name} Organization: {payload.organization_name} Run ID: {payload.run_id} Run Message: {payload.run_message} STEP 1 — DETECT DRIFT: Compare Terraform configuration against actual Azure state... STEP 2 — CORRELATE WITH INCIDENTS: Check Azure Activity Log and App Insights... STEP 3 — CLASSIFY SEVERITY: Rate each drift item as Benign, Risky, or Critical... STEP 4 — INVESTIGATE ROOT CAUSE: Read the application source code... STEP 5 — GENERATE DRIFT REPORT: Produce a structured summary... STEP 6 — RECOMMEND SMART REMEDIATION: Context-aware recommendations... STEP 7 — NOTIFY TEAM: Post a summary to Microsoft Teams...

HTTP Triggers page showing tfc-drift-handler with status On and 3 completed runsThe HTTP Trigger dashboard showingtfc-drift-handleractive with 3 completed runs.

Step 4: Connect GitHub and Teams

We connect two integrations in the SRE Agent Connectors settings:

Code Repository: GitHub — so the agent can read application source code during investigations
Notification: Microsoft Teams — so the agent can post drift reports to the team channel

Connectors page showing Teams and GitHub connectedBoth connectors show "Connected" status — the agent can read source code and notify the team.

The Incident Story

Act 1: The Latency Bug

Our demo app has a subtle but devastating bug. The /api/data endpoint calls processLargeDatasetSync() — a function that sorts an array on every iteration, creating an O(n² log n) blocking operation.

On a B1 App Service Plan (single vCPU), this blocks the Node.js event loop entirely. Under load, response times spike from milliseconds to 25-58 seconds, with 502 Bad Gateway errors from the Azure load balancer.

Act 2: The On-Call Response

An on-call engineer sees the latency alerts and responds — not through Terraform, but directly through the Azure Portal and CLI. They:

Add diagnostic tags — manual_update=True, changed_by=portal_user (benign)
Downgrade TLS from 1.2 to 1.0 while troubleshooting (risky — security regression)
Scale the App Service Plan from B1 to S1 to throw more compute at the problem (critical — cost increase from ~$13/mo to ~$73/mo)

Azure Portal showing iacdemo-webapp with TLS warning, unauthorized tags, and S1 SKUThe Azure Portal tells the story: a TLS security warning banner across the top, unauthorizedmanual_updateandchanged_bytags, and the App Service Plan upgraded to S1. Three types of drift, all from a single incident response.

The incident is partially mitigated — S1 has more compute, so latency drops from catastrophic to merely bad. Everyone goes back to sleep. Nobody updates Terraform.

Act 3: The Drift Check Fires

The next morning, a nightly speculative Terraform plan runs and detects 3 drifted attributes. The notification webhook fires, flowing through the Logic App auth bridge to the SRE Agent HTTP Trigger.

The agent wakes up and begins its investigation.

What the Agent Found

Layer 1: Drift Detection

The agent compares Terraform configuration against Azure reality and produces a severity-classified drift report:

Drift Report showing Critical, Risky, and Benign drift with Expected vs Actual columnsThe agent's drift report — organized by severity. The "Incident Correlation" column (partially visible) is what makes this more than aterraform planwrapper.

Three drift items detected:

Critical: App Service Plan SKU changed from B1 (~$13/mo) to S1 (~$73/mo) — a +462% cost increase
Risky: Minimum TLS version downgraded from 1.2 to 1.0 — a security regression vulnerable to BEAST and POODLE attacks
Benign: Additional tags (changed_by: portal_user, manual_update: True) — cosmetic, no functional impact

Layer 2: Incident Correlation

Here's where the agent goes beyond simple drift detection. It queries Application Insights and discovers a performance incident correlated with the SKU change:

Application Insights analysis showing /api/data endpoint with 25,919ms avg latency and 57,697ms P95The agent found thatGET /api/datais averaging 25,919ms with a P95 of 57,697ms — affecting 97.6% of all requests. It also discovered that the/api/dataendpoint exists in production butnotin the repository source code.

Key findings from the incident correlation:

97.6% of requests (40 of 41) were impacted by high latency
The /api/data endpoint does not exist in the repository source code — the deployed application has diverged from the codebase
The endpoint likely contains a blocking synchronous pattern — Node.js runs on a single event loop, and any synchronous blocking call would explain 26-58s response times
The SKU scale-up from B1→S1 was an attempt to mitigate latency by adding more compute, but scaling cannot fix application-level blocking code on a single-threaded Node.js server

Layer 3: Smart Remediation

This is the insight that separates an autonomous agent from a reporting tool. Instead of blindly recommending "revert all drift," the agent produces context-aware remediation recommendations:

Smart Remediation Recommendations showing different actions for Tags, TLS, and SKU driftThree different recommendations based on context: safe to revert (tags), revert immediately for security (TLS), and critically — do NOT revert the SKU until the code is fixed.

The agent's remediation logic:

Tags (Benign) → Safe to revert anytime via terraform apply -target
TLS 1.0 (Risky) → Revert immediately — the TLS downgrade is a security risk unrelated to the incident
SKU S1 (Critical) → DO NOT revert until the /api/data performance root cause is fixed

Agent explaining "Do NOT revert the SKU from S1 back to B1 yet" with recommended action sequence and code fixThe agent explainswhythe SKU shouldn't be reverted: "Reverting to B1 while the/api/datablocking code is still deployed would worsen the performance incident." It then provides a 5-step action sequence and a suggested async code pattern.

This is the logic an experienced SRE would apply. Blindly running terraform apply to revert all drift would scale the app back down to B1 while the blocking code is still deployed — turning a mitigated incident into an active outage.

Layer 4: Investigation Summary

The agent produces a complete summary tying everything together:

Investigation summary showing drift table, key findings including actor and performance incident, remediation recommendations, and actions takenThe final summary includes: who made the changes (identified via Activity Log), the performance incident details, the code-infrastructure mismatch finding, and three actions taken — Teams notification, skill improvement, and PR creation.

Key findings in the summary:

Actor: surivineela@microsoft.com made all changes via Azure Portal at ~23:19 UTC
Performance incident: /api/data averaging 25-57s latency, affecting 97.6% of requests
Code-infrastructure mismatch: /api/data exists in production but not in the repository source code
Root cause: SKU scale-up was emergency incident response, not unauthorized drift

Layer 5: Teams Notification

The agent posts a structured drift report to the team's Microsoft Teams channel:

Teams channel showing "Terraform Drift Detected" notification with drift table, performance incident, root cause, and recommended actionsThe Teams notification includes the severity-coded drift table, the performance incident context, the root cause explanation, and the 5-step recommended action sequence — all posted automatically, with a link back to the full SRE Agent investigation thread.

The on-call engineer opens Teams in the morning and sees everything they need: what drifted, why it drifted, and exactly what to do about it — without logging into any dashboard.

The Payoff: A Self-Improving Agent

Here's where the demo surprised us. After completing the investigation, the agent did two things we didn't explicitly ask for.

The Agent Improved Its Own Skill

The agent performed an Execution Review — analyzing what worked and what didn't during its investigation — and found 5 gaps in its own terraform-drift-analysis.md skill file:

Execution Review showing what worked well, 5 gaps found in the skill, and the agent editing terraform-drift-analysis.mdThe agent identified gaps including "No incident correlation guidance," "No smart remediation logic," and "No Activity Log integration" — then updated its own skill file with these learnings for next time.

What worked well:

Drift detection via az CLI comparison against Terraform HCL was straightforward
Activity Log correlation identified the actor and timing
Application Insights telemetry revealed the performance incident driving the SKU change

Gaps it found and fixed:

No incident correlation guidance — the skill didn't instruct checking App Insights
No code-infrastructure mismatch detection — no guidance to verify deployed code matches the repository
No smart remediation logic — didn't warn against reverting critical drift during active incidents
Report template missing incident correlation column
No Activity Log integration guidance — didn't instruct checking who made changes and when

The agent then edited its own skill file to incorporate these learnings. Next time it runs a drift analysis, it will include incident correlation, code-infra mismatch checks, and smart remediation logic by default.

This is a learning loop — every investigation makes the agent better at future investigations.

The Agent Created a PR

Without being asked, the agent identified the root cause code issue and proactively created a pull request to fix it:

GitHub PR #1 "Improve terraform-drift-analysis skill with incident correlation and smart remediation" showing code changes to server.jsPR #1 — the agent modified bothserver.js(adding safety constants and capping delay values) andterraform-drift-analysis.md(incorporating the learnings from the investigation). Two commits, two files changed, +103/-10 lines.

The PR includes:

App safety fixes: Adding MAX_DELAY_MS and SERVER_TIMEOUT_MS constants to prevent unbounded latency
Skill improvements: Incorporating incident correlation, code-infra mismatch detection, and smart remediation logic

From a single webhook: drift detected → incident correlated → root cause found → team notified → skill improved → fix shipped.

Key Takeaways

Drift detection is not enough. Knowing that B1 changed to S1 is table stakes. Knowing it changed because of a latency incident, and that reverting it would cause an outage — that's the insight that matters.
Context-aware remediation prevents outages. Blindly running terraform apply after drift would have scaled the app back to B1 while blocking code was still deployed. The agent's "DO NOT revert SKU" recommendation is the difference between fixing drift and causing a P1.
Skills create a learning loop. The agent's self-review and skill improvement means every investigation makes the next one better — without human intervention.
HTTP Triggers connect any platform. The auth bridge pattern (Logic App + Managed Identity) works for Terraform Cloud, but the same architecture applies to any webhook source: GitHub Actions, Jenkins, Datadog, PagerDuty, custom internal tools.
The agent acts, not just reports. From a single webhook: drift detected, incident correlated, root cause identified, team notified via Teams, skill improved, and PR created. End-to-end in one autonomous session.

Getting Started

HTTP Triggers are available now in Azure SRE Agent:

Create a Skill — Teach the agent your operational runbook (in this case, drift analysis with severity classification and smart remediation)
Create an HTTP Trigger — Define your agent prompt with {payload.X} placeholders and connect it to a skill
Set Up an Auth Bridge — Deploy a Logic App with Managed Identity to handle Azure AD token acquisition
Connect Your Source — Point Terraform Cloud (or any webhook-capable platform) at the Logic App URL
Connect GitHub + Teams — Give the agent access to source code and team notifications

Within minutes, you'll have an autonomous pipeline that turns infrastructure drift events into fully contextualized investigations — with incident correlation, root cause analysis, and smart remediation recommendations.

The full implementation guide, Terraform files, skill definitions, and demo scripts are available in this repository.

Explaining what GitHub Copilot Modernization can (and cannot do)

PabloLopes — Thu, 16 Apr 2026 21:45:41 GMT

In the last post, we looked at the workflow: assess, plan, execute. You get reports you can review and the agent makes changes you can inspect.

If you don’t know, GitHub Copilot Modernization is the new agentic tool that supports you to in modernizing older applications. Could it support you with that old 4.8 Framework app, even that forgotten VB.NET script?

You're probably not modernizing one small app. It is probably a handful of projects, each with its own stack of blockers. Different frameworks, different databases, different dependencies frozen in time because nobody wants to touch them.

GitHub Copilot modernization handles two big categories: upgrading .NET projects to newer versions and migrating .NET apps to Azure. But what does that look like?

Upgrading .NET Projects

Let’s say, you've got an ASP.NET app running on .NET Framework 4.8 or it's a web API stuck on .NET Core 3.1. Unfortunately, getting it to .NET 9 or 10 isn't just updating a target framework property.

Here's what the upgrade workflow handles in Visual Studio:

Assessment first. - The agent examines your project structure, dependencies, and code patterns. It generates an Assessment Report UI, which shows both the app information, to create the plan, and the Cloud Readiness, for Azure deployment.

Then planning. - Once you approve the assessment, it moves to planning. Here you get upgrade strategies, refactoring approaches, dependency upgrade paths, and risk mitigations documented in a plan.md file at .appmod/.migration, you can check and edit that Markdown before moving forward or ask in the Copilot Chat window to change it.

# .NET 10.0 Upgrade Plan ## Execution Steps Execute steps below sequentially one by one in the order they are listed. 1. Validate that a .NET 10.0 SDK required for this upgrade is installed on the machine and if not, help to get it installed. 2. Ensure that the SDK version specified in global.json files is compatible with the .NET 10.0 upgrade. 3. Upgrade src\eShopLite.StoreFx\eShopLite.StoreFx.csproj ## Settings This section contains settings and data used by execution steps. ### Excluded projects No projects are excluded from this upgrade. ### Aggregate NuGet packages modifications across all projects NuGet packages used across all selected projects or their dependencies that need version update in projects that reference them

Then execution. - After you approve the plan, and the agent breaks it into discrete tasks in a tasks.md file. Each task gets validation criteria. As it works, it updates the file with checkboxes and completion percentages so you can track progress. It makes code changes, verifies builds, runs tests. If it hits a problem, it tries to identify the cause and apply a fix.

Go to the GitHub Copilot Chat window and type:

The plan and progress tracker look good to me. Go ahead with the migration.

It usually creates Git commits for each portion so you can review what changed or roll back if you need to. In case you don’t have a need for the Git commits for the change, you can ask the agent at the start to not commit anything.

The agent primarily focuses on ASP.NET, ASP.NET Core, Blazor, Razor Pages, MVC, and Web API. It can also handle Azure Functions, WPF, Windows Forms, console apps, class libraries, and test projects.

What It Handles Well (and What It Doesn't)

The agent is good at code-level transformations: updating TargetFramework in .csproj files, upgrading NuGet packages, replacing deprecated APIs with their modern equivalents, fixing breaking changes like removed BinaryFormatter methods, running builds, and validating test suites. It can handle repetitive work across multiple projects in a solution without you needing to track every dependency manually.

It's also solid at applying predefined Azure migration patterns, swapping plaintext credentials for managed identity, replacing file I/O with Azure Blob Storage calls, moving authentication from on-prem Active Directory to Microsoft Entra ID. These are structured transformations with clear before-and-after code patterns.

But here's where you may need to pay closer attention:

Language and framework coverage: It works with C# projects mainly. If your codebase includes complex Entity Framework migrations that rely on hand-tuned database scripts, the agent won't rewrite those for you. It also won't handle third-party UI framework patterns that don't map cleanly to ASP.NET Core conventions that have breaking changes between .NET Framework and later .NET versions. Web Forms migration is underway.

Configuration and infrastructure: The agent doesn't migrate IIS-specific web.config settings that don't have direct equivalents in Kestrel or ASP.NET Core. It won't automatically set up a CI/CD pipeline or any modernization features; for that, you need to implement it with Copilot’s help. If you've got frontend frameworks bundled with ASP.NET (like an older Angular app served through MVC), you'll need to separate and upgrade that layer yourself.

Learning and memory: The agent uses your code as context during the session, and if you correct a fix or update the plan, it tries to apply that learning within the same session. But those corrections don't persist across future upgrades. You can encode internal standards using custom skills, but that requires deliberate setup.

Offline and deployment: There's no offline mode. The agent needs connectivity to run. And while it can help prepare your app for Azure deployment, it doesn't manage the actual infrastructure provisioning or ongoing operations, that's still on you.

Guarantees: The suggestions aren't guaranteed to follow best practices. The agent won't always pick the best migration path. It won't catch every edge case. You're reviewing the work; pay attention to the results before putting it into production.

What it does handle: the tedious parts. Reading dependency graphs. Finding all the places a deprecated API is used. Updating project files. Writing boilerplate for managed identity. Fixing compilation errors that follow a predictable pattern.

Where to Start

If you've been staring at a modernization backlog, pick one project. See what it comes up with! You don't have to commit to upgrading your entire portfolio. Try it on one project and see if it saves you time. Modernization at scale still happens application by application, repo by repo, and decision by decision. GitHub Copilot modernization just makes each one a little less painful. Experiment with it!

Managing Multi‑Tenant Azure Resource with SRE Agent and Lighthouse

Pranab_Mandal — Thu, 16 Apr 2026 04:58:53 GMT

Azure SRE Agent is an AI‑powered reliability assistant that helps teams diagnose and resolve production issues faster while reducing operational toil. It analyzes logs, metrics, alerts, and deployment data to perform root cause analysis and recommend or execute mitigations with human approval. It’s capable of integrating with azure services across subscriptions and resource groups that you need to monitor and manage. Today’s enterprise customers live in a multi-tenant world, and there are multiple reasons to that due to acquisitions, complex corporate structures, managed service providers, or IT partners. Azure Lighthouse enables enterprise IT teams and managed service providers to manage resources across multiple azure tenants from a single control plane.

In this demo I will walk you through how to set up Azure SRE agent to manage and monitor multi-tenant resources delegated through Azure Lighthouse.

Navigate to the Azure SRE agent and select Create agent. Fill in the required details along with the deployment region and deploy the SRE agent.

Once the deployment is complete, hit Set up your agent. Select the Azure resources you would like your agent to analyze like resource groups or subscriptions.

This will land you to the popup window that allows you to select the subscriptions and resource groups that you would like SRE agent to monitor and manage. You can then select the subscriptions and resource groups under the same tenant that you want SRE agent to manage; Great, So far so good 👍

As a Managed Service Provider (MSP) you have multiple tenants that you are managing via Azure Lighthouse, and you need to have SRE agent access to those.

So, to demo this will need to set up Azure Lighthouse with correct set of roles and configuration to delegate access to management subscription where the Centralized SRE agent is running.

From Azure portal search Lighthouse. Navigate to the Lighthouse home page and select Manage your customers. On My customers Overview select Create ARM Template

Provide a Name and Description. Select subscriptions on a Delegated scope. Select + Add authorization which will take you to Add authorization window. Select Principal type, I am selecting User for demo purposes. The pop-up window will allow Select users from the list.

Select the checkbox next to the desired user who you want to delegate the subscription and hit Select

Then select the Role that you would like to assign the user from the managing tenant to the delegated tenant and select add. You can add multiple roles by adding additional authorization to the selected user. This step is important to make sure the delegated tenant is assigned with the right role in order for SRE Agents to add it as Azure source.

Azure SRE agent requires an Owner or User Administrator RBAC role to assign the subscription to the list of managed resources. If an appropriate role is not assigned, you will see an error when selecting the delegated subscriptions in SRE agent Managed resources.

As per Lighthouse role support Owner role isn’t supported and User access Administrator role is supported, but only for limited purpose. Refer Azure Lighthouse documentation for additional information. If role is not defined correctly, you might see an error stating: 🛑Failed to add Role assignment “The 'delegatedRoleDefinitionIds' property is required when using certain roleDefinitionIds for authorization.

To allow a principalId to assign roles to a managed identity in the customer tenant, set its roleDefinitionId to User Access Administrator. Download the ARM template and add specific Azure built-in roles that you want to grant in the delegatedRoleDefinitionIds property. You can include any supported Azure built-in role except for User Access Administrator or Owner. This example shows a principalId with User Access Administrator role that can assign two built in roles to managed identities in the customer tenant: Contributor and Log Analytics Contributor.

{ "principalId": "00000000-0000-0000-0000-000000000000", "principalIdDisplayName": "Policy Automation Account", "roleDefinitionId": "18d7d88d-d35e-4fb5-a5c3-7773c20a72d9", "delegatedRoleDefinitionIds": [ "b24988ac-6180-42a0-ab88-20f7382dd24c", "92aaf0da-9dab-42b6-94a3-d43ce8d16293" ] }

In addition SRE agent would require certain roles at the managed identity level in order to access and operate on those services. Locate SRE agent User assigned managed identity and add roles to the service principal. For the demo purpose I am assigning Reader, Monitoring Reader, and Log Analytics Reader role.

Here is the sample ARM template used for this demo.

{ "$schema": "https://schema.management.azure.com/schemas/2019-08-01/subscriptionDeploymentTemplate.json#", "contentVersion": "1.0.0.0", "parameters": { "mspOfferName": { "type": "string", "metadata": { "description": "Specify a unique name for your offer" }, "defaultValue": "lighthouse-sre-demo" }, "mspOfferDescription": { "type": "string", "metadata": { "description": "Name of the Managed Service Provider offering" }, "defaultValue": "lighthouse-sre-demo" } }, "variables": { "mspRegistrationName": "[guid(parameters('mspOfferName'))]", "mspAssignmentName": "[guid(parameters('mspOfferName'))]", "managedByTenantId": "6e03bca1-4300-400d-9e80-000000000000", "authorizations": [ { "principalId": "504adfc5-da83-47d4-8709-000000000000", "roleDefinitionId": "e40ec5ca-96e0-45a2-b4ff-59039f2c2b59", "principalIdDisplayName": "Pranab Mandal" }, { "principalId": "504adfc5-da83-47d4-8709-000000000000", "roleDefinitionId": "18d7d88d-d35e-4fb5-a5c3-7773c20a72d9", "delegatedRoleDefinitionIds": [ "b24988ac-6180-42a0-ab88-20f7382dd24c", "92aaf0da-9dab-42b6-94a3-d43ce8d16293" ], "principalIdDisplayName": "Pranab Mandal" }, { "principalId": "504adfc5-da83-47d4-8709-000000000000", "roleDefinitionId": "b24988ac-6180-42a0-ab88-20f7382dd24c", "principalIdDisplayName": "Pranab Mandal" }, { "principalId": "0374ff5c-5272-49fa-878a-000000000000", "roleDefinitionId": "acdd72a7-3385-48ef-bd42-f606fba81ae7", "principalIdDisplayName": "sre-agent-ext-sub1-4n4y4v5jjdtuu" }, { "principalId": "0374ff5c-5272-49fa-878a-000000000000", "roleDefinitionId": "43d0d8ad-25c7-4714-9337-8ba259a9fe05", "principalIdDisplayName": "sre-agent-ext-sub1-4n4y4v5jjdtuu" }, { "principalId": "0374ff5c-5272-49fa-878a-000000000000", "roleDefinitionId": "73c42c96-874c-492b-b04d-ab87d138a893", "principalIdDisplayName": "sre-agent-ext-sub1-4n4y4v5jjdtuu" } ] }, "resources": [ { "type": "Microsoft.ManagedServices/registrationDefinitions", "apiVersion": "2022-10-01", "name": "[variables('mspRegistrationName')]", "properties": { "registrationDefinitionName": "[parameters('mspOfferName')]", "description": "[parameters('mspOfferDescription')]", "managedByTenantId": "[variables('managedByTenantId')]", "authorizations": "[variables('authorizations')]" } }, { "type": "Microsoft.ManagedServices/registrationAssignments", "apiVersion": "2022-10-01", "name": "[variables('mspAssignmentName')]", "dependsOn": [ "[resourceId('Microsoft.ManagedServices/registrationDefinitions/', variables('mspRegistrationName'))]" ], "properties": { "registrationDefinitionId": "[resourceId('Microsoft.ManagedServices/registrationDefinitions/', variables('mspRegistrationName'))]" } } ], "outputs": { "mspOfferName": { "type": "string", "value": "[concat('Managed by', ' ', parameters('mspOfferName'))]" }, "authorizations": { "type": "array", "value": "[variables('authorizations')]" } } }

Login to the customers tenant and navigate to the service provides from the Azure Portal. From the Service Providers overview screen, select Service provider offers from the left navigation pane. From the top menu, select the Add offer drop down and select Add via template.

In the Upload Offer Template window drag and drop or upload the template file that was created in the earlier step and hit Upload. Once the file is uploaded, select Review + Create. This will take a few minutes to deploy the template, and a successful deployment page should be displayed.

Navigate to Delegations from Lighthouse overview and validate if you see the delegated subscription and the assigned role. Once the Lighthouse delegation is set up sign in to the managing tenant and navigate to the deployed SRE agent. Navigate to Azure resources from top menu or via Settings > Managed resources. Navigate to Add subscriptions to select customers subscriptions that you need SRE agent to manage.

Adding subscription will automatically add required permission for the agent.

Once the appropriate roles are added, the subscriptions are ready for the agent to manage and monitor resources within them.

Summary - Benefits

This blog post demonstrates how Azure SRE Agent can be used to centrally monitor and manage Azure resources across multiple tenants by integrating it with Azure Lighthouse, a common requirement for enterprises and managed service providers operating in complex, multi-tenant environments. It walks through:

Centralized SRE operations across multiple Azure tenants
Secure, role-based access using delegated resource management
Reduced operational overhead for MSPs and enterprise IT teams
Unified visibility into resource health and reliability across customer environments

Using an AI Agent to Troubleshoot and Fix Azure Function App Issues

theringe — Thu, 16 Apr 2026 03:11:32 GMT

TOC

Preparation
Troubleshooting Workflow
Conclusion

Preparation

Topic: Required tools

AI agent: for example, Copilot CLI / OpenCode / Hermes / OpenClaw, etc. In this example, we use Copilot CLI.
Model access: for example, Anthropic Claude Opus.
Relevant skills: this example does not use skills, but using relevant skills can speed up troubleshooting.

Topic: Compliant with your organization

Enterprise-level projects are sensitive, so you must confirm with the appropriate stakeholders before using them.
Enterprise environments may also have strict standards for AI agent usage.

Topic: Network limitations

If the process involves restarting the Function App container or restarting related settings, communication between the user and the agent may be interrupted, and you will need to use /resume.
If the agent needs internet access for investigation, the app must have outbound connectivity.
If the Kudu container cannot be used because of network issues, this type of investigation cannot be carried out.

Topic: Permission limitations

If you are using Azure blessed images, according to the official documentation, the containers use the fixed password Docker!. However, if you are using a custom container, you will need to provide an additional login method.
For resources the agent does not already have permission to investigate, you will need to enable SAMI and assign the appropriate RBAC roles.

Troubleshooting Workflow

Let’s use a classic case where an HTTP trigger cannot be tested from the Azure Portal. As you can see, when clicking Test/Run in the Azure Portal, an error message appears.

At the same time, however, the home page does not show any abnormal status.

At this point, we first obtain the Function App’s SAMI and assign it the Owner role for the entire resource group. This is only for demonstration purposes. In practice, you should follow the principle of least privilege and scope permissions down to only the specific resources and operations that are actually required.

Next, go to the Kudu container, which is the always-on maintenance container dedicated to the app.

Install and enable Copilot CLI.

Then we can describe the problem we are encountering.

After the agent processes the issue and interacts with you further, it can generate a reasonable investigation report. In this example, it appears that the Function App’s Storage Account access key had been rotated previously, but the Function App had not updated the corresponding environment variable.

Once we understand the issue, we could perform the follow-up actions ourselves. However, to demonstrate the agent’s capabilities, you can also allow it to fix the problem directly, provided that you have granted the corresponding permissions through SAMI.

During the process, the container restart will disconnect the session, so you will need to return to the Kudu container and resume the previous session so it can continue.

Finally, it will inform you that the issue has been fixed, and then you can validate the result.

This is the validation result, and it looks like the repair was successful.

Conclusion

After each repair, we can even extract the experience from that case into a skill and store it in a Storage Account for future reuse. In this way, we can not only reduce the agent’s initial investigation time for similar issues, but also save tokens. This makes both time and cost management more efficient.

Gemma 4 on Azure Container Apps Serverless GPU

simonjj — Wed, 15 Apr 2026 16:20:22 GMT

Every prompt you send to a hosted AI service leaves your tenant. Your code, your architecture decisions, your proprietary logic — all of it crosses a network boundary you don't control. For teams building in regulated industries or handling sensitive IP, that's not a philosophical concern. It's a compliance blocker.

What if you could spin up a fully private AI coding agent — running on your own GPU, in your own Azure subscription — with a single command?

That's exactly what this template does. One azd up, 15 minutes, and you have Google's Gemma 4 running on Azure Container Apps serverless GPU with an OpenAI-compatible API, protected by auth, and ready to power OpenCode as your terminal-based coding agent. No data leaves your environment. No third-party model provider sees your code. Full control.

Why Self-Hosted AI on ACA?

Azure Container Apps serverless GPU gives you on-demand GPU compute without managing VMs, Kubernetes clusters, or GPU drivers. You get a container, a GPU, and an HTTPS endpoint — Azure handles the rest.

Here's what makes this approach different from calling a hosted model API:

Complete data privacy — your code and prompts never leave your Azure subscription. No PII exposure, no data leakage, no third-party processing. For teams navigating HIPAA, SOC 2, or internal IP policies, this is the simplest path to compliant AI-assisted development.
Predictable costs — you pay for GPU compute time, not per-token. Run as many prompts as you want against your deployed model.
No rate limits — the GPU is yours. No throttling, no queue, no waiting for capacity.
Model flexibility — swap models in minutes. Start with the 4B parameter Gemma 4 for fast iteration, scale up to 26B for complex reasoning tasks.

This isn't a tradeoff between convenience and privacy. ACA serverless GPU makes self-hosted AI as easy to deploy as any SaaS endpoint — but the data stays yours.

What You're Building

What does the configuration look like to run Gemma 4 + Ollama securely on ACA serverless GPU

The template deploys two containers into an Azure Container Apps environment:

Ollama + Gemma 4 — running on a serverless GPU (NVIDIA T4 or A100), serving an OpenAI-compatible API
Nginx auth proxy — a lightweight reverse proxy that adds basic authentication and exposes the endpoint over HTTPS

The Ollama container pulls the Gemma 4 model on first start, so there's nothing to pre-build or upload. The nginx proxy runs on the free Consumption profile — only the Ollama container needs GPU.

After deployment, you get a single HTTPS endpoint that works with curl, any OpenAI-compatible SDK, or OpenCode — a terminal-based AI coding agent that turns the whole thing into a private GitHub Copilot alternative.

Step 1: Deploy with `azd up`

You need the Azure CLI and Azure Developer CLI (azd) installed.

git clone https://github.com/simonjj/gemma4-on-aca.git
cd gemma4-on-aca
azd up

The setup walks you through three choices:

GPU selection — T4 (16 GB VRAM) for smaller models, or A100 (80 GB VRAM) for the full Gemma 4 lineup.

Model selection — depends on your GPU choice. The defaults are tuned for the best quality-to-speed ratio on each GPU tier.

Proxy password — protects your endpoint with basic auth.

Region availability: Serverless GPUs are available in various regoins such as australiaeast, brazilsouth, canadacentral, eastus, italynorth, swedencentral, uksouth, westus, and westus3. Pick one of these when prompted for location.

That's it. Provisioning takes about 10 minutes — mostly waiting for the ACA environment to create and the model to download.

The deployment output

Choose Your Model

Gemma 4 ships in four sizes. The right choice depends on your GPU and workload:

Model	Params	Architecture	Context	Modalities	Disk Size
`gemma4:e2b`	~2B	Dense	128K	Text, Image, Audio	~7 GB
`gemma4:e4b`	~4B	Dense	128K	Text, Image, Audio	~10 GB
`gemma4:26b`	26B	MoE (4B active)	256K	Text, Image	~18 GB
`gemma4:31b`	31B	Dense	256K	Text, Image	~20 GB

Real-World Performance on ACA

We benchmarked every model on both GPU tiers using Ollama v0.20 with Q4_K_M quantization and 32K context in Sweden Central:

Model	GPU	Tokens/sec	TTFT	Notes
`gemma4:e2b`	T4	~81	~15ms	Fastest on T4
`gemma4:e4b`	T4	~51	~17ms	Default T4 choice — best quality/speed
`gemma4:e2b`	A100	~184	~9ms	Ultra-fast
`gemma4:e4b`	A100	~129	~12ms	Great for lighter workloads
`gemma4:26b`	A100	~113	~14ms	Default A100 choice — strong reasoning
`gemma4:31b`	A100	~40	~30ms	Highest quality, slower

51 tokens/second on a T4 with the 4B model is fast enough for interactive coding assistance. The 26B model on A100 delivers 113 tokens/second with noticeably better reasoning — ideal for complex refactoring, architecture questions, and multi-file changes.

The 26B and 31B models require A100 — they don't fit in T4's 16 GB VRAM.

Step 2: Verify Your Endpoint

After azd up completes, the post-provision hook prints your endpoint URL. Test it:

curl -u admin:<YOUR_PASSWORD> \
  https://<YOUR_PROXY_ENDPOINT>/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemma4:e4b",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

You should get a JSON response with Gemma 4's reply. The endpoint is fully OpenAI-compatible — it works with any tool or SDK that speaks the OpenAI API format.

Step 3: Connect OpenCode

Here's where it gets powerful. OpenCode is a terminal-based AI coding agent — think GitHub Copilot, but running in your terminal and pointing at whatever model backend you choose.

The azd up post-provision hook automatically generates an opencode.json in your project directory with the correct endpoint and credentials. If you need to create it manually:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "gemma4-aca": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Gemma 4 on ACA",
      "options": {
        "baseURL": "https://<YOUR_PROXY_ENDPOINT>/v1",
        "headers": {
          "Authorization": "Basic <BASE64_OF_admin:YOUR_PASSWORD>"
        }
      },
      "models": {
        "gemma4:e4b": {
          "name": "Gemma 4 e4b (4B)"
        }
      }
    }
  }
}

Generate the Base64 value: echo -n "admin:YOUR_PASSWORD" | base64

Now run it:

opencode run -m "gemma4-aca/gemma4:e4b" "Write a binary search in Rust"

That command sends your prompt to Gemma 4 running on your ACA GPU, and streams the response back to your terminal. Every token is generated on your infrastructure. Nothing leaves your subscription.

For interactive sessions, launch the TUI:

opencode

Select your model with /models, pick Gemma 4, and start coding. OpenCode supports file editing, code generation, refactoring, and multi-turn conversations — all powered by your private Gemma 4 instance.

The Privacy Case

This matters most for teams that can't send code to external APIs:

HIPAA-regulated healthcare apps — patient data in code, schema definitions, and test fixtures stays in your Azure subscription
Financial services — proprietary trading algorithms and risk models never leave your network boundary
Defense and government — classified or CUI-adjacent codebases get AI assistance without external data processing agreements
Startups with sensitive IP — your secret sauce stays secret, even while you use AI to build faster

With ACA serverless GPU, you're not running a VM or managing a Kubernetes cluster to get this privacy. It's a managed container with a GPU attached. Azure handles the infrastructure, you own the data boundary.

Clean Up

When you're done:

azd down

This tears down all Azure resources. Since ACA serverless GPU bills only while your containers are running, you can also scale to zero replicas to pause costs without destroying the environment.

Get Started

📖 gemma4-on-aca on GitHub — clone it, run azd up, and you're live
🤖 OpenCode — the terminal AI agent that connects to your Gemma 4 endpoint
📌 Gemma 4 docs — model architecture and capabilities
📌 ACA serverless GPU — GPU regions and workload profile details

Govern AI Agents on App Service with the Microsoft Agent Governance Toolkit

jordanselig — Mon, 13 Apr 2026 21:15:44 GMT

Part 3 of 3 — Multi-Agent AI on Azure App Service

In Blog 1, we built a multi-agent travel planner with Microsoft Agent Framework 1.0 on App Service. In Blog 2, we added observability with OpenTelemetry and the new Application Insights Agents view. Now in Part 3, we secure those agents for production with the Microsoft Agent Governance Toolkit.

This post assumes you've followed the guidance in Blog 1 to deploy the multi-agent travel planner to Azure App Service. If you haven't deployed the app yet, start there first.

The governance gap

Our travel planner works. It's observable. But here's the question I'm hearing from customers: "How do I make sure my agents don't do something they shouldn't?"

It's a fair question. Our six agents — Coordinator, Currency Converter, Weather Advisor, Local Knowledge, Itinerary Planner, and Budget Optimizer — can call external APIs, process user data, and make autonomous decisions. In a demo, that's impressive. In production, that's a risk surface.

Consider what can go wrong with ungoverned agents:

Unauthorized API calls — An agent calls an external API it was never intended to use, leaking data or incurring costs
Sensitive data exposure — An agent passes PII to a third-party service without consent controls
Runaway token spend — A recursive agent loop burns through your OpenAI budget in minutes
Tool misuse — A prompt injection tricks an agent into executing a tool it shouldn't
Cascading failures — One agent's error propagates through the entire multi-agent workflow

These aren't theoretical. In December 2025, OWASP published the Top 10 for Agentic Applications — the first formal taxonomy of risks specific to autonomous AI agents, including goal hijacking, tool misuse, identity abuse, memory poisoning, and rogue agents. Regulators are paying attention too: the EU AI Act's high-risk AI obligations take effect in August 2026, and the Colorado AI Act becomes enforceable in June 2026.

The bottom line: if you're running agents in production, you need governance. Not eventually — now.

What the Agent Governance Toolkit does

The Agent Governance Toolkit is an open-source project (MIT license) from Microsoft that brings runtime security governance to autonomous AI agents. It's the first toolkit to address all 10 OWASP agentic AI risks with deterministic, sub-millisecond policy enforcement.

The toolkit is organized into 7 packages:

Package	What it does	Think of it as...
Agent OS	Stateless policy engine, intercepts every action before execution (<0.1ms p99)	The kernel for AI agents
Agent Mesh	Cryptographic identity (DIDs), inter-agent trust protocol, dynamic trust scoring	mTLS for agents
Agent Runtime	Execution rings (like CPU privilege levels), saga orchestration, kill switch	Process isolation for agents
Agent SRE	SLOs, error budgets, circuit breakers, chaos engineering	SRE practices for agents
Agent Compliance	Automated governance verification, regulatory mapping (EU AI Act, HIPAA, SOC2)	Compliance-as-code
Agent Marketplace	Plugin lifecycle management, Ed25519 signing, supply-chain security	Package manager security
Agent Lightning	RL training governance with policy-enforced runners	Safe training guardrails

The toolkit is available in Python, TypeScript, Rust, Go, and .NET. It's framework-agnostic — it works with MAF, LangChain, CrewAI, Google ADK, and more. For our ASP.NET Core travel planner, we'll use the .NET SDK via NuGet (Microsoft.AgentGovernance).

For this blog, we're focusing on three packages:

Agent OS — the policy engine that intercepts and evaluates every agent action
Agent Compliance — regulatory mapping and audit trail generation
Agent SRE — SLOs and circuit breakers for agent reliability

How easy it was to add governance

Here's the part that surprised me. I expected adding governance to a production agent system to be a multi-hour effort — new infrastructure, complex configuration, extensive refactoring. Instead, it took about 30 minutes.

Here's exactly what we changed:

Step 1: Add NuGet packages

Three packages added to TravelPlanner.Shared.csproj:

Step 2: Create the policy file

One new file: governance-policies.yaml in the project root. This is where all your governance rules live:

apiVersion: governance.toolkit/v1 name: travel-planner-governance description: Policy enforcement for the multi-agent travel planner on App Service scope: global defaultAction: deny rules: - name: allow-currency-conversion condition: "tool == 'ConvertCurrency'" action: allow priority: 10 description: Allow Currency Converter agent to call Frankfurter exchange rate API - name: allow-weather-forecast condition: "tool == 'GetWeatherForecast'" action: allow priority: 10 description: Allow Weather Advisor agent to call NWS forecast API - name: allow-weather-alerts condition: "tool == 'GetWeatherAlerts'" action: allow priority: 10 description: Allow Weather Advisor agent to check NWS weather alerts

Step 3: One line in BaseAgent.cs

This is the moment. Here's our BaseAgent.cs before:

Agent = new ChatClientAgent(
    chatClient, instructions: Instructions, 
    name: AgentName, description: Description)
    .AsBuilder()
    .UseOpenTelemetry(sourceName: AgentName)
    .Build();

And after:

var kernel = serviceProvider.GetService<GovernanceKernel>();
if (kernel is not null)
    builder.UseGovernance(kernel, AgentName);

Agent = builder.Build();

One line of intent, two lines of null-safety. The .UseGovernance(kernel, AgentName) call intercepts every tool/function invocation in the agent's pipeline, evaluating it against the loaded policies before execution. If the GovernanceKernel isn't registered (governance disabled), agents work exactly as before — no crash, no code change needed.

Here's the full updated constructor using IServiceProvider to optionally resolve governance:

using AgentGovernance;
using Microsoft.Extensions.DependencyInjection;

public abstract class BaseAgent : IAgent
{
    protected readonly ILogger Logger;
    protected readonly AgentOptions Options;
    protected readonly AIAgent Agent;

    // Constructor for simple agents without tools
    protected BaseAgent(
        ILogger logger,
        IOptions<AgentOptions> options,
        IChatClient chatClient,
        IServiceProvider serviceProvider)
    {
        Logger = logger;
        Options = options.Value;

        var builder = new ChatClientAgent(
            chatClient, instructions: Instructions,
            name: AgentName, description: Description)
            .AsBuilder()
            .UseOpenTelemetry(sourceName: AgentName);

        var kernel = serviceProvider.GetService<GovernanceKernel>();
        if (kernel is not null)
            builder.UseGovernance(kernel, AgentName);

        Agent = builder.Build();
    }

    // Constructor for agents with tools
    protected BaseAgent(
        ILogger logger,
        IOptions<AgentOptions> options,
        IChatClient chatClient,
        ChatOptions chatOptions,
        IServiceProvider serviceProvider)
    {
        Logger = logger;
        Options = options.Value;

        var builder = new ChatClientAgent(
            chatClient, instructions: Instructions,
            name: AgentName, description: Description,
            tools: chatOptions.Tools?.ToList())
            .AsBuilder()
            .UseOpenTelemetry(sourceName: AgentName);

        var kernel = serviceProvider.GetService<GovernanceKernel>();
        if (kernel is not null)
            builder.UseGovernance(kernel, AgentName);

        Agent = builder.Build();
    }
    
    // ... rest unchanged
}

Step 4: DI registrations in Program.cs

A few lines to wire up governance in the dependency injection container:

using AgentGovernance;

// ... existing builder setup ...

// Configure OpenTelemetry with Azure Monitor (existing — from Blog 2)
builder.Services.AddOpenTelemetry().UseAzureMonitor();

// NEW: Configure Agent Governance Toolkit
// Load policy from YAML, register as singleton. Agents resolve via IServiceProvider.
var policyPath = Path.Combine(builder.Environment.ContentRootPath, "governance-policies.yaml");
if (File.Exists(policyPath))
{
    try
    {
        var yaml = File.ReadAllText(policyPath);
        var kernel = new GovernanceKernel(new GovernanceOptions 
        { 
            EnableAudit = true, 
            EnableMetrics = true 
        });
        kernel.LoadPolicyFromYaml(yaml);
        builder.Services.AddSingleton(kernel);
        Console.WriteLine($"[Governance] Loaded policies from {policyPath}");
    }
    catch (Exception ex)
    {
        Console.WriteLine($"[Governance] Failed to load: {ex.Message}. Running without governance.");
    }
}

That's it. Your agents are now governed.

Let me repeat that because it's the core message of this blog: we added production governance to a six-agent system by adding one NuGet package, creating one YAML policy file, adding a few lines to our base agent class, and registering the governance kernel in DI. No new infrastructure. No complex rewiring. No multi-sprint project. If you followed Blog 1 and Blog 2, you can do this in 30 minutes.

Policy flexibility deep-dive

The YAML policy language is intentionally simple to start with, but it supports real complexity when you need it. Let's walk through what each policy in our file does.

API allowlists and blocklists

Our travel planner calls two external APIs: Frankfurter (currency exchange) and the National Weather Service. The defaultAction: deny combined with explicit allow rules ensures agents can only call these approved tools. If an agent attempts to call any other function — whether through a prompt injection or a bug — the call is blocked before it executes:

defaultAction: deny
rules:
  - name: allow-currency-conversion
    condition: "tool == 'ConvertCurrency'"
    action: allow
    priority: 10
  - name: allow-weather-forecast
    condition: "tool == 'GetWeatherForecast'"
    action: allow
    priority: 10

When a blocked call happens, you'll see output like this in your logs:

[Governance] Tool call 'DeleteDatabase' blocked for agent 'LocalKnowledgeAgent': 
  No matching rules; default action is deny.

Condition language

The condition field supports equality checks, pattern matching, and boolean logic. You can match on tool name, agent ID, or any key in the evaluation context:

# Match a specific tool
condition: "tool == 'ConvertCurrency'"

# Match multiple tools with OR
condition: "tool == 'GetWeatherForecast' or tool == 'GetWeatherAlerts'"

# Match by agent
condition: "agent == 'CurrencyConverterAgent' and tool == 'ConvertCurrency'"

Priority and conflict resolution

When multiple rules match, the toolkit evaluates by priority (higher number = higher priority). A deny rule at priority 100 will override an allow rule at priority 10. This lets you layer broad allows with specific denies:

rules:
  - name: allow-all-weather-tools
    condition: "tool == 'GetWeatherForecast' or tool == 'GetWeatherAlerts'"
    action: allow
    priority: 10
  - name: block-during-maintenance
    condition: "tool == 'GetWeatherForecast'"
    action: deny
    priority: 100
    description: Temporarily block NWS calls during API maintenance

Advanced: OPA Rego and Cedar

The YAML policy language handles most scenarios, but for teams with advanced needs, the toolkit also supports OPA Rego and Cedar policy languages. You can mix them — use YAML for simple rules and Rego for complex conditional logic:

# policies/advanced.rego — Example: time-based access control
package travel_planner.governance

default allow_tool_call = false

allow_tool_call {
    input.agent == "CurrencyConverterAgent"
    input.tool == "get_exchange_rate"
    time.weekday(time.now_ns()) != "Sunday"  # Markets closed
}

Start simple with YAML. Add complexity only when you need it.

Why App Service for governed agent workloads

You might be wondering: why does hosting platform matter for governance? It matters a lot. The governance toolkit handles the application-level policies, but a production agent system also needs platform-level security, networking, identity, and deployment controls. App Service gives you these out of the box.

Managed Identity

Governance policies enforce what agents can access. Managed Identity handles how they authenticate — without secrets to manage, rotate, or leak. Our travel planner already uses DefaultAzureCredential for Azure OpenAI, Cosmos DB, and Service Bus. Governance layers on top of this identity foundation.

VNet Integration + Private Endpoints

The governance toolkit enforces API allowlists at the application level. App Service's VNet integration and private endpoints enforce network boundaries at the infrastructure level. This is defense in depth: even if a governance policy is misconfigured, the network layer prevents unauthorized egress. Your agents can only reach the networks you've explicitly allowed.

Easy Auth

App Service's built-in authentication (Easy Auth) protects your agent APIs without custom code. Before a request even reaches your governance engine, App Service has already validated the caller's identity. No custom auth middleware. No JWT parsing. Just toggle it on.

Deployment Slots

This is underrated for governance. With deployment slots, you can test new governance policies in a staging slot before swapping to production. Deploy updated governance-policies.yaml to staging, run your test suite, verify the policies work as expected, and then swap. Zero-downtime policy updates with full rollback capability.

App Insights integration

Governance audit events flow into the same Application Insights instance we configured in Blog 2. This means your governance decisions appear alongside your OTel traces in the Agents view. One pane of glass for agent behavior and governance enforcement.

Always-on + WebJobs

Our travel planner uses WebJobs for long-running agent workflows. With App Service's Always-on feature, those workflows stay warm, and governance is continuous — no cold-start gaps where agents run unmonitored.

azd deployment

One command deploys the full governed stack — application code, governance policies, infrastructure, and monitoring:

azd up

App Service gives you the enterprise production features governance needs — identity, networking, observability, safe deployment — out of the box. The governance toolkit handles agent-level policy enforcement; App Service handles platform-level security. Together, they're a complete governed agent platform.

Governance audit events in App Insights

In Blog 2, we set up OpenTelemetry and the Application Insights Agents view to monitor agent behavior. With the governance toolkit, those same traces now include governance audit events — every policy decision is recorded as a span attribute on the agent's trace.

When you open a trace in the Agents view, you'll see governance events inline:

Policy: api-allowlist → ALLOWED — CurrencyConverterAgent called Frankfurter API, permitted
Policy: token-budget → ALLOWED — Request used 3,200 tokens, within per-request limit of 8,000
Policy: rate-limit → THROTTLED — WeatherAdvisorAgent exceeded 60 calls/min, request delayed

For deeper analysis, use KQL to query governance events directly. Here's a query that finds all policy violations in the last 24 hours:

// Find all governance policy violations in the last 24 hours
traces
| where timestamp > ago(24h)
| where customDimensions["governance.decision"] != "ALLOWED"
| extend 
    agentName = tostring(customDimensions["agent.name"]),
    policyName = tostring(customDimensions["governance.policy"]),
    decision = tostring(customDimensions["governance.decision"]),
    violationReason = tostring(customDimensions["governance.reason"]),
    targetUrl = tostring(customDimensions["tool.target_url"])
| project timestamp, agentName, policyName, decision, violationReason, targetUrl
| order by timestamp desc

And here's one for tracking token budget consumption across agents:

// Token budget consumption by agent over the last hour
customMetrics
| where timestamp > ago(1h)
| where name == "governance.tokens.consumed"
| extend agentName = tostring(customDimensions["agent.name"])
| summarize 
    totalTokens = sum(value),
    avgTokensPerRequest = avg(value),
    maxTokensPerRequest = max(value)
    by agentName, bin(timestamp, 5m)
| order by totalTokens desc

This is the power of integrating governance with your existing observability stack. You don't need a separate governance dashboard — everything lives in the same App Insights workspace you already know.

SRE for agents

The Agent SRE package brings Site Reliability Engineering practices to agent systems. This was the part that got me most excited, because it addresses a question I hear constantly: "How do I know my agents are actually reliable?"

Service Level Objectives (SLOs)

We defined SLOs in our policy file:

slos:
  - name: weather-agent-latency
    agent: "WeatherAdvisorAgent"
    metric: latency-p99
    target: 5000ms
    window: 5m

This says: "The Weather Advisor Agent must respond within 5 seconds at the 99th percentile, measured over a 5-minute rolling window." When the SLO is breached, the toolkit emits an alert event and can trigger automated responses.

Circuit breakers

Circuit breakers prevent cascading failures. If an agent fails 5 times in a row, the circuit opens, and subsequent requests get a fast failure response instead of waiting for another timeout:

circuit-breakers:
  - agent: "*"
    failure-threshold: 5
    recovery-timeout: 30s
    half-open-max-calls: 2

After 30 seconds, the circuit enters a half-open state, allowing 2 test calls through. If those succeed, the circuit closes and normal operation resumes. If they fail, the circuit opens again. This pattern is battle-tested in microservices — now it protects your agents too.

Error budgets

Error budgets tie SLOs to business decisions. If your Coordinator Agent's success rate target is 99.5% over a 15-minute window, that means you have an error budget of 0.5%. When the budget is consumed, the toolkit can automatically reduce agent autonomy — for example, requiring human approval for high-risk actions until the error budget recovers.

SRE practices turn agent reliability from a hope into a measurable, enforceable contract.

Architecture

Here's how everything fits together after adding governance:


┌─────────────────────────────────────────────────────────────────┐
│                     Azure App Service                           │
│  ┌──────────────┐    ┌─────────────────────────────────────┐    │
│  │   Frontend   │───▶│           ASP.NET Core API          │    │
│  │   (Static)   │    │                                     │    │
│  └──────────────┘    │  ┌─────────────────────────────┐    │    │
│                      │  │     Coordinator Agent       │    │    │
│                      │  │  ┌───────┐  ┌────────────┐  │    │    │
│                      │  │  │ OTel  │─▶│ Governance │  │    │    │
│                      │  │  └───────┘  │   Engine   │  │    │    │
│                      │  │             │ ┌────────┐ │  │    │    │
│                      │  │             │ │Policies│ │  │    │    │
│                      │  │             │ └────────┘ │  │    │    │
│                      │  │             └─────┬──────┘  │    │    │
│                      │  └───────────────────┼─────────┘    │    │
│                      │  ┌───────────────────┼──────────┐   │    │
│                      │  │  Specialist Agents │         │   │    │
│                      │  │  (Currency, Weather, etc.)   │   │    │
│                      │  │  Each with OTel + Governance │   │    │
│                      │  └───────────────────┼──────────┘   │    │
│                      └──────────────────────┼──────────────┘    │
│                                             │                   │
│  ┌────────────┐  ┌───────────┐  ┌───────────┼─────────┐         │
│  │  Managed   │  │   VNet    │  │ App Insights        │         │
│  │  Identity  │  │Integration│  │ (Traces +           │         │
│  │ (no keys)  │  │(network   │  │  Governance Audit)  │         │
│  │            │  │ boundary) │  │                     │         │
│  └────────────┘  └───────────┘  └─────────────────────┘         │
└──────────────────────────────┬──────────────────────────────────┘
                               │ Only allowed APIs
                               ▼
                    ┌──────────────────────┐
                    │   External APIs      │
                    │  ✅ Frankfurter API  │
                    │  ✅ NWS Weather API  │
                    │  ❌ Everything else  │
                    └──────────────────────┘

The key insight: governance is a transparent layer in the agent pipeline. It sits between the agent's decision and the action's execution. The agent code doesn't know or care about governance — it just builds the agent with .UseGovernance() and the policy engine handles the rest.

Bring it to your own agents

We've shown governance with Microsoft Agent Framework on .NET, but the toolkit is framework-agnostic. Here's how to add it to other popular frameworks:

LangChain (Python)

from agent_governance import PolicyEngine, GovernanceCallbackHandler

policy_engine = PolicyEngine.from_yaml("governance-policies.yaml")

# Add governance as a LangChain callback handler
agent = create_react_agent(
    llm=llm,
    tools=tools,
    callbacks=[GovernanceCallbackHandler(policy_engine)]
)

CrewAI (Python)

from agent_governance import PolicyEngine
from agent_governance.integrations.crewai import GovernanceTaskDecorator

policy_engine = PolicyEngine.from_yaml("governance-policies.yaml")

# Add governance as a CrewAI task decorator
@GovernanceTaskDecorator(policy_engine)
def research_task(agent, context):
    return agent.execute(context)

Google ADK (Python)

from agent_governance import PolicyEngine
from agent_governance.integrations.google_adk import GovernancePlugin

policy_engine = PolicyEngine.from_yaml("governance-policies.yaml")

# Add governance as a Google ADK plugin
agent = Agent(
    model="gemini-2.0-flash",
    tools=[...],
    plugins=[GovernancePlugin(policy_engine)]
)

TypeScript / Node.js

import { PolicyEngine } from '@microsoft/agentmesh-sdk';

const policyEngine = PolicyEngine.fromYaml('governance-policies.yaml');

// Use as middleware in your agent pipeline
agent.use(policyEngine.middleware());

Every integration hooks into the framework's native extension points — callbacks, decorators, plugins, middleware — so adding governance doesn't require rewriting your agent code. Install the package, point it at your policy file, and you're governed.

What's next

This wraps up our three-part series on building production-ready multi-agent AI applications on Azure App Service:

Blog 1: Build — Deploy a multi-agent travel planner with Microsoft Agent Framework 1.0
Blog 2: Monitor — Add observability with OpenTelemetry and the Application Insights Agents view
Blog 3: Govern — Secure agents for production with the Agent Governance Toolkit (you are here)

The progression is intentional: first make it work, then make it visible, then make it safe. And the consistent theme across all three parts is that App Service makes each step easier — managed hosting for Blog 1, integrated monitoring for Blog 2, and platform-level security features for Blog 3.

Next steps for your agents

Explore the Agent Governance Toolkit — star the repo, browse the 20 tutorials, try the demo
Customize policies for your compliance needs — start with our YAML template and adapt it to your domain. Healthcare teams: enable HIPAA mappings. Finance teams: add SOC2 controls.
Explore Agent Mesh for multi-agent trust — if you have agents communicating across services or trust boundaries, Agent Mesh's cryptographic identity and trust scoring add another layer of defense
Deploy the sample — clone our travel planner repo, run azd up, and see governed agents in action

AI agents are becoming autonomous decision-makers in high-stakes domains. The question isn't whether we need governance — it's whether we build it proactively, before incidents force our hand. With the Agent Governance Toolkit and Azure App Service, you can add production governance to your agents today. In about 30 minutes.

Introducing Wildcard Roles in Azure Web PubSub: simpler, smarter permissions for real-time apps

kevinguo — Mon, 13 Apr 2026 07:11:33 GMT

Real-time interactivity is now a baseline expectation across industries, from collaborative dashboards to trading platforms, IoT monitoring, and live data visualizations. Developers need a way to broadcast data instantly to connected clients without worrying about connection management, scaling, or infrastructure.

That’s where Azure Web PubSub comes in. It provides a fully managed service that enables real-time messaging over WebSocket. Your applications can send and receive live updates instantly, without managing servers or message fan-out manually.

Now Azure Web PubSub is introducing a new capability that makes permission management simpler and more scalable: using wildcard pattern to define client permissions in groups.

Understanding Azure Web PubSub

Azure Web PubSub architecture showing client connections and message flow between backend and clients through the service. The dashed line indicates stateless APIs call, while the solid line indicates persistent, long-lived WebSocket connections.

Azure Web PubSub allows you to add real-time capabilities to your app. At a high level:

Your backend generates a client access token and hands it to a connecting client.
The client connects to Azure Web PubSub over WebSocket using that token.
Once connected, both the backend and client can send and receive messages through the service.
Clients can be organized into groups (for example, all clients in the same trading room or dashboard).

Groups allow you to target specific audiences efficiently: sending messages to all users in `dashboard.operations` or receiving updates from `market.NASDAQ.MSFT`.

To maintain security, every token defines a set of roles that specify what the client can do. The code illustrates what your backend needs to specify when generating a client access token for the scenarios mentioned.

// Arguments omitted for simplicity const WebPubSubServiceClient = new WebPubSubServiceClient(); WebPubSubServiceClient.getClientAccessToken({ roles: [ "webpubsub.joinLeaveGroup.dashboard.operations", "webpubsub.sendToGroups.dashboard.operations", "webpubsub.sendToGroups.market.NASDAQ.MSFT", ], });

The Current Permission Model: literal roles

Until now, Azure Web PubSub used literal group roles to define client permissions precisely.

For example:

roles: ["webpubsub.joinLeaveGroup.room123", "webpubsub.sendToGroup.room123"];

These roles are clear and secure: the client can only join and send to a specific group, `room123`.

However, as your application scales and dynamically creates many groups — for example, hundreds of trading accounts, projects, or classrooms — issuing one role per group becomes cumbersome. Your backend needs to:

Track all the groups a user is authorized for
Generate large tokens containing many role strings
Refresh those tokens every time group access changes

The New Capability: wildcard patterns for group roles

Wildcard roles let you express permissions using patterns instead of individual hardcoded names. With a single role, you can authorize access to many related groups.

For example:

roles: ["webpubsub.joinLeaveGroups.room.*", "webpubsub.sendToGroups.room.*"];

This allows the client to join or send messages to any group whose name starts with `room:`.

Real-World Examples

Industry	Example	Benefit of wildcard roles
Finance	Risk monitoring bots subscribing to all trading accounts	One role covers `account:*` groups
Gaming	Matchmaking service observing all `lobby:*` rooms	Simplifies admin tools
Education	Teacher dashboard viewing all `class:*` groups	Fewer roles, easier permission management
Collaboration	Logging all messages across `project:*` for auditing purpose	Centralized monitoring without large tokens

Read the documentation for all supported wildcard patterns.

Why This Matters for Developers

Wildcard roles simplify the permission model for dynamic or large-scale systems:

Simpler token management – You no longer need to issue or refresh tokens every time a client’s group list changes.
Smaller tokens – One pattern replaces many literal roles, reducing token size.
Dynamic authorization – When permissions for a client change (for example, they’re assigned to new groups that match existing patterns), there’s no need to regenerate tokens.

Deep Dive: financial trading platform

Let’s look at how wildcard roles can simplify real-time event management in a trading platform where financial assets, like stocks, are traded. See the complete code here.

The Setup

The platform includes:

A trading dashboard for managers of a trading team
A trading dashboard for human traders on a trading team
Two risk analysis bots:
- A hardcoded risk bot that applies strict predefined rules
- An LLM-based risk bot that uses AI to detect unusual behavior

On the platform, each trading account managed by one or more human traders, is given its own Web PubSub group:

`account.1234.trades` – Trade updates
`account.1234.orders` – Order events
`market.NYSE` – Market data

The backend publishes events to these groups whenever a new order or trade occurs and clients that subscribe to these groups can receive real-time data.

Before: literal roles

Previously, each risk bot would need literal roles for every account:

roles: [ "webpubsub.joinLeaveGroup.account.1234.trades", "webpubsub.joinLeaveGroup.account.5678.trades", "webpubsub.joinLeaveGroup.account.9012.orders", ];

If new accounts were created, new tokens had to be issued to include their roles.

Now: wildcard roles

With the new feature, each risk bot can receive a single, compact token:

roles: [ "webpubsub.joinLeaveGroups.account.*", "webpubsub.joinLeaveGroups.market.*", ];

Now, the bot automatically gains access to all existing and future account and market groups matching those patterns, without any token regeneration.

How the Risk Bots Work

Component	Behavior
Hardcoded risk bot	Implements deterministic rules: e.g., if position size > 100 of a company's stock, trigger alert.
LLM risk bot	Uses AI models to identify anomalies, fraudulent behavior, or market stress.
Backend publisher	Emits order and trade events to `account:` and `market:` groups.

When a trade event is published:

Both bots receive it in real time through wildcard subscriptions.
Each evaluates the event differently.
If a risk is detected, they publish an alert to `alerts.risk.*`.

Traders still receive messages for only their specific account group — using literal roles to ensure isolation:

roles: ["webpubsub.joinLeaveGroup.account.1234.trades"];

This demonstrates a clean separation:

Automation and monitoring use wildcard roles for flexibility.
End users use literal roles for strict access control.

Developer Experience: cleaner and more scalable

With wildcard roles, developers can design real-time architectures that are both expressive and efficient:

Simplified token issuance
Reduced backend logic for permission changes
Better scalability for dynamic environments
Flexible system-level actors (bots, dashboards, monitors)

Together, these improvements reduce operational complexity while keeping access control transparent and secure. Whether you’re building a trading platform, a game server, or a collaborative dashboard, this new capability helps you scale real-time systems with less friction and more control.

PHP 8.5 is now available on Azure App Service for Linux

TulikaC — Fri, 10 Apr 2026 10:11:11 GMT

PHP 8.5 is now available on Azure App Service for Linux across all public regions. You can create a new PHP 8.5 app through the Azure portal, automate it with the Azure CLI, or deploy using ARM/Bicep templates.

PHP 8.5 brings several useful runtime improvements. It includes better diagnostics, with fatal errors now providing a backtrace, which can make troubleshooting easier. It also adds the pipe operator (|>) for cleaner, more readable code, along with broader improvements in syntax, performance, and type safety. You can take advantage of these improvements while continuing to use the deployment and management experience you already know in App Service.

For the full list of features, deprecations, and migration notes, see the official PHP 8.5 release page: https://www.php.net/releases/8.5/en.php

Getting started

A simpler way to deploy your code to Azure App Service for Linux

TulikaC — Fri, 10 Apr 2026 09:48:40 GMT

We’ve added a new deployment experience for Azure App Service for Linux that makes it easier to get your code running on your web app.

To get started, go to the Kudu/SCM site for your app:

<sitename>.scm.azurewebsites.net

From there, open the new Deployments experience.

You can now deploy your app by simply dragging and dropping a zip file containing your code. Once your file is uploaded, App Service shows you the contents of the zip so you can quickly verify what you’re about to deploy.

If your application is already built and ready to run, you also have the option to skip server-side build. Otherwise, App Service can handle the build step for you.

When you’re ready, select Deploy.

From there, the deployment starts right away, and you can follow each phase of the process as it happens. The experience shows clear progress through upload, build, and deployment, along with deployment logs to help you understand what’s happening behind the scenes.

After the deployment succeeds, you can also view runtime logs, which makes it easier to confirm that your app has started successfully.

This experience is ideal if you’re getting started with Azure App Service and want the quickest path from code to a running app. For production workloads and teams with established release processes, you’ll typically continue using an automated CI/CD pipeline (for example, GitHub Actions or Azure DevOps) for repeatable deployments.

We’re continuing to improve the developer experience on App Service for Linux. Give it a try and let us know what you think.

Monitor AI Agents on App Service with OpenTelemetry and the New Application Insights Agents View

jordanselig — Tue, 14 Apr 2026 16:28:22 GMT

Part 2 of 3: In Blog 1, we deployed a multi-agent travel planner on Azure App Service using the Microsoft Agent Framework (MAF) 1.0 GA. This post dives deep into how we instrumented those agents with OpenTelemetry and lit up the brand-new Agents (Preview) view in Application Insights.

📋 Prerequisite: This post assumes you've followed the guidance in Blog 1 to deploy the multi-agent travel planner to Azure App Service. If you haven't deployed the app yet, start there first — you'll need a running App Service with the agents, Service Bus, Cosmos DB, and Azure OpenAI provisioned before the monitoring steps in this post will work.

Deploying Agents Is Only Half the Battle

In Blog 1, we walked through deploying a multi-agent travel planning application on Azure App Service. Six specialized agents — a Coordinator, Currency Converter, Weather Advisor, Local Knowledge Expert, Itinerary Planner, and Budget Optimizer — work together to generate comprehensive travel plans. The architecture uses an ASP.NET Core API backed by a WebJob for async processing, Azure Service Bus for messaging, and Azure OpenAI for the brains.

But here's the thing: deploying agents to production is only half the battle. Once they're running, you need answers to questions like:

Which agent is consuming the most tokens?
How long does the Itinerary Planner take compared to the Weather Advisor?
Is the Coordinator making too many LLM calls per workflow?
When something goes wrong, which agent in the pipeline failed?

Traditional APM gives you HTTP latencies and exception rates. That's table stakes. For AI agents, you need to see inside the agent — the model calls, the tool invocations, the token spend. And that's exactly what Application Insights' new Agents (Preview) view delivers, powered by OpenTelemetry and the GenAI semantic conventions.

Let's break down how it all works.

The Agents (Preview) View in Application Insights

Azure Application Insights now includes a dedicated Agents (Preview) blade that provides unified monitoring purpose-built for AI agents. It's not just a generic dashboard — it understands agent concepts natively. Whether your agents are built with Microsoft Agent Framework, Azure AI Foundry, Copilot Studio, or a third-party framework, this view lights up as long as your telemetry follows the GenAI semantic conventions.

Here's what you get out of the box:

Agent dropdown filter — A dropdown populated by gen_ai.agent.name values from your telemetry. In our travel planner, this shows all six agents: "Travel Planning Coordinator", "Currency Conversion Specialist", "Weather & Packing Advisor", "Local Expert & Cultural Guide", "Itinerary Planning Expert", and "Budget Optimization Specialist". You can filter the entire dashboard to one agent or view them all.
Token usage metrics — Visualizations of input and output token consumption, broken down by agent. Instantly see which agents are the most expensive to run.
Operational metrics — Latency distributions, error rates, and throughput for each agent. Spot performance regressions before users notice.
End-to-end transaction details — Click into any trace to see the full workflow: which agents were invoked, what tools they called, how long each step took. The "simple view" renders agent steps in a story-like format that's remarkably easy to follow.
Grafana integration — One-click export to Azure Managed Grafana for custom dashboards and alerting.

The key insight: this view isn't magic. It works because the telemetry is structured using well-defined semantic conventions. Let's look at those next.

📖 Docs: Application Insights Agents (Preview) view documentation

GenAI Semantic Conventions — The Foundation

The entire Agents view is powered by the OpenTelemetry GenAI semantic conventions. These are a standardized set of span attributes that describe AI agent behavior in a way that any observability backend can understand. Think of them as the "contract" between your instrumented code and Application Insights.

Let's walk through the key attributes and why each one matters:

`gen_ai.agent.name`

This is the human-readable name of the agent. In our travel planner, each agent sets this via the name parameter when constructing the MAF ChatClientAgent — for example, "Weather & Packing Advisor" or "Budget Optimization Specialist". This is what populates the agent dropdown in the Agents view. Without this attribute, Application Insights would have no way to distinguish one agent from another in your telemetry. It's the single most important attribute for agent-level monitoring.

`gen_ai.agent.description`

A brief description of what the agent does. Our Weather Advisor, for example, is described as "Provides weather forecasts, packing recommendations, and activity suggestions based on destination weather conditions." This metadata helps operators and on-call engineers quickly understand an agent's role without diving into source code. It shows up in trace details and helps contextualize what you're looking at when debugging.

`gen_ai.agent.id`

A unique identifier for the agent instance. In MAF, this is typically an auto-generated GUID. While gen_ai.agent.name is the human-friendly label, gen_ai.agent.id is the machine-stable identifier. If you rename an agent, the ID stays the same, which is important for tracking agent behavior across code deployments.

`gen_ai.operation.name`

The type of operation being performed. Values include "chat" for standard LLM calls and "execute_tool" for tool/function invocations. In our travel planner, when the Weather Advisor calls the GetWeatherForecast function via NWS, or when the Currency Converter calls ConvertCurrency via the Frankfurter API, those tool calls get their own spans with gen_ai.operation.name = "execute_tool". This lets you measure LLM think-time separately from tool execution time — a critical distinction for performance optimization.

`gen_ai.request.model` / `gen_ai.response.model`

The model used for the request and the model that actually served the response (these can differ when providers do model routing). In our case, both are "gpt-4o" since that's what we deploy via Azure OpenAI. These attributes let you track model usage across agents, spot unexpected model assignments, and correlate performance changes with model updates.

`gen_ai.usage.input_tokens` / `gen_ai.usage.output_tokens`

Token consumption per LLM call. This is what powers the token usage visualizations in the Agents view. The Coordinator agent, which aggregates results from all five specialist agents, tends to have higher output token counts because it's synthesizing a full travel plan. The Currency Converter, which makes focused API calls, uses fewer tokens overall. These attributes let you answer the question "which agent is costing me the most?" — and more importantly, let you set alerts when token usage spikes unexpectedly.

`gen_ai.system`

The AI system or provider. In our case, this is "openai" (set by the Azure OpenAI client instrumentation). If you're using multiple AI providers — say, Azure OpenAI for planning and a local model for classification — this attribute lets you filter and compare.

Together, these attributes create a rich, structured view of agent behavior that goes far beyond generic tracing. They're the reason Application Insights can render agent-specific dashboards with token breakdowns, latency distributions, and end-to-end workflow views. Without these conventions, all you'd see is opaque HTTP calls to an OpenAI endpoint.

💡 Key takeaway: The GenAI semantic conventions are what transform generic distributed traces into agent-aware observability. They're the bridge between your code and the Agents view. Any framework that emits these attributes — MAF, Semantic Kernel, LangChain — can light up this dashboard.

Two Layers of OpenTelemetry Instrumentation

Our travel planner sample instruments at two distinct levels, each capturing different aspects of agent behavior. Let's look at both.

Layer 1: IChatClient-Level Instrumentation

The first layer instruments at the IChatClient level using Microsoft.Extensions.AI. This is where we wrap the Azure OpenAI chat client with OpenTelemetry:

var client = new AzureOpenAIClient(azureOpenAIEndpoint, new DefaultAzureCredential());
// Wrap with OpenTelemetry to emit GenAI semantic convention spans
return client.GetChatClient(modelDeploymentName).AsIChatClient()
    .AsBuilder()
    .UseOpenTelemetry()
    .Build();

This single .UseOpenTelemetry() call intercepts every LLM call and emits spans with:

gen_ai.system — the AI provider (e.g., "openai")
gen_ai.request.model / gen_ai.response.model — which model was used
gen_ai.usage.input_tokens / gen_ai.usage.output_tokens — token consumption per call
gen_ai.operation.name — the operation type ("chat")

Think of this as the "LLM layer" — it captures what the model is doing regardless of which agent called it. It's model-centric telemetry.

Layer 2: Agent-Level Instrumentation

The second layer instruments at the agent level using MAF 1.0 GA's built-in OpenTelemetry support. This happens in the BaseAgent class that all our agents inherit from:

Agent = new ChatClientAgent(
    chatClient,
    instructions: Instructions,
    name: AgentName,
    description: Description,
    tools: chatOptions.Tools?.ToList())
    .AsBuilder()
    .UseOpenTelemetry(sourceName: AgentName)
    .Build();

The .UseOpenTelemetry(sourceName: AgentName) call on the MAF agent builder emits a different set of spans:

gen_ai.agent.name — the human-readable agent name (e.g., "Weather & Packing Advisor")
gen_ai.agent.description — what the agent does
gen_ai.agent.id — the unique agent identifier
Agent invocation traces — spans that represent the full lifecycle of an agent call

This is the "agent layer" — it captures which agent is doing the work and provides the identity information that powers the Agents view dropdown and per-agent filtering.

Why Both Layers?

When both layers are active, you get the richest possible telemetry. The agent-level spans nest around the LLM-level spans, creating a trace hierarchy that looks like:

Agent: "Weather & Packing Advisor" (gen_ai.agent.name)
  └── chat (gen_ai.operation.name)
        ├── model: gpt-4o, input_tokens: 450, output_tokens: 120
        └── execute_tool: GetWeatherForecast
              └── chat (follow-up with tool results)
                    └── model: gpt-4o, input_tokens: 680, output_tokens: 350

There is a tradeoff: with both layers active, you may see some span duplication since both the IChatClient wrapper and the MAF agent wrapper emit spans for the same underlying LLM call. If you find the telemetry too noisy, you can disable one layer:

Agent layer only (remove .UseOpenTelemetry() from the IChatClient) — You get agent identity but lose per-call token breakdowns.
IChatClient layer only (remove .UseOpenTelemetry() from the agent builder) — You get detailed LLM metrics but lose agent identity in the Agents view.

For the fullest experience with the Agents (Preview) view, we recommend keeping both layers active. The official sample uses both, and the Agents view is designed to handle the overlapping spans gracefully.

📖 Docs: MAF Observability Guide

Exporting Telemetry to Application Insights

Emitting OpenTelemetry spans is only useful if they land somewhere you can query them. The good news is that Azure App Service and Application Insights have deep native integration — App Service can auto-instrument your app, forward platform logs, and surface health metrics out of the box. For a full overview of monitoring capabilities, see Monitor Azure App Service.

For our AI agent scenario, we go beyond the built-in platform telemetry. We need the GenAI semantic convention spans that we configured in the previous sections to flow into App Insights so the Agents (Preview) view can render them. Our travel planner has two host processes — the ASP.NET Core API and a WebJob — and each requires a slightly different exporter setup.

ASP.NET Core API — Azure Monitor OpenTelemetry Distro

For the API, it's a single line. The Azure Monitor OpenTelemetry Distro handles everything:

// Configure OpenTelemetry with Azure Monitor for traces, metrics, and logs.
// The APPLICATIONINSIGHTS_CONNECTION_STRING env var is auto-discovered.
builder.Services.AddOpenTelemetry().UseAzureMonitor();

That's it. The distro automatically:

Discovers the APPLICATIONINSIGHTS_CONNECTION_STRING environment variable
Configures trace, metric, and log exporters to Application Insights
Sets up appropriate sampling and batching
Registers standard ASP.NET Core HTTP instrumentation

This is the recommended approach for any ASP.NET Core application. One NuGet package (Azure.Monitor.OpenTelemetry.AspNetCore), one line of code, zero configuration files.

WebJob — Manual Exporter Setup

The WebJob is a non-ASP.NET Core host (it uses Host.CreateApplicationBuilder), so the distro's convenience method isn't available. Instead, we configure the exporters explicitly:

// Configure OpenTelemetry with Azure Monitor for the WebJob (non-ASP.NET Core host).
// The APPLICATIONINSIGHTS_CONNECTION_STRING env var is auto-discovered.
builder.Services.AddOpenTelemetry()
    .ConfigureResource(r => r.AddService("TravelPlanner.WebJob"))
    .WithTracing(t => t
        .AddSource("*")
        .AddAzureMonitorTraceExporter())
    .WithMetrics(m => m
        .AddMeter("*")
        .AddAzureMonitorMetricExporter());

builder.Logging.AddOpenTelemetry(o => o.AddAzureMonitorLogExporter());

A few things to note:

.AddSource("*") — Subscribes to all trace sources, including the ones emitted by MAF's .UseOpenTelemetry(sourceName: AgentName). In production, you might narrow this to specific source names for performance.
.AddMeter("*") — Similarly captures all metrics, including the GenAI metrics emitted by the instrumentation layers.
.ConfigureResource(r => r.AddService("TravelPlanner.WebJob")) — Tags all telemetry with the service name so you can distinguish API vs. WebJob telemetry in Application Insights.
The connection string is still auto-discovered from the APPLICATIONINSIGHTS_CONNECTION_STRING environment variable — no need to pass it explicitly.

The key difference between these two approaches is ceremony, not capability. Both send the same GenAI spans to Application Insights; the Agents view works identically regardless of which exporter setup you use.

📖 Docs: Azure Monitor OpenTelemetry Distro

Infrastructure as Code — Provisioning the Monitoring Stack

The monitoring infrastructure is provisioned via Bicep modules alongside the rest of the application's Azure resources. Here's how it fits together.

Log Analytics Workspace

infra/core/monitor/loganalytics.bicep creates the Log Analytics workspace that backs Application Insights:

resource logAnalyticsWorkspace 'Microsoft.OperationalInsights/workspaces@2023-09-01' = {
  name: name
  location: location
  tags: tags
  properties: {
    sku: {
      name: 'PerGB2018'
    }
    retentionInDays: 30
  }
}

Application Insights

infra/core/monitor/appinsights.bicep creates a workspace-based Application Insights resource connected to Log Analytics:

resource appInsights 'Microsoft.Insights/components@2020-02-02' = {
  name: name
  location: location
  tags: tags
  kind: 'web'
  properties: {
    Application_Type: 'web'
    WorkspaceResourceId: logAnalyticsWorkspaceId
  }
}

output connectionString string = appInsights.properties.ConnectionString

Wiring It All Together

In infra/main.bicep, the Application Insights connection string is passed as an app setting to the App Service:

appSettings: {
  APPLICATIONINSIGHTS_CONNECTION_STRING: appInsights.outputs.connectionString
  // ... other app settings
}

This is the critical glue: when the app starts, the OpenTelemetry distro (or manual exporters) auto-discover this environment variable and start sending telemetry to your Application Insights resource. No connection strings in code, no configuration files — it's all infrastructure-driven.

The same connection string is available to both the API and the WebJob since they run on the same App Service. All agent telemetry from both host processes flows into a single Application Insights resource, giving you a unified view across the entire application.

See It in Action

Once the application is deployed and processing travel plan requests, here's how to explore the agent telemetry in Application Insights.

Step 1: Open the Agents (Preview) View

In the Azure portal, navigate to your Application Insights resource. In the left nav, look for Agents (Preview) under the Investigations section. This opens the unified agent monitoring dashboard.

Step 2: Filter by Agent

The agent dropdown at the top of the page is populated by the gen_ai.agent.name values in your telemetry. You'll see all six agents listed:

Travel Planning Coordinator
Currency Conversion Specialist
Weather & Packing Advisor
Local Expert & Cultural Guide
Itinerary Planning Expert
Budget Optimization Specialist

Select a specific agent to filter the entire dashboard — token usage, latency, error rate — down to that one agent.

Step 3: Review Token Usage

The token usage tile shows total input and output token consumption over your selected time range. Compare agents to find your biggest spenders. In our testing, the Coordinator agent consistently uses the most output tokens because it aggregates and synthesizes results from all five specialists.

Step 4: Drill into Traces

Click "View Traces with Agent Runs" to see all agent executions. Each row represents a workflow run. You can filter by time range, status (success/failure), and specific agent.

Step 5: End-to-End Transaction Details

Click any trace to open the end-to-end transaction details. The "simple view" renders the agent workflow as a story — showing each step, which agent handled it, how long it took, and what tools were called. For a full travel plan, you'll see the Coordinator dispatch work to each specialist, tool calls to the NWS weather API and Frankfurter currency API, and the final aggregation step.

Grafana Dashboards

The Agents (Preview) view in Application Insights is great for ad-hoc investigation. For ongoing monitoring and alerting, Azure Managed Grafana provides prebuilt dashboards specifically designed for agent workloads.

From the Agents view, click "Explore in Grafana" to jump directly into these dashboards:

Agent Framework Dashboard — Per-agent metrics including token usage trends, latency percentiles, error rates, and throughput over time. Pin this to your operations wall.
Agent Framework Workflow Dashboard — Workflow-level metrics showing how multi-agent orchestrations perform end-to-end. See how long complete travel plans take, identify bottleneck agents, and track success rates.

These dashboards query the same underlying data in Log Analytics, so there's zero additional instrumentation needed. If your telemetry lights up the Agents view, it lights up Grafana too.

Key Packages Summary

Here are the NuGet packages that make this work, pulled from the actual project files:

Package	Version	Purpose
`Azure.Monitor.OpenTelemetry.AspNetCore`	1.3.0	Azure Monitor OTEL Distro for ASP.NET Core (API). One-line setup for traces, metrics, and logs.
`Azure.Monitor.OpenTelemetry.Exporter`	1.3.0	Azure Monitor OTEL exporter for non-ASP.NET Core hosts (WebJob). Trace, metric, and log exporters.
`Microsoft.Agents.AI`	1.0.0	MAF 1.0 GA — `ChatClientAgent`, `.UseOpenTelemetry()` for agent-level instrumentation.
`Microsoft.Extensions.AI`	10.4.1	`IChatClient` abstraction with `.UseOpenTelemetry()` for LLM-level instrumentation.
`OpenTelemetry.Extensions.Hosting`	1.11.2	OTEL dependency injection integration for `Host.CreateApplicationBuilder` (WebJob).
`Microsoft.Extensions.AI.OpenAI`	10.4.1	OpenAI/Azure OpenAI adapter for `IChatClient`. Bridges the Azure OpenAI SDK to the M.E.AI abstraction.

Wrapping Up

Let's zoom out. In this three-part series, so far we've gone from zero to a fully observable, production-grade multi-agent AI application on Azure App Service:

Blog 1 covered deploying the multi-agent travel planner with MAF 1.0 GA — the agents, the architecture, the infrastructure.
Blog 2 (this post) showed how to instrument those agents with OpenTelemetry, explained the GenAI semantic conventions that make agent-aware monitoring possible, and walked through the new Agents (Preview) view in Application Insights.
Blog 3 will show you how to secure those agents for production with the Microsoft Agent Governance Toolkit.

The pattern is straightforward:

Add .UseOpenTelemetry() at the IChatClient level for LLM metrics.
Add .UseOpenTelemetry(sourceName: AgentName) at the MAF agent level for agent identity.
Export to Application Insights via the Azure Monitor distro (one line) or manual exporters.
Wire the connection string through Bicep and environment variables.
Open the Agents (Preview) view and start monitoring.

With MAF 1.0 GA's built-in OpenTelemetry support and Application Insights' new Agents view, you get production-grade observability for AI agents with minimal code. The GenAI semantic conventions ensure your telemetry is structured, portable, and understood by any compliant backend. And because it's all standard OpenTelemetry, you're not locked into any single vendor — swap the exporter and your telemetry goes to Jaeger, Grafana, Datadog, or wherever you need it.

Now go see what your agents are up to and check out Blog 3.

Resources

Sample repository: seligj95/app-service-multi-agent-maf-otel
App Insights Agents (Preview) view: Documentation
GenAI Semantic Conventions: OpenTelemetry GenAI Registry
MAF Observability Guide: Microsoft Agent Framework Observability
Azure Monitor OpenTelemetry Distro: Enable OpenTelemetry for .NET
Grafana Agent Framework Dashboard: aka.ms/amg/dash/af-agent
Grafana Workflow Dashboard: aka.ms/amg/dash/af-workflow
Blog 1: Deploy Multi-Agent AI Apps on Azure App Service with MAF 1.0 GA
Blog 3: Govern AI Agents on App Service with the Microsoft Agent Governance Toolkit

Build Multi-Agent AI Apps on Azure App Service with Microsoft Agent Framework 1.0

jordanselig — Tue, 14 Apr 2026 16:33:13 GMT

Part 1 of 3 — Multi-Agent AI on Azure App Service

This is part 1 of a 3 part series on deploying and working with multi-agent AI on Azure App Service. Follow allong to learn how to deploy, manage, observe, and secure your agents on Azure App Service.

A couple of months ago, we published a three-part series showing how to build multi-agent AI systems on Azure App Service using preview packages from the Microsoft Agent Framework (MAF) (formerly AutoGen / Semantic Kernel Agents). The series walked through async processing, the request-reply pattern, and client-side multi-agent orchestration — all running on App Service.

Since then, Microsoft Agent Framework has reached 1.0 GA — unifying AutoGen and Semantic Kernel into a single, production-ready agent platform. This post is a fresh start with the GA bits. We'll rebuild our travel-planner sample on the stable API surface, call out the breaking changes from preview, and get you up and running fast.

All of the code is in the companion repo: seligj95/app-service-multi-agent-maf-otel.

What Changed in MAF 1.0 GA

The 1.0 release is more than a version bump. Here's what moved:

Unified platform. AutoGen and Semantic Kernel agent capabilities have converged into Microsoft.Agents.AI. One package, one API surface.
Stable APIs with long-term support. The 1.0 contract is now locked for servicing. No more preview churn.
Breaking change — Instructions on options removed. In preview, you set instructions through ChatClientAgentOptions.Instructions. In GA, pass them directly to the ChatClientAgent constructor.
Breaking change — RunAsync parameter rename. The thread parameter is now session (type AgentSession). If you were using named arguments, this is a compile error.
Microsoft.Extensions.AI upgraded. The framework moved from the 9.x preview of Microsoft.Extensions.AI to the stable 10.4.1 release.
OpenTelemetry integration built in. The builder pipeline now includes UseOpenTelemetry() out of the box — more on that in Blog 2.

Our project references reflect the GA stack:

<PackageReference Include="Microsoft.Agents.AI" Version="1.0.0" />
<PackageReference Include="Microsoft.Extensions.AI" Version="10.4.1" />
<PackageReference Include="Azure.AI.OpenAI" Version="2.1.0" />

Why Azure App Service for AI Agents?

If you're building with Microsoft Agent Framework, you need somewhere to run your agents. You could reach for Kubernetes, containers, or serverless — but for most agent workloads, Azure App Service is the sweet spot. Here's why:

No infrastructure management — App Service is fully managed. No clusters to configure, no container orchestration to learn. Deploy your .NET or Python agent code and it just runs.
Always On — Agent workflows can take minutes. App Service's Always On feature (on Premium tiers) ensures your background workers never go cold, so agents are ready to process requests instantly.
WebJobs for background processing — Long-running agent workflows don't belong in HTTP request handlers. App Service's built-in WebJob support gives you a dedicated background worker that shares the same deployment, configuration, and managed identity — no separate compute resource needed.
Managed Identity everywhere — Zero secrets in your code. App Service's system-assigned managed identity authenticates to Azure OpenAI, Service Bus, Cosmos DB, and Application Insights automatically. No connection strings, no API keys, no rotation headaches.
Built-in observability — Native integration with Application Insights and OpenTelemetry means you can see exactly what your agents are doing in production (more on this in Part 2).
Enterprise-ready — VNet integration, deployment slots for safe rollouts, custom domains, auto-scaling rules, and built-in authentication. All the things you'll need when your agent POC becomes a production service.
Cost-effective — A single P0v4 instance (~$75/month) hosts both your API and WebJob worker. Compare that to running separate container apps or a Kubernetes cluster for the same workload.

The bottom line: App Service lets you focus on building your agents, not managing infrastructure. And since MAF supports both .NET and Python — both first-class citizens on App Service — you're covered regardless of your language preference.

Architecture Overview

The sample is a travel planner that coordinates six specialized agents to build a personalized trip itinerary. Users fill out a form (destination, dates, budget, interests), and the system returns a comprehensive travel plan complete with weather forecasts, currency advice, a day-by-day itinerary, and a budget breakdown.

The Six Agents

Currency Converter — calls the Frankfurter API for real-time exchange rates
Weather Advisor — calls the National Weather Service API for forecasts and packing tips
Local Knowledge Expert — cultural insights, customs, and hidden gems
Itinerary Planner — day-by-day scheduling with timing and costs
Budget Optimizer — allocates spend across categories and suggests savings
Coordinator — assembles everything into a polished final plan

Four-Phase Workflow

Phase	Agents	Execution
1 — Parallel Gathering	Currency, Weather, Local Knowledge	`Task.WhenAll`
2 — Itinerary	Itinerary Planner	Sequential (uses Phase 1 context)
3 — Budget	Budget Optimizer	Sequential (uses Phase 2 output)
4 — Assembly	Coordinator	Final synthesis

Infrastructure

Azure App Service (P0v4) — hosts the API and a continuous WebJob for background processing
Azure Service Bus — decouples the API from heavy AI work (async request-reply)
Azure Cosmos DB — stores task state, results, and per-agent chat histories (24-hour TTL)
Azure OpenAI (GPT-4o) — powers all agent LLM calls
Application Insights + Log Analytics — monitoring and diagnostics

ChatClientAgent Deep Dive

At the core of every agent is ChatClientAgent from Microsoft.Agents.AI. It wraps an IChatClient (from Microsoft.Extensions.AI) with instructions, a name, a description, and optionally a set of tools. This is client-side orchestration — you control the chat history, lifecycle, and execution order. No server-side Foundry agent resources are created.

Here's the BaseAgent pattern used by all six agents in the sample:

// BaseAgent.cs — constructor for agents with tools
Agent = new ChatClientAgent(
    chatClient,
    instructions: Instructions,
    name: AgentName,
    description: Description,
    tools: chatOptions.Tools?.ToList())
    .AsBuilder()
    .UseOpenTelemetry(sourceName: AgentName)
    .Build();

Notice the builder pipeline: .AsBuilder().UseOpenTelemetry(...).Build(). This opts every agent into the framework's built-in OpenTelemetry instrumentation with a single line. We'll explore what that telemetry looks like in Blog 2.

Invoking an agent is equally straightforward:

// BaseAgent.cs — InvokeAsync
public async Task<ChatMessage> InvokeAsync(
    IList<ChatMessage> chatHistory,
    CancellationToken cancellationToken = default)
{
    var response = await Agent.RunAsync(
        chatHistory, session: null, options: null, cancellationToken);

    return response.Messages.LastOrDefault()
        ?? new ChatMessage(ChatRole.Assistant, "No response generated.");
}

Key things to note:

session: null — this is the renamed parameter (was thread in preview). We pass null because we manage chat history ourselves.
The agent receives the full chatHistory list, so context accumulates across turns.
Simple agents (Local Knowledge, Itinerary Planner, Budget Optimizer, Coordinator) use the tool-less constructor; agents that call external APIs (Currency, Weather) use the constructor that accepts ChatOptions with tools.

Tool Integration

Two of our agents — Weather Advisor and Currency Converter — call real external APIs through the MAF tool-calling pipeline. Tools are registered using AIFunctionFactory.Create() from Microsoft.Extensions.AI.

Here's how the WeatherAdvisorAgent wires up its tool:

// WeatherAdvisorAgent.cs
private static ChatOptions CreateChatOptions(
    IWeatherService weatherService, ILogger logger)
{
    var chatOptions = new ChatOptions
    {
        Tools = new List<AITool>
        {
            AIFunctionFactory.Create(
                GetWeatherForecastFunction(weatherService, logger))
        }
    };
    return chatOptions;
}

GetWeatherForecastFunction returns a Func<double, double, int, Task<string>> that the model can call with latitude, longitude, and number of days. Under the hood, it hits the National Weather Service API and returns a formatted forecast string. The Currency Converter follows the same pattern with the Frankfurter API.

This is one of the nicest parts of the GA API: you write a plain C# method, wrap it with AIFunctionFactory.Create(), and the framework handles the JSON schema generation, function-call parsing, and response routing automatically.

Multi-Phase Workflow Orchestration

The TravelPlanningWorkflow class coordinates all six agents. The key insight is that the orchestration is just C# code — no YAML, no graph DSL, no special runtime. You decide when agents run, what context they receive, and how results flow between phases.

// Phase 1: Parallel Information Gathering
var gatheringTasks = new[]
{
    GatherCurrencyInfoAsync(request, state, progress, cancellationToken),
    GatherWeatherInfoAsync(request, state, progress, cancellationToken),
    GatherLocalKnowledgeAsync(request, state, progress, cancellationToken)
};
await Task.WhenAll(gatheringTasks);

After Phase 1 completes, results are stored in a WorkflowState object — a simple dictionary-backed container that holds per-agent chat histories and contextual data:

// WorkflowState.cs
public Dictionary<string, object> Context { get; set; } = new();
public Dictionary<string, List<ChatMessage>> AgentChatHistories { get; set; } = new();

Phases 2–4 run sequentially, each pulling context from the previous phase. For example, the Itinerary Planner receives weather and local knowledge gathered in Phase 1:

var localKnowledge = state.GetFromContext<string>("LocalKnowledge") ?? "";
var weatherAdvice = state.GetFromContext<string>("WeatherAdvice") ?? "";

var itineraryChatHistory = state.GetChatHistory("ItineraryPlanner");
itineraryChatHistory.Add(new ChatMessage(ChatRole.User,
    $"Create a detailed {days}-day itinerary for {request.Destination}..."
    + $"\n\nWEATHER INFORMATION:\n{weatherAdvice}"
    + $"\n\nLOCAL KNOWLEDGE & TIPS:\n{localKnowledge}"));

var itineraryResponse = await _itineraryAgent.InvokeAsync(
    itineraryChatHistory, cancellationToken);

This pattern — parallel fan-out followed by sequential context enrichment — is simple, testable, and easy to extend. Need a seventh agent? Add it to the appropriate phase and wire it into WorkflowState.

Async Request-Reply Pattern

A multi-agent workflow with six LLM calls (some with tool invocations) can easily run 30–60 seconds. That's well beyond typical HTTP timeout expectations and not a great user experience for a synchronous request. We use the Async Request-Reply pattern to handle this:

The API receives the travel plan request and immediately queues a message to Service Bus.
It stores an initial task record in Cosmos DB with status queued and returns a taskId to the client.
A continuous WebJob (running as a separate process on the same App Service plan) picks up the message, executes the full multi-agent workflow, and writes the result back to Cosmos DB.
The client polls the API for status updates until the task reaches completed.

This pattern keeps the API responsive, makes the heavy work retriable (Service Bus handles retries and dead-lettering), and lets the WebJob run independently — you can restart it without affecting the API. We covered this pattern in detail in the previous series, so we won't repeat the plumbing here.

Deploy with `azd`

The repo is wired up with the Azure Developer CLI for one-command provisioning and deployment:

git clone https://github.com/seligj95/app-service-multi-agent-maf-otel.git
cd app-service-multi-agent-maf-otel
azd auth login
azd up

azd up provisions the following resources via Bicep:

Azure App Service (P0v4 Windows) with a continuous WebJob
Azure Service Bus namespace and queue
Azure Cosmos DB account, database, and containers
Azure AI Services (Azure OpenAI with GPT-4o deployment)
Application Insights and Log Analytics workspace
Managed Identity with all necessary role assignments

After deployment completes, azd outputs the App Service URL. Open it in your browser, fill in the travel form, and watch six agents collaborate on your trip plan in real time.

What's Next

We now have a production-ready multi-agent app running on App Service with the GA Microsoft Agent Framework. But how do you actually observe what these agents are doing? When six agents are making LLM calls, invoking tools, and passing context between phases — you need visibility into every step.

In the next post, we'll dive deep into how we instrumented these agents with OpenTelemetry and the new Agents (Preview) view in Application Insights — giving you full visibility into agent runs, token usage, tool calls, and model performance. You already saw the .UseOpenTelemetry() call in the builder pipeline; Blog 2 shows what that telemetry looks like end to end and how to light up the new Agents experience in the Azure portal.

Stay tuned!

Resources

Deploying to Azure Web App from Azure DevOps Using UAMI

theringe — Thu, 09 Apr 2026 05:53:29 GMT

TOC

UAMI Configuration
App Configuration
Azure DevOps Configuration
Logs

UAMI Configuration

Create a User Assigned Managed Identity with no additional configuration.
This identity will be mentioned in later steps, especially at Object ID.

App Configuration

On an existing Azure Web App, enable Diagnostic Settings and configure it to retain certain types of logs, such as Access Audit Logs.
These logs will be discussed in the final section of this article.

Next, navigate to Access Control (IAM) and assign the previously created User Assigned Managed Identity the Website Contributor role.

Azure DevOps Configuration

Go to Azure DevOps → Project Settings → Service Connections, and create a new ARM (Azure Resource Manager) connection.

While creating the connection:

Select the corresponding User Assigned Managed Identity
Grant it appropriate permissions at the Resource Group level

During this process, you will be prompted to sign in again using your own account.
This authentication will later be reflected in the deployment logs discussed below.

Assuming the following deployment template is used in the pipeline, you will notice that additional steps appear in the deployment process compared to traditional service principal–based authentication.

Logs

A few minutes after deployment, related log records will appear.

In the AppServiceAuditLogs table, you can observe that the deployment initiator is shown as the Object ID from UAMI, and the Source is listed as Azure (DevOps).

This indicates that the User Assigned Managed Identity is authorized under my user context, while the deployment action itself is initiated by Azure DevOps.

Build and Host MCP Apps on Azure App Service

jordanselig — Wed, 08 Apr 2026 18:10:03 GMT

MCP Apps are here, and they're a game-changer for building AI tools with interactive UIs. If you've been following the Model Context Protocol (MCP) ecosystem, you've probably heard about the MCP Apps spec — the first official MCP extension that lets your tools return rich, interactive UIs that render directly inside AI chat clients like Claude Desktop, ChatGPT, VS Code Copilot, Goose, and Postman.

And here's the best part: you can host them on Azure App Service. In this post, I'll walk you through building a weather widget MCP App and deploying it to App Service. You'll have a production-ready MCP server serving interactive UIs in under 10 minutes.

What Are MCP Apps?

MCP Apps extend the Model Context Protocol by combining tools (the functions your AI client can call) with UI resources (the interactive interfaces that display the results). The pattern is simple:

A tool declares a _meta.ui.resourceUri in its metadata
When the tool is invoked, the MCP host fetches that UI resource
The UI renders in a sandboxed iframe inside the chat client

The key insight? MCP Apps are just web apps — HTML, JavaScript, and CSS served through MCP. And that's exactly what App Service does best.

The MCP Apps spec supports cross-client rendering, so the same UI works in Claude Desktop, VS Code Copilot, ChatGPT, and other MCP-enabled clients. Your weather widget, map viewer, or data dashboard becomes a universal component in the AI ecosystem.

Why App Service for MCP Apps?

Azure App Service is a natural fit for hosting MCP Apps. Here's why:

Always On — No cold starts. Your UI resources are served instantly, every time.
Easy Auth — Secure your MCP endpoint with Entra ID authentication out of the box, no code required.
Custom domains + TLS — Professional MCP server endpoints with your own domain and managed certificates.
Deployment slots — Canary and staged rollouts for MCP App updates without downtime.
Sidecars — Run backend services (Redis, message queues, monitoring agents) alongside your MCP server.
App Insights — Built-in telemetry to see which tools and UIs are being invoked, response times, and error rates.

Now, these are all capabilities you can add to a production MCP App, but the sample we're building today keeps things simple. We're focusing on the core pattern: serving MCP tools with interactive UIs from App Service. The production features are there when you need them.

When to Use Functions vs App Service for MCP Apps

Before we dive into the code, let's talk about Azure Functions. The Functions team has done great work with their MCP Apps quickstart, and if serverless is your preferred model, that's a fantastic option. Functions and App Service both host MCP Apps beautifully — they just serve different needs.

	Azure Functions	Azure App Service
Best for	New, purpose-built MCP Apps that benefit from serverless scaling	MCP Apps that need always-on hosting, persistent state, or are part of larger web apps
Scaling	Scale to zero, pay per invocation	Dedicated plans, always running
Cold start	Possible (mitigated by premium plan)	None (Always On)
Deployment	`azd up` with Functions template	`azd up` with App Service template
MCP Apps quickstart	Available	This blog post!
Additional capabilities	Event-driven triggers, durable functions	Easy Auth, custom domains, deployment slots, sidecars

Think of it this way: if you're building a new MCP App from scratch and want serverless economics, go with Functions. If you're adding MCP capabilities to an existing web app, need zero cold starts, or want production features like Easy Auth and deployment slots, App Service is your friend.

Build the Weather Widget MCP App

Let's build a simple MCP App that fetches weather data from the Open-Meteo API and displays it in an interactive widget. The sample uses ASP.NET Core for the MCP server and Vite for the frontend UI.

Here's the structure:

app-service-mcp-app-sample/
├── src/
│   ├── Program.cs              # MCP server setup
│   ├── WeatherTool.cs          # Weather tool with UI metadata
│   ├── WeatherUIResource.cs    # MCP resource serving the UI
│   ├── WeatherService.cs       # Open-Meteo API integration
│   └── app/                    # Vite frontend (weather widget)
│       └── src/
│           └── weather-app.ts  # MCP Apps SDK integration
├── .vscode/
│   └── mcp.json                # VS Code MCP server config
├── azure.yaml                  # Azure Developer CLI config
└── infra/                      # Bicep infrastructure

Program.cs — MCP Server Setup

The MCP server is an ASP.NET Core app that registers tools and UI resources:

using ModelContextProtocol;

var builder = WebApplication.CreateBuilder(args);

// Register WeatherService
builder.Services.AddSingleton<WeatherService>(sp =>
    new WeatherService(WeatherService.CreateDefaultClient()));

// Add MCP Server with HTTP transport, tools, and resources
builder.Services.AddMcpServer()
    .WithHttpTransport(t => t.Stateless = true)
    .WithTools<WeatherTool>()
    .WithResources<WeatherUIResource>();

var app = builder.Build();

// Map MCP endpoints (no auth required for this sample)
app.MapMcp("/mcp").AllowAnonymous();

app.Run();

AddMcpServer() configures the MCP protocol handler. WithHttpTransport() enables Streamable HTTP with stateless mode (no session management needed). WithTools<WeatherTool>() registers our weather tool, and WithResources<WeatherUIResource>() registers the UI resource that the MCP host will fetch and render. MapMcp("/mcp") maps the MCP endpoint at /mcp.

WeatherTool.cs — Tool with UI Metadata

The WeatherTool class defines the tool and uses the [McpMeta] attribute to declare a ui metadata block containing the resourceUri. This tells the MCP host where to fetch the interactive UI:

using System.ComponentModel;
using ModelContextProtocol.Server;

[McpServerToolType]
public class WeatherTool
{
    private readonly WeatherService _weatherService;

    public WeatherTool(WeatherService weatherService)
    {
        _weatherService = weatherService;
    }

    [McpServerTool]
    [Description("Get current weather for a location via Open-Meteo. Returns weather data that displays in an interactive widget.")]
    [McpMeta("ui", JsonValue = """{"resourceUri": "ui://weather/index.html"}""")]
    public async Task<object> GetWeather(
        [Description("City name to check weather for (e.g., Seattle, New York, Miami)")]
        string location)
    {
        var result = await _weatherService.GetCurrentWeatherAsync(location);
        return result;
    }
}

The key line is the [McpMeta("ui", ...)] attribute. This adds _meta.ui.resourceUri to the tool definition, pointing to the ui://weather/index.html resource. When the AI client calls this tool, the host fetches that resource and renders it in a sandboxed iframe alongside the tool result.

WeatherUIResource.cs — UI Resource

The UI resource class serves the bundled HTML as an MCP resource with the ui:// scheme and text/html;profile=mcp-app MIME type required by the MCP Apps spec:

using ModelContextProtocol.Protocol;
using ModelContextProtocol.Server;

[McpServerResourceType]
public class WeatherUIResource
{
    [McpServerResource(
        UriTemplate = "ui://weather/index.html",
        Name = "weather_ui",
        MimeType = "text/html;profile=mcp-app")]
    public static ResourceContents GetWeatherUI()
    {
        var filePath = Path.Combine(
            AppContext.BaseDirectory, "app", "dist", "index.html");
        var html = File.ReadAllText(filePath);

        return new TextResourceContents
        {
            Uri = "ui://weather/index.html",
            MimeType = "text/html;profile=mcp-app",
            Text = html
        };
    }
}

The [McpServerResource] attribute registers this method as the handler for the ui://weather/index.html resource. When the host fetches it, the bundled single-file HTML (built by Vite) is returned with the correct MIME type.

WeatherService.cs — Open-Meteo API Integration

The WeatherService class handles geocoding and weather data from the Open-Meteo API. Nothing MCP-specific here — it's just a standard HTTP client that geocodes a city name and fetches current weather observations.

The UI Resource (Vite Frontend)

The app/ directory contains a TypeScript app built with Vite that renders the weather widget. It uses the @modelcontextprotocol/ext-apps SDK to communicate with the host:

import { App } from "@modelcontextprotocol/ext-apps";

const app = new App({ name: "Weather Widget", version: "1.0.0" });

// Handle tool results from the server
app.ontoolresult = (params) => {
  const data = parseToolResultContent(params.content);
  if (data) render(data);
};

// Adapt to host theme (light/dark)
app.onhostcontextchanged = (ctx) => {
  if (ctx.theme) applyTheme(ctx.theme);
};

await app.connect();

The SDK's App class handles the postMessage communication with the host. When the tool returns weather data, ontoolresult fires and the widget renders the temperature, conditions, humidity, and wind. The app also adapts to the host's theme so it looks native in both light and dark mode.

The frontend is bundled into a single index.html file using Vite and the vite-plugin-singlefile plugin, which inlines all JavaScript and CSS. This makes it easy to serve as a single MCP resource.

Run Locally

To run the sample locally, you'll need the .NET 9 SDK and Node.js 18+ installed. Clone the repo and run:

# Clone the repo
git clone https://github.com/seligj95/app-service-mcp-app-sample.git
cd app-service-mcp-app-sample

# Build the frontend
cd src/app
npm install
npm run build

# Run the MCP server
cd ..
dotnet run

The server starts on http://localhost:5000. Now connect from VS Code Copilot:

Open your workspace in VS Code

The sample includes a .vscode/mcp.json that configures the local MCP server:

{
  "servers": {
    "local-mcp-appservice": {
      "type": "http",
      "url": "http://localhost:5000/mcp"
    }
  }
}

Open the GitHub Copilot Chat panel
Ask: "What's the weather in Seattle?"

Copilot will invoke the GetWeather tool, and the interactive weather widget will render inline in the chat:

Weather widget MCP App rendering inline in VS Code Copilot Chat

Deploy to Azure

Deploying to Azure is even easier. The sample includes an azure.yaml file and Bicep templates for App Service, so you can deploy with a single command:

cd app-service-mcp-app-sample
azd auth login
azd up

azd up will:

Provision an App Service plan and web app in your subscription
Build the .NET app and Vite frontend
Deploy the app to App Service
Output the public MCP endpoint URL

After deployment, azd will output a URL like https://app-abc123.azurewebsites.net. Update your .vscode/mcp.json to point to the remote server:

{
  "servers": {
    "remote-weather-app": {
      "type": "http",
      "url": "https://app-abc123.azurewebsites.net/mcp"
    }
  }
}

From that point forward, your MCP App is live. Any AI client that supports MCP Apps can invoke your weather tool and render the interactive widget — no local server required.

What's Next?

You've now built and deployed an MCP App to Azure App Service. Here's what you can explore next:

Read the MCP Apps spec to understand the full capabilities of the extension, including input forms, persistent state, and multi-step workflows.
Check out the ext-apps examples on GitHub — there are samples for map viewers, PDF renderers, system monitors, and more.
Try the Azure Functions MCP Apps quickstart if you want to build a serverless MCP App.
Learn about hosting remote MCP servers in App Service for more patterns and best practices.
Clone the sample repo and customize it for your own use cases.

And remember: App Service gives you a full production hosting platform for your MCP Apps. You can add Easy Auth to secure your endpoints with Entra ID, wire up App Insights for telemetry, configure custom domains and TLS certificates, and set up deployment slots for blue/green rollouts. These features make App Service a great choice when you're ready to take your MCP App to production.

If you build something cool with MCP Apps and App Service, let me know — I'd love to see what you create!

3 Ways to Get More from Azure SRE Agent

dchelupati — Tue, 07 Apr 2026 23:33:10 GMT

When you first set up Azure SRE Agent, it’s tempting to give it everything. Connect all your alert sources, route every severity, set up scheduled tasks to poll your channels every 30 seconds. The agent can handle all of it.

But a few simple configuration choices can help you get more value from every token the agent uses. Each investigation creates a conversation thread, and each thread consumes tokens. With the right setup, you can make sure the agent is spending those tokens on the work that has the highest impact.

The pattern that works best: start focused, see results, and expand from there. Here are three ways to do that.

1. Start with the incidents that matter most

It's natural to want full coverage from day one. But in practice, starting narrow and expanding works better. When you route only high-severity or high-impact incidents to the agent first, you get to see the quality of its investigations on the work that matters most. Once you trust the output, expanding to broader coverage is a confident decision, not a leap of faith.

The mechanism for this is your **incident response plan**. Instead of relying on a default handler that routes everything, create a targeted response plan with filters that match the incidents you want the agent to investigate.

Incident response plan filters: severity, title keywords, and exclusions.

Getting started:

Go to Response plan configuration and create a new incident response plan.
Set the Severity filter. A good starting point is Sev0 through Sev2. These are the incidents where deep investigation has the highest impact.
Use Title contains to focus on specific incident patterns, or Title does not contain to exclude known noisy alerts.
Preview the filter results to see which past incidents would have matched.

As you see results and get comfortable, widen the filters. Add Sev3. Remove title exclusions. Bring in more incident sources. The agent will handle the volume, and you'll know what the cost looks like because you've been watching it grow incrementally.

If you already have an agent running with broad filters, it's worth reviewing your response plan. A quick check on your severity and title filters can make sure the agent is spending its time on the incidents you care about.

2. Replace high-frequency polling with smarter patterns

Scheduled tasks are one of the most powerful features of the agent, but they're also where cost can quietly balloon. The reason is simple: a scheduled task runs on a timer whether there's anything to find. An incident investigation fires once per incident. A task polling every 2 minutes fires 720 times a day, and most of those runs may find nothing new.

High-frequency polling is generally a weak engineering pattern regardless of cost. It wastes compute, creates unnecessary load, and in the case of an AI agent, burns tokens checking for changes that haven't happened. Better patterns exist.

Prefer push over poll. If the source system can send a signal (an alert, a webhook, a ticket), use that to trigger the agent. Push-based workflows fire only when something happens. This is cheaper and faster than polling.

When polling is the right fit, batch it. Instead of checking every 2 minutes, run a thorough check every hour. One consolidated report from 24 daily runs is more useful than 720 micro-checks that mostly say "nothing changed." The hourly report shows trends. The 2-minute poll shows snapshots.

Consider HTTP triggers. If you have an external system that knows when work is needed (a deployment pipeline, a CI/CD tool, a monitoring platform), use an HTTP trigger to invoke the agent on demand. The agent only runs when there's actually something to do.

Match frequency to the operational cadence. A Teams channel monitor works fine at 5-minute intervals. Humans don't type that fast. A health summary runs once a day. A shift-handoff report runs once per shift. Ask: how quickly do I actually need to detect this change? The answer is almost always slower than the timer you first set.

3. Keep threads fresh

Here's a detail that's easy to miss: every time a scheduled task runs, it adds to the same conversation thread. The agent reads the full thread history before responding. So a task that runs hourly accumulates 24 conversations a day in the same thread. After a week, the agent is reading through hundreds of prior exchanges before it even starts on the new work.

The work stays the same. The cost per run keeps climbing. It's the equivalent of reopening a document and reading the entire thing from page one every time you want to add a sentence at the end.

The fix is one setting. When creating or editing a scheduled task, set "Message grouping for updates" to "New chat thread for each run."

That gives the agent a clean context on every execution. No accumulated history, no growing cost. One dropdown, predictable token usage on every run.

The pattern

Start small with incident routing, expand as you see results. Replace high-frequency polling with push signals, batching, and HTTP triggers. Keep scheduled task threads fresh with "New chat thread for each run."

The agent is built to handle whatever you throw at it. These patterns just make sure you're getting the most value for what you spend.

Azure Monitor in Azure SRE Agent: Autonomous Alert Investigation and Intelligent Merging

Vineela-Suri — Tue, 07 Apr 2026 22:20:53 GMT

Azure Monitor is great at telling you something is wrong. But once the alert fires, the real work begins — someone has to open the portal, triage it, dig into logs, and figure out what happened. That takes time. And while they're investigating, the same alert keeps firing every few minutes, stacking up duplicates of a problem that's already being looked at.

This is exactly what Azure SRE Agent's Azure Monitor integration addresses. The agent picks up alerts as they fire, investigates autonomously, and remediates when it can — all without waiting for a human to get involved. And when that same alert fires again while the investigation is still underway, the agent merges it into the existing thread rather than creating a new one.

In this blog, we'll walk through the full Azure Monitor experience in SRE Agent with a live AKS + Redis scenario — how alerts get picked up, what the agent does with them, how merging handles the noise, and why one often-overlooked setting (auto-resolve) makes a bigger difference than you'd expect.

Key Takeaways

Set up Incident Response Plans to scope which alerts the agent handles — filter by severity, title patterns, and resource type. Start with review mode, then promote to autonomous once you trust the agent's behavior for that failure pattern.

Recurring alerts merge into one thread automatically — when the same alert rule fires repeatedly, the agent merges subsequent firings into the existing investigation instead of creating duplicates.

Turn auto-resolve OFF for persistent failures (bad credentials, misconfigurations, resource exhaustion) so all firings merge into one thread. Turn it ON for transient issues (traffic spikes, brief timeouts) so each gets a fresh investigation.

Design alert rules around failure categories, not components — one alert rule = one investigation thread. Structure rules by symptom (Redis errors, HTTP errors, pod health) to give the agent focused, non-overlapping threads.

Attach Custom Response Plans for specialized handling — route specific alert patterns to custom-agents with custom instructions, tools, and runbooks.

It Starts with Any Azure Monitor Alert

Before we get to the demo, a quick note on what SRE Agent actually watches. The agent queries the Azure Alerts Management REST API, which returns every fired alert regardless of signal type. Log search alerts, metric alerts, activity log alerts, smart detection, service health, Prometheus — all of them come through the same API, and the agent processes them all the same way. You don't need to configure connectors or webhooks per alert type. If it fires in Azure Monitor, the agent can see it.

What you do need to configure is which alerts the agent should care about. That's where Incident Response Plans come in.

Setting Up: Incident Response Plans and Alert Rules

We start by heading to Settings > Incident Platform > Azure Monitor and creating an Incident Response Plan. Response Plans et you scope the agent's attention by severity, alert name patterns, target resource types, and — importantly — whether the agent should act autonomously or wait for human approval.

Action: Match the agent mode to your confidence in the remediation, not just the severity. Use autonomous mode for well-understood failure patterns where the fix is predictable and safe (e.g., rolling back a bad config, restarting a pod). Use review mode for anything where you want a human to validate before the agent acts — especially Sev0/Sev1 alerts that touch critical systems. You can always start in review mode and promote to autonomous once you've validated the agent's behavior.

Incident Response Plan configuration for Azure Monitor, showing severity, agent mode, and title pattern fields

For our demo, we created a Sev1 response plan in autonomous mode — meaning the agent would pick up any Sev1 alert and immediately start investigating and remediating, no approval needed.

On the Azure Monitor side, we set up three log-based alert rules against our AKS cluster's Log Analytics workspace. The star of the show was a Redis connection error alert — a custom log search query looking for WRONGPASS, ECONNREFUSED, and other Redis failure signatures in ContainerLog:

Each rule evaluates every 5 minutes with a 15-minute aggregation window. If the query returns any results, the alert fires. Simple enough.

Breaking Redis (On Purpose)

Our test app is a Node.js journal app on AKS, backed by Azure Cache for Redis. To create a realistic failure scenario, we updated the Redis password in the Kubernetes secret to a wrong value. The app pods picked up the bad credential, Redis connections started failing, and error logs started flowing.

Within minutes, the Redis connection error alert fired.

What Happened Next

Here's where it gets interesting. We didn't touch anything — we just watched.

The agent's scanner polls the Azure Monitor Alerts API every 60 seconds. It spotted the new alert (state: "New", condition: "Fired"), matched it against our Sev1 Incident Response Plan, and immediately acknowledged it in Azure Monitor — flipping the state to "Acknowledged" so other systems and humans know someone's on it.

Then it created a new investigation thread. The thread included everything the agent needed to get started: the alert ID, rule name, severity, description, affected resource, subscription, resource group, and a deep-link back to the Azure Portal alert.

From there, the agent went to work autonomously. It queried container logs, identified the Redis WRONGPASS errors, traced them to the bad secret, retrieved the correct access key from Azure Cache for Redis, updated the Kubernetes secret, and triggered a pod rollout. By the time we checked the thread, it was already marked "Completed."

No pages. No human investigation. No context-switching.

But the Alert Kept Firing...

Here's the thing — our alert rule evaluates every 5 minutes. Between the first firing and the agent completing the fix, the alert fired again. And again. Seven times total over 35 minutes.

Without intelligent handling, that would mean seven separate investigation threads. Seven notifications. Seven disruptions.

SRE Agent handles this with alert merging. When a subsequent firing comes in for the same alert rule, the agent checks: is there already an active thread for this rule, created within the last 7 days, that hasn't been resolved or closed? If yes, the new firing gets silently merged into the existing thread — the total alert count goes up, the "Last fired" timestamp updates, and that's it. No new thread, no new notification, no interruption to the ongoing investigation.

How merging decides: new thread or merge?

Condition Result
Same alert rule, existing thread still active Merged — alert count increments, no new thread
Same alert rule, existing thread resolved/closed New thread — fresh investigation starts
Different alert rule New thread — always separate

Condition	Result
Same alert rule, existing thread still active	Merged — alert count increments, no new thread
Same alert rule, existing thread resolved/closed	New thread — fresh investigation starts
Different alert rule	New thread — always separate

Five minutes after the first alert, the second firing came in and that continued. The agent finished the fix and closed the thread, and the final tally was one thread, seven merged alerts — spanning 35 minutes of continuous firings.

On the Azure Portal side, you can see all seven individual alert instances. Each one was acknowledged by the agent.

7 Redis Connection Error Alert entries, all Sev1, Fired condition, Closed by user, spanning 8:50 PM to 9:21 PM

Seven firings. One investigation. One fix. That's the merge in action.

The Auto-Resolve Twist

Now here's the part we didn't expect to matter as much as it did.

Azure Monitor has a setting called "Automatically resolve alerts". When enabled, Azure Monitor automatically transitions an alert to "Resolved" once the underlying condition clears — for example, when the Redis errors stop because the pod restarted.

For our first scenario above, we had auto-resolve turned off. That's why the alert stayed in "Fired" state across all seven evaluation cycles, and all seven firings merged cleanly into one thread.

But what happens if auto-resolve is on? We turned it on and ran the same scenario again:

Here's what happened:

Redis broke. Alert fired. Agent picked it up and created a thread.
The agent investigated, found the bad Redis password, fixed it.
With Redis working again, error logs stopped. We noticed that the condition cleared and closed all the 7 alerts manually.
We broke Redis a second time (simulating a recurrence). The alert fired again — but the previous alert was already closed/resolved. The merge check found no active thread. A brand-new thread was created, reinvestigated and mitigated.

Two threads for the same alert rule, right there on the Incidents page:

And on the Azure Monitor side, the newest alert shows "Resolved" condition — that's the auto-resolve doing its thing:

For a persistent failure like a Redis misconfiguration, this is clearly worse. You get a new investigation thread every break-fix cycle instead of one continuous investigation.

So, Should You Just Turn Auto-Resolve Off?

No. It depends on what kind of failure the alert is watching for.

Quick Reference: Auto-Resolve Decision Guide

Auto-Resolve OFF Auto-Resolve ON
Use when Problem persists until fixed Problem is transient and self-correcting
Examples Bad credentials, misconfigurations, CrashLoopBackOff, connection pool exhaustion, IOPS limits OOM kills during traffic spikes, brief latency from neighboring deployments, one-off job timeouts
Merge behavior All repeat firings merge into one thread Each break-fix cycle creates a new thread
Best for Agent is actively managing the alert lifecycle Each occurrence may have a different root cause
Tradeoff Alerts stay in "Fired/Acknowledged" state in Azure Monitor until the agent closes them More threads, but each gets a clean investigation

	Auto-Resolve OFF	Auto-Resolve ON
Use when	Problem persists until fixed	Problem is transient and self-correcting
Examples	Bad credentials, misconfigurations, CrashLoopBackOff, connection pool exhaustion, IOPS limits	OOM kills during traffic spikes, brief latency from neighboring deployments, one-off job timeouts
Merge behavior	All repeat firings merge into one thread	Each break-fix cycle creates a new thread
Best for	Agent is actively managing the alert lifecycle	Each occurrence may have a different root cause
Tradeoff	Alerts stay in "Fired/Acknowledged" state in Azure Monitor until the agent closes them	More threads, but each gets a clean investigation

Turn auto-resolve OFF when you want repeated firings from the same alert rule to stay in a single investigation thread until the alert is explicitly resolved or closed in Azure Monitor. This works best for persistent issues such as a Kubernetes deployment stuck in CrashLoopBackOff because of a bad image tag, a database connection pool exhausted due to a leaked connection, or a storage account hitting its IOPS limit under sustained load.

Turn auto-resolve ON when you want a new investigation thread after the previous occurrence has been resolved or closed in Azure Monitor. This works best for episodic or self-clearing issues such as a pod getting OOM-killed during a temporary traffic spike, a brief latency increases during a neighboring service’s deployment, or a scheduled job that times out once due to short-lived resource contention.

The key question is: when this alert fires again, is it the same ongoing problem or a new one? If it's the same problem, turn auto-resolve off and let the merges do their job. If it's a new problem, leave auto-resolve on and let the agent investigate fresh.

Note: These behaviors describe how SRE Agent groups alert investigations and may differ from how Azure Monitor documents native alert state behavior.

A Few Things We Learned Along the Way

Design alert rules around symptoms, not components. Each alert rule maps to one investigation thread. We structured ours around failure categories — root cause signal (Redis errors, Sev1), blast radius signal (HTTP errors, Sev2), infrastructure signal (unhealthy pods, Sev2). This gave the agent focused threads without overlap.

Incident Response Plans let you tier your response. Not every alert needs the agent to go fix things immediately. We used a Sev1 filter in autonomous mode for the Redis alert, but you could set up a Sev2 filter in review mode — the agent investigates and provides analysis but waits for human approval before taking action.

Response Plans specialize the agent. For specific alert patterns, you can give the agent custom instructions, specialized tools, and a tailored system prompt. A Redis alert can route to a custom-agent loaded with Redis-specific runbooks; a Kubernetes alert can route to one with deep kubectl expertise.

Best Practices Checklist

Here's what we learned distilled into concrete actions:

Alert Rule Design

Do	Don't
Design rules around failure categories (root cause, blast radius, infra health)	Create one alert per component — you'll get overlapping threads
Set evaluation frequency and aggregation window to match the failure pattern	Use the same frequency for everything — transient vs. persistent issues need different cadences

Example rule structure from our test:

Root cause signal — Redis WRONGPASS/ECONNREFUSED errors → Sev1
Blast radius signal — HTTP 5xx response codes → Sev2
Infrastructure signal — KubeEvents Reason="Unhealthy" → Sev2

Incident Response Plan Setup

Do	Don't
Create separate response plans per severity tier	Use one catch-all filter for everything
Start with review mode — especially for Sev0/Sev1 where wrong fixes are costly	Jump straight to autonomous mode on critical alerts without validating agent behavior first
Promote to autonomous mode once you've validated the agent handles a specific failure pattern correctly	Assume severity alone determines the right mode — it's about confidence in the remediation

Response Plans

Do	Don't
Attach custom response plans to specific alert patterns for specialized handling	Leave every alert to the agent's general knowledge
Include custom instructions, tools, and runbooks relevant to the failure type	Write generic instructions — the more specific, the better the investigation
Route Redis alerts to a Redis-specialized custom-agent; K8s alerts to one with kubectl expertise	Assume one agent configuration fits all failure types

Getting Started

Head to sre.azure.com and open your agent
Make sure the agent's managed identity has Monitoring Reader on your target subscriptions
Go to Settings > Incident Platform > Azure Monitor and create your Incident Response Plans
Review the auto-resolve setting on your alert rules — turn it off for persistent issues, leave it on for transient ones (see the decision guide above)
Start with a test response plan using Title Contains to target a specific alert rule — validate agent behavior before broadening
Watch the Incidents page and review the agent's investigation threads before expanding to more alert rules

Learn More

Agentic IIS Migration to Managed Instance on Azure App Service

Gaurav-Seth — Mon, 06 Apr 2026 23:58:35 GMT

Introduction

Enterprises running ASP.NET Framework workloads on Windows Server with IIS face a familiar dilemma: modernize or stay put. The applications work, the infrastructure is stable, and nobody wants to be the person who breaks production during a cloud migration. But the cost of maintaining aging on-premises servers, patching Windows, and managing IIS keeps climbing.

Azure App Service has long been the lift-and-shift destination for these workloads. But what about applications that depend on Windows registry keys, COM components, SMTP relay, MSMQ queues, local file system access, or custom fonts? These OS-level dependencies have historically been migration blockers — forcing teams into expensive re-architecture or keeping them anchored to VMs.

Managed Instance on Azure App Service changes this equation entirely. And the IIS Migration MCP Server makes migration guided, intelligent, and safe — with AI agents that know what to ask, what to check, and what to generate at every step.

What Is Managed Instance on Azure App Service?

Managed Instance on App Service is Azure's answer to applications that need OS-level customization beyond what standard App Service provides. It runs on the PremiumV4 (PV4) SKU with IsCustomMode=true, giving your app access to:

Capability	What It Enables
Registry Adapters	Redirect Windows Registry reads to Azure Key Vault secrets — no code changes
Storage Adapters	Mount Azure Files, local SSD, or private VNET storage as drive letters (e.g., D:\, E:\)
install.ps1 Startup Script	Run PowerShell at instance startup to install Windows features (SMTP, MSMQ), register COM components, install MSI packages, deploy custom fonts
Custom Mode	Full access to the Windows instance for configuration beyond standard PaaS guardrails

The key constraint: Managed Instance on App Service requires PV4 SKU with IsCustomMode=true. No other SKU combination supports it.

Why Managed Instance Matters for Legacy Apps

Consider a classic enterprise ASP.NET application that:

Reads license keys from HKLM\SOFTWARE\MyApp in the Windows Registry
Uses a COM component for PDF generation registered via regsvr32
Sends email through a local SMTP relay
Writes reports to D:\Reports\ on a local drive
Uses a custom corporate font for PDF rendering

With standard App Service, you'd need to rewrite every one of these dependencies. With Managed Instance on App Service, you can:

Map registry reads to Key Vault secrets via Registry Adapters
Mount Azure Files as D:\ via Storage Adapters
Enable SMTP Server via install.ps1
Register the COM DLL via install.ps1 (regsvr32)
Install the custom font via install.ps1

Please note that when you are migrating your web applications to Managed Instance on Azure App Service in majority of the use cases "Zero application code changes may be required " but depending on your specific web app some code changes may be necessary.

Microsoft Learn Resources

Why Agentic Migration? The Case for AI-Guided IIS Migration

The Problem with Traditional Migration

Microsoft provides excellent PowerShell scripts for IIS migration — Get-SiteReadiness.ps1, Get-SitePackage.ps1, Generate-MigrationSettings.ps1, and Invoke-SiteMigration.ps1. They're free, well-tested, and reliable. So why wrap them in an AI-powered system?

Because the scripts are powerful but not intelligent. They execute what you tell them to. They don't tell you what to do.

Here's what a traditional migration looks like:

Run readiness checks — get a wall of JSON with cryptic check IDs like ContentSizeCheck, ConfigErrorCheck, GACCheck
Manually interpret 15+ readiness checks per site across dozens of sites
Decide whether each site needs Managed Instance or standard App Service (how?)
Figure out which dependencies need registry adapters vs. storage adapters vs. install.ps1 (the "Managed Instance provisioning split")
Write the install.ps1 script by hand for each combination of OS features
Author ARM templates for adapter configurations (Key Vault references, storage mount specs, RBAC assignments)
Wire together PackageResults.json → MigrationSettings.json with correct Managed Instance fields (Tier=PremiumV4, IsCustomMode=true)
Hope you didn't misconfigure anything before deploying to Azure

Even experienced Azure engineers find this time-consuming, error-prone, and tedious — especially across a fleet of 20, 50, or 100+ IIS sites.

What Agentic Migration Changes

The IIS Migration MCP Server introduces an AI orchestration layer that transforms this manual grind into a guided conversation:

Traditional Approach	Agentic Approach
Read raw JSON output from scripts	AI summarizes readiness as tables with plain-English descriptions
Memorize 15 check types and their severity	AI enriches each check with title, description, recommendation, and documentation links
Manually decide Managed Instance vs App Service	recommend_target analyzes all signals and recommends with confidence + reasoning
Write install.ps1 from scratch	generate_install_script builds it from detected features
Author ARM templates manually	generate_adapter_arm_template generates full templates with RBAC guidance
Wire JSON artifacts between phases by hand	Agents pass readiness_results_path → package_results_path → migration_settings_path automatically
Pray you set PV4 + IsCustomMode correctly	Enforced automatically — every tool validates Managed Instance constraints
Deploy and find out what broke	confirm_migration presents a full cost/resource summary before touching Azure

The core value proposition: the AI knows the Managed Instance provisioning split. It knows that registry access needs an ARM template with Key Vault-backed adapters, while SMTP needs an install.ps1 section enabling the Windows SMTP Server feature. You don't need to know this. The system detects it from your IIS configuration and AppCat analysis, then generates exactly the right artifacts.

Human-in-the-Loop Safety

Agentic doesn't mean autonomous. The system has explicit gates:

Phase 1 → Phase 2: "Do you want to assess these sites, or skip to packaging?"
Phase 3: "Here's my recommendation — Managed Instance for Site A (COM + Registry), standard for Site B. Agree?"
Phase 4: "Review MigrationSettings.json before proceeding"
Phase 5: "This will create billable Azure resources. Type 'yes' to confirm"

The AI accelerates the workflow; the human retains control over every decision.

Quick Start

Clone and set up the MCP server git clone https://github.com//iis-migration-mcp.git

cd iis-migration-mcp python -m venv .venv .venv\Scripts\activate pip

install -r requirements.txt

# Download Microsoft's migration scripts (NOT included in this repo) # From: https://appmigration.microsoft.com/api/download/psscripts/AppServiceMigrationScripts.zip

# Unzip to C:\MigrationScripts (or your preferred path)

# Start using in VS Code with Copilot

# 1. Copy .vscode/mcp.json.example → .vscode/mcp.json

# 2. Open folder in VS Code

# 3. In Copilot Chat: "Configure scripts path to C:\MigrationScripts"

# 4. Then: @iis-migrate "Discover my IIS sites"

The server also works with any MCP-compatible client — Claude Desktop, Cursor, Copilot CLI, or custom integrations — via stdio transport.

Architecture: How the MCP Server Works

The system is built on the Model Context Protocol (MCP), an open protocol that lets AI assistants like GitHub Copilot, Claude, or Cursor call external tools through a standardized interface.

┌──────────────────────────────────────────────────────────────────┐

│ VS Code + Copilot Chat │

│ @iis-migrate orchestrator agent │

│ ├── iis-discover (Phase 1) │

│ ├── iis-assess (Phase 2) │

│ ├── iis-recommend (Phase 3) │

│ ├── iis-deploy-plan (Phase 4) │

│ └── iis-execute (Phase 5) │

└─────────────┬────────────────────────────────────────────────────┘

│ stdio JSON-RPC (MCP Transport)

▼

│ FastMCP Server (server.py) │

│ 13 Python Tool Modules (tools/*.py) │

│ └── ps_runner.py (Python → PowerShell bridge) │

│ └── Downloaded PowerShell Scripts (user-configured) │

│ ├── Local IIS (discovery, packaging) │

│ └── Azure ARM API (deployment) │

└──────────────────────────────────────────────────────────────────┘

The server exposes 13 MCP tools organized across 5 phases, orchestrated by 6 Copilot agents (1 orchestrator + 5 specialist subagents).

Important: The PowerShell migration scripts are not included in this repository. Users must download them from GitHub and configure the path using the configure_scripts_path tool. This ensures you always use the latest version of Microsoft's scripts, avoiding version mismatch issues.

The 13 MCP Tools: Complete Reference

Phase 0 — Setup

configure_scripts_path

Purpose: Point the server to Microsoft's downloaded migration PowerShell scripts.

Before any migration work, you need to download the scripts from GitHub, unzip them, and tell the server where they are.

"Configure scripts path to C:\MigrationScripts"

Phase 1 — Discovery

1. discover_iis_sites

Purpose: Scan the local IIS server and run readiness checks on every web site.

This is the entry point for every migration. It calls Get-SiteReadiness.ps1 under the hood, which:

Enumerates all IIS web sites, application pools, bindings, and virtual directories
Runs 15 readiness checks per site (config errors, HTTPS bindings, non-HTTP protocols, TCP ports, location tags, app pool settings, app pool identity, virtual directories, content size, global modules, ISAPI filters, authentication, framework version, connection strings, and more)
Detects source code artifacts (.sln, .csproj, .cs, .vb) near site physical paths

Output: ReadinessResults.json with per-site status:

Status	Meaning
READY	No issues detected — clear for migration
READY_WITH_WARNINGS	Minor issues that won't block migration
READY_WITH_ISSUES	Non-fatal issues that need attention
BLOCKED	Fatal issues (e.g., content > 2GB) — cannot migrate as-is

Requires: Administrator privileges, IIS installed.

2. choose_assessment_mode

Purpose: Route each discovered site into the appropriate next step.

After discovery, you decide the path for each site:

assess_all: Run detailed assessment on all non-blocked sites
package_and_migrate: Skip assessment, proceed directly to packaging (for sites you already know well)

The tool classifies each site into one of five actions:

assess_config_only — IIS/web.config analysis
assess_config_and_source — Config + AppCat source code analysis (when source is detected)
package — Skip to packaging
blocked — Fatal errors, cannot proceed
skip — User chose to exclude

Phase 2 — Assessment

3. assess_site_readiness

Purpose: Get a detailed, human-readable readiness assessment for a specific site.

Takes the raw readiness data from Phase 1 and enriches each check with:

Title: Plain-English name (e.g., "Global Assembly Cache (GAC) Dependencies")
Description: What the check found and why it matters
Recommendation: Specific guidance on how to resolve the issue
Category: Grouping (Configuration, Security, Compatibility)
Documentation Link: Microsoft Learn URL for further reading

This enrichment comes from WebAppCheckResources.resx, an XML resource file that maps check IDs to detailed metadata. Without this tool, you'd see GACCheck: FAIL — with it, you see the full context.

Output: Overall status, enriched failed/warning checks, framework version, pipeline mode, binding details.

4. assess_source_code

Purpose: Analyze an Azure Migrate application and code assessment for .NET JSON report to identify Managed Instance-relevant source code dependencies.

If your application has source code and you've run the assessment tool against it, this tool parses the results and maps findings to migration actions:

Dependency Detected	Migration Action
Windows Registry access	Registry Adapter (ARM template)
Local file system I/O / hardcoded paths	Storage Adapter (ARM template)
SMTP usage	install.ps1 (SMTP Server feature)
COM Interop	install.ps1 (regsvr32/RegAsm)
Global Assembly Cache (GAC)	install.ps1 (GAC install)
Message Queuing (MSMQ)	install.ps1 (MSMQ feature)
Certificate access	Key Vault integration

The tool matches rules from the assessment output against known Managed Instance-relevant patterns. For a complete list of rules and categories, see Interpret the analysis results.

Output: Issues categorized as mandatory/optional/potential, plus install_script_features and adapter_features lists that feed directly into Phase 3 tools.

Phase 3 — Recommendation & Provisioning

5. suggest_migration_approach

Purpose: Recommend the right migration tool/approach for the scenario.

This is a routing tool that considers:

Source code available? → Recommend the App Modernization MCP server for code-level changes
No source code? → Recommend this IIS Migration MCP (lift-and-shift)
OS customization needed? → Highlight Managed Instance on App Service as the target

6. recommend_target

Purpose: Recommend the Azure deployment target for each site based on all assessment data.

This is the intelligence center of the system. It analyzes config assessments and source code findings to recommend:

Target	When Recommended	SKU
MI_AppService	Registry, COM, MSMQ, SMTP, local file I/O, GAC, or Windows Service dependencies detected	PremiumV4 (PV4)
AppService	Standard web app, no OS-level dependencies	PremiumV2 (PV2)
ContainerApps	Microservices architecture or container-first preference	N/A

Each recommendation comes with:

Confidence: high or medium
Reasoning: Full explanation of why this target was chosen
Managed Instance reasons: Specific dependencies that require Managed Instance
Blockers: Issues that prevent migration entirely
install_script_features: What the install.ps1 needs to enable
adapter_features: What the ARM template needs to configure
Provisioning guidance: Step-by-step instructions for what to do next

7. generate_install_script

Purpose: Generate an install.ps1 PowerShell script for OS-level feature enablement on Managed Instance.

This handles the OS-level side of the Managed Instance provisioning split. It generates a startup script that includes sections for:

Feature	What the Script Does
SMTP	Install-WindowsFeature SMTP-Server, configure smart host relay
MSMQ	Install MSMQ, create application queues
COM/MSI	Run msiexec for MSI installers, regsvr32/RegAsm for COM registration
Crystal Reports	Install SAP Crystal Reports runtime MSI
Custom Fonts	Copy .ttf/.otf to C:\Windows\Fonts, register in registry

The script can auto-detect needed features from config and source assessments, or you can specify them manually.

8. generate_adapter_arm_template

Purpose: Generate an ARM template for Managed Instance registry and storage adapters.

This handles the platform-level side of the Managed Instance provisioning split. It generates a deployable ARM template that configures:

Registry Adapters (Key Vault-backed):

Map Windows Registry paths (e.g., HKLM\SOFTWARE\MyApp\LicenseKey) to Key Vault secrets
Your application reads the registry as before; Managed Instance redirects the read to Key Vault transparently

Storage Adapters (three types):

Type	Description	Credentials
AzureFiles	Mount Azure Files SMB share as a drive letter	Storage account key in Key Vault
Custom	Mount storage over private endpoint via VNET	Requires VNET integration
LocalStorage	Allocate local SSD on the Managed Instance as a drive letter	None needed

The template also includes:

Managed Identity configuration
RBAC role assignments guidance (Key Vault Secrets User, Storage File Data SMB Share Contributor, etc.)
Deployment CLI commands ready to copy-paste

Phase 4 — Deployment Planning & Packaging

9. plan_deployment

Purpose: Plan the Azure App Service deployment — plans, SKUs, site assignments.

Collects your Azure details (subscription, resource group, region) and creates a validated deployment plan:

Assigns sites to App Service Plans
Enforces PV4 + IsCustomMode=true for Managed Instance — won't let you accidentally use the wrong SKU
Supports single_plan (all sites on one plan) or multi_plan (separate plans)
Optionally queries Azure for existing Managed Instance plans you can reuse

10. package_site

Purpose: Package IIS site content into ZIP files for deployment.

Calls Get-SitePackage.ps1 to:

Compress site binaries + web.config into deployment-ready ZIPs
Optionally inject install.ps1 into the package (so it deploys alongside the app)
Handle sites with non-fatal issues (configurable)

Size limit: 2 GB per site (enforced by System.IO.Compression).

11. generate_migration_settings

Purpose: Create the MigrationSettings.json deployment configuration.

This is the final configuration artifact. It calls Generate-MigrationSettings.ps1 and then post-processes the output to inject Managed Instance-specific fields:

Important: The Managed Instance on App Service Plan is not automatically created by the migration tools. You must pre-create the Managed Instance on App Service Plan (PV4 SKU with IsCustomMode=true) in the Azure portal or via CLI before generating migration settings. When running generate_migration_settings, provide the name of your existing Managed Instance plan so the settings file references it correctly.

{ "AppServicePlan": "mi-plan-eastus", "Tier": "PremiumV4", "IsCustomMode": true, "InstallScriptPath": "install.ps1", "Region": "eastus", "Sites": [ { "IISSiteName": "MyLegacyApp", "AzureSiteName": "mylegacyapp-azure", "SitePackagePath": "packagedsites/MyLegacyApp_Content.zip" } ] }

Phase 5 — Execution

12. confirm_migration

Purpose: Present a full migration summary and require explicit human confirmation.

Before touching Azure, this tool displays:

Total plans and sites to be created
SKU and pricing tier per plan
Whether Managed Instance is configured
Cost warning for PV4 pricing
Resource group, region, and subscription details

Nothing proceeds until the user explicitly confirms.

13. migrate_sites

Purpose: Deploy everything to Azure App Service. This creates billable resources.

Calls Invoke-SiteMigration.ps1, which:

Sets Azure subscription context
Creates/validates resource groups
Creates App Service Plans (PV4 with IsCustomMode for Managed Instance)
Creates Web Apps
Configures .NET version, 32-bit mode, pipeline mode from the original IIS settings
Sets up virtual directories and applications
Disables basic authentication (FTP + SCM) for security
Deploys ZIP packages via Azure REST API

Output: MigrationResults.json with per-site Azure URLs, Resource IDs, and deployment status.

The 6 Copilot Agents

The MCP tools are orchestrated by a team of specialized Copilot agents — each responsible for a specific phase of the migration lifecycle.

@iis-migrate — The Orchestrator

The root agent that guides the entire migration. It:

Tracks progress across all 5 phases using a todo list
Delegates work to specialist subagents
Gates between phases — asks before transitioning
Enforces the Managed Instance constraint (PV4 + IsCustomMode) at every decision point
Never skips the Phase 5 confirmation gate

Usage: Open Copilot Chat and type @iis-migrate I want to migrate my IIS applications to Azure

iis-discover — Discovery Specialist

Handles Phase 1. Runs discover_iis_sites, presents a summary table of all sites with their readiness status, and asks whether to assess or skip to packaging. Returns readiness_results_path and per-site routing plans.

iis-assess — Assessment Specialist

Handles Phase 2. Runs assess_site_readiness for every site, and assess_source_code when AppCat results are available. Merges findings, highlights Managed Instance-relevant issues, and produces the adapter/install features lists that drive Phase 3.

iis-recommend — Recommendation Specialist

Handles Phase 3. Runs recommend_target for each site, then conditionally generates install.ps1 and ARM adapter templates. Presents all recommendations with confidence levels and reasoning, and allows you to edit generated artifacts.

iis-deploy-plan — Deployment Planning Specialist

Handles Phase 4. Collects Azure details, runs plan_deployment, package_site, and generate_migration_settings. Validates Managed Instance configuration, allows review and editing of MigrationSettings.json. Does not execute migration.

iis-execute — Execution Specialist

Handles Phase 5 only. Runs confirm_migration to present the final summary, then only proceeds with migrate_sites after receiving explicit "yes" confirmation. Reports results with Azure URLs and deployment status.

The Managed Instance Provisioning Split: A Critical Concept

One of the most important ideas Managed Instance introduces is the provisioning split — the division of OS dependencies into two categories that are configured through different mechanisms:

┌──────────────────────────────────────────────────────────────┐ │ MANAGED INSTANCE PROVISIONING SPLIT │ ├─────────────────────────────┬────────────────────────────────┤ │ ARM Template │ install.ps1 │ │ (Platform-Level) │ (OS-Level) │ ├─────────────────────────────┼────────────────────────────────┤ │ Registry Adapters │ COM/MSI Registration │ │ → Key Vault secrets │ → regsvr32, RegAsm, msiexec │ │ │ │ │ Storage Mounts │ SMTP Server Feature │ │ → Azure Files │ → Install-WindowsFeature │ │ → Local SSD │ │ │ → VNET private storage │ MSMQ │ │ │ → Message queue setup │ │ │ │ │ │ Crystal Reports Runtime │ │ │ → SAP MSI installer │ │ │ │ │ │ Custom Fonts │ │ │ → Copy to C:\Windows\Fonts │ └─────────────────────────────┴────────────────────────────────┘

The MCP server handles this split automatically:

assess_source_code detects which dependencies fall into which category
recommend_target reports both adapter_features and install_script_features
generate_adapter_arm_template builds the ARM template for platform features
generate_install_script builds the PowerShell startup script for OS features

You don't need to remember which goes where — the system decides and generates the right artifacts.

End-to-End Walkthrough: From Discovery to Running on Managed Instance

Here's what a complete migration conversation looks like:

You: "@iis-migrate I want to migrate my IIS applications to Azure"

Phase 1 — Discovery:

Agent runs discover_iis_sites, presents a table:

Site	Status	Framework	Source Code?
HRPortal	READY_WITH_ISSUES	v4.8	Yes (.sln found)
PayrollAPI	READY	v4.8	No
IntranetCMS	BLOCKED (>2GB)	v4.7.2	No

Phase 2 — Assessment:

Agent runs assess_site_readiness for HRPortal — finds GACCheck and RegistryCheck failures. Runs assess_source_code using AppCat report — confirms COM interop, registry access, and SMTP usage.

Phase 3 — Recommendation:

Agent runs recommend_target:

HRPortal → MI_AppService (high confidence) — COM, registry, SMTP dependencies
PayrollAPI → AppService (high confidence) — no OS dependencies

Generates install.ps1 for HRPortal (SMTP + COM sections). Generates ARM template with registry adapter (Key Vault-backed) for HRPortal.

Phase 4 — Deployment Planning:

Agent collects subscription/RG/region, validates PV4 availability. Packages both sites. Generates MigrationSettings.json with two plans:

mi-plan-hrportal (PremiumV4, IsCustomMode=true) — HRPortal
std-plan-payrollapi (PremiumV2) — PayrollAPI

Phase 5 — Execution:

Agent shows full summary with cost projection. You type "yes". Sites deploy. You get Azure URLs within minutes.

Prerequisites & Setup

Requirement	Purpose
Windows Server with IIS	Source server for discovery and packaging
PowerShell 5.1	Runs migration scripts (ships with Windows)
Python 3.10+	MCP server runtime
Administrator privileges	Required for IIS discovery, packaging, and migration
Azure subscription	Target for deployment (execution phase only)
Azure PowerShell (Az module)	Deploy to Azure (execution phase only)
Migration Scripts ZIP	Microsoft's PowerShell migration scripts
AppCat CLI	Source code analysis (optional)
FastMCP (mcp[cli]>=1.0.0)	MCP server framework

Data Flow & Artifacts

Every phase produces JSON artifacts that chain into the next phase:

Phase 1: discover_iis_sites ──→ ReadinessResults.json

│

Phase 2: assess_site_readiness ◄──────┘

assess_source_code ───→ Assessment JSONs

│

Phase 3: recommend_target ◄───────────┘

generate_install_script ──→ install.ps1

generate_adapter_arm ─────→ mi-adapters-template.json

│

Phase 4: package_site ────────────→ PackageResults.json + site ZIPs

generate_migration_settings → MigrationSettings.json

│

Phase 5: confirm_migration ◄──────────┘

migrate_sites ───────────→ MigrationResults.json

│

▼

Apps live on Azure

*.azurewebsites.net

Each artifact is inspectable, editable, and auditable — providing a complete record of what was assessed, recommended, and deployed.

Error Handling

The MCP server classifies errors into actionable categories:

Error	Cause	Resolution
ELEVATION_REQUIRED	Not running as Administrator	Restart VS Code / terminal as Admin
IIS_NOT_FOUND	IIS or WebAdministration module missing	Install IIS role + WebAdministration
AZURE_NOT_AUTHENTICATED	Not logged into Azure PowerShell	Run Connect-AzAccount
SCRIPT_NOT_FOUND	Migration scripts path not configured	Run configure_scripts_path
SCRIPT_TIMEOUT	PowerShell script exceeded time limit	Check IIS server responsiveness
OUTPUT_NOT_FOUND	Expected JSON output wasn't created	Verify script execution succeeded

Conclusion

The IIS Migration MCP Server turns what used to be a multi-week, expert-driven project into a guided conversation. It combines Microsoft's battle-tested migration PowerShell scripts with AI orchestration that understands the nuances of Managed Instance on App Service — the provisioning split, the PV4 constraint, the adapter configurations, and the OS-level customizations.

Whether you're migrating 1 site or 10, agentic migration reduces risk, eliminates guesswork, and produces auditable artifacts at every step. The human stays in control; the AI handles the complexity.

Get started: Download the migration scripts, set up the MCP server, and ask @iis-migrate to discover your IIS sites. The agents will take it from there.

This project is compatible with any MCP-enabled client: VS Code GitHub Copilot, Claude Desktop, Cursor, and more. The intelligence travels with the server, not the client.

Azure Red Hat OpenShift: Managed Identity and Workload Identity now generally available

MelanieKraintz007 — Sat, 11 Apr 2026 23:03:17 GMT

Azure Red Hat OpenShift now supports managed identities and workload identities as a generally available capability, so you can run OpenShift clusters and applications on Azure without long-lived service principal credentials.

ARO, identity, and Azure governance

With GA support for managed identities and workload identities, Azure Red Hat OpenShift uses short‑lived credentials and least‑privilege access to help organizations strengthen their security posture. This approach reduces reliance on long‑lived credentials and overly broad permissions, supporting enterprise security requirements while improving how identity is managed for OpenShift workloads.

As an Azure-native service, Azure Red Hat OpenShift also integrates directly with Microsoft Entra workload identities, Azure RBAC strengthening your overall security and identity management posture.

Platform identity: managed identities for ARO operators

At the platform layer, ARO now uses multiple user assigned managed identities rather than a single service principal with broad rights. Each identity is mapped to a specific ARO component and associated with a dedicated built-in ARO role, so permissions are scoped according to least privilege principles and aligned with Azure RBAC best practices.

You can wire this model in several ways: create identities and role assignments up front and reference them during deployment or use the Azure portal “all-in-one” experience to have identities and assignments created for you as part of cluster creation. Clusters can be deployed using the Azure portal or ARM/Bicep templates, and the native az aro (the formal az aro is available in version 2.84.0 or higher) commands provide a similar end-to-end experience for CLI-driven environments.

For an architectural deep dive into operators, scopes, and role assignment patterns, see Understand managed identities in Azure Red Hat OpenShift.

Application access: workload identity for Azure services

Workload identity uses Kubernetes-native OIDC federation so pods can securely access an Azure Managed Identity, which remains the underlying identity governed by Azure Entra ID and Azure RBAC.

For applications running on ARO, this capability provides workload identity—a way for pods to obtain short-lived tokens for an Azure managed identity without storing secrets in the cluster. Using Microsoft Entra workload identities and OIDC federation, you bind a user assigned managed identity to a Kubernetes service account; workloads using that service account automatically receives tokens for the associated identity at runtime.

This enables very granular patterns: for example, granting a specific application read-only access to a single Key Vault, storage account, or Azure SQL database, without sharing credentials across namespaces or relying on a cluster wide service principal. Enterprise teams can use this to connect AI and data workloads on ARO to services like Azure OpenAI, Azure SQL, or Azure Storage—giving each app just the access it needs for inference, data access, or logging while staying within standard Azure governance controls. The Learn guide, Deploy and configure an application using workload identity on an Azure Red Hat OpenShift managed identity cluster, walks through the workflow end-to-end.

Existing preview clusters and how to start

If you deployed ARO clusters with managed identities during the preview, no changes are required: those clusters automatically transition to GA and are fully supported for production use, with no migration or redeployment needed. You can continue to upgrade them using the standard OpenShift mechanisms, following the managed identity update guidance in the documentation for upgrade steps.

Clusters built with the current service principal model continue to receive full support; however, there is not yet a migration path available to move from service principal to managed identity. To adopt managed identity, you deploy a new ARO cluster with managed identities enabled and migrate workloads to it.

To get started with new clusters, begin with Understand managed identities in Azure Red Hat OpenShift to review concepts and considerations, then create a cluster using the Azure portal, an ARM/Bicep template, or the ARO CLI. Joint Red Hat–Microsoft demos and videos provide an end-to-end view of the experience, from deploying a managed identity enabled cluster through configuring workload identity for applications consuming Azure services.

Resources:

Interactive Demo - Create a managed identity Azure Red Hat OpenShift cluster
Concepts / architecture
Understand managed identities in Azure Red Hat OpenShift
https://learn.microsoft.com/en-us/azure/openshift/howto-understand-managed-identities[learn.microsoft]
Cluster creation
Create an Azure Red Hat OpenShift cluster with managed identities
https://learn.microsoft.com/en-us/azure/openshift/howto-create-openshift-cluster[learn.microsoft]
Applications / workload identity
Deploy and configure an application using workload identity on an Azure Red Hat OpenShift managed identity cluster
https://learn.microsoft.com/en-us/azure/openshift/howto-deploy-configure-application[learn.microsoft]
Automation (ARM/Bicep)
Deploy an Azure Red Hat OpenShift cluster with an ARM template or Bicep
https://learn.microsoft.com/en-us/azure/openshift/quickstart-openshift-arm-bicep-template[learn.microsoft]
Joint story from Red Hat
Managed Identity and Workload Identity support in Azure Red Hat OpenShift
https://www.redhat.com/en/blog/general-availability-managed-identity-and-workload-identity-microsoft-azure-red-hat-openshift

Apps on Azure Blog articles

AKS App Routing's Next Chapter: Gateway API with Istio

What Is It?

How It Differs From the Istio Service Mesh Add-On

Current Limitations

Getting Started

Migrating From NGINX Ingress

Inventory Your Ingress Resources

Convert Ingress Resources to Gateway API

Handle DNS and TLS

Deploy and Validate

Cut Over DNS and Clean Up

Upgrades and Lifecycle

What Should You Do Now?

Autonomous AKS Incident Response with Azure SRE Agent: From Alert to Verified Recovery in Minutes

Core concepts

Demo environment

How the agent was configured

Two incidents, two response modes

Incident 1. CPU starvation (alert driven, ~8 min MTTR)

Incident 2. OOMKilled (chat driven, ~4 min MTTR)

Side by side comparison

Why this matters

Teams + GitHub follow-up

Key takeaways

Three things to carry forward

Next steps

New in Azure SRE Agent: Log Analytics and Application Insights Connectors

Why This Matters

What You Get

Setup

Backed by Azure MCP

What It Looks Like

Things to Know

Learn More

Event-Driven IaC Operations with Azure SRE Agent: Terraform Drift Detection via HTTP Triggers

What Happens After terraform plan Finds Drift?

The Pipeline: Detection to Resolution in One Webhook

Setting Up the Pipeline

Step 1: Deploy the Infrastructure with Terraform

Step 2: Create the Drift Analysis Skill

Step 3: Create the HTTP Trigger

Step 4: Connect GitHub and Teams

The Incident Story

Act 1: The Latency Bug

Act 2: The On-Call Response

Act 3: The Drift Check Fires

What the Agent Found

Layer 1: Drift Detection

Layer 2: Incident Correlation

Layer 3: Smart Remediation

Layer 4: Investigation Summary

Layer 5: Teams Notification

The Payoff: A Self-Improving Agent

The Agent Improved Its Own Skill

The Agent Created a PR

Key Takeaways

Getting Started

Explaining what GitHub Copilot Modernization can (and cannot do)

Upgrading .NET Projects

What It Handles Well (and What It Doesn't)

Where to Start

Managing Multi‑Tenant Azure Resource with SRE Agent and Lighthouse

Summary - Benefits

Using an AI Agent to Troubleshoot and Fix Azure Function App Issues

Preparation

Troubleshooting Workflow

Conclusion

Gemma 4 on Azure Container Apps Serverless GPU

Why Self-Hosted AI on ACA?

What You're Building

Step 1: Deploy with azd up

Choose Your Model

Real-World Performance on ACA

Step 2: Verify Your Endpoint

Step 3: Connect OpenCode

The Privacy Case

Clean Up

Get Started

Govern AI Agents on App Service with the Microsoft Agent Governance Toolkit

Step 1: Deploy with `azd up`

`gen_ai.agent.name`

`gen_ai.agent.description`

`gen_ai.agent.id`

`gen_ai.operation.name`

`gen_ai.request.model` / `gen_ai.response.model`

`gen_ai.usage.input_tokens` / `gen_ai.usage.output_tokens`

`gen_ai.system`