Azure Networking Blog articles

Introducing Azure Front Door edge actions - Bringing secure, programmable logic to the edge

akhilkarmalkar — Mon, 20 Jul 2026 15:32:26 GMT

Modern web applications are expected to feel instant, secure, and personalized, no matter where users connect from or what device they use. Delivering that experience requires more than moving static content closer to users. It increasingly depends on making smart decisions at the edge, from validating requests and selecting the right origin to personalizing responses and stopping suspicious traffic before it reaches the backend.

Because of this, over the last decade, compute resources for web applications have steadily moved closer to users. What began with content caching at the edge quickly evolved into dynamic routing, security enforcement, and intelligent application traffic management. Today, enterprises expect even more: programmable compute at the edge! Developers and platform teams want the flexibility to run lightweight logic at the edge without giving up the scale, security, and operational confidence they expect from Azure.

Azure Front Door through its rulesets offers a robust, declarative framework for applying common traffic management and security policies, such as redirects, header manipulation, and conditional routing. They are optimized for scenarios where behavior can be defined statically and evaluated efficiently at scale.

As customer workloads become more dynamic, and application logic increasingly moves closer to the user, additional capabilities are often required beyond declarative configuration.

Meet Azure Front Door edge actions, now in public preview

We are excited to announce Azure Front Door edge actions which represent Microsoft’s next step in this evolution. With edge actions, customers can execute custom logic directly at Microsoft’s global edge, enabling new classes of real-time personalization, security, and resiliency scenarios, without pushing complexity back to origins!

Edge actions allow customers to author lightweight JavaScript functions that execute as part of Azure Front Door request processing. In the current public preview, edge actions are invoked during the client request phase and are attached to Azure Front Door routes through rulesets.

This tight integration means edge actions work seamlessly with existing Azure Front Door capabilities, including Web Application Firewall (WAF), caching, and routing - while extending them with programmable logic. Customers can incrementally adopt edge actions without re-architecting their applications or delivery pipelines.

What can you build with edge actions today

Edge actions in public preview are optimized for lightweight, latency-sensitive scenarios that benefit from immediate decision-making. Common use cases include:

A/B experimentation and canary rollouts: Evaluate request context at the edge and route users to different application experiences or releases without adding origin-side decision logic.

Request and response header manipulation: Add, remove, or transform headers to support security controls, experimentation, routing, and application modernization patterns.

Request rejection: Stop unwanted, malformed, or unauthorized requests before they consume origin resources.

Dynamic origin selection: Choose the best origin based on request attributes, health signals, geography, device type, or business logic.

URL rewrite and redirect: Adapt paths or redirect users at the edge to simplify migrations, campaign launches, localization, and application routing.

Authentication and authorization scenarios: Validate tokens or request attributes close to the user, helping protect applications before traffic reaches backend services.

Please refer to Azure/EdgeActionsSamples to start building edge actions today.

Designed for industry leading security-first edge compute

Running customer-defined code at the edge introduces unique security challenges. Edge environments now process untrusted user code at massive scales, making strong isolation non-negotiable for Azure.

Enter Hyperlight: Microsoft’s foundation for secure execution. With Hyperlight, each edge action code runs in its own lightweight, hardware-backed micro-VM - like a secure apartment, isolated from neighbors. Unlike traditional virtual machines, Hyperlight micro‑VMs are extremely lightweight and do not have a general-purpose guest operating system. Because of their small footprint, they provide a much more reduced attack surface with a hardware-backed isolation boundary between every instance of customer code, the host infrastructure, and other tenants.

This model allows Azure Front Door to combine the safety properties of hardware virtualization with the performance characteristics required for edge compute. Customer code is isolated by design to reduce blast radius to a single instance, helping protect both the platform and neighboring workloads.

How edge actions fit into the request path 

When a client request reaches Azure Front Door, it follows Azure Front Door’s standard routing pipeline. The request is first evaluated against any configured WAF policies and routing rules. If a rule is configured to invoke an edge action, execution is handed off to the edge actions runtime (Hyperlight) at the edge POP.

At execution time, Azure Front Door provides the edge action with a constrained, immutable context that includes request metadata, server variables, origin health information, and geo or device signals. The edge action can then modify the request, generate a response, or influence routing decisions, all within strict execution and resource limits designed to preserve platform stability.

Ship safely with versions and execution filters

Edge actions are built for operational confidence. Each edge action supports multiple versions of code, and customers can control which version executes using execution filters. This enables header-based routing, canary deployments, and A/B testing scenarios without downtime.

Teams can gradually introduce new logic, validate behavior with real traffic, and roll back instantly if needed, bringing modern DevOps practices directly to the edge.

Observe and operate with confidence

Edge actions integrate with Azure’s existing observability stack. Customers can emit logs from their edge action code and correlate execution results with Azure Front Door access and routing logs. This unified view simplifies troubleshooting and provides visibility into edge execution behavior in production environments.

Pricing

Please refer to the pricing document for details regarding Edge actions pricing, including applicable billing meters and usage-based charges on invocations and execution time.

What’s next for edge actions

The public preview of edge actions represents the foundation of a broader roadmap. Over time, Microsoft plans to expand invocation points, capabilities, and integrations - while continuing to prioritize security, performance, and operational simplicity. Stay tuned for scenarios like response invocations, image/video optimizations, and edge inferencing during general availability.

By combining programmable edge logic with Hyperlight-based isolation, Azure Front Door edge actions mark a significant milestone in Microsoft’s edge strategy - enabling customers to build faster, safer, and more adaptive applications on a global scale.

Start building with Azure Front Door edge actions today! Explore the documentation and bring your complex edge scenarios to life.

Azure Front Door: Resiliency Series – Part 3: Tenant isolation

AbhishekTiwari — Fri, 10 Jul 2026 22:08:29 GMT

Abhishek Tiwari, Vice President of Engineering, Azure Networking
Amit Srivastava, Partner Director of PM, Azure Networking
Varun Chawla, Partner Director of Engineering, Azure Networking

Azure Front Door serves hundreds of thousands of tenants from hundreds of edge locations, densely sharing the globally distributed edge fleet. That density is exactly what allows us to deliver global scale, performance, and cost efficiency. It also means that, without strong isolation, a single tenant’s incompatible configuration or anomalous traffic can, in the worst case, affect many other tenants. The October 2025 incidents reinforced how important it is to contain this class of risk. Our goal for this tenant isolation is simple to state and hard to achieve: no single tenant’s configuration or traffic should be able to impact any other tenant.

In Part 1 of our three part mini blog series, we outlined our four‑pillar strategy for improving the resiliency of Azure Front Door: configuration resiliency, data plane resiliency, tenant isolation, and accelerated Recovery Time Objective (RTO). Part 1 detailed how we would make configuration propagation safer and how the data plane keeps serving from a ‘last‑known‑good’ (LKG) configuration, even if an incompatible configuration change is propagated to the data plane. Part 2 turned to recovery, showing how we bring the system back to full operation in an accelerated, predictable, and in a bounded timeframe. In this final part, we turn to the tenant isolation pillar that ensures that any single tenant configuration or traffic issues are limited in scope to that tenant alone and do not impact other tenants. We will also show how Azure Front Door achieves single-tenant containment through configuration isolation, lazy loading, and a micro-cellular layered ingress-sharding architecture.

Repair status: all outstanding items now complete

Before we dive into tenant isolation, here is a final update on the overall repair items from the two October 2025 incidents (you can review the details in our Azure Incident Retrospective sessions for the October 9th and October 29th incidents). We are pleased to report that all outstanding work across every pillar is now complete and fully deployed in production – including the tenant isolation work described in this post. With these safeguards in place, we have also returned configuration propagation latency to pre‑incident levels while keeping platform stability our top priority. In the table below, “Completed” means broadly deployed in production.

Learning category	Goal	Repairs	Status
Safe customer configuration deployment	Incompatible configuration never propagates beyond ‘EUAP or canary regions’	Control plane and data plane defect fixes; forced synchronous configuration processing; additional stages with extended bake time; early detection of crash state	Completed
Data plane resiliency	Configuration processing cannot impact data plane availability	Manage data‑plane lifecycle to prevent outages caused by configuration‑processing defects; isolated work‑process in every data plane server to process and load the configuration	Completed
100% Azure Front Door resiliency posture for Microsoft internal services	Microsoft operates an isolated, independent Active/Active fleet with automatic failover for critical Azure services	Phase 1: onboarded critical services batch impacted on Oct 29th outage running on a day‑old configuration; Phase 2: automation & hardening of operations, auto‑failover and self‑management of onboarding for additional services	Completed
Recovery improvements	Data plane crash recovery in under 10 minutes	Data plane boot‑up time optimized via local cache; recovery time accelerated to under 10 minutes	Completed
Tenant isolation	No configuration or traffic regression can impact other tenants	Micro‑cellular Azure Front Door with ingress layered shards	Completed

Why isolation at edge scale is deceptively hard

Traditional isolation techniques such as dedicating separate hardware to each tenant or running every tenant inside its own virtual machine are impractical at the edge. Edge sites are constrained on space, power, and capacity. The entire premise of a modern, multi-tenant application delivery platform is that any tenant can be served from any site closest to the user. We cannot simply partition hundreds of thousands of tenants onto dedicated machines without giving up either proximity, scale, or the efficiency that make the edge fast and cost efficient. Isolation therefore, must be achieved in software, inside a multi-tenant fleet.

What we already do today

Azure Front Door already includes several layers of tenant isolation and partitioning. However, the incidents in October clearly highlighted that these techniques were not enough. Prior to the incidents, our protection mechanisms included:

Infrastructure partitioning. Edge sites were organized into physically isolated primary and fallback traffic rings.
Noisy‑neighbor protection. Fair‑share resource allocation, rate limiting, and anomaly-based load protection kept any single tenant from monopolizing shared resources such as CPU, memory, or network bandwidth on an Azure Front Door server.
Circuit breakers. Circuit breakers shed costly work first and can disable a risky per‑tenant feature before it exhausts shared resources on a server.
Real‑time crash protection. A crash‑analysis system correlates crash signatures across machines and can pinpoint and block crash patterns caused by tenant IPs or traffic patterns.

While these protections are valuable, many of them are reactive and proved insufficient during the October incidents. The next generation of isolation makes single‑tenant containment a fundamental part of the platform which governs how configurations are loaded, and how traffic is served.

Configuration isolation: loading only what is needed

Part 2 introduced ‘lazy loading’ as a recovery optimization technique. It is also an important configuration‑isolation mechanism. Historically, every worker on every edge server had to be ready to serve any tenant, which meant each worker loaded a large set of tenant configurations. A single incompatible configuration could therefore ripple across many workers.

With lazy loading, a worker loads a tenant’s configuration and its TLS certificates only when it actually receives traffic for that tenant. The practical consequence for isolation is powerful: a faulty configuration can only affect the workers that have loaded that specific tenant, never the entire server or fleet. Combined with per‑tenant validation on load, and the Food Taster safeguard from Part 1 (a sacrificial process that pretests every configuration change in isolation), configuration problems are caught early and contained to the smallest possible footprint.

Figure 1: With lazy loading, an incompatible configuration is contained to the workers serving that tenant, instead of poisoning the whole server.

Tenant isolation: a micro-cellular, layered ingress sharding architecture

Configuration isolation limits the blast radius of an incompatible configuration. Traffic isolation addresses the other half of the problem: a tenant whose anomalous traffic incident, like a sudden surge, a pathological request pattern, or malicious activity, could degrade a shared worker. Our approach is a micro‑cellular architecture that combines multiple concepts working together.

Worker‑process isolation. Each edge server already runs many independent worker processes. Instead of letting every worker serve every tenant, we assign tenants to specific groups of workers. Those worker group (shards) become the unit of isolation: if a tenant destabilizes its shard, the impact is contained to that shard’s workers while the rest of the server keeps serving normally.
Ingress sharding. Rather than a handful of fixed shards, we compose shards from overlapping subsets of a server’s workers. Even a modest number of workers can be combined into an enormous number of distinct, overlapping shards – giving us a very large number of fine‑grained fault domains without dedicating hardware to every tenant.

Figure 2: Tenants are randomly assigned to different shards on each server (layer). Even when a good tenant shares the noisy tenant’s shard on one layer, routing steers its traffic to healthy shards on the others.

Multi‑layer ingress sharding. This is where ‘layered’ comes in. Each edge server is treated as an independent layer, and each tenant is assigned to a different, randomly chosen shard on every server. Because assignments are independent from one server to the next, two tenants that happen to share a shard on one server are extremely unlikely to share a shard again on another server. The chance of any good tenant repeatedly colliding with a noisy tenant across many servers becomes vanishingly small.
Intelligent ingress routing. Tying it together is a routing layer that terminates each incoming connection, identifies the tenant, and steers the request to that tenant’s assigned, healthy shard. If a shard is unhealthy or saturated, traffic is directed to the tenant’s healthy shards on other layers.

Figure 3: Intelligent Ingress Routing

The combined effect is that when a noisy tenant overwhelms or crashes its shard on one server, only that shard is affected. Because every other tenant is spread across a different, randomized set of shards, they continue to find healthy paths, and the routing layer moves their traffic accordingly. A worst-case availability problem is downgraded to, at most, a small and redistributable capacity problem, that the routing layer smooths over.

An in-depth technical analysis of layered ingress sharding is available here for reference.

Shrinking the blast radius

Taken together, these mechanisms fundamentally change the shape of failure. In a uniform, fully shared fleet, an incompatible tenant can, in the worst case, affect a large share of the tenants on a machine, in an edge location, or beyond. With configuration isolation and layered ingress sharding, the same failure is confined to a subset of workers serving the offending tenant. Our target for this tenant isolation pillar is effectively single‑tenant containment: a configuration or traffic anomaly caused by one tenant should never cause issues to any other tenants.

Figure 4: From a shared fleet where a single tenant can affect many, to micro‑cellular shards that confine impact to the offending tenant.

Validating isolation in practice

As with our recovery work, we don’t simply design these boundaries and assume they hold, we test them. Through deliberate fault‑injections, we have pushed noisy and faulty tenants into the system and confirmed that impact stayed contained to the offending shard, that healthy tenants kept serving, and that the routing layer steered around unhealthy shards as intended. This turns isolation from a design claim into a well-drilled, and repeatable outcome.

Closing

This post concludes our three-part mini blog series on Azure Front Door resiliency. We have shared how we are making configuration propagation safer (Part 1), recovering faster when failures do occur (Part 2), and containing the blast radius stemming from any single tenant through configuration isolation and a micro‑cellular, layered‑sharding architecture (Part 3).

Resiliency, however, is not a project with an end date. It is an ongoing commitment. While this series concludes the blog series of our response to the October 2025 incidents, our investments in Azure Front Door’s resiliency, isolation, and recovery will continue. As we make further improvements, we will keep sharing them with you.

We deeply value our customers’ trust in Azure Front Door. We remain committed to exceeding expectations for security, reliability, and transparency.

Azure Firewall explicit proxy Migration Guide

devanshirastogi — Fri, 19 Jun 2026 18:21:40 GMT

Purpose of the blog

This blog outlines the key upcoming changes to Azure Firewall explicit proxy and provides detailed migration guidance for customers using PAC file–based configurations. It also covers the supported deployment options for enabling explicit proxy after the changes are released, including the Azure portal, PowerShell, and Azure CLI.

Who is this article for?

This article is intended for customers currently using Azure Firewall explicit proxy in preview. If you use PAC file–based proxy configuration, follow the steps below to configure the new PAC file SAS URL retrieval method, which will become the standard approach going forward.

Azure Firewall explicit proxy

Azure Firewall operates in a transparent proxy mode by default. In this mode, you use a user-defined route (UDR) configuration to send traffic to the firewall. The firewall intercepts that traffic inline and passes it to the destination.

When you set up explicit proxy on the outbound path, you can configure a proxy setting on the sending application (such as a web browser) with Azure Firewall configured as the proxy. As a result, traffic from the sending application goes to the firewall's private IP address and therefore egresses directly from the firewall without using a UDR.

The Azure Firewall explicit proxy feature is in Preview at the time this article was published.

Upcoming changes to the explicit proxy feature in Azure Firewall

PAC (Proxy Auto-Configuration) file size is now limited to 256 KB.
Support HTTP and HTTPS traffic over a single HTTP proxy port.
Removal of the previous dual-port configuration requirement (explicit proxy v1).
Ability to enable explicit proxy directly using Firewall Policy creation in the Azure portal.
Following general availability (GA), explicit proxy will require both a PAC file SAS URL and Managed Identity (MSI), along with the appropriate role assignments to meet Microsoft security standards.

Follow the steps below to migrate to the new PAC file retrieval model that uses customer-managed Azure Storage and Managed Identity authentication.

Step 1: Create a PAC File SAS URL

Create an Azure Storage container by following the steps in Manage blob containers using the Azure portal - Azure Storage | Microsoft Learn. Note: Use a subscription in which required permissions to add roles exist.
Upload the PAC file to the storage container.

Select the uploaded file and copy the file URL. Example URL: "https://eproxypstestresources.blob.core.windows.net/explicitproxycontainer/proxy.pac"

Step 2: Create a Managed Identity and assign required roles

Navigate to Managed Identity blade and create a Managed Identity. See Manage user-assigned managed identities using the Azure portal - Managed identities for Azure resources | Microsoft Learn for more details.
Go to the storage account resource created in the previous step and navigate to Access Control (IAM). Select Add to add the role assignment.
Go to the Add role assignment page and search for Storage Blob Data Contributor and Storage Blob Data Reader, then select both.

Go to Members → Managed Identity and select the identity created earlier. Review the changes and click Assign in Review + Assign blade.

Verify that your changes are reflected under Role Assignments by searching for the managed identity. Note: Make sure that the Managed Identity created has prefix "PacFileMSI-".

Configuration using portal, PowerShell and Azure CLI

Portal configuration

After obtaining the PAC file SAS URL and Managed Identity, enable the PAC file in the explicit proxy configuration by:

providing the PAC file SAS URL, and
selecting the Managed Identity created in the previous steps.

PowerShell configuration

To securely use explicit proxy, customers must provide:

the PAC file SAS URL, and
a Managed Identity with the required permissions to access the PAC file from the customer-managed Blob Storage account.

Create Firewall Policy with explicit proxy settings:$exProxy = New-AzFirewallPolicyExplicitProxy ` -EnableExplicitProxy ` -HttpPort 100 ` -EnablePacFile ` -PacFilePort 130 ` -PacFile "https://sampleurlfortesting.blob.core.windows.net/container/proxy.pac"
```
Update Firewall Policy with explicit proxy configuration:
```
New-AzFirewallPolicy ` -Name "fp1" ` -ResourceGroupName "TestRg" ` -ExplicitProxy $exProxy ` -UserAssignedIdentityId "/subscriptions/e7eb2257-46e4-4826-94df-153853fea38f/resourcegroups/testrg/providers/Microsoft.ManagedIdentity/userAssignedIdentities/PacFileMSI-eproxyidentity"

Azure CLI configuration

Create Firewall Policy with explicit proxy settings:az network firewall policy create -g "testrg" -n "testfwpolicy" --sku Premium --explicit-proxy enable-explicit-proxy=true http-port=9001 enable-pac-file=true pac-file-port=122 pac-file="https://eproxypstestresources.blob.core.windows.net/explicitproxycontainer/proxy.pac" --identity "Identity_ID"
Update Firewall Policy with Explicit Proxy Configuration:az network firewall policy update -g "testrg" -n "testfwpolicy" --explicit-proxy enable-explicit-proxy=true http-port=9001 enable-pac-file=true pac-file-port=124 pac-file="https://eproxypstestresources.blob.core.windows.net/explicitproxycontainer/proxy.pac" --identity "Identity_ID"

Migrating from MSEE Hairpin Routing to AVNM Mesh for Large-Scale VNet-to-VNet Connectivity

Jay-Li — Fri, 19 Jun 2026 07:55:59 GMT

Introduction

A common pattern in large Azure deployments is to route VNet-to-VNet traffic through Microsoft Enterprise Edge (MSEE) routers. This happens when spoke VNets in a hub-and-spoke topology communicate with each other by hairpinning through the ExpressRoute circuit: traffic exits the Azure data center, traverses MSEE, and re-enters the data center to reach the destination VNet.

This pattern works, but it was not designed as the long-term connectivity model for east-west traffic. With Azure Virtual Network Manager (AVNM) mesh connectivity and recent scale improvements — including high-scale mesh up to 5,000 VNets and High-Scale Private Endpoints (HSPE) up to 20,000 Private Endpoints across connected VNets — enterprises can migrate to a direct, in-datacenter routing model that removes MSEE dependency for VNet-to-VNet traffic.

This article explains why the migration is useful, what the new scale limits are, how to enable the required features, and how to execute the migration with minimal disruption.

Who This Is For

This migration is most relevant if you have 50 or more spoke VNets communicating east-west through ExpressRoute hairpin routing, are approaching VNet peering limits, want to reduce ExpressRoute utilization from internal traffic, or need a simpler centrally managed connectivity model. Even if most east-west flows must continue through a hub firewall for inspection, you can still simplify connectivity management for the flows allowed to go direct.

Why MSEE Hairpin Routing for VNet-to-VNet Is Not Recommended

When spoke VNets communicate through MSEE, traffic follows a suboptimal path:

Spoke A → Hub VNet → ExpressRoute Gateway → MSEE → ExpressRoute Gateway → Hub VNet → Spoke B

Single point of failure. MSEE becomes a shared dependency for east-west traffic. An MSEE outage or capacity constraint can affect every VNet pair in the topology.
Lower-latency path. Traffic no longer needs to leave the data center and return for spoke-to-spoke communication. Direct mesh keeps east-west traffic on an in-datacenter path, which is generally lower latency than hairpinning through MSEE.
Bandwidth constraints. MSEE circuits have finite bandwidth. Routing east-west traffic through them competes with north-south on-premises traffic and can saturate the circuit.
Operational risk at scale. Large deployments place significant load on MSEE infrastructure, creating scalability, reliability, and operational concerns for environments with thousands of VNets.

Why Manual Peering Was Not a Practical Alternative

The obvious alternative — creating direct VNet peerings between every spoke pair — solves the MSEE dependency but introduces its own operational complexity:

Combinatorial growth. Connecting N spokes requires N × (N - 1) / 2 peering relationships. For 100 spokes, that is 4,950 peerings. For 1,000 spokes, it is nearly 500,000.
Management overhead. Each peering must be individually provisioned, monitored, and maintained. This increases drift, audit, and operational overhead.

How AVNM Mesh Solves This

AVNM mesh provides group-based connectivity. You define a set of VNets as a network group, apply a mesh connectivity configuration, and AVNM establishes bi-directional connectivity across all members.

Traffic between meshed VNets stays within the Azure data center: no MSEE traversal, no hub hop, and no manual peering management.

Define once, connect all. A single mesh configuration connects every VNet in the group to every other VNet.
Centralized management. Add or remove VNets from the group; AVNM reconciles connectivity automatically.
Direct spoke-to-spoke paths. Traffic flows directly between VNets, bypassing both the hub and MSEE.
Dynamic membership. Use Azure Policy to auto-enroll new VNets based on tags or resource group conditions.

How to Migrate Large Scale Topology

High-scale mesh lets user migrate larger topology — up to 5,000 VNets and 20,000 Private Endpoints in a mesh.

Scale — Standard Mesh vs. High-Scale Mesh

Dimension	Standard Mesh	High-Scale Mesh
VNets per mesh	Up to 250 (soft limit, can request increase)	Up to 5,000
Private Endpoints per mesh	Up to 2,000	Up to 20,000 (with HSPE enabled)
Private Endpoints per VNet	Up to 1,000	Up to 5,000 (with HSPE enabled)

Enabling High-Scale Private Endpoints (HSPE)

As a mesh footprint grows, so does the number of Private Endpoints deployed across the connected VNets. The default platform limits — 1,000 Private Endpoints per VNet and 2,000 across connected VNets and mesh — can be reached quickly in large environments. Enabling HSPE raises these limits to 5,000 and 20,000, respectively.

For large-scale mesh migrations, enable HSPE proactively if the environment is expected to grow toward standard Private Endpoint limits by following the steps below.

Step 1 — Prepare Each VNet

Ensure Private Endpoint Network Policies are set to Enabled or RouteTableEnabled on all subnets containing Private Endpoints. This is a prerequisite.
Set the VNet-level property PrivateEndpointVNetPolicies to Basic. This activates HSPE on the VNet.

Step 2 — Enable HSPE on the Mesh Configuration

In the AVNM mesh connectivity configuration, enable the option for high-scale private endpoints. AVNM validates that all VNets in the mesh are HSPE-enabled. If any VNet is missing the configuration, the deployment is blocked with a clear error.

Step 3 — Deploy

Deploy the connectivity configuration. AVNM applies HSPE across the mesh.

Behavior Changes When Enabling HSPE

Brief connection reset. Enabling or disabling HSPE triggers a one-time, approximately 1-second connection reset for existing Private Endpoint connections in the VNet. Plan this during a maintenance window.
Per-PE Bytes In / Out monitoring is no longer available. HSPE treats each Private Endpoint IP like any other IP in the VNet, which removes per-PE traffic counters. If you depend on per-PE metrics, evaluate alternatives before enabling.
On-premises PE traffic billing changes. PE traffic originating from on-premises appears as an aggregate bill on the gateway VNet, not on the individual Private Endpoint resource. The total bill does not change.

How to Avoid Downtime

Mesh coexists with existing peerings. AVNM does not delete manually created peerings unless explicitly configured to do so.
Traffic shifts automatically. Once mesh is deployed, spoke-to-spoke traffic routes directly. The MSEE hairpin path remains available if mesh is removed.
No reconfiguration of hub components. Firewalls, gateways, and NVAs in the hub continue to function. North-south on-premises traffic still flows through the hub gateway.
Rollback is simple while the legacy path remains. Keep the MSEE hairpin path in place during validation so affected VNets can fall back by removing them from the network group or undeploying the mesh configuration.

Security and East-West Traffic Inspection

A common concern is whether direct mesh connectivity bypasses hub firewalls or network virtual appliances. The answer depends on how inspection is enforced today.

Mesh provides connectivity, not routing policy. If spoke subnets have UDRs that direct traffic through a hub NVA or firewall, those UDRs continue to apply and can keep inspected flows on the firewall path.
Security Admin Rules provide centralized segmentation. For flows that do not require firewall inspection, AVNM Security Admin Rules can enforce network-level allow or deny policies across network groups.
Use both where appropriate. Mesh can provide direct connectivity for approved flows while Security Admin Rules enforce segmentation boundaries where required.

Recommendation: Before migrating, inventory which spoke-to-spoke flows currently traverses the firewall. Decide per flow whether to maintain inspection by keeping UDRs in place or allowing a direct mesh path by removing the UDR for that flow pair.

Migration Order and Process

The migration from MSEE hairpin routing to AVNM mesh is non-disruptive by design. Mesh connectivity overlays on top of existing peerings and takes routing precedence for east-west traffic. You do not need to tear down the existing hub-and-spoke topology first.

Recommended Steps

Design the mesh topology. Group VNets by region into mesh groups. If you expect more than 250 VNets per mesh, register the AllowHighScaleConnectedGroup feature in advance.
Create a Network Manager and define Network Groups. Ensure that the AVNM scope covers all relevant subscriptions. Use static membership for initial migration or dynamic membership through Azure Policy for ongoing enrollment.
Enable HSPE on every VNet in the mesh. Follow the HSPE enablement steps if you need to have more than 2,000 Pes in a mesh. Schedule the change during a maintenance window to account for the brief connection reset.
Create the mesh connectivity configuration in AVNM. Select the network groups, enable mesh topology, enable high-scale private endpoints, and enable global mesh if cross-region connectivity is required.
Deploy incrementally. Start with a pilot region or non-critical environment. Validate effective routes, spoke-to-spoke connectivity, spoke-to-hub-to-on-premises connectivity, Private Endpoint reachability, and expected VNet flow log patterns before expanding to production regions.

Route Behavior During and After Migration

During migration, mesh and MSEE can coexist. Mesh-connected VNets receive direct routes for mesh-connected destinations, while existing ExpressRoute gateway routes continue to serve on-premises destinations. UDRs still override system routes, so forced-tunneling and firewall inspection patterns remain in effect when UDRs are present.

Mesh destinations. Traffic between mesh-connected VNets goes directly instead of hairpinning through MSEE when no UDR overrides the route.
On-premises destinations. ExpressRoute continues to provide north-south connectivity to on-premises networks.
Gateway transit. Spokes can continue to reach on-premises through the hub gateway when the design uses gateway transit.

Infrastructure-as-Code Considerations

If VNet peerings are managed through Terraform, Bicep, or ARM templates, treat AVNM mesh as the new source of truth only after validation.

Deploy mesh first. AVNM mesh can coexist with existing peerings, so do not remove peering resources from IaC before validating the mesh path.
Validate the traffic path. Use effective routes, Connection Monitor, and flow logs to confirm traffic is using mesh where expected.
Guard against drift. Review pipeline state and lifecycle settings before decommissioning old peerings, especially in environments where multiple teams manage network resources.
Codify AVNM. Manage the Network Manager, network groups, configuration, and deployment through IaC so mesh becomes the governed connectivity model.

DNS Resolution Across Mesh

Mesh connectivity does not change DNS resolution behavior by itself. If spoke VNets are already linked to Private DNS Zones hosted or managed through the hub, those links continue to determine name resolution. If spokes use custom DNS servers in the hub, verify that any UDR changes made during migration do not unintentionally alter the DNS traffic path.

Migration Example — Two Hub-and-Spoke Topologies into a Single Mesh

This example shows how an enterprise can migrate two regional hub-and-spoke environments into one centrally managed AVNM mesh while preserving the existing MSEE path during validation.

Current State — Two Hub-and-Spoke Topologies with MSEE Hairpin

Contoso Corp operates a large Azure environment in the East US region with two hub-and-spoke topologies:

	Topology A	Topology B
Hub VNet	Hub-A	Hub-B
Spoke VNets	500	500
ExpressRoute Gateway	ER-GW-A in Hub-A	ER-GW-B in Hub-B
ExpressRoute Circuit	Shared circuit, connected to both gateways	Same shared circuit
Avg. Private Endpoints per spoke	~8 (4,000 total)	~12 (6,000 total)
Total Private Endpoints	10,000 across both topologies

How traffic flows today:

Spoke-to-spoke within Topology A: Spoke-A-01 → Hub-A → ER-GW-A → MSEE → ER-GW-A → Hub-A → Spoke-A-02
Spoke-to-spoke across topologies: Spoke-A-01 → Hub-A → ER-GW-A → MSEE → ER-GW-B → Hub-B → Spoke-B-01

Every spoke-to-spoke packet — whether within the same topology or across topologies — exits the data center, traverses MSEE, and re-enters. With 1,000 spokes generating east-west traffic, MSEE becomes a shared single point of failure and adds latency to every flow.

Target State — A Single AVNM Mesh with 1,000 VNets

Contoso's goal is to consolidate all 1,000 spoke VNets into a single AVNM mesh, removing MSEE from the east-west traffic path.

Spoke-to-spoke traffic, any pair: Spoke-A-01 → directly→ Spoke-B-01. Traffic stays in the data center and uses the direct mesh path.
MSEE role: MSEE carries north-south on-premises traffic. East-west load is removed from the ExpressRoute hairpin path.

Migration Execution

Phase 0 — Pre-Work

Register the feature. Since the mesh contains 1,000 VNets, above the 250 standard limit, Contoso registers the AllowHighScaleConnectedGroup feature flag on the subscription. This enables high-scale mesh support for up to 5,000 VNets.
Inventory Private Endpoints. With 10,000 Private Endpoints across 1,000 VNets, Contoso exceeds the standard mesh Private Endpoint limit of 2,000. HSPE must be enabled.

Phase 1 — Enable HSPE on All 1,000 Spoke VNets

Batch 1: Enable HSPE on all 500 spoke VNets in Topology A during the maintenance window by following the instructions.
Batch 2: Apply the same configuration to all 500 spokes in Topology B.
Expected impact: Each VNet may experience a brief connection reset for existing Private Endpoint connections when HSPE is enabled. Schedule the change during a maintenance window.

Phase 2 — Create the AVNM Mesh

Create a Network Manager scoped to the management group containing all 1,000 spoke VNets.
Define a single network group called eastus-mesh-all-spokes with dynamic membership using an Azure Policy and the tags.
Create a mesh connectivity configuration. Set topology to Mesh, network group to eastus-mesh-all-spokes, high-scale private endpoints to Enabled, and global mesh to Not needed because all VNets are in the same region.
Save the configuration as a draft and do not deploy yet.

Phase 3 — Incremental Deployment

Wave 1 — Pilot: Contoso deploys the mesh configuration to 50 dev/test spokes, either by using a temporary network group or by tagging only those VNets initially. Validation includes effective routes showing ConnectedGroup as the next-hop type for meshed spoke prefixes, spoke-to-spoke connectivity through the direct mesh path, Private Endpoint reachability across meshed spokes, unchanged on-premises connectivity through the hub and MSEE, and VNet flow logs that confirm expected direct spoke-to-spoke flows.

Wave 2 — Light production traffic VNets: After a successful pilot, Contoso tags light production traffic VNets. The dynamic network group picks them up automatically. Contoso redeploys the connectivity configuration, runs the same validation checklist, and monitors traffic to confirm that east-west traffic from these VNets is moving away from the ExpressRoute hairpin path.

Wave 3 — All remaining production VNets: Contoso tags all remaining spokes and redeploys the configuration. At this point, all 1,000 spokes are in the mesh.

No downtime migration: During Waves 2 and 3, existing MSEE hairpin routing remains functional. VNets not yet in the mesh continue to communicate through MSEE. VNets already in the mesh communicate directly through the mesh. This avoids a planned connectivity gap during migration.

Phase 4 — Post-Migration Validation

After deployment, confirm that mesh is active and traffic is taking the expected path before decommissioning the legacy route.

Effective routes. Verify spoke subnets show direct routes to peer VNet prefixes instead of routing through the gateway or MSEE.
Connection Monitor. Track representative spoke-to-spoke flows and compare latency and reachability before and after migration.
VNet flow logs. Confirm east-west traffic matches the expected mesh path and is not still traversing the ExpressRoute gateway path.
Network Watcher topology. Visualize the resulting connectivity model and identify any VNets not enrolled in the target network group.

If traffic is still hairpinning after mesh deployment, check for UDRs overriding system routes, spokes missing from the network group, deployment not committed to the target region, or IaC pipelines recreating legacy peerings.

Rollback

Quick rollback while MSEE remains in place. Remove affected VNets from the network group or undeploy the mesh connectivity configuration. AVNM removes only the connectivity it created, and traffic can fall back to the existing MSEE hairpin path.
Rollback after decommissioning legacy paths. If old peerings or route dependencies have already been removed, rollback may require reprovisioning those resources and should be treated as a longer change window.
Recommendation. Keep the MSEE hairpin path available for at least two weeks after mesh deployment, monitor traffic patterns, and only then remove the legacy path.

Before and After Summary

Metric	Before (MSEE Hairpin)	After (AVNM HSPE Mesh)
Spoke-to-spoke latency in the same region	Higher due to the MSEE hairpin path	Lower-latency direct in-datacenter path; actual latency depends on workload, region, and network conditions
Traffic path for east-west	Spoke → Hub → MSEE → Hub → Spoke	Spoke → Spoke directly through the Mesh
MSEE dependency for east-west	Yes, shared dependency	No MSEE dependency for east-west traffic
Manual peerings required	0 when using hairpin routing, but 499,500 if built manually for 1,000 spokes	0 manual spoke-to-spoke peerings; AVNM manages connectivity
Private Endpoints supported	2,000 per mesh under standard limits	20,000 per mesh with HSPE
Rollback complexity	Not applicable to the current hairpin model	Remove VNets from the network group or undeploy the connectivity configuration
Migration downtime	Not applicable	Designed for no planned downtime when deployed incrementally and validated carefully

Closing Notes

Migrating to AVNM mesh does not require tearing down your existing network. The hub gateway, firewalls, and NVAs continue to function as they do today. What changes is that east-west spoke-to-spoke traffic stops leaving the data center unnecessarily.

MSEE is not the right tool for the east-west fabric. Removing internal traffic from the ExpressRoute circuit is a reliability and capacity improvement.
AVNM mesh replaces combinatorial complexity with group-based intent. The operational model scales with the number of groups, not the number of VNets.
High-scale mesh and HSPE remove the ceiling — up to 5,000 VNets and 20,000 Private Endpoints per mesh.
The migration is incremental and reversible. Mesh coexists with existing paths, and you can validate wave by wave before decommissioning the legacy route.

Start with a pilot mesh in a non-critical environment, validate the traffic shift, and expand from there.

Resources

ICMP Support for Azure StandardV2 NAT Gateway

malaikanazim — Wed, 17 Jun 2026 17:44:22 GMT

Our customers, across all industries, depend on consistent outbound connectivity and observability to operate and troubleshoot their cloud workloads at scale. As Azure environments grow in size and complexity, teams increasingly rely on standard network diagnostic tools such as ping to quickly validate reachability and identify connectivity issues.

When customers route outbound traffic through Azure NAT Gateway, they benefit from centralized and scalable outbound connectivity designed to prevent SNAT port exhaustion, while simplifying management by removing the need for per‑resource public IPs. Historically, outbound connectivity through Azure NAT Gateway supported TCP and UDP traffic only for Azure workloads.

We are pleased to announce that Azure StandardV2 NAT Gateway now supports outbound ping through ICMP Echo Request and Echo Reply traffic. Workloads that rely on NAT Gateway for internet egress can now use ping to validate reachability, improving operational visibility and simplifying network troubleshooting while preserving the scalability and resiliency characteristics of NAT Gateway.

What’s now supported?

Azure StandardV2 NAT Gateway supports outbound ICMP Echo Request and Echo Reply traffic (ping) for IPv4 and IPv6. Other ICMP message types are not currently supported.

With this update, ICMP traffic sent from workloads behind a NAT Gateway can be translated to a public IP address and reliably receive responses from external endpoints. This brings ICMP behavior in line with existing TCP and UDP outbound flows, enabling consistent network diagnostics across all common transport types.

This capability is available automatically when workloads egress through a StandardV2 NAT Gateway - no additional configuration or rule changes are required.

Scenarios enabled by ICMP support

Support for ICMP through StandardV2 NAT Gateway unlocks several common operational scenarios:

Basic reachability testing – Operators can use ping from virtual machines, VM scale sets, or AKS nodes behind NAT Gateway to verify outbound connectivity to internet endpoints.
Health validation and troubleshooting – Teams can perform quick checks during incident triage without deploying jump hosts, public IPs per VM, or temporary firewall exceptions.

These scenarios are especially valuable in large‑scale environments where workloads rely on centralized outbound connectivity via NAT Gateway.

How ICMP works through StandardV2 NAT Gateway

Unlike TCP or UDP, ICMP does not use ports to differentiate flows. Instead, ICMP messages include an identifier field, which is used by diagnostic tools to correlate requests and responses.

A common troubleshooting scenario is when workloads deployed behind an Azure StandardV2 NAT Gateway are unable to reach external endpoints. Since these workloads typically do not have public IP addresses, it can be difficult to quickly determine whether the issue is related to basic network reachability or higher-layer configuration. In such cases, ICMP‑based diagnostics such as ping can be used to validate outbound connectivity and help isolate the source of the issue.

When a workload sends an outbound ICMP packet through a NAT Gateway:

NAT Gateway translates the source private IP to one of its associated public IP addresses.
The platform tracks the ICMP flow using the ICMP identifier and other packet attributes.
Return ICMP responses from the external destination are matched to the originating flow.
The response is translated back to the original private IP and delivered to the workload.

Flow chart showing handling of outbound ICMP Echo Request and Echo Reply traffic through Azure NAT Gateway

This stateful handling allows replies to be routed correctly even when multiple workloads are sending ICMP traffic concurrently through the same NAT Gateway.

Using ICMP for end-to-end outbound troubleshooting

To validate outbound connectivity using ICMP from workloads deployed behind StandardV2 NAT Gateway:

Go to your virtual network in the Azure portal and select the subnet hosting your workload.
Verify that a StandardV2 NAT Gateway is associated with the subnet.
The workload does not require a public IP address.
Connect to the workload (for example, a virtual machine) running in the subnet.
Run an ICMP ping to an external destination to validate outbound reachability.

Successful ping from a workload behind an Azure StandardV2 NAT Gateway demonstrating outbound connectivity and ICMP Echo reply.
Review the results.
Successful ICMP echo replies confirm that outbound connectivity through StandardV2 NAT Gateway is functioning and that ICMP traffic is translated and returned correctly

Summary

With ICMP Echo support, Azure StandardV2 NAT Gateway provides a more complete outbound networking experience by enabling essential diagnostic tools to function reliably through NAT. Customers can validate reachability, perform basic troubleshooting, and diagnose connectivity issues while continuing to benefit from the simplicity and scale of centralized outbound connectivity.

Learn More

Network security perimeter for Azure Service Bus & also now available in Azure Gov. Regions

shashankamalladi — Tue, 16 Jun 2026 10:56:27 GMT

TL; DR

Network security perimeter support for Azure Service Bus is now Generally Available. With this, you can now place your Service Bus namespace inside a central security boundary and apply perimeter-based governance for inbound/outbound network access—while keeping key PaaS-to-PaaS scenarios secure and auditable.

Introduction

We’re excited to announce that network security perimeter support for Azure Service Bus is now Generally Available (GA).

This milestone brings one of Azure’s most widely used messaging services into the network security perimeter ecosystem, enabling customers to define a centralized security boundary across messaging and data services.

Alongside this, we are also expanding network security perimeter’s reach — network security perimeter is now available in Azure Government regions, including:

Texas
Arizona
Virginia
DoD East
DoD Central

This ensures that customers operating in regulated, sovereign, and mission-critical environments can adopt network security perimeter while meeting their compliance and regional requirements.

Why this matters?

Modern applications rely heavily on messaging layers like Service Bus. These systems often connect microservices, data platforms, key management systems and external integrations. As architectures scale, managing network access individually becomes complex and error prone.

Network security perimeter changes this by introducing a perimeter-based access model, where:

Communication is restricted by default
Access must be explicitly allowed
Governance is applied consistently across services

With Service Bus now onboarded and network security perimeter extending into Azure Gov regions, customers can apply this model across both commercial and sovereign environments.

What you can do with Service Bus + network security perimeter

Confine communication within a security boundary

Service Bus namespaces communicate only with resources inside the perimeter by default—blocking unintended access.

Secure PaaS-to-PaaS communication

Enable secure interactions between Service Bus, Azure Key Vault (for CMK scenarios) and other network security perimeter-enabled services (What is a network security perimeter? - Azure Private Link | Microsoft Learn)

Define explicit access controls

Inbound rules → IP ranges and subscriptions
Outbound rules → FQDN-based filtering

Enable audit and compliance visibility

Diagnostic logs capture all access attempts, supporting compliance and investigation workflows.

Use Private Link seamlessly

Private endpoint traffic continues to work without additional configuration inside the perimeter.

Azure Government Availability

With this update, network security perimeter is now available in key Azure Government regions (Texas Arizona Virginia DoD East DoD Central), enabling:

Consistent security across clouds

Apply the same network security perimeter model across public Azure regions and Azure Government environments.

Support for regulated workloads

Customers in federal, defence, and highly regulated industries can now:

Enforce perimeter-based governance
Reduce exposure risks
Meet compliance requirements

Enable secure cross-service patterns in Gov clouds

Azure Government boundaries now support scenarios like:

CMK with Key Vault
Service-to-service messaging
Controlled external access

More details of onboarded PaaS services are detailed in What is a network security perimeter? - Azure Private Link | Microsoft Learn

What’s next

Service Bus GA further strengthens network security perimeter’s growing coverage across Azure PaaS services. We will continue to:

Expand PaaS service onboarding
Improve access rule capabilities (e.g. Service tag-based access, identity-based access)

Pod CIDR Expansion Generally Available and IP Address Planning on Azure CNI Overlay

Sam_Foo — Fri, 05 Jun 2026 00:14:24 GMT

In networking with Azure CNI Overlay, the cluster-wide pod CIDR is logically partitioned into smaller “node” blocks where each node is assigned a fixed CIDR slice (/24) by Azure. This decouples pod networking from the VNet address space entirely because pods receive addresses from a private CIDR that is separate from the VNet.

By default, Azure CNI Overlay uses a pod CIDR of 10.244.0.0/16 which provides 65,536 addresses. Since each node consumes 256 addresses from the /24 slice, the default cluster has a node scaling limit of 65,536 divided by 256, or 256 nodes. Choosing a pod CIDR at cluster creation effectively sets an upper bound on how many nodes the cluster can accommodate.

Pod CIDR	Per-Node Block	Max Nodes Supported
/16	/24	256
/15	/24	512
/14	/24	1,024

What does Pod CIDR Expansion enable?

Even with careful upfront planning, long-lived clusters grow in ways that are difficult to anticipate. For organizations using Azure CNI Overlay, this previous represents a difficult migration without meticulous IP planning.

Pod CIDR expansion allows you to expand the existing CIDR without downtime or node reimaging. Instead of being locked to the range chosen at cluster creation, operators can expand the available pod address space with minimal operational burden.

Choosing a Pod CIDR

In addition to node scaling limits, there are other considerations for pod CIDR planning:

Overlapping pod CIDRs across clusters – even though pod IPs are not directly routable between clusters, overlapping CIDRs can cause problems with observability tooling or cross-cluster networking on top of the overlay. Careful planning can prevent having to recreate the cluster down the road.
Accounting for system node pools – each system node also consumes a /24 block. IP address planning should factor nodes running cluster control plane components in addition to existing workloads.

Learn More

Simplify Virtual WAN Spoke Connectivity at Scale with Azure Virtual Network Manager

Jay-Li — Tue, 26 May 2026 20:44:54 GMT

With Azure Virtual Network Manager (AVNM) integration, organizations using Virtual WAN for transitive connectivity can simplify spoke connectivity and policy management across large-scale hub-and-spoke deployments. By using a Virtual WAN hub as the hub in an AVNM hub-and-spoke topology, organizations can define connectivity and routing intent once at the network group level and apply it consistently across large numbers of spoke VNets. This reduces repetitive per-spoke connection and routing configuration, helps maintain operational consistency as deployments expand, and makes it easier to manage hub-and-spoke environments at scale. Together, AVNM’s centralized, group-based orchestration and Virtual WAN’s managed routing, security integration, and hybrid connectivity provide a more streamlined way to simplify operations and scale with confidence.

What is Azure Virtual Network Manager?

Azure Virtual Network Manager is a management service that lets you group, configure, and deploy network connectivity and security policies across virtual networks at scale. Instead of configuring VNet peering and access rules on each virtual network individually, you define network groups — logical collections of virtual networks based on static selection or dynamic Azure Policy conditions — and apply connectivity configurations and security admin rules to those groups.

Key capabilities include:

Hub-and-spoke and mesh topologies — Define how virtual networks in a network group connect to a central hub or to each other.
Network groups — Group VNets statically or dynamically (using tags, subscriptions, resource group names, or other Azure Policy conditions).
Security admin rules — Author and enforce access control lists across all VNets in a network group, providing a centralized layer of defense that complements NSGs and firewalls.
Region-scoped deployment — Deploy configurations to specific Azure regions, enabling incremental rollout and controlled blast radius.

AVNM operates as an overlay management layer — it orchestrates VNet peering, connectivity, and security rules without replacing the underlying networking primitives.

What is Azure Virtual WAN?

Azure Virtual WAN as a service brings together routing, security, VPN, ExpressRoute, and transitive connectivity in a hub-and-spoke architecture. A Virtual WAN hub is a managed regional resource that acts as a central transit point for branch connectivity, remote users, private enterprise connectivity, spoke virtual networks, and private traffic routing through security services.

Site-to-site VPN connectivity (branch offices, SD-WAN devices)
Point-to-site VPN connectivity (remote users)
ExpressRoute private connectivity (on-premises datacenters)
VNet-to-VNet transitive connectivity (spoke virtual networks)
Routing, firewall, and encryption for private traffic

All hubs in a Standard Virtual WAN are connected in a full mesh over the Microsoft backbone, enabling any-to-any connectivity between spokes, branches, and remote users across regions. Virtual WAN removes the need to manually manage complex route tables and transit VNets — routing is handled by the hub's built-in router.

What this integration enables

When you select a Virtual WAN hub as the hub in an AVNM connectivity configuration, AVNM handles the spoke-to-hub wiring for you. For each virtual network in your selected network groups:

If the VNet is not yet connected to the Virtual WAN hub, AVNM creates the Virtual Network connection to Virtual WAN hub and applies a consistent routing configuration with Virtual WAN connection policy.
If the VNet is already connected, AVNM updates the existing Virtual Network connection to utilize the routing properties in the Virtual WAN connection policy.

A connection policy is a hub-level Virtual WAN resource that defines shared routing behavior for the virtual network connections it governs, including route table association and propagation, route maps, internet security settings, and propagated labels. Because the policy applies these settings consistently across governed connections, it helps standardize routing and overrides conflicting settings configured directly on individual connections.

How it works

The setup follows AVNM's standard workflow:

Create a network group. Add virtual networks as members — either statically (by selecting specific VNets) or dynamically (using Azure Policy conditions such as tags or resource group names).
Create a connectivity configuration. Choose hub-and-spoke topology, select your Virtual WAN hub as the hub, and select or create a connection policy.
Deploy. Commit the configuration to your target regions. AVNM connects all VNets in the network groups to the Virtual WAN hub and applies the connection policy in parallel.

You can also enable direct connectivity within a spoke network group. When enabled, VNet-to-VNet traffic within that group routes directly between virtual networks instead of transiting the Virtual WAN hub — useful for latency-sensitive or high-throughput east-west workloads. By default, direct connectivity is regional; enable global mesh to extend it across Azure regions.

Key use cases

Bulk spoke onboarding

Connect many virtual networks to a Virtual WAN hub in one operation. All connections are orchestrated in parallel by AVNM, and the pre-defined routing configuration is automatically applied.

Policy-based dynamic onboarding

Use Azure Policy to define network group membership conditions. When a new virtual network matches those conditions—for example, a VNet tagged env:prod—it is automatically added to the network group. On the next deployment, AVNM connects it to the Virtual WAN hub with the correct routing configuration, reducing manual onboarding effort.

Batch routing configuration updates

Push routing changes to all virtual networks in a network group as a single, fully parallelized operation. This significantly reduces maintenance window duration for network-wide changes and makes rollback straightforward.

Incremental deployment

Segment your network into precise update domains by creating separate network groups — for example, by environment (staging, dev, production) or by region. Deploy connection policies to each group or region independently. This lets you test changes on a smaller subset before applying them broadly, minimizing blast radius.

Mesh for selective inspection bypass

If you use routing intent to send all private traffic through a firewall in the Virtual WAN hub, certain high-throughput or latency-sensitive flows (such as database replication) may benefit from bypassing that inspection. Enable direct connectivity in AVNM to create a mesh between selected spokes, allowing VNet-to-VNet traffic to route directly while all other traffic continues through the hub firewall.

Security admin rules at scale

Define network groups for your Virtual WAN spokes, then use AVNM security admin rules to author and deploy access control lists across those spokes. This provides an additional layer of defense alongside next-generation firewalls in the Virtual WAN hub.

Getting started

Prerequisites:

An existing Azure Virtual Network Manager instance
An existing Azure Virtual WAN and Virtual WAN hub
One or more virtual networks to use as spoke members

To configure:

Go to your Network Manager instance in the Azure portal.
Create a network group and add your spoke VNets.
Create a connectivity configuration → select hub-and-spoke → select your Virtual WAN hub → select or create a connection policy → add spoke network groups.
Deploy the configuration to your target regions.
In your Virtual WAN resource, verify that the expected spoke VNet connections are in a connected state. Review effective routes in the virtual hub to confirm routing behavior matches the selected connection policy.

For detailed step-by-step instructions, see Configure Azure Virtual WAN hub for Azure Virtual Network Manager.

For more on connection policy, see Connection policy in Azure Virtual WAN.

Learn more

Summarized Gateway Prefixes for Route Advertisement in Azure Virtual Networks

Jay-Li — Wed, 20 May 2026 19:10:30 GMT

Background

Many Azure deployments follow a hub-and-spoke topology: one VNet is designated as the hub and holds the connection to on-premises (via ExpressRoute Gateway, VPN Gateway, or both), and workload VNets — the spokes — peer to the hub to reach on-premises and shared services. This centralizes gateway connectivity so many workloads can share a single ExpressRoute or VPN Gateway.

However, in large hub-and-spoke topologies, ExpressRoute and VPN Gateway limits on advertised prefixes (for example, 1,000 IPv4 and 100 IPv6 prefixes) can be reached. Because each spoke adds its own address prefixes to that count, these limits are approached quickly, constraining how far the topology can scale.

What's New

With Summarized Gateway Prefixes, customers can now advertise a single covering prefix (for example, 10.0.0.0/16) instead of many smaller CIDRs (for example, multiple /24s) – dramatically reducing advertised route count and enabling larger-scale Azure environments.

A new property, summarizedGatewayPrefixes, is now available on the Virtual Network resource in public preview. When configured on a hub VNet, it controls what your ExpressRoute Gateway and VPN Gateway advertise to on-premises, replacing the default behavior of advertising all individual hub and spoke VNet CIDRs with a set of aggregated prefixes you define.

For example, instead of advertising 10.0.1.0/24, 10.0.2.0/24, 10.0.3.0/24, and so on for each spoke, you can advertise a single 10.0.0.0/16.

Key Benefits

Fewer advertised routes — Replace hundreds of individual spoke CIDRs with a small set of summarized prefixes.
Scales with your topology — Supports deployments with 500+ spokes without requiring address plan redesigns or VNet splits.
IPv4 and IPv6 — Summarize both address families.
Works with both gateway types — Supported on ExpressRoute Gateway and VPN Gateway.
Simple configuration — A single property on the VNet resource. No additional services or dependencies.
Backward compatible — If the property is left empty, behavior is unchanged: all hub and peered spoke address spaces are advertised as before.

How It Works

Default behavior

ExpressRoute Gateway and VPN Gateway advertise all address spaces of the hub VNet and all address spaces of peered spoke VNets to on-premises.

With summarizedGatewayPrefixes configured

The gateways advertise the summarized prefixes instead of the hub VNet's individual address spaces.
For each peered spoke, if the spoke's address space falls within a summarized prefix, the spoke's individual CIDRs are suppressed from advertisement.

Spoke address spaces not covered by a summarized prefix continue to be advertised individually.

Example:

Without Summarization	With Summarization
10.0.1.0/24, 10.0.2.0/24, 10.0.3.0/24, …	10.0.0.0/16
Hundreds of prefixes	One prefix

Getting Started

Open the hub VNet (the VNet containing your GatewaySubnet) in the Azure portal.
Go to Address space → Advertised gateway prefixes.
Add one or more IPv4 or IPv6 CIDR prefixes that cover the address spaces you want to summarize.

Navigate to your virtual network and verify that the summarized prefixes appear.

Things to Know

The property is set on the hub VNet (the VNet with the GatewaySubnet).
The summarized prefixes list can include prefixes outside the VNet's own address space.
Avoid overlap among prefixes within the list, but overlap with peered VNet address spaces is expected in hub-and-spoke designs.
For dual-stack (IPv4 + IPv6) VNets, define both IPv4 and IPv6 summarized prefixes explicitly.

Deploy with Confidence: Using Rule Impact Analyzer in Azure Virtual Network Manager

Jay-Li — Fri, 22 May 2026 06:36:58 GMT

Introduction

In a previous blog post, we described how Azure Virtual Network Manager (AVNM) enables central teams to enforce security admin rules across hundreds of virtual networks—bring consistency and governance to complex enterprise environments.

But enforcement at scale introduces a new challenge: deployment confidence. Security admin rules take priority over NSG rules and can span subscriptions and management groups. That makes them powerful—but a single misconfigured rule can disrupt critical traffic across your entire network. Governance teams need a way to understand the real-world impact of a rule before it reaches production—not after.

This is exactly the problem Azure Virtual Network Manager now solves with the Rule Impact Analyzer—a capability that simulates proposed security admin rules against your real network traffic, so you can see exactly what will change, what won't, and deploy with confidence instead of guesswork.

The Challenge: Understanding Rule Impact Before Deployment

As enterprises scale up their use of security admin rules, a visibility gap emerges. Consider a common scenario: a central governance team needs to block high-risk ports across all production virtual networks. The rules are well-intentioned, but the team has no visibility into which existing traffic flows would be affected. Without a way to preview the impact, teams face an uncomfortable tradeoff—move quickly and risk disruption, or slow down manual review across every affected network.

The Rule Impact Analyzer is designed to close this gap—giving teams with a clear, data-driven view of what a rule of change will do before it reaches production.

What Is the Rule Impact Analyzer?

The Rule Impact Analyzer is a joint capability of Azure Virtual Network Manager and Azure Network Watcher. It lets you simulate proposed security admin rules against traffic data derived from virtual network (VNet) flow logs and Traffic Analytics in your environment.

Instead of relying on manual review, the analyzer evaluates proposed rules against observed traffic and classifies each flow:

Affected — The proposed rule would change the current evaluation outcome for this flow (e.g., traffic that is currently allowed would be blocked).
Not Affected — The flow would continue as-is; the rule does not apply.
Indeterminate — The flow cannot be conclusively evaluated (e.g., insufficient traffic data).

This gives governance teams and network administrators a clear, data-driven view of what a rule of change will do—before it reaches production.

Note: The analysis is based on traffic data available through flow logs and Traffic Analytics. Results reflect recorded traffic patterns; traffic that has not yet been observed will not appear in results.

The Customer Journey: From Rule Authoring to Validated Deployment

The Rule Impact Analyzer fits naturally into the lifecycle of security admin rule management:

This workflow lets teams author rules, simulate impact, review results, and refine policies before committing a single change to production. Teams can cycle through simulation as many times as needed.

Key Capabilities

Predicted Impact Visibility

See briefly how your proposed security admin rules would affect existing traffic flows. Results are based on Traffic Analytics data, helping teams make informed deployment decisions.

Flow-Level Drill-Down

Go beyond summary counts. Inspect specific source and destination paths, see which rule affects each flow, and identify legitimate traffic that would be unintentionally blocked. This makes it easy to pinpoint issues and refine your rules.

Configurable Scope

You don't have to analyze everything at once. Target your analysis to specific:

Rule collections or individual security admin rules
Network groups or specific virtual networks

This lets you focus on the areas that matter most, whether you're validating a single rule change or assessing a broad policy rollout.

Controlled Iteration

Modify your security admin rules, re-run the analysis, and repeat—as many times as you need. Deploy only when the simulated impact matches your intended connectivity outcome.

Inbound and Outbound Evaluation

The analyzer evaluates both inbound and outbound traffic directions, giving you full visibility into the rule's impact across your network.

Real-World Scenario: Locking Down Internet-Exposed Management Ports at Scale

Let’s look at a real-world scenario as an example. Your organization runs hundreds of VNets across multiple subscriptions. Over time, different teams have created NSG rules that allow inbound SSH (port 22) and RDP (port 3389) from broad source ranges — some even from 0.0.0.0/0. Your security team mandates: block all inbound management-port access except from trusted bastion subnets.

The challenge? You can't just flip a switch. Blocking the wrong traffic could be risky, and you want to know the impact of applying the security rules.

With Rule Impact Analyzer, you can:

Define the proposed security admin rule — deny inbound TCP 22/3389 from all sources except your bastion subnet prefix
Simulate before you commit — see exactly which VNets, subnets, and NICs currently have traffic matching the rule, and which existing NSG rules would be overridden
Identify conflicts — spot cases where a team's NSG "Allow" rule would be superseded by your new admin-level "Deny," so you can coordinate before deployment
Deploy with confidence — roll out the rule knowing the blast radius is fully understood, not guessed

Before Rule Impact Analyzer, this required manually auditing NSG rules across every subscription, cross-referencing with resource inventories, and hoping nothing was missed. Now, a single simulation gives you a complete picture in minutes — turning a week-long audit into a self-service workflow.

How It Works: Architecture and Design

Rule Impact Analyzer uses existing Azure networking telemetry and analytics components. It does not require a separate data collection pipeline.

The following diagram provides an interactive version of the architecture:

Step 1: Traffic Analytics as Ground Truth. The analyzer queries your existing VNet flow logs through Traffic Analytics. No new agents, log pipelines, or storage accounts are required.

Step 2: Log Analytics as the Query Engine. Traffic Analytics data resides in your Log Analytics workspace. The Rule Impact Analyzer runs Kusto Query Language (KQL) queries to retrieve the observed flows relevant to your analysis scope.

Step 3: AVNM Rule Evaluation Engine. The retrieved flows are evaluated using AVNM's own enforcement logic—the same priority ordering, allow/deny behavior, and scope resolution used in production. This ensures that what you see in the analyzer matches what would happen when rules are enforced.

Step 4: Results Correlation and Surfacing. Each flow is classified and surfaced in the Azure Portal with drill-down capabilities—from summary impact counts down to individual flow paths and the specific rules affecting them.

What Means for You

Uses existing infrastructure. If you already have Traffic Analytics enabled, there is nothing new to deploy.
No data duplication. Queries run in place within your existing Log Analytics workspace, under your existing RBAC and data retention policies.
Transparent costs. Only standard Log Analytics query costs apply—no hidden charges or separate billing.

Getting Started

You can access Rule Impact Analyzer from two entry points in the Azure Portal:

From Azure Virtual Network Manager: Navigate to your security admin configuration → select a rule collection → launch the Rule Impact Analyzer.
From Azure Network Watcher: Navigate to Monitoring → Traffic Analytics → Rule Analyzer.

Both paths lead to the same analysis experience, so you can start with whichever tool fits your workflow.

Prerequisites

Before using the Rule Impact Analyzer, ensure the following are in place:

VNet flow logs are enabled on the virtual networks you want to analyze.
Traffic Analytics is configured and sends data to a Log Analytics workspace.
You have the necessary RBAC permissions to access the AVNM security admin configuration and the Log Analytics workspace.

Steps

Enable VNet flow logs and Traffic Analytics on your target virtual networks. Learn more about Traffic Analytics.
Author or update your security admin rules in Azure Virtual Network Manager. Learn more about AVNM security admin rules.
Launch the Rule Impact Analyzer from either portal entry point, configure your scope (rule collections, network groups, or specific VNets), and run the analysis.
Review, refine, and deploy. Iterate your rules until the simulated impact matches your intended outcome, then deploy with confidence.

The screenshot below shows the Rule Impact Analyzer in the Azure Portal. After running a simulation, you can see a summary of predicted traffic impact—total paths analyzed, how many are affected or not affected—along with a detailed results table to drill into individual flows and identify which rule impacts each one.

Why It Matters

Outage Prevention

For organizations rolling out network isolation policies at scale, Rule Impact Analyzer acts as a safety net. By simulating rule impact against recorded traffic patterns, teams can catch misconfigurations before they reach production.

Faster Rule Adoption

Without the analyzer, deploying new admin rules often requires lengthy manual review cycles. With self-service impact analysis, governance teams can validate and deploy rules faster—without waiting for manual approval.

Aligning with Behavior

Security policies express intent—what traffic should or shouldn't be allowed. Rule Impact Analyzer validates whether a proposed rule achieves that intent against your observed traffic, closing the loop between policy design and operational behavior.

Conclusion

The AVNM Rule Impact Analyzer closes the gap between policy intent and deployment confidence. Simulating rules against observed traffic—with no additional infrastructure required—governance teams can validate impact before enforcement.

Enforcement without visibility is a risk. Visibility without enforcement is incomplete. This capability brings both together.

We welcome your feedback as you start using this capability. Share your experience through the Azure Portal feedback button or your Microsoft account team.

Learn more:

Authors:

Deepak Bansal, Corporate Vice President and Technical Fellow, Microsoft Azure, Xinyan Zan, Vice President, Ashish Bhargava, Principal Software Development Manager, and Jay Li, Senior Product Manager

Understanding and building an Azure Hybrid Meshed Hub-Spoke Topology

Svenbaeck — Mon, 18 May 2026 07:15:26 GMT

A meshed hybrid hub-spoke topology

Azure offers two main approaches to build network architectures. This article focuses on traditional networking (using VNets, peering, route tables, etc.), rather than Azure Virtual WAN.

Why a hub-spoke topology?

A hub‑spoke topology is the only way to control traffic flows while maintaining scalability, because it enforces a central point of connectivity and policy enforcement:

Centralized traffic control / inspection: All connectivity (to on‑premises, the internet, and between spokes) is anchored through the hub. The hub hosts shared services such as firewalls or NVAs, providing a single control point where traffic is inspected, filtered, and governed consistently.
Avoids uncontrolled lateral communication: Spokes do not connect arbitrarily to each other. All connectivity is routed through the hub, preventing uncontrolled east‑west communication and ensuring traffic follows defined security and routing policies.
Inherent scalability by design: New workloads are added by introducing additional spokes. The core network design remains unchanged, enabling linear scaling without the complexity of full-mesh connectivity.

In summary, the hub‑spoke model provides centralized control combined with scalable, decoupled workload networks—something that flat or full-mesh designs struggle to achieve.

From hub-spoke to meshed multi-region

In a hub‑spoke topology, it’s important to keep in mind that the hub is implemented as an Azure Virtual Network (VNet) and VNets are scoped to a single region. This means that in a multi‑region setup, you’ll always need at least one hub per region. Each of these hubs hosts shared services like firewalls, NVAs, and DNS, acting as the central point for connectivity and traffic control.

Extending dependencies across regions—for example by connecting spokes to a hub in another region—is generally not recommended. It creates tight coupling between regions, which goes against the goal of keeping regions independent. A well-designed multi‑region architecture aims for regional self‑containment to improve resilience and fault isolation. Relying on a remote hub can lead to issues like failure propagation between regions, higher latency for inspected traffic and more complex routing and operations. It can also introduce organizational challenges when different regions are managed by separate teams, reducing agility and increasing operational risk.

For this reason, meshed hub‑spoke architectures should use hubs that are deployed within each region. Connectivity between regions should be established directly between the hubs, not through spokes. In a meshed design, hubs are typically connected in a full‑mesh peering model, allowing for controlled and predictable inter‑region communication while still maintaining regional independence.

Within a single region, it can also make sense to deploy multiple hubs to create isolated environments. This is especially useful when you need to separate workloads based on security requirements, regulatory needs, or organizational boundaries. Each hub can then have its own dedicated set of connectivity and inspection services.

Finally, each spoke VNet connects to just one hub. This keeps routing simple and predictable, ensures that all traffic passes through the correct inspection and policy enforcement layers, and reinforces the hub’s role as the central control point for network traffic within the region.

Integrating hybrid connectivity

In most enterprise scenarios, Azure doesn’t operate in isolation—it needs to connect to external networks such as on‑premises datacenters or other cloud environments. This hybrid connectivity is typically set up using services like Azure ExpressRoute, Azure VPN Gateway or third‑party SD‑WAN solutions. In a (meshed) hub‑spoke topology, these connectivity components are best deployed in the hub VNet, since the hub acts as the central point where all inbound and outbound traffic comes together.

By centralizing external connectivity in the hub, all traffic—whether entering or leaving Azure—can be routed, inspected and governed in a consistent way using shared services like firewalls or NVAs. It also avoids the need to duplicate gateways and connectivity components across multiple spokes, which helps reduce cost and operational overhead.

This approach also simplifies routing and policy management. Spokes can rely on the hub’s shared connectivity instead of maintaining their own connections to external networks. Overall, this reinforces the hub’s role as the single, controlled integration point between Azure and the broader network landscape.

Implementation fundamentals

With the overall architecture in place, the next step is to understand how Azure actually handles routing and traffic control in this kind of design.

When working with a hub‑spoke topology in Azure, it’s important to realize that a virtual network (VNet) doesn’t behave like a traditional router. While you can associate Azure Route Tables with subnets, those routes only apply to traffic originating from within that subnet. Traffic entering the VNet from outside isn’t automatically re‑routed. This is also why VNet peering is non‑transitive by design: peered VNets can communicate directly, but they won’t forward traffic for other networks.

To enable controlled routing between spokes—and between Azure and external networks such as ExpressRoute or VPN—you need a component in the hub that can actively receive and forward traffic. In most cases, this is handled by an Azure Firewall or a network virtual appliance (NVA) deployed in the hub. These components act as an explicit routing hop: they receive traffic, inspect or process it based on defined policies and then send it back into the virtual network so Azure’s routing engine can continue forwarding it.

In a secure hub‑spoke design, the firewall plays a dual role. It not only provides centralized traffic inspection and enforces security policies, but also acts as the mechanism that enables transitive communication between spokes and external networks. This combination of control and connectivity is a key part of the architecture. Of course, this only works as intended if the firewall is configured with the right rules to allow or block traffic according to your security requirements.

While it’s technically possible to implement routing using a basic virtual machine or even a Virtual Network Gateway, these approaches don’t meet typical enterprise requirements. They lack built‑in capabilities like advanced traffic inspection, high availability, autoscaling and centralized policy management. Purpose‑built solutions such as Azure Firewall or mature third‑party NVAs are designed to provide not just routing, but also integrated security, consistency, and scalability. For that reason, they’re generally the only realistic choice for production‑grade hub‑spoke environments where both control and resilience matter.

Design principles for building the topology

The diagram below shows the topology for a hybrid meshed hub-spoke, with 2 hubs and an Azure Firewall (any other 3rd party Firewall could be used as well).

Ensuring correct connectivity in a hub-and-spoke topology may initially appear complex, but in practice it comes down to understanding and correctly applying four key design principles:

controlled routing in the GatewaySubnet
controlled routing in each spoke
proper peering of spokes to the hub
meshing the hubs.

Before looking at these in detail, it is important to understand a fundamental behavior of Azure Virtual Network (VNet) peering. When two VNets are peered, Azure automatically exchanges their address spaces (CIDR ranges) and injects these prefixes as system routes into the effective route tables of all subnets. As a result, resources in one VNet can communicate directly with resources in the other using private IP addressing, without any additional routing configuration. This built-in route propagation is what makes VNet peering an efficient and low-latency connectivity mechanism in Azure.

However, this default behavior is not always aligned with the requirements of a hub-and-spoke topology. In this model, network services such as firewalls, inspection and routing control are typically centralized in the hub VNet. If communication between spokes is allowed to follow the automatically injected system routes, traffic could bypass these centralized controls, which would undermine design objectives such as inspection, segmentation and governance.

For this reason, although VNet peering provides seamless connectivity by default, additional configuration is required in a hub-and-spoke architecture. This is usually achieved through Azure Route Tables, network virtual appliances (NVAs) or Azure Firewall, ensuring that traffic between spokes is routed through the hub as intended. This approach enables a controlled routing model that is essential for maintaining security and architectural consistency in enterprise-scale Azure environments.

Design principle 1: Controlled routing in the GatewaySubnet

In hybrid connectivity scenarios, traffic originating from on-premises environments over VPN or ExpressRoute is first terminated by the Azure Virtual Network Gateway. From there, the traffic is injected into the Azure network using the routing context of the GatewaySubnet. By default, this process relies on system routes that are automatically populated through VNet peering. As a result, when the destination resides in a spoke VNet, the traffic is forwarded directly to that spoke, since its address space has already been learned and installed as a system route. While this behavior is efficient, it also means that traffic will bypass centralized security controls in the hub, such as Azure Firewall.

To ensure that all incoming traffic is properly inspected, this default routing behavior needs to be adjusted. This is done by associating a custom Azure Route Table with the GatewaySubnet and defining user-defined routes for each spoke address range. These routes should point to the private IP address of the firewall as the next hop, effectively overriding the system routes created by VNet peering. Because Azure gives precedence to user-defined routes over system routes, traffic that would normally go directly to the spoke is instead redirected through the firewall before reaching its destination.

It is important that these user-defined routes precisely match the CIDR ranges defined for the spoke VNets! Any mismatch, such as using broader or more specific prefixes, can lead to unexpected routing behavior and may introduce issues such as asymmetric traffic flows or packet loss. For instance, if a spoke uses address spaces like 10.10.10.0/24 and 192.168.10.0/24, these exact prefixes must be reflected in the route table. Only by aligning the custom routes with the advertised address ranges can you ensure predictable routing and consistent inspection through the firewall.

If the hub VNet hosts additional resources beyond an Azure Firewall or third-party network virtual appliance that also require traffic inspection, the corresponding CIDR ranges—either for the specific subnets or for the entire hub VNet—should be included as routes in the route table associated with the GatewaySubnet. These routes should be configured in the same way as those for spoke VNets, ensuring that traffic destined for these resources is directed through the intended inspection point. A typical example is Azure DNS Private Resolver, which can include both inbound and outbound endpoints deployed in dedicated subnets. When such endpoints are present in the hub, their associated subnet address ranges must also be added to the route table for the GatewaySubnet. This ensures that traffic to and from these endpoints is routed through the designated inspection path, maintaining consistent enforcement of security controls.

Design principle 2: Controlled routing in every spoke

In a hub-and-spoke architecture, traffic flows should follow the intended security model. Workloads within the same spoke VNet are usually treated as part of the same trust boundary, so traffic between resources in that spoke can flow directly over the Azure backbone without needing to pass through centralized controls. Network Security Groups (NSGs) should still be used at the subnet level to provide granular, stateful filtering, but routing this traffic through a central firewall is typically not required.

The situation changes when traffic leaves the local VNet. As soon as traffic is destined for another spoke, the hub, or on-premises networks, it crosses a trust boundary and needs to be inspected centrally. To enforce this, Azure’s default routing behavior must be overridden by associating an Azure Route Table with each subnet in the spoke VNets.

In most cases, this route table can be kept simple by defining a single default route that sends all outbound, non-local traffic to the firewall in the hub:

Destination: 0.0.0.0/0
Next hop: Private IP address of the hub firewall (Virtual Appliance)

With this configuration in place, all traffic that is not local to the spoke is forced through the hub, ensuring that communication between VNets and towards external networks is inspected and controlled.

From a management perspective, the same route table can often be reused across multiple subnets or even multiple VNets within the same subscription, which helps keep the design consistent and easy to maintain. It’s worth noting, however, that Azure requires route tables and the subnets they’re associated with to be in the same subscription, as this association is enforced by the platform.

There is one additional setting that is often overlooked but plays an important role in getting routing right in a hub-and-spoke design. Azure route tables include an option called “Propagate gateway routes”, which controls whether routes learned by a Virtual Network Gateway are added to the effective routes of the associated subnets. By default, routes learned via BGP (for example from ExpressRoute or VPN) or defined through a Local Network Gateway are propagated not only within the hub VNet, but also across VNet peerings. This means that spoke VNets can automatically learn routes to on-premises or external networks and may send traffic directly to the gateway, bypassing the firewall in the hub.

To avoid this and keep traffic flowing through the centralized security controls, this setting should be disabled on the route tables used by the spoke subnets. When “Propagate gateway routes” is set to No, routes learned by the gateway are no longer injected into the spokes. As a result, traffic to those destinations cannot take a direct path and instead follows the user-defined default route (0.0.0.0/0) toward the hub firewall, where it can be properly inspected.

When combined with the default route to the firewall, this setup ensures that traffic—whether it is going to other VNets, on-premises environments, or external networks—always follows a controlled and predictable path through the hub. This helps maintain consistent security enforcement and avoids unexpected routing behavior in larger or hybrid deployments.

Design principle 3: Peering the spokes to the hub

Virtual Network (VNet) peering in Azure is often seen as a simple, single configuration, but in reality it is directional by design. To fully connect two VNets, you need two separate peering configurations—one in each direction—and both must be configured correctly to ensure not only connectivity, but also proper routing behavior. Each peering exposes four key settings and getting these right is especially important in a hub-and-spoke architecture.

For basic connectivity, the first two settings—“allow virtual network access” and “allow forwarded traffic”—should be enabled on both peerings. These ensure that traffic can flow between VNets and support scenarios where traffic is routed through a central component, such as a firewall in the hub. The other two settings depend on the direction of the peering. In a typical hub-and-spoke setup, the Virtual Network Gateway (or Azure Route Server) is deployed in the hub. This means the peering from the spoke to the hub must enable “use remote gateways”, while the peering from the hub to the spoke must enable “allow gateway transit.”

At first, this might seem to contradict the idea that spokes should not directly use the gateway. However, these settings influence control plane behavior and don't enable unrestricted traffic flow. They are required so the gateway can learn and advertise spoke address ranges via BGP to external networks, such as those connected over VPN or ExpressRoute. Whether those routes are actually used in the spokes is still controlled through the “propagate gateway routes” setting on the route tables, allowing you to enforce routing through the firewall as intended.

Even if you are not currently using BGP—for example, in environments relying on static routing—it is still a good practice to configure peerings this way. Doing so makes the design future-proof, allowing you to introduce dynamic routing later without changes to the peering model. This approach keeps the architecture consistent and avoids unnecessary rework as the environment evolves.

Design principle 4: Meshing the hubs

When you extend a hub-and-spoke design across multiple regions, you typically introduce multiple hubs, each managing its own regional spokes. In this setup, it becomes important to connect the hubs to each other, which is done by fully meshing the hub VNets using VNet peering. At the same time, a key principle remains unchanged: each spoke should connect to only one hub in the same region. This keeps the architecture simple, scalable and easier to reason about from a routing perspective.

When configuring connectivity between hubs, it’s important to note that VNet peering settings differ from the typical hub–spoke configuration. For inter-hub peerings, only “allow virtual network access” and “allow forwarded traffic” should be enabled. The remaining options—“allow gateway transit” and “use remote gateways”—should be left disabled, as gateway sharing is not required between hubs and would even be blocked in the configuration.

Just connecting the hubs with peering is not enough to guarantee correct traffic flow. To ensure traffic moves between regions in a controlled and secure way, you need additional routing logic. Each hub should have an Azure Route Table assigned to its FirewallSubnet (or the subnet hosting the 3rd party NVAs) defining how traffic towards other hub-and-spoke environments is handled. This ensures that inter-region traffic is always routed through the appropriate hub firewall, instead of flowing directly across the Azure backbone.

At this point, IP address planning becomes critical. Without a clear addressing strategy, routing quickly becomes complex and hard to maintain. A common best practice is to assign a single “master” CIDR range per region, and then allocate all VNets in that region—both hub and spokes—from that range. This creates a clean, hierarchical addressing model that simplifies routing decisions.

With this approach in place, route tables can remain relatively simple. Instead of adding routes for every individual spoke, you only need one route per remote region. The destination is the master CIDR range of that region and the next hop is the private IP of the firewall in the corresponding hub. Because all hubs are peered with each other, these address ranges and firewall endpoints are automatically known through peering, allowing for consistent and predictable routing.

Overall, this design keeps routing logic straightforward while ensuring that all inter-region traffic is inspected in the correct hub, preserving the security model and making it easy to scale as new regions are added.

Conclusion

When the four design principles described in this article are applied consistently, a hub-and-spoke architecture becomes a strong, scalable and easy-to-operate foundation for your network. By combining controlled routing, centralized inspection and clear traffic flows, the model delivers both solid security and predictable behavior, even in complex environments.

More importantly, the concepts covered here go beyond just one specific design. They represent the key building blocks of Azure networking, including routing, peering and traffic control. Understanding these fundamentals not only helps you implement hub-and-spoke topologies correctly, but also gives you a solid base for designing and running reliable, enterprise-grade network architectures in Azure.

To make this easier to apply in practice, the table below summarizes the main concepts from this article and how they translate into actual configuration. It can be useful both when setting up a hub-and-spoke topology and when troubleshooting existing environments.

Area	Configuration	Key Setting / Value	Purpose
Hub VNet	Deploy shared services	Azure Firewall or NVA in hub	Central inspection + routing
	Deploy connectivity	VPN Gateway / ExpressRoute in hub	Centralize hybrid connectivity
GatewaySubnet	Associate Route Table	UDRs for each spoke CIDR → Firewall IP	Force inbound traffic through firewall
Spoke Subnets	Associate Route Table	0.0.0.0/0 → Firewall (Virtual Appliance)	Force all outbound traffic via hub
	Route Table setting	Propagate gateway routes = Disabled	Prevent bypass of firewall via gateway
VNet Peering (Spoke → Hub)	Setting	Allow VNet access = Yes	Basic connectivity
	Setting	Allow forwarded traffic = Yes	Support transitive routing via firewall
	Setting	Allow gateway transit = Yes	Allow spoke to leverage hub gateway
	Setting	Use remote gateways = No	-
VNet Peering (Hub → Spoke)	Setting	Allow VNet access = Yes	Basic connectivity
	Setting	Allow forwarded traffic = Yes	Support routing through firewall
	Setting	Allow gateway transit = No	-
	Setting	Use remote gateways = Yes	Advertise spoke prefixes via hub gateway
VNet Peering (Hub→ Hub)	Setting	Allow VNet access = Yes	Basic connectivity
	Setting	Allow forwarded traffic = Yes	Support transitive routing via firewall
	Setting	Allow gateway transit = No	-
	Setting	Use remote gateways = No	-
Hub FirewallSubnet	Associate Route Table	Route remote region CIDR → remote hub firewall IP	Ensure inter-region/hub routing
Addressing strategy	CIDR planning	Assign master CIDR per region	Simplify routing and reduce UDR complexity
Spoke design rule	Peering constraint	Each spoke connected to one hub only	Prevent routing ambiguity

Metrics Filtering and Log Aggregation Now GA for Advanced Container Networking Services

chandanAggarwal — Tue, 19 May 2026 18:13:31 GMT

We are thrilled to announce that Advanced Container Networking Services (ACNS) for Azure Kubernetes Service (AKS) now delivers two powerful observability features in General Availability: container network metrics filtering and container network log filtering and aggregation. Together, these capabilities set a new standard for Kubernetes network observability, giving you high-fidelity visibility at dramatically lower cost and noise. These capabilities fundamentally redefine how network observability works at scale while delivering up to 97% cost reduction.

Why this is a Milestone?

Most Kubernetes observability solutions face a fundamental tension: collect everything and drown in noise and cost, or sample and miss the signals that matter, with new features of Advanced Container Networking Services that tradeoff has been eliminated.

With this release, Azure becomes the first cloud provider to deliver on-node metrics filtering and flow log aggregation for Kubernetes networking, capabilities now also contributed to the upstream Hubble project, making them available to the broader open-source community.

For AKS customers running Cilium-based clusters, this means:

Every flow you care about is captured. Everything else is dropped at the source.
Log volume is compressed by up to 45% through aggregation, without losing security verdicts or error context.
Costs scale with what you monitor, not with cluster size.

What’s been improved in observability?

This release introduces two capabilities that work together: container network metrics filtering and container network log filtering and aggregation. Both are available on AKS clusters with the Cilium data plane and give you precise controls to keep observability costs predictable while maintaining the visibility you need.

Container Network Metrics Filtering

Container network metrics are generated for all pods by default whenever Advanced Container Networking Services is enabled. With metrics filtering, you now control what gets collected at the point of ingestion, on the node, before anything is scraped or transmitted.

A single ContainerNetworkMetric CRD per cluster defines which metric types (dns, flow, tcp, drop), namespaces, pod labels, and protocols to ingest. It supports both include and exclude filters, so you can maintain broad collection while carving out specific workloads or namespaces. Anything that doesn't match is dropped on the node. Changes reconcile in a few seconds, with no Cilium agent or Prometheus restarts required.

Container Network Log Filtering and Aggregation

Unlike metrics, container network logs are not generated automatically. You start capturing network flows only after applying a ContainerNetworkLog CRD that defines exactly which traffic to capture-by namespace, pod, service, protocol, or verdict. Only matching flows are logged, giving you a precise, targeted view rather than a fire hose.

This is where Azure's first-to-market innovation comes in. Flow log aggregation, now built into Advanced Container Networking Services and contributed upstream to Hubble for the open-source community, groups similar flows into summarized records every 30 seconds. The result is dramatically reduced data volume while preserving security verdicts, service identity, and error context. What previously required custom post-processing pipelines is now built directly into the platform before storage costs are incurred.

Every matched flow log captures: source and destination pods, namespaces, ports, protocols, traffic direction, and policy verdicts.

Logs are stored in a Log Analytics workspace (ContainerNetworkLogs table) with a choice of using the Analytics or Basic tier. Built-in Azure portal dashboards are available for both tiers. Logs can also be exported to external log collectors such as Splunk or Datadog.

First to Market: Azure and the upstream Hubble Contribution

Advanced Container Networking Services built-in filtering and aggregation capabilities were engineered from the ground up to solve real production observability challenges at scale. Rather than keeping this innovation proprietary, Azure contributed the log aggregation and filtering capabilities to the upstream Hubble project, the observability layer of the Cilium ecosystem.

This means:

AKS customers get a fully managed, Azure-native experience with portal dashboards, Log Analytics integration, and Grafana visualization, out of the box.
The broader open-source community gains access to the same filtering and aggregation primitives through upstream Hubble.

Azure is the first to ship this capability in a managed Kubernetes service, and the first to give it back to the community.

Key Benefits

💰 Lower observability cost. Metrics filtering drops unwanted data on the node before Prometheus ever scrapes it. Flow log aggregation compresses log data by up to 97% in lab testing. Your cost scales with what you choose to monitor, not with cluster size.

📉 Less noise, more signal. Metrics filtering carves out the namespaces and workloads that matter, so dashboards show only relevant signals. Log filters scope collection to specific pods and verdicts. Engineers start every investigation with data that's already relevant.

⚡ Faster root-cause isolation. Every metric carries source and destination pod context. Targeted flow logs add the forensic detail, which policy, destination, or port is involved. Together, they cut mean time to resolution from hours of guesswork to minutes of structured investigation.

🔒 Full signal, zero gaps. Within the scope you define, every flow is captured and every pattern is preserved. Aggregation compresses volume without losing security verdicts or error context.

Who Benefits

Platform engineers managing multi-tenant clusters can scope data collection per namespace, so each team gets visibility into their own traffic without contributing to a shared cost pool.

SREs can isolate packet drops, TCP resets, or DNS failures to a specific workload in minutes, starting with data that's already scoped to what matters.

Decision-makers evaluating observability spend get predictable, controllable ingestion costs that scale with intent, not infrastructure size.

How to optimize metrics and logs with filtering?

Enable Advanced Container Networking Services ( ACNS) on your AKS cluster with the Cilium data plane:

az aks create --enable-acns

Or on an existing cluster:

az aks update --resource-group $RESOURCE_GROUP --name $CLUSTER --enable-acns

Apply a ContainerNetworkMetric CRD to filter which metrics are collected on each node. Start by excluding noisy system namespaces, then scope to business-critical workloads.
Apply a ContainerNetworkLog CRD to define which flows to capture.
Enable Azure Monitor integration with --enable-container-network-logs to send logs to a Log Analytics workspace, or export logs from the node to an external logging system such as Splunk or Datadog.
Check your dashboards. Open your cluster in the Azure portal and go to Monitor > Insights > Networking for bytes, drops, DNS errors, and flows. For flow logs, use the built-in Azure portal dashboards available for both Basic and Analytics tiers.

Conclusion

Kubernetes network observability has long meant choosing between visibility and cost. With container network metrics filtering and log filtering and aggregation now GA in Advanced Container Networking Services (ACNS) and contributed to upstream Hubble for the open-source community, that tradeoff is gone.

Azure is first to market with this capability. AKS customers get it fully managed, out of the box, with built-in dashboards with Log Analytics integration. And the broader Cilium ecosystem gets it through upstream Hubble.

High-fidelity visibility. Lower cost. No compromise.

Learn more:

Container network metrics overview: Container network metrics overview - Azure Kubernetes Service | Microsoft Learn
Container network logs overview: Container Network Logs Overview - Azure Kubernetes Service | Microsoft Learn
Configure container network metrics filtering: Configure Container network metrics filtering for Azure Kubernetes Service (AKS) - Azure Kubernetes Service | Microsoft Learn
Set up container network logs: Set up container network logs - Azure Kubernetes Service | Microsoft Learn

Private subnets by default in Azure Virtual Networks: What changed and how to use NAT Gateway

aimeelittleton — Wed, 22 Apr 2026 19:09:37 GMT

Azure is evolving to better support secure‑by‑default cloud architectures.

Starting with API version 2025‑07‑01 (released after March 31, 2026), newly created virtual networks now default to using private subnets. This change removes the long‑standing platform behavior of automatically enabling outbound internet access through implicit public IPs, also known as default outbound access (DOA).

As a result: newly deployed virtual machines will not have public outbound connectivity unless explicitly configured.

What changed?

Previously, Azure automatically assigned a hidden Microsoft‑owned public IP to virtual machines deployed without an explicit outbound method (such as NAT Gateway, Load Balancer outbound rules, or instance‑level public IPs). This allowed public outbound connectivity without requiring customer configuration.

While convenient, this model introduced challenges:

Security – Implicit internet access conflicts with Zero Trust principles.

Reliability – Platform‑managed outbound IPs can change unexpectedly.

Operational consistency – VMSS instances or multi‑NIC VMs may egress using different default outbound IPs.

With API version 2025‑07‑01 and later:

Subnets in newly created VNets are private by default.

The subnet property `defaultOutboundAccess` is set to false.

Azure no longer assigns implicit outbound public IPs.

This applies across deployment methods including Portal, ARM/Bicep, CLI, and PowerShell. Portal has started using the new model as of April 1, 2026.

Note: This change has not yet applied to Terraform.

Am I impacted by this change?

Deployment scenario	Behavior
Existing VNets or VMs using DOA	✅ Unchanged
New VMs in existing VNets	✅ Unchanged
Subnets already using explicit outbound	✅ Continue using configured outbound method
New VMs in new VNets (with subnets created using API 07-01-2025 or later)	🔒 Subnets private by default
New VMs in private subnets without explicit outbound configured	❌ No public outbound connectivity

Existing workloads are not impacted.

If required, you can still create new subnets without the private setting by choosing the appropriate configuration option during creation. See the FAQ section of this blog for more information. However, we strongly recommend transitioning to an explicit outbound method so that:

Your workloads won’t be affected by public IP address changes.

You have greater control over how your VMs connect to public endpoints.

Your VMs use traceable IP resources that you own.

When is outbound connectivity required?

If your virtual network contains virtual machines, you must configure explicit outbound connectivity. Here are common scenarios that require it:

Virtual machine operating system activation and updates, such as Windows or Linux.

Pulling container images from public registries (Docker Hub or Microsoft Container Registry).

Accessing 3rd party SaaS or public APIs

Virtual machine scale sets using flexible orchestration mode are always secure by default and therefore require an explicit outbound method.

Private subnets don’t apply to delegated or managed subnets that host PaaS services. In these cases, the service handles outbound connectivity—see the service-specific documentation for details.

Recommended outbound connectivity method: StandardV2 NAT Gateway

Azure now recommends using an explicit outbound connectivity method such as:

NAT Gateway

Load Balancer outbound rules

Public IP assigned to the VM

Network Virtual Appliance (NVA) / Firewall

Among these, Azure StandardV2 NAT Gateway is the recommended method for outbound connectivity for scalable and resilient outbound connectivity.

StandardV2 NAT Gateway:

Provides zone‑redundancy by default in supported regions

Supports up to 100 Gbps throughput

Provides dual-stack support with IPv4 and IPv6 public IPs

Uses customer‑owned static public IPs

Enables outbound connectivity without allowing inbound internet access

Requires no route table configuration when associated to a subnet

When configured, NAT Gateway automatically becomes the subnet’s default outbound path and takes precedence over:

Load Balancer outbound rules
VM instance‑level public IPs

Note: UDRs for 0.0.0.0/0 traffic directed to virtual appliances/Firewall takes precedence over NAT gateway.

Flow chart showing priority order for different outbound methods

Migrate from Default Outbound Access to NAT Gateway

To transition from DOA to Azure’s recommended method of outbound, StandardV2 NAT Gateway:

Go to your virtual network in the portal, and select the subnet you want to modify.
In the Edit subnet menu, select the ‘Enable private subnet’ checkbox under the Private subnet section

Enabling private subnet can also be done through other supported clients, below is an example for CLI, in which the default-outbound parameter is set to false:

az network vnet subnet update \ --resource-group rgname \ --name subnetname \ --vnet-name vnetname \ --default-outbound false

3. Deploy a StandardV2 NAT gateway resource.

4. Associate one or more StandardV2 public IP addresses or prefixes.

5. Attach the NAT gateway to the target subnet.

Once associated:

All new outbound traffic from that subnet uses NAT Gateway automatically

VM‑level public IPs are no longer required

Existing outbound connections are not interrupted

Note: Enabling private subnet on an existing subnet will not affect any VMs already using default outbound IPs. Private subnet ensures that only new VMs don’t receive a default outbound public IP.

For step-by-step guidance, see migrate default outbound access to NAT Gateway.

FAQ

1. Will my existing workloads lose outbound connectivity?

No. Workloads currently using default outbound IPs are not impacted by this change. The private subnet by default update only affects:

Newly created VNets

New subnets created using the updated API, 2025-07-01

New virtual machines deployed into those subnets using the updated API

VMs and subnets using an explicit outbound connectivity method like a NAT gateway, NVA / Firewall, a VM instance level public IP or Load balancer outbound rules is not impacted by this change.

2. Why can’t my new VM reach the internet or other public endpoints within Microsoft (e.g. VM activation, updates)?

New subnets are private by default. If your deployment does not include an explicit outbound method — such as a NAT Gateway, Public IP, Load Balancer outbound rule, or NVA/Firewall— outbound connectivity is not automatically enabled.

3. My workload has a dependency on default outbound IPs and isn’t ready to move to private subnets, what should I do?

You can opt-out of the default private subnet setting by disabling the private subnet feature. You can do this in the portal by unselecting the private subnet checkbox:

Disabling private subnet can also be done through other supported clients, below is an example for CLI, in which the default-outbound parameter is set to true:

az network vnet subnet update \ --resource-group rgname \ --name subnetname \ --vnet-name vnetname \ --default-outbound true

4. Why do I see an alert showing that I have a default outbound IP on my VM?

There's a NIC-level parameter `defaultOutboundConnectivityEnabled` that tracks whether a default outbound IP is allocated to a VM/Virtual Machine Scale Set instance. If detected, the Azure portal displays a notification banner and will generate Azure Advisor recommendations about disabling default outbound connectivity for your VMs / VMSS.

5. How do I clear this alert?

To remove the default outbound IP and clear the alert:

Configure a StandardV2 NAT gateway (or other explicit outbound method).
Set your subnet to be private or by setting the subnet property defaultOutboundAccess = false using one of the supported clients.
Stop and deallocate any applicable virtual machines (this will remove the default outbound IP currently associated with the VM).

6. I have a NAT gateway (or UDR pointing to an NVA) configured for my private subnet, why do I still see this alert?

In some cases, a default outbound IP is still assigned to virtual machines in a non-private subnet, even when an explicit outbound method—such as a NAT gateway or a UDR directing traffic to an NVA/firewall—is configured.

This does not mean that the default outbound IP is used for egress traffic.

To fully remove the assignment (and clear the alert):

Set the subnet to private

Stop and deallocate the affected virtual machines

Summary

The move to private subnets by default improves the security posture of Azure networking deployments by removing implicit outbound internet access.

Customers deploying new workloads must now explicitly configure outbound connectivity.
StandardV2 NAT Gateway provides a scalable, resilient method for enabling outbound internet access without exposing workloads to inbound connections or relying on platform‑managed IPs.

Learn more

Default Outbound Access

StandardV2 NAT Gateway

Migrate Default Outbound Access to StandardV2 NAT Gateway

Azure VNet Data Gateway for Secure Power BI & Power Platform Access in Enterprises

kirankumar_manchiwar04 — Wed, 22 Apr 2026 16:30:19 GMT

What Is a VNet data gateway?

The VNet data gateway is a Microsoft‑managed gateway service that runs inside a delegated subnet of an Azure Virtual Network. It allows supported Microsoft cloud services—such as Power BI, Power Platform dataflows, and Microsoft Fabric workloads—to securely connect to data sources that are protected using private networking.

Key characteristics:

No customer‑managed VM or container

No OS, patching, or gateway software upgrades

Gateway lifecycle fully managed by Microsoft

Traffic stays on the Azure backbone network

Works seamlessly with Private Endpoints

This makes it ideal for enterprise and regulated environments where security and operational efficiency are equally important.

Why Enterprises need VNet data gateway

Eliminates gateway infrastructure management

Traditional gateways require:

Virtual machines

High availability setup

OS patching and scaling

Monitoring and troubleshooting

With the VNet data gateway:

Microsoft manages compute lifecycle

No VM or gateway software to maintain

No HA or load balancer design needed

✅ Result: Significant reduction in operational and maintenance overhead for platform and infrastructure teams.

Secure access to private Azure resources

Most enterprise Azure environments use:

Private Endpoints

NSGs and route tables

Firewalls blocking public access

The VNet data gateway:

Is injected into a delegated subnet in your VNet

Uses private IP addressing

Enforces NSG and UDR rules

Communicates with Microsoft services over a Microsoft‑managed internal tunnel

✅ Result: Data sources remain fully private—no public endpoints or inbound ports required.

Designed for Power Platform & Power BI at Scale

The gateway supports secure access for:

Power BI semantic models

Power BI paginated reports

Microsoft Fabric Dataflow Gen2

Fabric pipelines and copy jobs

Because it’s cloud‑native and centrally managed, the VNet data gateway scales well in large enterprises standardizing on Power Platform and Fabric.

High‑level architecture overview

At runtime, the VNet data gateway works as follows:

A query is initiated from Power BI / Power Platform
Query details and credentials are sent to the Microsoft Power Platform VNet service
A containerized gateway instance is injected into the delegated subnet
The gateway connects to the private data source using private networking
Results are sent back to Power BI or Power Platform via a Microsoft‑managed internal tunnel

Key security highlights:

No inbound connectivity

No public IP exposure

Traffic remains on Azure backbone

Full enforcement of NSGs and routing rules

Key Enterprise benefits

Least management overhead – no gateway servers

Zero Trust aligned – private-only connectivity

Fully managed by Microsoft

Enterprise-grade security & governance

Works with Azure Private Endpoint architectures

When to Use VNet Data Gateway

Scenario	Recommendation
Azure private PaaS services	✅ VNet data gateway
Private Endpoint–only access	✅ VNet data gateway
Zero Trust network model	✅ VNet data gateway
Minimal ops & maintenance	✅ VNet data gateway
On‑prem only, no Azure	❌ Traditional gateway

Step‑by‑step configuration: VNet data gateway (Enterprise setup)

High‑level flow (What you will configure)

Register required Azure resource provider
Prepare Azure Virtual Network and subnet
Configure private connectivity to data source
Create the VNet data gateway
Create and bind data source connections
Validate with Power BI / Power Platform workloads

Step 1: Register Microsoft.PowerPlatform resource provider

Why this step is required

The VNet data gateway is a Microsoft‑managed service that is injected into your Azure VNet. Azure must explicitly allow Power Platform to deploy managed infrastructure into your subscription.

Configuration steps

Sign in to Azure portal
Navigate to Subscriptions
Select the subscription that hosts the target VNet
Go to Resource providers
Search for Microsoft.PowerPlatform
Click Register

✅ Status must show Registered

This step enables subnet delegation to Power Platform services.

Step 2: Prepare the Azure Virtual Network

Why this step is required

The gateway runs inside your VNet. It must be placed in a dedicated, delegated subnet to maintain isolation and security boundaries.

Requirements

VNet can be in any Azure region

Subnet must be exclusive to VNet data gateway

Subnet must have outbound connectivity to the data source

Configuration steps

Go to Azure portal → virtual networks
Select your existing VNet (or create one)
Navigate to Subnets → + Subnet
Configure:
- Subnet name: snet-vnet-datagateway
- Address range: /27 or larger (recommended)

- Subnet delegation:
  Microsoft.PowerPlatform/vnetaccesslinks
- Save the subnet

⚠️ Do not place any VMs, app gateway, or other workloads in this subnet.

Step 3: Configure private connectivity to the data source

Why this step is required

Enterprises typically block public access to PaaS services. The VNet data gateway is designed to work natively with private endpoints.

Example: Azure SQL / SQL Managed Instance

Create Private Endpoint for the data service
Attach it to the same VNet (can be different subnet)
Create or link a Private DNS Zone, for example:
privatelink.database.windows.net
Link the Private DNS Zone to the VNet
Ensure DNS resolution from the delegated subnet resolves to private IP

✅ This ensures all traffic remains private and internal.

Step 4: Create the VNet data gateway

Why this step is required

This is where the actual Microsoft‑managed gateway is logically created and associated with your VNet.

Configuration steps

You can do this from either Power BI Service or Power Platform Admin Center.

Using Power Platform Admin Center

Go to https://admin.powerplatform.microsoft.com

Select Data → Gateways

Click + New → Virtual network data gateway

Provide:

- Gateway name

- Azure subscription

- Resource group

- Virtual network

- Delegated subnet

1. Click Create

📌 Notes:

Gateway metadata is stored in Power BI tenant home region

Gateway runtime executes in the VNet region

No VM or scale settings are required

Step 5: Create and configure data source connections

Why this step is required

The gateway exists, but Power BI / Power Platform must know which data sources can be accessed via it.

Configuration steps (Power BI example)

Go to Power BI Service

- Navigate to Settings → Manage connections and gateways

- Select the newly created VNet data gateway

- Click + New connection

- Provide:

- - Data source type (Azure SQL, Storage, Databricks, etc.)

- - Server / endpoint name (private DNS name)

- - Authentication (SQL / Entra ID)
Save the connection
Assign users or security groups

✅ This step enables governance and access control.

Step 6: Use the gateway in Power BI / Power Platform

Power BI

Open dataset or semantic model settings
Under Gateway connection, select:
Use a data gateway

Choose the VNet data gateway
Apply changes
Refresh or run queries

Power Platform / Fabric

Select the same connection when configuring:

Dataflows Gen2

Fabric pipelines

Copy jobs

Step 7: Validate and test

Validation Checklist

✅ DNS resolves to private IP
✅ No public endpoint access enabled
✅ NSGs allow outbound traffic to data source
✅ Dataset refresh succeeds
✅ No gateway VM exists in subscription

Optional:

Enable logging and auditing from Power BI / Fabric

Monitor gateway health in Admin Center

Key Enterprise design guidance (Best practices)

Use one gateway per environment tier (Prod / Non‑Prod)

Use dedicated VNets for data access where possible

Use Private Endpoint only (avoid service endpoints)

Control access via AAD groups, not individuals

Avoid mixing gateway subnet with other workloads

Conclusion: For enterprises looking to consume Power Platform, Power BI, and Microsoft Fabric securely while keeping operational overhead close to zero, the VNet data gateway is the recommended approach.

It removes gateway infrastructure complexity, strengthens security posture, and aligns perfectly with modern Azure landing zone and Zero Trust architectures.

Introducing the Container Network Insights Agent for AKS: Now in Public Preview

chandanAggarwal — Thu, 16 Apr 2026 21:05:45 GMT

We are thrilled to announce public preview of Container Network Insights Agent - Agentic AI network troubleshooting for your workloads running in Azure Kubernetes Service (AKS).

The Challenge

AKS networking is layered by design. Azure CNI, eBPF, Cilium, CoreDNS, NetworkPolicy, CiliumNetworkPolicy, Hubble. Each layer contributes capabilities, and some of these can fail silently in ways the surrounding layers cannot observe.

When something breaks, the evidence usually exists. Operators already have the tools such as Azure Monitor for metrics, Container Insights for cluster health, Prometheus and Grafana for dashboarding, Cilium and Hubble for pod network observation, and Kubectl for direct inspection. However, correlating different signals and identifying the root cause takes time.

Imagine this scenario: An application performance alert fires. The on-call engineer checks dashboards, reviews events, inspects pod health. Each tool shows its own slice. But the root cause usually lives in the relationship between signals, not in any single tool. So the real work begins to manually cross-reference Hubble flows, NetworkPolicy specs, DNS state, node-level stats, and verdicts. Each check is a separate query, a separate context switch, a separate mental model of how the layers interact.

This process is manual, it is slow, needs domain knowledge, and does not scale. Mean time to resolution (MTTR) stays high not because engineers lack skill, but because the investigation surface is wide and the interactions between the layers are complex.

The solution: Container Network Insights Agent

Container Network Insights Agent is agentic AI to simplify and speed up AKS network troubleshooting

Rather than replacing your existing observability tools, the container network insights agent correlates signals on demand to help you quickly identify and resolve network issues. You describe a problem in natural language, and the agent runs a structured investigation across layers. It delivers a diagnosis with the evidence, the root cause, and the exact commands to fix it.

The container network insights agent gets its visibility through two data sources:

- AKS MCP server container network insight agent integrates with the AKS MCP (Model Context Protocol) server, a standardized and secure interface to kubectl, Cilium, and Hubble. Every diagnostic command runs through the same tools operators already use, via a well-defined protocol that enforces security boundaries. No ad-hoc scripts, no custom API integrations.

- Linux Networking plugin  For diagnostics that require visibility below the Kubernetes API layer, container network insights agent collects kernel-level telemetry directly from cluster nodes. This includes NIC ring buffer stats, kernel packet counters, SoftIRQ distribution, and socket buffer utilization. This is how it pinpoints packet drops and network saturation that surface-level metrics cannot explain.

When you describe a symptom, the container network insights agent:

- Classifies the issue and plans an investigation tailored to the symptom pattern

- Gathers evidence through the AKS MCP server and its Linux networking plugin across DNS, service routing, network policies, Cilium, and node-level statistics

- Reasons across layers to identify how a failure in one component manifests in another

- Delivers a structured report with pass/fail evidence, root cause analysis, and specific remediation guidance

The container network insight agent is scoped to AKS networking: DNS failures, packet drops, connectivity issues, policy conflicts, and Cilium dataplane health. It does not modify workloads or change configurations. All remediation guidance is advisory. The agent tells you what to run, and you decide whether to apply it.

What makes the container network insights agent different

Deep telemetry, not just surface metrics Most observability tools operate at the Kubernetes API level. container network insight agent goes deeper, collecting kernel-level network statistics, BPF program drop counters, and interface-level diagnostics that pinpoint exactly where packets are being lost and why. This is the difference between knowing something is wrong and knowing precisely what is causing it.

Cross-layer reasoning Networking incidents rarely have single-layer explanations. The container network insights agent correlates evidence from DNS, service routing, network policy, Cilium, and node-level statistics together. It surfaces causal relationships that span layers. For example: node-level RX drops caused by a Cilium policy denial triggered by a label mismatch after a routine Helm deployment, even though the pods themselves appear healthy.

Structured and auditable Every conclusion traces to a specific check, its output, and its pass/fail status. If all checks pass, container network insights agent reports no issue. It does not invent problems. Investigations are deterministic and reproducible. Results can be reviewed, shared, and rerun.

Guidance, not just findings The container network insights agent explains what the evidence means, identifies the root cause, and provides specific remediation commands. The analysis is done; the operator reviews and decides.

Where the container network insights agent fits

The container network insights agent is not another monitoring tool. It does not collect continuous metrics or replace dashboards. Your existing observability stack, including Azure Monitor, Prometheus, Grafana, Container Insights, and your log pipelines, keeps doing what it does. The agent complements those tools by adding an intelligence layer that turns fragmented signals into actionable diagnosis. Your alerting detects the problem; this agent helps you understand it.

Safe by Design

The container network insights agent is built for production clusters.

- Read-only access Minimal RBAC scoped to pods, services, endpoints, nodes, namespaces, network policies, and Cilium resources. container network insight agent deploys a temporary debug DaemonSet only for packet-drop diagnostics that require host-level stats.

- Advisory remediation only The container network insights agent tells you what to run. It never executes changes.

- Evidence-backed conclusions Every root cause traces to a specific failed check. No speculation.

- Scoped and enforced The agent handles AKS networking questions only. It does not respond to off-topic requests. Prompt injection defenses are built in.

- Credentials stay in the cluster The container network insights agent authenticates via managed identity with workload identity federation. No secrets, no static credentials. Only a session ID cookie reaches the browser.

Get Started

Container network insights agent is available in Public Preview in **Central US, East US, East US 2, UK South, and West US 2**.

The agent deploys as an AKS cluster extension and uses your own Azure OpenAI resource, giving you control over model configuration and data residency. Full capabilities require Cilium and Advanced Container Networking Services. DNS and packet drop diagnostics work on all supported AKS clusters.

To try it:

- Review the Container Network Insights Agent overview on Microsoft Learn 

https://learn.microsoft.com/en-us/azure/aks/container-network-insights-agent-overview 

- Follow the quickstart to deploy container network insights agent and run your first diagnostic

- Share feedback via the Azure feedback channel or the thumbs-up and thumbs-down feedback controls on each response

Your feedback shapes the roadmap. If the agent gets something wrong or misses a scenario you encounter, we want to hear about it.

Enabling fallback to internet for Azure Private DNS Zones in hybrid architectures

kirankumar_manchiwar04 — Wed, 15 Apr 2026 15:57:15 GMT

Introduction

Azure Private Endpoint enables secure connectivity to Azure PaaS services such as:

Azure SQL Managed Instance

Azure Container Registry

Azure Key Vault

Azure Storage Account

through private IP addresses within a virtual network.

When Private Endpoint is enabled for a service, Azure DNS automatically changes the name resolution path using

CNAME Redirection

Example:
myserver.database.windows.net 

↓

myserver.privatelink.database.windows.net

↓

Private IP

Azure Private DNS Zones are then used to resolve this Private Endpoint FQDN within the VNet.

However, this introduces a critical DNS limitation in:

Hybrid cloud architectures (AWS → Azure SQL MI)

Multiregion deployments (DR region access)

Crosstenant / Crosssubscription access

MultiVNet isolated networks

If the Private DNS zone does not contain a corresponding record, Azure DNS returns:

NXDOMAIN (NonExistent Domain)

When a DNS resolver receives a negative response (NXDOMAIN), it sends no DNS response to the DNS client and the query fails.

This results in:

❌ Application connectivity failure
❌ Database connection timeout
❌ AKS pod DNS resolution errors
❌ DR failover application outage

Problem statement

In traditional Private Endpoint DNS resolution:

DNS query is sent from the application.

Azure DNS checks linked Private DNS Zone.

If no matching record exists: NXDOMAIN returned

DNS queries for Azure Private Link and network isolation scenarios across different tenants and resource groups have unique name resolution paths which can affect the ability to reach Private Linkenabled resources outside a tenant's control.

Azure does not retry resolution using public DNS by default.

Therefore:

Public Endpoint resolution never occurs

DNS query fails permanently

Application cannot connect

Microsoft native solution

Fallback to internet (NxDomainRedirect)

Azure introduced a DNS resolution policy:

resolutionPolicy = NxDomainRedirect

This property enables public recursion via Azure’s recursive resolver fleet when an authoritative NXDOMAIN response is received for a Private Link zone.

When enabled:

✅ Azure DNS retries the query
✅ Public endpoint resolution occurs
✅ Application connectivity continues
✅ No custom DNS forwarder required

Fallback policy is configured at:

Private DNS Zone → virtualnetwork link

Resolution policy is enabled at the virtual network link level with the NxDomainRedirect setting.

In the Azure portal this appears as:

         Enable fallback to internet

How it works

Without fallback:

Application → Azure DNS

         → Private DNS Zone

         → Record missing

         → NXDOMAIN returned

         → Connection failure

With fallback enabled:

Application → Azure DNS

         → Private DNS Zone

         → Record missing

         → NXDOMAIN returned

         → Azure recursive resolver

         → Public DNS resolution

         → Public endpoint IP returned

         → Connection successful

Azure recursive resolver retries the query using the public endpoint QNAME each time NXDOMAIN is received from the private zone scope

Real world use case

AWS Application Connecting to Azure SQL Managed Instance

You are running:

SQL MI in Azure

Private Endpoint enabled

Private DNS Zone: privatelink.database.windows.net

AWS application tries to connect:

my-mi.database.windows.net

If DR region DNS record is not available:

Without fallback:

DNS query → NXDOMAIN → App failure

With fallback enabled:

DNS query → Retry public DNS → Connection success

Step-by-step configuration

Method 1 – Azure portal

Go to:

Private DNS Zones

Select your Private Link DNS Zone:

Example:

privatelink.database.windows.net

Select:

Virtual network links

Open your linked VNet

Enable:

✅ Enable fallback to internet

Click:

Save

Method 2 – Azure CLI

You can configure fallback policy using:

az network private-dns link vnet update \

      --resource-group RG-Network \

      --zone-name privatelink.database.windows.net \

      --name VNET-Link \

      --resolution-policy NxDomainRedirect

Validation steps

Run from Azure VM:

nslookup my-mi.database.windows.net

Expected:

✔ Private IP (if available)
✔ Public IP (if fallback triggered)

Security considerations

Fallback to internet:

✅ Does NOT expose data
✅ Only impacts DNS resolution
✅ Network traffic still governed by:

Azure Firewall

Service Endpoint Policies

DNS resolution fallback only triggers on NXDOMAIN and does not change networklevel firewall controls.

When should you enable this?

Recommended in:

Hybrid AWS → Azure connectivity

Multiregion DR deployments

AKS accessing Private Endpoint services

CrossTenant connectivity

Private Link + VPN / ExpressRoute scenarios

Conclusion

Fallback to Internet using NxDomainRedirect provides:

Seamless hybrid connectivity

Reduced DNS complexity

No custom forwarders

Improved application resilience

and simplifies DNS resolution for modern Private Endpointenabled architectures.

A demonstration of Virtual Network TAP

Marc de Droog — Wed, 15 Apr 2026 10:30:01 GMT

Azure Virtual Network Terminal Access Point (VTAP), at the time of writing in April 2026 in public preview in select regions, copies network traffic from source Virtual Machines to a collector or traffic analytics tool, running as a Network Virtual Appliance (NVA). VTAP creates a full copy of all traffic sent and received by Virtual Machine Network Interface Card(s) (NICs) designated as VTAP source(s). This includes packet payload content - in contrast to VNET Flow Logs, which only collect traffic meta data. Traffic collectors and analytics tools are 3rd party partner products, available from the Azure Marketplace, amongst which are the major Network Detection and Response solutions.

VTAP is an agentless, cloud-native traffic tap at the Azure network infrastructure level. It is entirely out-of-band; it has no impact on the source VM's network performance and the source VM is unaware of the tap. Tapped traffic is VXLAN-encapsulated and delivered to the collector NVA, in the same VNET as the source VMs, or in a peered VNET.

This post demonstrates the basic functionality of VTAP: copying traffic into and out of a source VM, to a destination VM.

The demo consists of 3 three Windows VMs in one VNET, each running a basic web server that responds with the VM's name. Another VNET contains the target - a Windows VM on which Wireshark is installed, to inspect traffic forwarded by VTAP. This demo does not use 3rd party VTAP partner solutions from the Marketplace.

The lab for this demonstration is available on Github: Virtual Network TAP.

The VTAP resource is configured with the target VM's NIC as the destination.

All traffic captured from sources is VXLAN-encapsulated and sent to the destination on UDP port 4789 (this cannot be changed).

We use a single source to easier inspect the traffic flows in Wireshark; we will see that communication from the other VMs to our source VM is captured and copied to the destination. In a real world scenario, multiple or all of the VMs in an environment could be set up as TAP sources.

The source VM, vm1, generates traffic through a script that continuously polls vm2 and vm3 on http://10.0.2.5 and http://10.0.2.6, and https://ipconfig.io.

On the destination VM, we use Wireshark to observe captured traffic.

The filter on UDP port 4789 causes Wireshark to only capture the VXLAN encapsulated traffic forwarded by VTAP.

Wireshark automatically decodes VXLAN and displays the actual traffic to and from vm1, which is set up as the (only) VTAP source. Wireshark's capture panel shows the decapsulated TCP and HTTP exchanges, including the TCP handshake, between vm1 and the other VMs, and https://ipconfig.io.

Expanding the lines in the detail panel below the capture panel shows the details of the VXLAN encapsulation. The outer IP packets, encapsulating the VXLAN frames in UDP, originate from the source VM's IP address, 10.0.2.4, and have the target VM's address, 10.1.1.4, as the destination.

The VXLAN frames contain all the details of the original Ethernet frames sent from and received by the source VM, and the IP packets within those. The Wireshark trace shows the full exchange between vm1 and the destinations it speaks with.

This brief demonstration uses Wireshark to simply visualize the operation of VTAP. The partner solutions available from the Azure Marketplace operate on the captured traffic to implement their specific functionality.

Connecting an ExpressRoute circuit to Megaport Virtual Edge

Marc de Droog — Mon, 13 Apr 2026 13:18:18 GMT

Megaport is an ExpressRoute partner in many locations.

The Megaport Cloud Router (MCR) allows ExpressRoute customers to connect leased lines to their on-premise locations, and to connect other Cloud Providers. MCR is easy to set up and operate, it even automatically configures the ExpressRoute Private Peering on both the Megaport and Azure sides, but it does not have a command line interface and does not permit advanced configuration.

For advanced scenario's, Megaport Virtual Edge (MVE) provides a platform to run fully configurable Network Virtual Appliances (NVAs) from a variety of vendors.

This post describes how to connect ExpressRoute to MVE running a Cisco 8000v NVA.

Create the Expressroute Circuit

In the Azure portal, create an ExpressRoute circuit with Standard Resiliency in a Peering location where Megaport is available.

When the circuit deployment is completed, copy the Service key.

Create MVE and ExpressRoute connections

Select Cisco C8000 as the Vendor / Product.

On the next screen:

Select the Location where the MVE is to be deployed - use the ExpressRoute peering location.
Select the MVE size.

On the following screen:

Select Autonomous under Appliance Mode.
Paste a 2048-bit RS SSH public key in the box.
Under Virtual Interfaces (vNICs), add vNICs as needed. One ExpressRoute circuit requires 2 vNICs, one for each path.
vNIC0 will be used to connect a Megaport Internet VXC for SSH access to the device.

On the following screen, give the MVE a name under Finalize Details in the left bar, verify the Summary, and and click Add MVE.

Clicking Create Megaport Internet in the pop up that now appears lets you directly to provision an internet VXC:

Select the location with the lowest latest latency to the MVE - this will be at the top of the list.

On the next screen:

Leave the name as proposed or change as needed.
Set Rate Limit to 20 Mbps (lowest possible, this is for SSH access only).
Leave A-vNIC set to vNIC-0.
Leave Preferred A-End VLAN at Untagged.

On the next screen verify the configuration and click Add VXC.

On the main Services page, the MVE and Internet VXC now show with the note "Order pending".

Click +Connection in the MVE box to connect a VXC to the ExpressRoute Circuit.

Under Choose Destination Type select Cloud.
Then select Microsoft Azure as the Provider.
Paste in the circuit's Service Key and select Port for the Primary path.
Click Next.

On the next screen:

Give the connection a name.
Leave the Rate Limit as proposed, this is set to the bandwidth of the circuit.
At A-end vNIC, select vNIC-1 (do not leave this at vNIC-0!).
At Preferred A-End VLAN, turn off Untag and enter a VLAN number. This will be used to set the sub-interface in the MVE configuration later.

Scroll down to Azure peering VLAN.

Leave Configure Azure Peering VLAN turned on.
Enter the same VLAN ID that will be used in the configuration of the Private Peering on the Azure end.
Click Next.

Verify the configuration summary and click Add VXC.

Repeat the process to add the Secondary path, terminating on vNIC-2. Enter a different VLAN ID for Preferred A-End VLAN. Enter the same VLAN ID that will be used in the Private Peering under Azure peering VLAN.

When the second ExpressRoute VXC is configured, click Review Order in the right hand bar of the Services screen.

When the validation completes, click Order Now.

This will provision the MVE and the VXC. It will take a few minutes for all services to come up.

In the Azure portal, the Provider Status of the ExpressRoute circuit will change to Provisioned.

Configure Private Peering

Go back to the ExpressRoute circuit in the Azure portal. The Provider Status will now be Provisioned, and the Private Peering can be enabled. Click on Peerings under Settings and then click Azure private.

Enter the Peer ASN and Primary and Secondary subnets. Under VLAN ID enter the same number as configured under Azure Peering VLAN in the Primary and Secondary VXC configurations in the Megaport portal.

Configure Cisco IOS

Establish an SSH session to the MVE. Use the public ip address from the internet VXC, and the private key that belongs with the public key used when deploying the MVE.

ssh -i <private-key-file> mveadmin@<public ip>

Configure interfaces:

interface GigabitEthernet2 no ip address no shutdown negotiation auto ! interface GigabitEthernet2.100 encapsulation dot1Q 100 ip address 192.168.0.1 255.255.255.252 ! interface GigabitEthernet3 no ip address no shutdown negotiation auto ! interface GigabitEthernet3.101 encapsulation dot1Q 101 ip address 192.168.0.5 255.255.255.252

Use the Preferred A-end VLAN values set in the primary and secondary VXCs to configure the encapsulation on the subinterfaces. Use the lower address of the /30 subnets configured on the Private Peering.

The higher IP addresses of the Private Peering should now respond to ping:

ping 192.168.0.2 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 192.168.0.2, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms

If ping does not work there likely is an ARP resolution issue. Run `show arp` and `debug arp` and check the ARP table of the Private Peering.

Configure BGP:

router bgp 64000 bgp log-neighbor-changes neighbor 192.168.0.2 remote-as 12076 neighbor 192.168.0.2 soft-reconfiguration inbound neighbor 192.168.0.6 remote-as 12076 neighbor 192.168.0.6 soft-reconfiguration inbound

Verify both neighbors show BGP state = Established:

sh ip bgp neighbor 192.168.0.2 BGP neighbor is 192.168.0.2, remote AS 12076, external link BGP version 4, remote router ID 192.168.0.2 BGP state = Established, up for 1d21h ...

This completes the basic configuration of ExpressRoute to MVE.

Announcing public preview: Cilium mTLS encryption for Azure Kubernetes Service

chandanAggarwal — Mon, 23 Mar 2026 01:50:18 GMT

We are thrilled to announce the public preview of Cilium mTLS encryption in Azure Kubernetes Service (AKS), delivered as part of Advanced Container Networking Services and powered by the Azure CNI dataplane built on Cilium.

This capability is the result of a close engineering collaboration between Microsoft and Isovalent (now part of Cisco). It brings transparent, workload‑level mutual TLS (mTLS) to AKS without sidecars, without application changes, and without introducing a separate service mesh stack.

This public preview represents a major step forward in delivering secure, high‑performance, and operationally simple networking for AKS customers. In this post, we’ll walk through how Cilium mTLS works, when to use it, and how to get started.

Why Cilium mTLS encryption matters

Traditionally, teams looking to in-transit traffic encryption in Kubernetes have had two primary options:

Node-level encryption (for example, WireGuard or virtual network encryption), which secures traffic in transit but lacks workload identity and authentication.

Service meshes, which provide strong identity and mTLS guarantees but introduce operational complexity.

This trade‑off has become increasingly problematic, as many teams want workload‑level encryption and authentication, but without the cost, overhead, and architectural impact of deploying and operating a full-service mesh.

Cilium mTLS closes this gap directly in the dataplane. It delivers transparent, inline mTLS encryption and authentication for pod‑to‑pod TCP traffic, enforced below the application layer. And implemented natively in the Azure CNI dataplane built on Cilium, so customers gain workload‑level security without introducing a separate service mesh, resulting in a simpler architecture with lower operational overhead.

To see how this works under the hood, the next section breaks down the Cilium mTLS architecture and follows a pod‑to‑pod TCP flow from interception to authentication and encryption.

Architecture and design: How Cilium mTLS works

Cilium mTLS achieves workload‑level authentication and encryption by combining three key components, each responsible for a specific part of the authentication and encryption lifecycle.

Cilium agent: Transparent traffic interception and wiring

Cilium agent which already exists on any cluster running with Azure CNI powered by cilium, is responsible for making mTLS invisible to applications. When a namespace is labelled with “io.cilium/mtls-enabled=true”, The Cilium agent enrolls all pods in that namespace. It enters each pod's network namespace and installs iptables rules that redirect outbound traffic to ztunnel on port 15001.

It is also responsible for passing workload metadata (such as pod IP and namespace context) to ztunnel.

Ztunnel: Node‑level mTLS enforcement

Ztunnel is an open source, lightweight, node‑level Layer 4 proxy that was originally created by Istio. Ztunnel runs as a DaemonSet, on the source node it looks up the destination workload via XDS (streamed from the Cilium agent) and establishes mutually authenticated TLS 1.3 sessions between source and destination nodes. Connections are held inline until authentication is complete, ensuring that traffic is never sent in plaintext.

The destination ztunnel decrypts the traffic and delivers it into the target pod, bypassing the interception rules via an in-pod mark. The application sees a normal plaintext connection — it is completely unaware encryption happened.

SPIRE: Workload identity and trust

SPIRE (SPIFFE Runtime Environment) provides the identity foundation for Cilium mTLS. SPIRE acts as the cluster Certificate Authority, issuing short‑lived X.509 certificates (SVIDs) that are automatically rotated and validated.

This is a key design principle of Cilium mTLS - trust is based on workload identity, not network topology.

Each workload receives a cryptographic identity derived from:

Kubernetes namespace

Kubernetes ServiceAccount

These identities are issued and rotated automatically by SPIRE and validated on both sides of every connection. As a result:

Identity remains stable across pod restarts and rescheduling

Authentication is decoupled from IP addresses

Trust decisions align naturally with Kubernetes RBAC and namespace boundaries

This enables a zero‑trust networking model that fits cleanly into existing AKS security practices.

End‑to‑End workflow example

To see how these components work together, consider a simple pod‑to‑pod connection:

A pod initiates a TCP connection to another pod.

Traffic intercepted inside the pod network namespace and redirected to the local ztunnel instance.

ztunnel retrieves the workload identity using certificates issued by SPIRE.

ztunnel establishes a mutually authenticated TLS session with the destination node’s ztunnel.

Traffic is encrypted and sent between pods.

The destination ztunnel decrypts the traffic and delivers it to the target pod.

Every packet from an enrolled pod is encrypted. There is no plaintext window, and no dropped first packets. The connection is held inline by ztunnel until the mTLS tunnel is established, then traffic flows bidirectionally through an HBONE (HTTP/2 CONNECT) tunnel.

Workload enrolment and scope

Cilium mTLS in AKS is opt‑in and scoped at the namespace level.

Platform teams enable mTLS by applying a single label to a namespace. From that point on:

All pods in that namespace participate in mTLS

Authentication and encryption are mandatory between enrolled workloads

Non-enrolled namespaces continue to operate unchanged

Encryption is applied only when both pods are enrolled. Traffic between enrolled and non‑enrolled workloads continues in plaintext without causing connectivity issues or hard failures. This model enables gradual rollout, staged migrations, and low-risk adoption across environments.

Getting started in AKS

Cilium mTLS encryption is available in public preview for AKS clusters that use:

Azure CNI powered by Cilium

Advanced Container Networking Services

You can enable mTLS:

When creating a new cluster, or

On an existing cluster by updating the Advanced Container Networking Services configuration

Once enabled, enrolling workloads is as simple as labelling a namespace.

👉 Learn more

Concepts: How Cilium mTLS works, architecture, and trust boundaries

How-to guide: Step-by-step instructions to enable and verify mTLS in AKS

Looking ahead

This public preview represents an important step forward in simplifying network security for AKS and reflects a deep collaboration between Microsoft and Isovalent to bring open, standards‑based innovation into production‑ready cloud platforms. We’re continuing to work closely with the community to improve the feature and move it toward general availability.

If you’re looking for workload‑level encryption without the overhead of a traditional service mesh, we invite you to try Cilium mTLS in AKS and share your experience.

Azure Front Door: Resiliency Series – Part 2: Faster recovery (RTO)

AbhishekTiwari — Tue, 14 Jul 2026 17:56:55 GMT

Abhishek Tiwari, Vice President of Engineering, Azure Networking
Amit Srivastava, Partner Director of PM, Azure Networking
Varun Chawla, Partner Director of Engineering, Azure Networking
Karthik Uthaman, Principal Engineer, Azure Networking

In Part 1 of this blog series, we outlined our four‑pillar strategy for resiliency in Azure Front Door: configuration resiliency, data plane resiliency, tenant isolation, and accelerated Recovery Time Objective (RTO). Together, these pillars help Azure Front Door remain continuously available and resilient at global scale.

Part 1 focused on the first two pillars: configuration and data plane resiliency. Our goal is to make configuration propagation safer, so incompatible changes never escape pre‑production environments. We discussed how incompatible configurations are blocked early, and how data plane resiliency ensures the system continues serving traffic from a last‑known‑good (LKG) configuration even if a bad change manages to propagate. We also introduced ‘Food Taster’, a dedicated sacrificial process running in each edge server’s data plane, that pretests every configuration change in isolation, before it ever reaches the live data plane.

In this post, we turn to the recovery pillar. We describe how we have made key enhancements to the Azure Front Door recovery path so the system can return to full operation in a predictable and bounded timeframe. For a global service like Azure Front Door, serving hundreds of thousands of tenants across 210+ edge sites worldwide, we set an explicit target: to be able to recover any edge site – or all edge sites – within approximately 10 minutes, even in worst‑case scenarios. In typical data plane crash scenarios, we expect recovery in under a second.

Repair status

The first blog post in this series mentioned the two Azure Front Door incidents from October 2025 – learn more by watching our Azure Incident Retrospective session recordings for the October 9^th incident and/or the October 29^th incident. Before diving into our platform investments for improving our Recovery Time Objectives (RTO), we wanted to provide a quick update on the overall repair items from these incidents. We are pleased to report that the work on configuration propagation and data plane resiliency is now complete and fully deployed across the platform (in the table below, “Completed” means broadly deployed in production). With this, we have reduced configuration propagation latency from ~45 minutes to ~20 minutes. We anticipate reducing this even further – to ~15 minutes by the end of April 2026, while ensuring that platform stability remains our top priority.

Learning category	Goal	Repairs	Status
Safe customer configuration deployment	Incompatible configuration never propagates beyond ‘EUAP or canary regions’	Control plane and data plane defect fixes Forced synchronous configuration processing Additional stages with extended bake time Early detection of crash state	Completed
Data plane resiliency	Configuration processing cannot impact data plane availability	Manage data-plane lifecycle to prevent outages caused by configuration-processing defects.	Completed
Data plane resiliency		Isolated work-process in every data plane server to process and load the configuration.	Completed
100% Azure Front Door resiliency posture for Microsoft internal services	Microsoft operates an isolated, independent Active/Active fleet with automatic failover for critical Azure services	Phase 1: Onboarded critical services batch impacted on Oct 29^th outage running on a day old configuration	Completed
		Phase 2: Automation & hardening of operations, auto-failover and self-management of Azure Front Door onboarding for additional services	March 2026
Recovery improvements	Data plane crash recovery in under 10 minutes	Data plane boot-up time optimized via local cache (~1 hour)	Completed
Recovery improvements	Data plane crash recovery in under 10 minutes	Accelerate recovery time < 10 minutes	April 2026
Tenant isolation	No configuration or traffic regression can impact other tenants	Micro cellular Azure Front Door with ingress layered shards	June 2026

Why recovery at edge scale is deceptively hard

To understand why recovery took as long as it did, it helps to first understand how the Azure Front Door data plane processes configuration.

Azure Front Door operates in 210+ edge sites with multiple servers per site. The data plane of each edge server hosts multiple processes. A master process orchestrates the lifecycle of multiple worker processes, that serve customer traffic. A separate configuration translator process runs alongside the data plane processes, and is responsible for converting customer configuration bundles from the control plane into optimized binary FlatBuffer files. This translation step, covering hundreds of thousands of tenants, represents hours of cumulative computation. A per edge server cache is kept locally at each server level – to enable a fast recovery of the data plane, if needed.

Once the configuration translator process produces these FlatBuffer files, each worker processes them independently and memory-maps them for zero-copy access. Configuration updates flow through a two-phase commit: new FlatBuffers are first loaded into a staging area and validated, then atomically swapped into production maps. In-flight requests continue using the old configuration, until the last request referencing them completes.

The data process recovery is designed to be resilient to different failure modes. A failure or crash at the worker process level has a typical recovery time of less than one second. Since each server has multiple such worker processes which serve customer traffic, this type of crash has no impact on the data plane. In the case of a master process crash, the system automatically tries to recover using the local cache. When the local cache is reused, the system is able to recover quickly – in approximately 60 minutes – since most of the configurations in the cache were already loaded into the data plane before the crash. However, in certain cases if the cache becomes unavailable or must be invalidated because of corruption, the recovery time increases significantly.

During the October 29^th incident, a data plane crash triggered a complete recovery sequence that took approximately 4.5 hours. This was not because restarting a process is slow, it is because a defect in the recovery process invalidated the local cache, which meant that “restart” meant rebuilding everything from scratch. The configuration translator process then had to re-fetch and re-translate every one of the hundreds of thousands of customer configurations, before workers could memory-map them and begin serving traffic.

This experience has crystallized three fundamental learnings related to our recovery path:

Expensive rework: A subset of crashes discarded all previously translated FlatBuffer artifacts, forcing the configuration translator process to repeat hours of conversion work that had already been validated and stored.
High restart costs: Every worker on every node had to wait for the configuration translator process to complete the full translation, before it could memory-map any configuration and begin serving requests.
Unbounded recovery time: Recovery time grew linearly with total tenant footprint rather than with active traffic, creating a ‘scale penalty’ as more tenants onboarded to the system.

Separately and together, the insight was clear: recovery must stop being proportional to the total configuration size.

Persisting ‘validated configurations’ across restarts

One of the key recovery improvements was strengthening how validated customer configurations are cached and reused across failures, rather than rebuilding configuration states from scratch during recovery. Azure Front Door already cached customer configurations on host‑mounted storage prior to the October incident. The platform enhancements post outage focused on making the local configuration cache resilient to crashes, partial failures, and bad tenant inputs.

Our goal was to ensure that recovery behavior is dominated by serving traffic safely, not by reconstructing configuration state. This led us to two explicit design goals…

Design goals

No category of crash should invalidate the configuration cache: Configuration cache invalidation must never be the default response to failures. Whether the failure is a worker crash, master crash, data plane restart, or coordinated recovery action, previously validated customer configurations should remain usable—unless there is a proven reason to discard it.
Bad tenant configuration must not poison the entire cache: A single faulty or incompatible tenant configuration should result in targeted eviction of that tenant’s configuration only—not wholesale cache invalidation across all tenants.

Platform enhancements

Previously, customer configurations persisted to host‑mounted storage, but certain failure paths treated the cache as unsafe and invalidated it entirely. In those cases, recovery implicitly meant reloading and reprocessing configuration for hundreds of thousands of tenants before traffic could resume, even though the vast majority of cached data was still valid.

We changed the recovery model to avoid invalidating customer configurations, with strict scoping around when and how cached entries are discarded:

Cached configurations are no longer invalidated based on crash type. Failures are assumed to be orthogonal to configuration correctness unless explicitly proven otherwise.
Cache eviction is granular and tenant‑scoped. If a cached configuration fails validation or load checks, only that tenant’s configuration is discarded and reloaded. All other tenant configurations remain available.

This ensures that recovery does not regress into a fleet‑wide rebuild due to localized or unrelated faults.

Safety and correctness

Durability is paired with strong correctness controls, to prevent unsafe configurations from being served:

Per‑tenant validation on load: Each cached tenant configuration is validated during the ‘load and verification’ phase, before being promoted for traffic serving. Therefore, failures are contained to that tenant.
Targeted re‑translation: When validation fails, only the affected tenant’s configuration is reloaded or reprocessed. Therefore, the cache for other tenants is left untouched.
Operational escape hatch: Operators retain the ability to explicitly instruct a clean rebuild of the configuration cache (with proper authorization), preserving control without compromising the default fast‑recovery path.

Resulting behavior

With these changes, recovery behavior now aligns with real‑world traffic patterns - configuration defects impact tenants locally and predictably, rather than globally. The system now prefers isolated tenant impact, and continued service using last-known-good over aggressive invalidation, both of which are critical for predictable recovery at the scale of Azure Front Door.

Making recovery scale with active traffic, not total tenants

Reusing configuration cache solves the problem of rebuilding configuration in its entirety, but even with a warm cache, the original startup path had a second bottleneck: eagerly loading a large volume of tenant configurations into memory before serving any traffic. At our scale, memory-mapping, parsing hundreds of thousands of FlatBuffers, constructing internal lookup maps, adding Transport Layer Security (TLS) certificates and configuration blocks for each tenant, collectively added almost an hour to startup time. This was the case even when a majority of those tenants had no active traffic at that moment.

We addressed this by fundamentally changing when configuration is loaded into workers. Rather than eagerly loading most of the tenants at startup across all edge locations, Azure Front Door now uses an Machine Learning (ML)-optimized lazy loading model.

In the new architecture, instead of loading a large number of tenant configurations, we only load a small subset of tenants that are known to be historically active in a given site, we call this the “warm tenants” list. The warm tenants list per edge site is created through a sophisticated traffic analysis pipeline that leverages ML. However, loading the warm tenants is not good enough, because when a request arrives and we don’t have the configuration in memory, we need to know two things. Firstly, is this a request from a real Azure Front Door tenant – and, if it is, where can I find the configuration?

To answer these questions, each worker maintains a hostmap that tracks the state of each tenant’s configuration. This hostmap is constructed during startup, as we process each tenant configuration – if the tenant is in the warm list, we will process and load their configuration fully; if not, then we will just add an entry into the hostmap where all their domain names are mapped to the configuration path location. When a request arrives for one of these tenants, the worker loads and validates that tenant’s configuration on demand, and immediately begins serving traffic. This allows a node to start serving its busiest tenants within a few minutes of startup, while additional tenants are loaded incrementally only when traffic actually arrives—allowing the system to progressively absorb cold tenants as demand increases.

The effect on recovery is transformative. Instead of recovery time scaling with the total number of tenants configured on a server, it scales with the number of tenants actively receiving traffic. In practice, even at our busiest edge sites, the active tenant set is a small fraction of the total.

Just as importantly, this modified form of lazy loading provides a natural failure isolation boundary. Most Edge sites won’t ever load a faulty configuration of an inactive tenant. When a request for an inactive tenant with an incompatible configuration arrives, impact is contained to a single worker.

The configuration load architecture now prefers serving as many customers as quickly as possible, rather than waiting until everything is ready before serving anyone. The above changes are slated to complete in April 2026 and will bring our RTO from the current ~1 hour to under 10 minutes – for complete recovery from a worst case scenario.

Continuous validation through Game Days

A critical element of our recovery confidence comes from GameDay fault-injection testing. We don’t simply design recovery mechanisms and assume they work—we break the system deliberately and observe how it responds. Since late 2025, we have conducted recurring GameDay drills that simulate the exact failure scenarios we are defending against:

Food Taster crash scenarios: Injecting deliberately faulty tenant configurations, to verify that they are caught and isolated with zero impact on live traffic. In our January 2026 GameDay, the Food Taster process crashed as expected, the system halted the update within approximately 5 seconds, and no customer traffic was affected.
Master process crash scenarios: Triggering master process crashes across test environments to verify that workers continue serving traffic, that the Local Config Shield engages within 10 seconds, and that the coordinated recovery tool restores full operation within the expected timeframe.
Multi-region failure drills: Simulating simultaneous failures across multiple regions to validate that global Config Shield mechanisms engage correctly, and that recovery procedures scale without requiring manual per-region intervention.
Fallback test drills for critical Azure services running behind Azure Front Door: In our February 2026 GameDay, we simulated the complete unavailability of Azure Front Door, and successfully validated failover for critical Azure services with no impact to traffic.

These drills have both surfaced corner cases and built operational confidence. They have transformed recovery from a theoretical plan into tested, repeatable muscle memory. As we noted in an internal communication to our team: “Game day testing is a deliberate shift from assuming resilience to actively proving it—turning reliability into an observed and repeatable outcome.”

Closing

Part 1 of this series emphasized preventing unsafe configurations from reaching the data plane, and data plane resiliency in case an incompatible configuration reaches production. This post has shown that prevention alone is not enough—when failures do occur, recovery must be fast, predictable, and bounded. By ensuring that the FlatBuffer cache is never invalidated, by loading only active tenants, and by building safe coordinated recovery tooling, we have transformed failure handling from a fleet-wide crisis into a controlled operation.

These recovery investments work in concert with the prevention mechanisms described in Part 1. Together, they ensure that the path from incident detection to full service restoration is measured in minutes, with customer traffic protected at every step.

In the next post of this series, we will cover the third pillar of our resiliency strategy: tenant isolation—how micro-cellular architecture and ingress-layered sharding can reduce the blast radius of any failure to a small subset, ensuring that one customer’s configuration or traffic anomaly never becomes everyone’s problem.

We deeply value our customers’ trust in Azure Front Door. We are committed to transparently sharing our progress on these resiliency investments, and to exceed expectations for safety, reliability, and operational readiness.