Azure Networking Blog articles

Introducing the Container Network Insights Agent for AKS: Now in Public Preview

chandanAggarwal — Thu, 16 Apr 2026 21:05:45 GMT

We are thrilled to announce public preview of Container Network Insights Agent - Agentic AI network troubleshooting for your workloads running in Azure Kubernetes Service (AKS).

The Challenge

AKS networking is layered by design. Azure CNI, eBPF, Cilium, CoreDNS, NetworkPolicy, CiliumNetworkPolicy, Hubble. Each layer contributes capabilities, and some of these can fail silently in ways the surrounding layers cannot observe.

When something breaks, the evidence usually exists. Operators already have the tools such as Azure Monitor for metrics, Container Insights for cluster health, Prometheus and Grafana for dashboarding, Cilium and Hubble for pod network observation, and Kubectl for direct inspection. However, correlating different signals and identifying the root cause takes time.

Imagine this scenario: An application performance alert fires. The on-call engineer checks dashboards, reviews events, inspects pod health. Each tool shows its own slice. But the root cause usually lives in the relationship between signals, not in any single tool. So the real work begins to manually cross-reference Hubble flows, NetworkPolicy specs, DNS state, node-level stats, and verdicts. Each check is a separate query, a separate context switch, a separate mental model of how the layers interact.

This process is manual, it is slow, needs domain knowledge, and does not scale. Mean time to resolution (MTTR) stays high not because engineers lack skill, but because the investigation surface is wide and the interactions between the layers are complex.

The solution: Container Network Insights Agent

Container Network Insights Agent is agentic AI to simplify and speed up AKS network troubleshooting

Rather than replacing your existing observability tools, the container network insights agent correlates signals on demand to help you quickly identify and resolve network issues. You describe a problem in natural language, and the agent runs a structured investigation across layers. It delivers a diagnosis with the evidence, the root cause, and the exact commands to fix it.

The container network insights agent gets its visibility through two data sources:

- AKS MCP server container network insight agent integrates with the AKS MCP (Model Context Protocol) server, a standardized and secure interface to kubectl, Cilium, and Hubble. Every diagnostic command runs through the same tools operators already use, via a well-defined protocol that enforces security boundaries. No ad-hoc scripts, no custom API integrations.

- Linux Networking plugin  For diagnostics that require visibility below the Kubernetes API layer, container network insights agent collects kernel-level telemetry directly from cluster nodes. This includes NIC ring buffer stats, kernel packet counters, SoftIRQ distribution, and socket buffer utilization. This is how it pinpoints packet drops and network saturation that surface-level metrics cannot explain.

When you describe a symptom, the container network insights agent:

- Classifies the issue and plans an investigation tailored to the symptom pattern

- Gathers evidence through the AKS MCP server and its Linux networking plugin across DNS, service routing, network policies, Cilium, and node-level statistics

- Reasons across layers to identify how a failure in one component manifests in another

- Delivers a structured report with pass/fail evidence, root cause analysis, and specific remediation guidance

The container network insight agent is scoped to AKS networking: DNS failures, packet drops, connectivity issues, policy conflicts, and Cilium dataplane health. It does not modify workloads or change configurations. All remediation guidance is advisory. The agent tells you what to run, and you decide whether to apply it.

What makes the container network insights agent different

Deep telemetry, not just surface metrics Most observability tools operate at the Kubernetes API level. container network insight agent goes deeper, collecting kernel-level network statistics, BPF program drop counters, and interface-level diagnostics that pinpoint exactly where packets are being lost and why. This is the difference between knowing something is wrong and knowing precisely what is causing it.

Cross-layer reasoning Networking incidents rarely have single-layer explanations. The container network insights agent correlates evidence from DNS, service routing, network policy, Cilium, and node-level statistics together. It surfaces causal relationships that span layers. For example: node-level RX drops caused by a Cilium policy denial triggered by a label mismatch after a routine Helm deployment, even though the pods themselves appear healthy.

Structured and auditable Every conclusion traces to a specific check, its output, and its pass/fail status. If all checks pass, container network insights agent reports no issue. It does not invent problems. Investigations are deterministic and reproducible. Results can be reviewed, shared, and rerun.

Guidance, not just findings The container network insights agent explains what the evidence means, identifies the root cause, and provides specific remediation commands. The analysis is done; the operator reviews and decides.

Where the container network insights agent fits

The container network insights agent is not another monitoring tool. It does not collect continuous metrics or replace dashboards. Your existing observability stack, including Azure Monitor, Prometheus, Grafana, Container Insights, and your log pipelines, keeps doing what it does. The agent complements those tools by adding an intelligence layer that turns fragmented signals into actionable diagnosis. Your alerting detects the problem; this agent helps you understand it.

Safe by Design

The container network insights agent is built for production clusters.

- Read-only access Minimal RBAC scoped to pods, services, endpoints, nodes, namespaces, network policies, and Cilium resources. container network insight agent deploys a temporary debug DaemonSet only for packet-drop diagnostics that require host-level stats.

- Advisory remediation only The container network insights agent tells you what to run. It never executes changes.

- Evidence-backed conclusions Every root cause traces to a specific failed check. No speculation.

- Scoped and enforced The agent handles AKS networking questions only. It does not respond to off-topic requests. Prompt injection defenses are built in.

- Credentials stay in the cluster The container network insights agent authenticates via managed identity with workload identity federation. No secrets, no static credentials. Only a session ID cookie reaches the browser.

Get Started

Container network insights agent is available in Public Preview in **Central US, East US, East US 2, UK South, and West US 2**.

The agent deploys as an AKS cluster extension and uses your own Azure OpenAI resource, giving you control over model configuration and data residency. Full capabilities require Cilium and Advanced Container Networking Services. DNS and packet drop diagnostics work on all supported AKS clusters.

To try it:

- Review the Container Network Insights Agent overview on Microsoft Learn 

https://learn.microsoft.com/en-us/azure/aks/container-network-insights-agent-overview 

- Follow the quickstart to deploy container network insights agent and run your first diagnostic

- Share feedback via the Azure feedback channel or the thumbs-up and thumbs-down feedback controls on each response

Your feedback shapes the roadmap. If the agent gets something wrong or misses a scenario you encounter, we want to hear about it.

Enabling fallback to internet for Azure Private DNS Zones in hybrid architectures

kirankumar_manchiwar04 — Wed, 15 Apr 2026 15:57:15 GMT

Introduction

Azure Private Endpoint enables secure connectivity to Azure PaaS services such as:

Azure SQL Managed Instance

Azure Container Registry

Azure Key Vault

Azure Storage Account

through private IP addresses within a virtual network.

When Private Endpoint is enabled for a service, Azure DNS automatically changes the name resolution path using

CNAME Redirection

Example:
myserver.database.windows.net 

↓

myserver.privatelink.database.windows.net

↓

Private IP

Azure Private DNS Zones are then used to resolve this Private Endpoint FQDN within the VNet.

However, this introduces a critical DNS limitation in:

Hybrid cloud architectures (AWS → Azure SQL MI)

Multiregion deployments (DR region access)

Crosstenant / Crosssubscription access

MultiVNet isolated networks

If the Private DNS zone does not contain a corresponding record, Azure DNS returns:

NXDOMAIN (NonExistent Domain)

When a DNS resolver receives a negative response (NXDOMAIN), it sends no DNS response to the DNS client and the query fails.

This results in:

❌ Application connectivity failure
❌ Database connection timeout
❌ AKS pod DNS resolution errors
❌ DR failover application outage

Problem statement

In traditional Private Endpoint DNS resolution:

DNS query is sent from the application.

Azure DNS checks linked Private DNS Zone.

If no matching record exists: NXDOMAIN returned

DNS queries for Azure Private Link and network isolation scenarios across different tenants and resource groups have unique name resolution paths which can affect the ability to reach Private Linkenabled resources outside a tenant's control.

Azure does not retry resolution using public DNS by default.

Therefore:

Public Endpoint resolution never occurs

DNS query fails permanently

Application cannot connect

Microsoft native solution

Fallback to internet (NxDomainRedirect)

Azure introduced a DNS resolution policy:

resolutionPolicy = NxDomainRedirect

This property enables public recursion via Azure’s recursive resolver fleet when an authoritative NXDOMAIN response is received for a Private Link zone.

When enabled:

✅ Azure DNS retries the query
✅ Public endpoint resolution occurs
✅ Application connectivity continues
✅ No custom DNS forwarder required

Fallback policy is configured at:

Private DNS Zone → virtualnetwork link

Resolution policy is enabled at the virtual network link level with the NxDomainRedirect setting.

In the Azure portal this appears as:

         Enable fallback to internet

How it works

Without fallback:

Application → Azure DNS

         → Private DNS Zone

         → Record missing

         → NXDOMAIN returned

         → Connection failure

With fallback enabled:

Application → Azure DNS

         → Private DNS Zone

         → Record missing

         → NXDOMAIN returned

         → Azure recursive resolver

         → Public DNS resolution

         → Public endpoint IP returned

         → Connection successful

Azure recursive resolver retries the query using the public endpoint QNAME each time NXDOMAIN is received from the private zone scope

Real world use case

AWS Application Connecting to Azure SQL Managed Instance

You are running:

SQL MI in Azure

Private Endpoint enabled

Private DNS Zone: privatelink.database.windows.net

AWS application tries to connect:

my-mi.database.windows.net

If DR region DNS record is not available:

Without fallback:

DNS query → NXDOMAIN → App failure

With fallback enabled:

DNS query → Retry public DNS → Connection success

Step-by-step configuration

Method 1 – Azure portal

Go to:

Private DNS Zones

Select your Private Link DNS Zone:

Example:

privatelink.database.windows.net

Select:

Virtual network links

Open your linked VNet

Enable:

✅ Enable fallback to internet

Click:

Save

Method 2 – Azure CLI

You can configure fallback policy using:

az network private-dns link vnet update \

      --resource-group RG-Network \

      --zone-name privatelink.database.windows.net \

      --name VNET-Link \

      --resolution-policy NxDomainRedirect

Validation steps

Run from Azure VM:

nslookup my-mi.database.windows.net

Expected:

✔ Private IP (if available)
✔ Public IP (if fallback triggered)

Security considerations

Fallback to internet:

✅ Does NOT expose data
✅ Only impacts DNS resolution
✅ Network traffic still governed by:

Azure Firewall

Service Endpoint Policies

DNS resolution fallback only triggers on NXDOMAIN and does not change networklevel firewall controls.

When should you enable this?

Recommended in:

Hybrid AWS → Azure connectivity

Multiregion DR deployments

AKS accessing Private Endpoint services

CrossTenant connectivity

Private Link + VPN / ExpressRoute scenarios

Conclusion

Fallback to Internet using NxDomainRedirect provides:

Seamless hybrid connectivity

Reduced DNS complexity

No custom forwarders

Improved application resilience

and simplifies DNS resolution for modern Private Endpointenabled architectures.

A demonstration of Virtual Network TAP

Marc de Droog — Wed, 15 Apr 2026 10:30:01 GMT

Azure Virtual Network Terminal Access Point (VTAP), at the time of writing in April 2026 in public preview in select regions, copies network traffic from source Virtual Machines to a collector or traffic analytics tool, running as a Network Virtual Appliance (NVA). VTAP creates a full copy of all traffic sent and received by Virtual Machine Network Interface Card(s) (NICs) designated as VTAP source(s). This includes packet payload content - in contrast to VNET Flow Logs, which only collect traffic meta data. Traffic collectors and analytics tools are 3rd party partner products, available from the Azure Marketplace, amongst which are the major Network Detection and Response solutions.

VTAP is an agentless, cloud-native traffic tap at the Azure network infrastructure level. It is entirely out-of-band; it has no impact on the source VM's network performance and the source VM is unaware of the tap. Tapped traffic is VXLAN-encapsulated and delivered to the collector NVA, in the same VNET as the source VMs, or in a peered VNET.

This post demonstrates the basic functionality of VTAP: copying traffic into and out of a source VM, to a destination VM.

The demo consists of 3 three Windows VMs in one VNET, each running a basic web server that responds with the VM's name. Another VNET contains the target - a Windows VM on which Wireshark is installed, to inspect traffic forwarded by VTAP. This demo does not use 3rd party VTAP partner solutions from the Marketplace.

The lab for this demonstration is available on Github: Virtual Network TAP.

The VTAP resource is configured with the target VM's NIC as the destination.

All traffic captured from sources is VXLAN-encapsulated and sent to the destination on UDP port 4789 (this cannot be changed).

We use a single source to easier inspect the traffic flows in Wireshark; we will see that communication from the other VMs to our source VM is captured and copied to the destination. In a real world scenario, multiple or all of the VMs in an environment could be set up as TAP sources.

The source VM, vm1, generates traffic through a script that continuously polls vm2 and vm3 on http://10.0.2.5 and http://10.0.2.6, and https://ipconfig.io.

On the destination VM, we use Wireshark to observe captured traffic.

The filter on UDP port 4789 causes Wireshark to only capture the VXLAN encapsulated traffic forwarded by VTAP.

Wireshark automatically decodes VXLAN and displays the actual traffic to and from vm1, which is set up as the (only) VTAP source. Wireshark's capture panel shows the decapsulated TCP and HTTP exchanges, including the TCP handshake, between vm1 and the other VMs, and https://ipconfig.io.

Expanding the lines in the detail panel below the capture panel shows the details of the VXLAN encapsulation. The outer IP packets, encapsulating the VXLAN frames in UDP, originate from the source VM's IP address, 10.0.2.4, and have the target VM's address, 10.1.1.4, as the destination.

The VXLAN frames contain all the details of the original Ethernet frames sent from and received by the source VM, and the IP packets within those. The Wireshark trace shows the full exchange between vm1 and the destinations it speaks with.

This brief demonstration uses Wireshark to simply visualize the operation of VTAP. The partner solutions available from the Azure Marketplace operate on the captured traffic to implement their specific functionality.

Connecting an ExpressRoute circuit to Megaport Virtual Edge

Marc de Droog — Mon, 13 Apr 2026 13:18:18 GMT

Megaport is an ExpressRoute partner in many locations.

The Megaport Cloud Router (MCR) allows ExpressRoute customers to connect leased lines to their on-premise locations, and to connect other Cloud Providers. MCR is easy to set up and operate, it even automatically configures the ExpressRoute Private Peering on both the Megaport and Azure sides, but it does not have a command line interface and does not permit advanced configuration.

For advanced scenario's, Megaport Virtual Edge (MVE) provides a platform to run fully configurable Network Virtual Appliances (NVAs) from a variety of vendors.

This post describes how to connect ExpressRoute to MVE running a Cisco 8000v NVA.

Create the Expressroute Circuit

In the Azure portal, create an ExpressRoute circuit with Standard Resiliency in a Peering location where Megaport is available.

When the circuit deployment is completed, copy the Service key.

Create MVE and ExpressRoute connections

Select Cisco C8000 as the Vendor / Product.

On the next screen:

Select the Location where the MVE is to be deployed - use the ExpressRoute peering location.
Select the MVE size.

On the following screen:

Select Autonomous under Appliance Mode.
Paste a 2048-bit RS SSH public key in the box.
Under Virtual Interfaces (vNICs), add vNICs as needed. One ExpressRoute circuit requires 2 vNICs, one for each path.
vNIC0 will be used to connect a Megaport Internet VXC for SSH access to the device.

On the following screen, give the MVE a name under Finalize Details in the left bar, verify the Summary, and and click Add MVE.

Clicking Create Megaport Internet in the pop up that now appears lets you directly to provision an internet VXC:

Select the location with the lowest latest latency to the MVE - this will be at the top of the list.

On the next screen:

Leave the name as proposed or change as needed.
Set Rate Limit to 20 Mbps (lowest possible, this is for SSH access only).
Leave A-vNIC set to vNIC-0.
Leave Preferred A-End VLAN at Untagged.

On the next screen verify the configuration and click Add VXC.

On the main Services page, the MVE and Internet VXC now show with the note "Order pending".

Click +Connection in the MVE box to connect a VXC to the ExpressRoute Circuit.

Under Choose Destination Type select Cloud.
Then select Microsoft Azure as the Provider.
Paste in the circuit's Service Key and select Port for the Primary path.
Click Next.

On the next screen:

Give the connection a name.
Leave the Rate Limit as proposed, this is set to the bandwidth of the circuit.
At A-end vNIC, select vNIC-1 (do not leave this at vNIC-0!).
At Preferred A-End VLAN, turn off Untag and enter a VLAN number. This will be used to set the sub-interface in the MVE configuration later.

Scroll down to Azure peering VLAN.

Leave Configure Azure Peering VLAN turned on.
Enter the same VLAN ID that will be used in the configuration of the Private Peering on the Azure end.
Click Next.

Verify the configuration summary and click Add VXC.

Repeat the process to add the Secondary path, terminating on vNIC-2. Enter a different VLAN ID for Preferred A-End VLAN. Enter the same VLAN ID that will be used in the Private Peering under Azure peering VLAN.

When the second ExpressRoute VXC is configured, click Review Order in the right hand bar of the Services screen.

When the validation completes, click Order Now.

This will provision the MVE and the VXC. It will take a few minutes for all services to come up.

In the Azure portal, the Provider Status of the ExpressRoute circuit will change to Provisioned.

Configure Private Peering

Go back to the ExpressRoute circuit in the Azure portal. The Provider Status will now be Provisioned, and the Private Peering can be enabled. Click on Peerings under Settings and then click Azure private.

Enter the Peer ASN and Primary and Secondary subnets. Under VLAN ID enter the same number as configured under Azure Peering VLAN in the Primary and Secondary VXC configurations in the Megaport portal.

Configure Cisco IOS

Establish an SSH session to the MVE. Use the public ip address from the internet VXC, and the private key that belongs with the public key used when deploying the MVE.

ssh -i <private-key-file> mveadmin@<public ip>

Configure interfaces:

interface GigabitEthernet2 no ip address no shutdown negotiation auto ! interface GigabitEthernet2.100 encapsulation dot1Q 100 ip address 192.168.0.1 255.255.255.252 ! interface GigabitEthernet3 no ip address no shutdown negotiation auto ! interface GigabitEthernet3.101 encapsulation dot1Q 101 ip address 192.168.0.5 255.255.255.252

Use the Preferred A-end VLAN values set in the primary and secondary VXCs to configure the encapsulation on the subinterfaces. Use the lower address of the /30 subnets configured on the Private Peering.

The higher IP addresses of the Private Peering should now respond to ping:

ping 192.168.0.2 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 192.168.0.2, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms

If ping does not work there likely is an ARP resolution issue. Run `show arp` and `debug arp` and check the ARP table of the Private Peering.

Configure BGP:

router bgp 64000 bgp log-neighbor-changes neighbor 192.168.0.2 remote-as 12076 neighbor 192.168.0.2 soft-reconfiguration inbound neighbor 192.168.0.6 remote-as 12076 neighbor 192.168.0.6 soft-reconfiguration inbound

Verify both neighbors show BGP state = Established:

sh ip bgp neighbor 192.168.0.2 BGP neighbor is 192.168.0.2, remote AS 12076, external link BGP version 4, remote router ID 192.168.0.2 BGP state = Established, up for 1d21h ...

This completes the basic configuration of ExpressRoute to MVE.

Announcing public preview: Cilium mTLS encryption for Azure Kubernetes Service

chandanAggarwal — Mon, 23 Mar 2026 01:50:18 GMT

We are thrilled to announce the public preview of Cilium mTLS encryption in Azure Kubernetes Service (AKS), delivered as part of Advanced Container Networking Services and powered by the Azure CNI dataplane built on Cilium.

This capability is the result of a close engineering collaboration between Microsoft and Isovalent (now part of Cisco). It brings transparent, workload‑level mutual TLS (mTLS) to AKS without sidecars, without application changes, and without introducing a separate service mesh stack.

This public preview represents a major step forward in delivering secure, high‑performance, and operationally simple networking for AKS customers. In this post, we’ll walk through how Cilium mTLS works, when to use it, and how to get started.

Why Cilium mTLS encryption matters

Traditionally, teams looking to in-transit traffic encryption in Kubernetes have had two primary options:

Node-level encryption (for example, WireGuard or virtual network encryption), which secures traffic in transit but lacks workload identity and authentication.

Service meshes, which provide strong identity and mTLS guarantees but introduce operational complexity.

This trade‑off has become increasingly problematic, as many teams want workload‑level encryption and authentication, but without the cost, overhead, and architectural impact of deploying and operating a full-service mesh.

Cilium mTLS closes this gap directly in the dataplane. It delivers transparent, inline mTLS encryption and authentication for pod‑to‑pod TCP traffic, enforced below the application layer. And implemented natively in the Azure CNI dataplane built on Cilium, so customers gain workload‑level security without introducing a separate service mesh, resulting in a simpler architecture with lower operational overhead.

To see how this works under the hood, the next section breaks down the Cilium mTLS architecture and follows a pod‑to‑pod TCP flow from interception to authentication and encryption.

Architecture and design: How Cilium mTLS works

Cilium mTLS achieves workload‑level authentication and encryption by combining three key components, each responsible for a specific part of the authentication and encryption lifecycle.

Cilium agent: Transparent traffic interception and wiring

Cilium agent which already exists on any cluster running with Azure CNI powered by cilium, is responsible for making mTLS invisible to applications. When a namespace is labelled with “io.cilium/mtls-enabled=true”, The Cilium agent enrolls all pods in that namespace. It enters each pod's network namespace and installs iptables rules that redirect outbound traffic to ztunnel on port 15001.

It is also responsible for passing workload metadata (such as pod IP and namespace context) to ztunnel.

Ztunnel: Node‑level mTLS enforcement

Ztunnel is an open source, lightweight, node‑level Layer 4 proxy that was originally created by Istio. Ztunnel runs as a DaemonSet, on the source node it looks up the destination workload via XDS (streamed from the Cilium agent) and establishes mutually authenticated TLS 1.3 sessions between source and destination nodes. Connections are held inline until authentication is complete, ensuring that traffic is never sent in plaintext.

The destination ztunnel decrypts the traffic and delivers it into the target pod, bypassing the interception rules via an in-pod mark. The application sees a normal plaintext connection — it is completely unaware encryption happened.

SPIRE: Workload identity and trust

SPIRE (SPIFFE Runtime Environment) provides the identity foundation for Cilium mTLS. SPIRE acts as the cluster Certificate Authority, issuing short‑lived X.509 certificates (SVIDs) that are automatically rotated and validated.

This is a key design principle of Cilium mTLS - trust is based on workload identity, not network topology.

Each workload receives a cryptographic identity derived from:

Kubernetes namespace

Kubernetes ServiceAccount

These identities are issued and rotated automatically by SPIRE and validated on both sides of every connection. As a result:

Identity remains stable across pod restarts and rescheduling

Authentication is decoupled from IP addresses

Trust decisions align naturally with Kubernetes RBAC and namespace boundaries

This enables a zero‑trust networking model that fits cleanly into existing AKS security practices.

End‑to‑End workflow example

To see how these components work together, consider a simple pod‑to‑pod connection:

A pod initiates a TCP connection to another pod.

Traffic intercepted inside the pod network namespace and redirected to the local ztunnel instance.

ztunnel retrieves the workload identity using certificates issued by SPIRE.

ztunnel establishes a mutually authenticated TLS session with the destination node’s ztunnel.

Traffic is encrypted and sent between pods.

The destination ztunnel decrypts the traffic and delivers it to the target pod.

Every packet from an enrolled pod is encrypted. There is no plaintext window, and no dropped first packets. The connection is held inline by ztunnel until the mTLS tunnel is established, then traffic flows bidirectionally through an HBONE (HTTP/2 CONNECT) tunnel.

Workload enrolment and scope

Cilium mTLS in AKS is opt‑in and scoped at the namespace level.

Platform teams enable mTLS by applying a single label to a namespace. From that point on:

All pods in that namespace participate in mTLS

Authentication and encryption are mandatory between enrolled workloads

Non-enrolled namespaces continue to operate unchanged

Encryption is applied only when both pods are enrolled. Traffic between enrolled and non‑enrolled workloads continues in plaintext without causing connectivity issues or hard failures. This model enables gradual rollout, staged migrations, and low-risk adoption across environments.

Getting started in AKS

Cilium mTLS encryption is available in public preview for AKS clusters that use:

Azure CNI powered by Cilium

Advanced Container Networking Services

You can enable mTLS:

When creating a new cluster, or

On an existing cluster by updating the Advanced Container Networking Services configuration

Once enabled, enrolling workloads is as simple as labelling a namespace.

👉 Learn more

Concepts: How Cilium mTLS works, architecture, and trust boundaries

How-to guide: Step-by-step instructions to enable and verify mTLS in AKS

Looking ahead

This public preview represents an important step forward in simplifying network security for AKS and reflects a deep collaboration between Microsoft and Isovalent to bring open, standards‑based innovation into production‑ready cloud platforms. We’re continuing to work closely with the community to improve the feature and move it toward general availability.

If you’re looking for workload‑level encryption without the overhead of a traditional service mesh, we invite you to try Cilium mTLS in AKS and share your experience.

Azure Front Door: Resiliency Series – Part 2: Faster recovery (RTO)

AbhishekTiwari — Thu, 19 Mar 2026 17:00:54 GMT

In Part 1 of this blog series, we outlined our four‑pillar strategy for resiliency in Azure Front Door: configuration resiliency, data plane resiliency, tenant isolation, and accelerated Recovery Time Objective (RTO). Together, these pillars help Azure Front Door remain continuously available and resilient at global scale.

Part 1 focused on the first two pillars: configuration and data plane resiliency. Our goal is to make configuration propagation safer, so incompatible changes never escape pre‑production environments. We discussed how incompatible configurations are blocked early, and how data plane resiliency ensures the system continues serving traffic from a last‑known‑good (LKG) configuration even if a bad change manages to propagate. We also introduced ‘Food Taster’, a dedicated sacrificial process running in each edge server’s data plane, that pretests every configuration change in isolation, before it ever reaches the live data plane.

In this post, we turn to the recovery pillar. We describe how we have made key enhancements to the Azure Front Door recovery path so the system can return to full operation in a predictable and bounded timeframe. For a global service like Azure Front Door, serving hundreds of thousands of tenants across 210+ edge sites worldwide, we set an explicit target: to be able to recover any edge site – or all edge sites – within approximately 10 minutes, even in worst‑case scenarios. In typical data plane crash scenarios, we expect recovery in under a second.

Repair status

The first blog post in this series mentioned the two Azure Front Door incidents from October 2025 – learn more by watching our Azure Incident Retrospective session recordings for the October 9^th incident and/or the October 29^th incident. Before diving into our platform investments for improving our Recovery Time Objectives (RTO), we wanted to provide a quick update on the overall repair items from these incidents. We are pleased to report that the work on configuration propagation and data plane resiliency is now complete and fully deployed across the platform (in the table below, “Completed” means broadly deployed in production). With this, we have reduced configuration propagation latency from ~45 minutes to ~20 minutes. We anticipate reducing this even further – to ~15 minutes by the end of April 2026, while ensuring that platform stability remains our top priority.

Learning category	Goal	Repairs	Status
Safe customer configuration deployment	Incompatible configuration never propagates beyond ‘EUAP or canary regions’	Control plane and data plane defect fixes Forced synchronous configuration processing Additional stages with extended bake time Early detection of crash state	Completed
Data plane resiliency	Configuration processing cannot impact data plane availability	Manage data-plane lifecycle to prevent outages caused by configuration-processing defects.	Completed
Data plane resiliency		Isolated work-process in every data plane server to process and load the configuration.	Completed
100% Azure Front Door resiliency posture for Microsoft internal services	Microsoft operates an isolated, independent Active/Active fleet with automatic failover for critical Azure services	Phase 1: Onboarded critical services batch impacted on Oct 29^th outage running on a day old configuration	Completed
		Phase 2: Automation & hardening of operations, auto-failover and self-management of Azure Front Door onboarding for additional services	March 2026
Recovery improvements	Data plane crash recovery in under 10 minutes	Data plane boot-up time optimized via local cache (~1 hour)	Completed
Recovery improvements	Data plane crash recovery in under 10 minutes	Accelerate recovery time < 10 minutes	April 2026
Tenant isolation	No configuration or traffic regression can impact other tenants	Micro cellular Azure Front Door with ingress layered shards	June 2026

Why recovery at edge scale is deceptively hard

To understand why recovery took as long as it did, it helps to first understand how the Azure Front Door data plane processes configuration.

Azure Front Door operates in 210+ edge sites with multiple servers per site. The data plane of each edge server hosts multiple processes. A master process orchestrates the lifecycle of multiple worker processes, that serve customer traffic. A separate configuration translator process runs alongside the data plane processes, and is responsible for converting customer configuration bundles from the control plane into optimized binary FlatBuffer files. This translation step, covering hundreds of thousands of tenants, represents hours of cumulative computation. A per edge server cache is kept locally at each server level – to enable a fast recovery of the data plane, if needed.

Once the configuration translator process produces these FlatBuffer files, each worker processes them independently and memory-maps them for zero-copy access. Configuration updates flow through a two-phase commit: new FlatBuffers are first loaded into a staging area and validated, then atomically swapped into production maps. In-flight requests continue using the old configuration, until the last request referencing them completes.

The data process recovery is designed to be resilient to different failure modes. A failure or crash at the worker process level has a typical recovery time of less than one second. Since each server has multiple such worker processes which serve customer traffic, this type of crash has no impact on the data plane. In the case of a master process crash, the system automatically tries to recover using the local cache. When the local cache is reused, the system is able to recover quickly – in approximately 60 minutes – since most of the configurations in the cache were already loaded into the data plane before the crash. However, in certain cases if the cache becomes unavailable or must be invalidated because of corruption, the recovery time increases significantly.

During the October 29^th incident, a data plane crash triggered a complete recovery sequence that took approximately 4.5 hours. This was not because restarting a process is slow, it is because a defect in the recovery process invalidated the local cache, which meant that “restart” meant rebuilding everything from scratch. The configuration translator process then had to re-fetch and re-translate every one of the hundreds of thousands of customer configurations, before workers could memory-map them and begin serving traffic.

This experience has crystallized three fundamental learnings related to our recovery path:

Expensive rework: A subset of crashes discarded all previously translated FlatBuffer artifacts, forcing the configuration translator process to repeat hours of conversion work that had already been validated and stored.
High restart costs: Every worker on every node had to wait for the configuration translator process to complete the full translation, before it could memory-map any configuration and begin serving requests.
Unbounded recovery time: Recovery time grew linearly with total tenant footprint rather than with active traffic, creating a ‘scale penalty’ as more tenants onboarded to the system.

Separately and together, the insight was clear: recovery must stop being proportional to the total configuration size.

Persisting ‘validated configurations’ across restarts

One of the key recovery improvements was strengthening how validated customer configurations are cached and reused across failures, rather than rebuilding configuration states from scratch during recovery. Azure Front Door already cached customer configurations on host‑mounted storage prior to the October incident. The platform enhancements post outage focused on making the local configuration cache resilient to crashes, partial failures, and bad tenant inputs.

Our goal was to ensure that recovery behavior is dominated by serving traffic safely, not by reconstructing configuration state. This led us to two explicit design goals…

Design goals

No category of crash should invalidate the configuration cache: Configuration cache invalidation must never be the default response to failures. Whether the failure is a worker crash, master crash, data plane restart, or coordinated recovery action, previously validated customer configurations should remain usable—unless there is a proven reason to discard it.
Bad tenant configuration must not poison the entire cache: A single faulty or incompatible tenant configuration should result in targeted eviction of that tenant’s configuration only—not wholesale cache invalidation across all tenants.

Platform enhancements

Previously, customer configurations persisted to host‑mounted storage, but certain failure paths treated the cache as unsafe and invalidated it entirely. In those cases, recovery implicitly meant reloading and reprocessing configuration for hundreds of thousands of tenants before traffic could resume, even though the vast majority of cached data was still valid.

We changed the recovery model to avoid invalidating customer configurations, with strict scoping around when and how cached entries are discarded:

Cached configurations are no longer invalidated based on crash type. Failures are assumed to be orthogonal to configuration correctness unless explicitly proven otherwise.
Cache eviction is granular and tenant‑scoped. If a cached configuration fails validation or load checks, only that tenant’s configuration is discarded and reloaded. All other tenant configurations remain available.

This ensures that recovery does not regress into a fleet‑wide rebuild due to localized or unrelated faults.

Safety and correctness

Durability is paired with strong correctness controls, to prevent unsafe configurations from being served:

Per‑tenant validation on load: Each cached tenant configuration is validated during the ‘load and verification’ phase, before being promoted for traffic serving. Therefore, failures are contained to that tenant.
Targeted re‑translation: When validation fails, only the affected tenant’s configuration is reloaded or reprocessed. Therefore, the cache for other tenants is left untouched.
Operational escape hatch: Operators retain the ability to explicitly instruct a clean rebuild of the configuration cache (with proper authorization), preserving control without compromising the default fast‑recovery path.

Resulting behavior

With these changes, recovery behavior now aligns with real‑world traffic patterns - configuration defects impact tenants locally and predictably, rather than globally. The system now prefers isolated tenant impact, and continued service using last-known-good over aggressive invalidation, both of which are critical for predictable recovery at the scale of Azure Front Door.

Making recovery scale with active traffic, not total tenants

Reusing configuration cache solves the problem of rebuilding configuration in its entirety, but even with a warm cache, the original startup path had a second bottleneck: eagerly loading a large volume of tenant configurations into memory before serving any traffic. At our scale, memory-mapping, parsing hundreds of thousands of FlatBuffers, constructing internal lookup maps, adding Transport Layer Security (TLS) certificates and configuration blocks for each tenant, collectively added almost an hour to startup time. This was the case even when a majority of those tenants had no active traffic at that moment.

We addressed this by fundamentally changing when configuration is loaded into workers. Rather than eagerly loading most of the tenants at startup across all edge locations, Azure Front Door now uses an Machine Learning (ML)-optimized lazy loading model.

In the new architecture, instead of loading a large number of tenant configurations, we only load a small subset of tenants that are known to be historically active in a given site, we call this the “warm tenants” list. The warm tenants list per edge site is created through a sophisticated traffic analysis pipeline that leverages ML. However, loading the warm tenants is not good enough, because when a request arrives and we don’t have the configuration in memory, we need to know two things. Firstly, is this a request from a real Azure Front Door tenant – and, if it is, where can I find the configuration?

To answer these questions, each worker maintains a hostmap that tracks the state of each tenant’s configuration. This hostmap is constructed during startup, as we process each tenant configuration – if the tenant is in the warm list, we will process and load their configuration fully; if not, then we will just add an entry into the hostmap where all their domain names are mapped to the configuration path location. When a request arrives for one of these tenants, the worker loads and validates that tenant’s configuration on demand, and immediately begins serving traffic. This allows a node to start serving its busiest tenants within a few minutes of startup, while additional tenants are loaded incrementally only when traffic actually arrives—allowing the system to progressively absorb cold tenants as demand increases.

The effect on recovery is transformative. Instead of recovery time scaling with the total number of tenants configured on a server, it scales with the number of tenants actively receiving traffic. In practice, even at our busiest edge sites, the active tenant set is a small fraction of the total.

Just as importantly, this modified form of lazy loading provides a natural failure isolation boundary. Most Edge sites won’t ever load a faulty configuration of an inactive tenant. When a request for an inactive tenant with an incompatible configuration arrives, impact is contained to a single worker.

The configuration load architecture now prefers serving as many customers as quickly as possible, rather than waiting until everything is ready before serving anyone. The above changes are slated to complete in April 2026 and will bring our RTO from the current ~1 hour to under 10 minutes – for complete recovery from a worst case scenario.

Continuous validation through Game Days

A critical element of our recovery confidence comes from GameDay fault-injection testing. We don’t simply design recovery mechanisms and assume they work—we break the system deliberately and observe how it responds. Since late 2025, we have conducted recurring GameDay drills that simulate the exact failure scenarios we are defending against:

Food Taster crash scenarios: Injecting deliberately faulty tenant configurations, to verify that they are caught and isolated with zero impact on live traffic. In our January 2026 GameDay, the Food Taster process crashed as expected, the system halted the update within approximately 5 seconds, and no customer traffic was affected.
Master process crash scenarios: Triggering master process crashes across test environments to verify that workers continue serving traffic, that the Local Config Shield engages within 10 seconds, and that the coordinated recovery tool restores full operation within the expected timeframe.
Multi-region failure drills: Simulating simultaneous failures across multiple regions to validate that global Config Shield mechanisms engage correctly, and that recovery procedures scale without requiring manual per-region intervention.
Fallback test drills for critical Azure services running behind Azure Front Door: In our February 2026 GameDay, we simulated the complete unavailability of Azure Front Door, and successfully validated failover for critical Azure services with no impact to traffic.

These drills have both surfaced corner cases and built operational confidence. They have transformed recovery from a theoretical plan into tested, repeatable muscle memory. As we noted in an internal communication to our team: “Game day testing is a deliberate shift from assuming resilience to actively proving it—turning reliability into an observed and repeatable outcome.”

Closing

Part 1 of this series emphasized preventing unsafe configurations from reaching the data plane, and data plane resiliency in case an incompatible configuration reaches production. This post has shown that prevention alone is not enough—when failures do occur, recovery must be fast, predictable, and bounded. By ensuring that the FlatBuffer cache is never invalidated, by loading only active tenants, and by building safe coordinated recovery tooling, we have transformed failure handling from a fleet-wide crisis into a controlled operation.

These recovery investments work in concert with the prevention mechanisms described in Part 1. Together, they ensure that the path from incident detection to full service restoration is measured in minutes, with customer traffic protected at every step.

In the next post of this series, we will cover the third pillar of our resiliency strategy: tenant isolation—how micro-cellular architecture and ingress-layered sharding can reduce the blast radius of any failure to a small subset, ensuring that one customer’s configuration or traffic anomaly never becomes everyone’s problem.

We deeply value our customers’ trust in Azure Front Door. We are committed to transparently sharing our progress on these resiliency investments, and to exceed expectations for safety, reliability, and operational readiness.

ExpressRoute Gateway Microsoft initiated migration

MekaylaMoore — Mon, 30 Mar 2026 18:20:25 GMT

Important:

Microsoft initiated Gateway migrations are temporarily paused. You will be notified when migrations resume.

Objective

The backend migration process is an automated upgrade performed by Microsoft to ensure your ExpressRoute gateways use the Standard IP SKU. This migration enhances gateway reliability and availability while maintaining service continuity. You receive notifications about scheduled maintenance windows and have options to control the migration timeline. For guidance on upgrading Basic SKU public IP addresses for other networking services, see Upgrading Basic to Standard SKU.

Important:

As of September 30, 2025, Basic SKU public IPs are retired. For more information, see the official announcement.
You can initiate the ExpressRoute gateway migration yourself at a time that best suits your business needs, before the Microsoft team performs the migration on your behalf. This gives you control over the migration timing. Please use the ExpressRoute Gateway Migration Tool to migrate your gateway Public IP to Standard SKU. This tool provides a guided workflow in the Azure portal and PowerShell, enabling a smooth migration with minimal service disruption.

Backend migration overview

The backend migration is scheduled during your preferred maintenance window. During this time, the Microsoft team performs the migration with minimal disruption. You don’t need to take any actions. The process includes the following steps:

Deploy new gateway: Azure provisions a second virtual network gateway in the same GatewaySubnet alongside your existing gateway. Microsoft automatically assigns a new Standard SKU public IP address to this gateway.
Transfer configuration: The process copies all existing configurations (connections, settings, routes) from the old gateway. Both gateways run in parallel during the transition to minimize downtime. You may experience brief connectivity interruptions may occur.
Clean up resources: After migration completes successfully and passes validation, Azure removes the old gateway and its associated connections. The new gateway includes a tag CreatedBy: GatewayMigrationByService to indicate it was created through the automated backend migration

Important:

To ensure a smooth backend migration, avoid making non-critical changes to your gateway resources or connected circuits during the migration process. If modifications are absolutely required, you can choose (after the Migrate stage complete) to either commit or abort the migration and make your changes.

Backend process details

This section provides an overview of the Azure portal experience during backend migration for an existing ExpressRoute gateway. It explains what to expect at each stage and what you see in the Azure portal as the migration progresses. To reduce risk and ensure service continuity, the process performs validation checks before and after every phase.

The backend migration follows four key stages:

Validate: Checks that your gateway and connected resources meet all migration requirements for the Basic to Standard public IP migration.
Prepare: Deploys the new gateway with Standard IP SKU alongside your existing gateway.
Migrate: Cuts over traffic from the old gateway to the new gateway with a Standard public IP.
Commit or abort: Finalizes the public IP SKU migration by removing the old gateway or reverts to the old gateway if needed.

These stages mirror the Gateway migration tool process, ensuring consistency across both migration approaches.

The Azure resource group RGA serves as a logical container that displays all associated resources as the process updates, creates, or removes them. Before the migration begins, RGA contains the following resources:

This image uses an example ExpressRoute gateway named ERGW-A with two connections (Conn-A and LAconn) in the resource group RGA.

Portal walkthrough

Before the backend migration starts, a banner appears in the Overview blade of the ExpressRoute gateway. It notifies you that the gateway uses the deprecated Basic IP SKU and will undergo backend migration between March 7, 2026, and April 30, 2026:

Validate stage

Once you start the migration, the banner in your gateway’s Overview page updates to indicate that migration is currently in progress.

In this initial stage, all resources are checked to ensure they are in a Passed state. If any prerequisites aren't met, validation fails and the Azure team doesn't proceed with the migration to avoid traffic disruptions. No resources are created or modified in this stage.

After the validation phase completes successfully, a notification appears indicating that validation passed and the migration can proceed to the Prepare stage.

Prepare stage

In this stage, the backend process provisions a new virtual network gateway in the same region and SKU type as the existing gateway. Azure automatically assigns a new public IP address and re-establishes all connections. This preparation step typically takes up to 45 minutes.

To indicate that the new gateway is created by migration, the backend mechanism appends _migrate to the original gateway name. During this phase, the existing gateway is locked to prevent configuration changes, but you retain the option to abort the migration, which deletes the newly created gateway and its connections.

After the Prepare stage starts, a notification appears showing that new resources are being deployed to the resource group:

Deployment status

In the resource group RGA, under Settings → Deployments, you can view the status of all newly deployed resources as part of the backend migration process.

In the resource group RGA under the Activity Log blade, you can see events related to the Prepare stage. These events are initiated by GatewayRP, which indicates they are part of the backend process:

Deployment verification

After the Prepare stage completes, you can verify the deployment details in the resource group RGA under Settings > Deployments. This section lists all components created as part of the backend migration workflow.

The new gateway ERGW-A_migrate is deployed successfully along with its corresponding connections: Conn-A_migrate and LAconn_migrate.

Gateway tag

The newly created gateway ERGW-A_migrate includes the tag CreatedBy: GatewayMigrationByService, which indicates it was provisioned by the backend migration process.

Migrate stage

After the Prepare stage finishes, the backend process starts the Migrate stage. During this stage, the process switches traffic from the existing gateway ERGW-A to the new gateway ERGW-A_migrate.

Gateway ERGW-A_migrate:

Old gateway (ERGW-A) handles traffic:

After the backend team initiates the traffic migration, the process switches traffic from the old gateway to the new gateway. This step can take up to 15 minutes and might cause brief connectivity interruptions.

New gateway (ERGW-A_migrate) handles traffic:

Commit stage

After migration, the Azure team monitors connectivity for 15 days to ensure everything is functioning as expected. The banner automatically updates to indicate completion of migration:

During this validation period, you can’t modify resources associated with both the old and new gateways. To resume normal CRUD operations without waiting 15 days, you have two options:

Commit: Finalize the migration and unlock resources.
Abort: Revert to the old gateway, which deletes the new gateway and its connections.

To initiate Commit before the 15-day window ends, type yes and select Commit in the portal.

When the commit is initiated from the backend, you will see “Committing migration. The operation may take some time to complete.”

The old gateway and its connections are deleted. The event shows as initiated by GatewayRP in the activity logs.

After old connections are deleted, the old gateway gets deleted.

Finally, the resource group RGA contains only resources only related to the migrated gateway
ERGW-A_migrate:

The ExpressRoute Gateway migration from Basic to Standard Public IP SKU is now complete.

Frequently asked questions

How long will Microsoft team wait before committing to the new gateway?

The Microsoft team waits around 15 days after migration to allow you time to validate connectivity and ensure all requirements are met. You can commit at any time during this 15-day period.

What is the traffic impact during migration? Is there packet loss or routing disruption?

Traffic is rerouted seamlessly during migration. Under normal conditions, no packet loss or routing disruption is expected. Brief connectivity interruptions (typically less than 1 minute) might occur during the traffic cutover phase.

Can we make any changes to ExpressRoute Gateway deployment during the migration?

Avoid making non-critical changes to the deployment (gateway resources, connected circuits, etc.). If modifications are absolutely required, you have the option (after the Migrate stage) to either commit or abort the migration.

Unlock outbound traffic insights with Azure StandardV2 NAT Gateway flow logs

cozhang — Fri, 06 Feb 2026 16:07:33 GMT

Recommended Outbound Connectivity

StandardV2 NAT Gateway is the next evolution of outbound connectivity in Azure. As the recommended solution for providing secure, reliable outbound Internet access, NAT Gateway continues to be the default choice for modern Azure deployments. With the highly anticipated general availability of the new StandardV2 SKU, customers gain access to the following highly requested upgrades:

Zone-redundancy: Automatically maintains outbound connectivity during single‑zone failures in AZ-enabled regions.
Enhanced performance: Up to 100 Gbps of throughput and 10 million packets per second - double the Standard SKU capacity.
Dual-stack support: Attach up to 16 IPv6 and 16 IPv4 public IP addresses for future ready connectivity.
Flow logs: Access historical logs of connections being established through your NAT gateway.

This blog will focus on how enabling StandardV2 NAT Gateway flow logs can be beneficial for your team along with some tips to get the most out of the data.

What are flow logs?

StandardV2 NAT Gateway flow logs are enabled through Diagnostic settings on your NAT gateway resource where the log data can be sent to Log Analytics, a storage account, or Event hub destination. “NatGatewayFlowlogV1” is the released log category, and it provides IP level information on traffic flowing through your StandardV2 NAT gateway.

Enable NATGateway Flow Logs through Diagnostics setting on your StandardV2 NAT gateway resource.

Schema output as seen on Log Analytics for a NAT gateway traffic flow.

Why should I use flow logs?

Security and compliance visibility

Prior to NAT gateway flow logs, customers could not see NAT gateway information when their virtual machines connect outbound. This made it difficult to:

Validate that only approved destinations were being accessed
Audit suspicious or unexpected outbound patterns
Satisfy compliance requirements that mandate traffic recording

Flow logs now provide visibility to the source IP -> NAT gateway outbound IP -> destination IP, along with details on sent/dropped packets and bytes.

Usage analytics

Flow logs allow you to answer usage questions such as:

Which VMs are generating the most outbound requests?
Which destinations receive the most traffic?
Is throughput growth caused by a specific workload pattern?

This level of insight is especially useful when debugging unexpected throughput increases, billing spikes, and connection bottlenecks.

To note: Flow logs only capture established connections. This means the TCP 3‑way handshake (SYN → SYN/ACK → ACK) or the UDP ephemeral session setup must complete. If a connection never establishes, for example due to NSG denial, routing mismatch, or SNAT exhaustion, it will not appear in flow logs.

Workflow of troubleshooting with flow logs

Let's walk through how you can leverage flow logs to troubleshoot a scenario where you are seeing intermittent connection drops.

Scenario: You have VMs that use a StandardV2 NAT gateway to reach the Internet. However, your VMs intermittently fail to reach github.com.

Step 1: Check NAT gateway health

Start with the datapath availability metric, which reflects the NAT gateway's overall health.

If metric > 90%, this confirms NAT gateway is healthy and is working as expected to send outbound traffic to the internet. Continue to Step 2.
If metric is lower, visit Troubleshoot Azure NAT Gateway connectivity - Azure NAT Gateway | Microsoft Learn for troubleshooting tips.

Step 2: Enable StandardV2 NAT Gateway Flow Logs

To further investigate the root cause, Enable StandardV2 NAT Gateway Flow Logs (NatGatewayFlowLogsV1 log category in Diagnostics Setting) for the NAT gateway resource providing outbound connectivity for the impacted VMs. It is recommended to enable Log Analytics as a destination as it allows you to easily query the data. For the detailed steps, visit Monitor with StandardV2 NAT Gateway Flow Logs - Azure NAT Gateway | Microsoft Learn.

Tip: You may enable flow logs even when not troubleshooting to ensure you’ll have historical data to reference when issues occur.

Step 3: Confirm whether the connection was established

Use Log Analytics to query for flows with source IP == VM private IP and destination IP == IP address(es) of github.com. The following query will generate a table and chart of the total packets sent per minute from your source IP to the destination IP through your NAT gateway in the last 24 hours.NatGatewayFlowlogsV1 | where TimeGenerated > ago(1d) | where SourceIP == '10.0.0.4' //and DestinationIP == <"github.com IP"> | summarize TotalPacketsSent = sum(PacketsSent) by TimeGenerated = bin(TimeGenerated, 1m), SourceIP, DestinationIP | order by TimeGenerated asc
If there are no records of this connection, it is likely an issue with establishing the connection because flow logs will only capture records of established connections. Take a look at SNAT connection metrics to determine whether it may be a SNAT port exhaustion issue or NSGs/UDRs that may be blocking the traffic.
If there are records of the connection, proceed with the next step.

Step 4: Check if there are any packets dropped

In Log Analytics, query for the total "PacketsSentDropped" and "PacketsReceivedDropped" per source/outbound/destination IP connection.

If "PacketsSentDropped" > 0 - NAT gateway dropped traffic sent from your VM.
If "PacketsReceivedDropped" > 0, NAT gateway dropped traffic received from destination IP, github.com in this case.
In both instances, it typically means the either the client or server is pushing more traffic through a single connection than is optimal, causing connection-level rate limiting.
To mitigate:

- Avoid relying on one connection and instead use multiple connections.
- Distribute traffic across multiple outbound IP addresses by assigning more public IP addresses to the NAT gateway resource.

Conclusion

StandardV2 NAT Gateway Flow Logs unlock a powerful new dimension of outbound visibility and they can help you:

Validate cybersecurity readiness
Audit outbound flows
Diagnose intermittent connectivity issues
Understand traffic patterns and optimize architecture

We are excited to see how you leverage this new capability with your StandardV2 NAT gateways!

Have more questions?

As always, for any feedback, please feel free to reach us by submitting your feedback. We look forward to hearing your thoughts and hope this announcement helps you build more resilient applications in Azure.

For more information on StandardV2 NAT Gateway Flow Logs and how to enable it, visit:

Manage StandardV2 NAT Gateway Flow Logs - Azure NAT Gateway | Microsoft Learn

Monitor with StandardV2 NAT Gateway Flow Logs - Azure NAT Gateway | Microsoft Learn

To see the most up-to-date pricing for flow logs, visit Azure NAT Gateway - Pricing | Microsoft Azure.

To learn more about StandardV2 NAT Gateway, visit What is Azure NAT Gateway? | Microsoft Learn.

Data Center Quantized Congestion Notification: Scaling congestion control for RoCE RDMA in Azure

VamsiVadlamuri — Tue, 13 Jan 2026 22:35:21 GMT

As cloud storage demands continue to grow, the need for ultra-fast, reliable networking becomes ever more critical. Microsoft Azure’s journey to empower its storage infrastructure with RDMA (Remote Direct Memory Access) has been transformative, but it’s not without challenges—especially when it comes to congestion control at scale. Azure’s deployment of RDMA at regional scale relies on DCQCN (Data Center Quantized Congestion Notification), a protocol that’s become central to Azure’s ability to deliver high-throughput, low-latency storage services across vast, heterogeneous data center regions.

Why congestion control matters in RDMA networks

RDMA offloads the network stack to NIC hardware, reducing CPU overhead and enabling near line-rate performance. However, as Azure scaled RDMA across clusters and regions, it faced new challenges:

Heterogeneous hardware: Different generations of RDMA NICs (Network Interface Cards) and switches, each with their own quirks.
Variable latency: Long-haul links between datacenters introduce large round-trip time (RTT) variations.
Congestion risks: High-speed, incast-like traffic patterns can easily overwhelm buffers, leading to packet loss and degraded performance.

To address these, Azure needed a congestion control protocol that could operate reliably across diverse hardware and network conditions. Traditional TCP congestion control mechanisms don’t apply here, so Azure leverages DCQCN combined with Priority Flow Control (PFC) to maintain high throughput, low latency, and near-zero packet loss.

How DCQCN works

DCQCN coordinates congestion control using three main entities:

Reaction point (RP): The sender adjusts its rate based on feedback.
Congestion point (CP): Switches mark packets using ECN when queues exceed thresholds.
Notification point (NP): The receiver sends Congestion Notification Packets (CNPs) upon receiving ECN-marked packets.

This feedback loop allows RDMA flows to dynamically adapt their sending rates, preventing congestion collapse while maintaining fairness.

When the switch detects congestion, it marks packets with ECN.
The receiver NIC (NP) observes ECN marks and sends CNPs to the sender.
The sender NIC (RP) reduces its sending rate upon receiving CNPs; otherwise, it increases the rate gradually.

Interoperability challenges across different hardware generations

Cloud infrastructure evolves incrementally, typically at the level of individual clusters or racks, as newer server hardware generations are introduced. Within a single region, clusters often differ in their NIC configurations. Our deployment includes three generations of commodity RDMA NICs—Gen1, Gen2, and Gen3—each implementing DCQCN with distinct design variations. These discrepancies create complex and often problematic interactions when NICs from different generations interoperate.

Gen1 NICs: Firmware-based DCQCN, NP-side CNP coalescing, burst-based rate limiting.
Gen2/Gen3 NICs: Hardware-based DCQCN, RP-side CNP coalescing, per-packet rate limiting.

Problem:

Gen2/Gen3 NICs sending to Gen1 can trigger excessive cache misses, slowing down Gen1’s receiver pipeline.
Gen1 sending to Gen2/Gen3 can cause excessive rate reductions due to frequent CNPs.

Azure’s solution:

Move CNP coalescing to NP side for Gen2/Gen3.
Implement per-QP CNP rate limiting, matching Gen1’s timer.
Enable per-burst rate limiting on Gen2/Gen3 to reduce cache pressure.

DCQCN tuning: Achieving fairness and performance

DCQCN is inherently RTT-fair—its rate adjustment is independent of round-trip time, making it suitable for Azure’s regional networks with RTTs ranging from microseconds to milliseconds.

Key Tuning Strategies:

Sparse ECN marking: Use large ECN marking thresholds (K_max - K_min) and low marking probabilities (P_max) for flows with large RTTs.
Joint buffer and DCQCN tuning: Tune switch buffer thresholds and DCQCN parameters together to avoid premature congestion signals and optimize throughput.
Global parameter settings: Azure’s NICs support only global DCQCN settings, so parameters must work well across all traffic types and RTTs.

Real-world results

High throughput & low latency:
- RDMA traffic runs at line rate with near-zero packet loss.
CPU savings:
- Freed CPU cores can be repurposed for customer VMs or application logic.
Performance metrics:
- RDMA reduces CPU utilization by up to 34.5% compared to TCP for storage frontend traffic.
- Large I/O requests (1 MB) see up to 23.8% latency reduction for reads and 15.6% for writes.
Scalability:
- As of November 2025, ~85% of Azure’s traffic is RDMA, supported in all public regions.

Conclusion

DCQCN is a cornerstone of Azure’s RDMA-enabled storage infrastructure, enabling reliable, high-performance cloud storage at scale. By combining ECN-based signaling with dynamic rate adjustments, DCQCN ensures high throughput, low latency, and near-zero packet loss—even across heterogeneous hardware and long-haul links. Its interoperability fixes and careful tuning make it a critical enabler for RDMA adoption in modern data centers, paving the way for efficient, scalable, and resilient cloud storage.

Azure Front Door: Implementing lessons learned following October outages

AbhishekTiwari — Fri, 19 Dec 2025 16:43:31 GMT

Abhishek Tiwari, Vice President of Engineering, Azure Networking
Amit Srivastava, Principal PM Manager, Azure Networking
Varun Chawla, Partner Director of Engineering

Introduction

Azure Front Door is Microsoft's advanced edge delivery platform encompassing Content Delivery Network (CDN), global security and traffic distribution into a single unified offering. By using Microsoft's extensive global edge network, Azure Front Door ensures efficient content delivery and advanced security through 210+ global and local points of presence (PoPs) strategically positioned closely to both end users and applications.

As the central global entry point from the internet onto customer applications, we power mission critical customer applications as well as many of Microsoft’s internal services. We have a highly distributed resilient architecture, which protects against failures at the server, rack, site and even at the regional level. This resiliency is achieved by the use of our intelligent traffic management layer which monitors failures and load balances traffic at server, rack or edge sites level within the primary ring, supplemented by a secondary-fallback ring which accepts traffic in case of primary traffic overflow or broad regional failures. We also deploy a traffic shield as a terminal safety net to ensure that in the event of a managed or unmanaged edge site going offline, end user traffic continues to flow to the next available edge site.

Like any large-scale CDN, we deploy each customer configuration across a globally distributed edge fleet, densely shared with thousands of other tenants. While this architecture enables global scale, it carries the risk that certain incompatible configurations, if not contained, can propagate broadly and quickly which can result in a large blast radius of impact. Here we describe how the two recent service incidents impacting Azure Front Door have reinforced the need to accelerate ongoing investments in hardening our resiliency, and tenant isolation strategy to mitigate likelihood and the scale of impact from this class of risk.

October incidents: recap and key learnings

Azure Front Door experienced two service incidents; on October 9^th and October 29^th, both with customer-impacting service degradation.

On October 9^th: A manual cleanup of stuck tenant metadata bypassed our configuration protection layer, allowing incompatible metadata to propagate beyond our canary edge sites. This metadata was created on October 7^th, from a control-plane defect triggered by a customer configuration change. While the protection system initially blocked the propagation, the manual override operation bypassed our safeguards. This incompatible configuration reached the next stage and activated a latent data-plane defect in a subset of edge sites, causing availability impact primarily across Europe (~6%) and Africa (~16%). You can learn more about this issue in detail at https://aka.ms/AIR/QNBQ-5W8

On October 29^th: A different sequence of configuration changes across two control-plane versions produced incompatible metadata. Because the failure mode in the data-plane was asynchronous, the health checks validations embedded in our protection systems were all passed during the rollout. The incompatible customer configuration metadata successfully propagated globally through a staged rollout and also updated the “last known good” (LKG) snapshot. Following this global rollout, the asynchronous process in data-plane exposed another defect which caused crashes. This impacted connectivity and DNS resolutions for all applications onboarded to our platform. Extended recovery time amplified impact on customer applications and Microsoft services. You can learn more about this issue in detail at https://aka.ms/AIR/YKYN-BWZ

We took away a number of clear and actionable lessons from these incidents, which are applicable not just to our service, but to any multi-tenant, high-density, globally distributed system.

Configuration resiliency – Valid configuration updates should propagate safely, consistently, and predictably across our global edge, while ensuring that incompatible or erroneous configuration never propagate beyond canary environments.
Data plane resiliency - Additionally, configuration processing in the data plane must not cause availability impact to any customer.
Tenant isolation – Traditional isolation techniques such as hardware partitioning and virtualization are impractical at edge sites. This requires innovative sharding techniques to ensure single tenant-level isolation – a must-have to reduce potential blast radius.
Accelerated and automated recovery time objective (RTO) – System should be able to automatically revert to last known good configuration in an acceptable RTO. In case of a service like Azure Front Door, we deem ~10 mins to be a practical RTO for our hundreds of thousands of customers at every edge site.

Post outage, given the severity of impact which allowed an incompatible configuration to propagate globally, we made the difficult decision to temporarily block configuration changes in order to expedite rollout of additional safeguards. Between October 29^th to November 5^th, we prioritized and deployed immediate hardening steps before opening up the configuration change. We are confident that the system is stable, and we are continuing to invest in additional safeguards to further strengthen the platform's resiliency.

Learning category	Goal	Repairs	Status
Safe customer configuration deployment	Incompatible configuration never propagates beyond Canary	· Control plane and data plane defect fixes · Forced synchronous configuration processing · Additional stages with extended bake time · Early detection of crash state	Completed
Data plane resiliency	Configuration processing cannot impact data plane availability	Manage data-plane lifecycle to prevent outages caused by configuration-processing defects.	Completed
Data plane resiliency		Isolated work-process in every data plane server to process and load the configuration.	January 2026
100% Azure Front Door resiliency posture for Microsoft internal services	Microsoft operates an isolated, independent Active/Active fleet with automatic failover for critical Azure services	Phase 1: Onboarded critical services batch impacted on Oct 29^th outage running on a day old configuration	Completed
		Phase 2: Automation & hardening of operations, auto-failover and self-management of Azure Front Door onboarding for additional services	March 2026
Recovery improvements	Data plane crash recovery in under 10 minutes	Data plane boot-up time optimized via local cache (~1 hour)	Completed
Recovery improvements	Data plane crash recovery in under 10 minutes	Accelerate recovery time < 10 minutes	March 2026
Tenant isolation	No configuration or traffic regression can impact other tenants	Micro cellular Azure Front Door with ingress layered shards	June 2026

This blog is the first in a multi-part series on Azure Front Door resiliency. In this blog, we will focus on configuration resiliency—how we are making the configuration pipeline safer and more robust. Subsequent blogs will cover tenant isolation and recovery improvements.

How our configuration propagation works

Azure Front Door configuration changes can be broadly classified into three distinct categories.

Service code & data – these include all aspects of Azure Front Door service like management plane, control plane, data plane, configuration propagation system. Azure Front Door follows a safe deployment practice (SDP) process to roll out newer versions of management, control or data plane over a period of approximately 2-3 weeks. This ensures that any regression in software does not have a global impact. However, latent bugs that escape pre-validation and SDP rollout can remain undetected until a specific combination of customer traffic patterns or configuration changes trigger the issue.
Web Application Firewall (WAF) & L7 DDoS platform data – These datasets are used by Azure Front Door to deliver security and load-balancing capabilities. Examples include GeoIP data, malicious attack signatures, and IP reputation signatures. Updates to these datasets occur daily through multiple SDP stages with an extended bake time of over 12 hours to minimize the risk of global impact during rollout. This dataset is shared across all customers and the platform, and it is validated immediately since it does not depend on variations in customer traffic or configuration steps.
Customer configuration data – Examples of these are any customer configuration change—whether a routing rule update, backend pool modification, WAF rule change, or security policy change. Due to the nature of these changes, it is expected across the edge delivery / CDN industry to propagate these changes globally in 5-10 mins. Both outages stemmed from issues within this category.

All configuration changes, including customer configuration data, are processed through a multi-stage pipeline designed to ensure correctness before global rollout across Azure Front Door’s 200+ edge locations. At a high level, Azure Front Door’s configuration propagation system has two distinct components -

Control plane – Accepts customer API/portal changes (create/update/delete for profiles, routes, WAF policies, origins, etc.) and translates them into internal configuration metadata which the data plane can understand.

Data plane – Globally distributed edge servers that terminate client traffic, apply routing/WAF logic, and proxy to origins using the configuration produced by the control plane.

Between these two halves sits a multi-stage configuration rollout pipeline with a dedicated protection system (known as ConfigShield):

Changes flow through multiple stages (pre-canary, canary, expanding waves to production) rather than going global at once.
Each stage is health-gated: the data plane must remain within strict error and latency thresholds before proceeding. Each stage’s health check also rechecks previous stage’s health for any regressions.
A successfully completed rollout updates a last known good (LKG) snapshot used for automated rollback.
Historically, rollout targeted global completion in roughly 5–10 minutes, in line with industry standards.

Customer configuration processing in Azure Front Door data plane stack

Customer configuration changes in Azure Front Door traverse multiple layers—from the control plane through the deployment system—before being converted into FlatBuffers at each Azure Front Door node. These FlatBuffers are then loaded by the Azure Front Door data plane stack, which runs as Kubernetes pods on every node.

FlatBuffer Composition: Each FlatBuffer references several sub-resources such as WAF and Rules Engine schematic files, SSL certificate objects, and URL signing secrets.
Data plane architecture:
o Master process: Accepts configuration changes (memory-mapped files with references) and manages the lifecycle of worker processes. o Workers: L7 proxy processes that serve customer traffic using the applied configuration.

Processing flow for each configuration update:

Load and apply in master: The transformed configuration is loaded and applied in the master process. Cleanup of unused references occurs synchronously except for certain categories à October 9 outage occurred during this step due to a crash triggered by incompatible metadata.
Apply to workers: Configuration is applied to all worker processes without memory overhead (FlatBuffers are memory-mapped).
Serve traffic: Workers start consuming new FlatBuffers for new requests; in-flight requests continue using old buffers. Old buffers are queued for cleanup post-completion.
Feedback to deployment service: Positive feedback signals readiness for rollout.Cleanup: FlatBuffers are freed asynchronously by the master process after all workers load updates à October 29 outage occurred during this step due to a latent bug in reference counting logic.

The October incidents showed we needed to strengthen key aspects of configuration validation, propagation safeguards, and runtime behavior. During the Azure Front Door incident on October 9^th, that protection system worked as intended but was later bypassed by our engineering team during a manual cleanup operation. During this Azure Front Door incident on October 29^th, the incompatible customer configuration metadata progressed through the protection system, before the delayed asynchronous processing task resulted in the crash.

Configuration propagation safeguards

Based on learnings from the incidents, we are implementing a comprehensive set of configuration resiliency improvements. These changes aim to guarantee that any sequence of configuration changes cannot trigger instability in the data plane, and to ensure quicker recovery in the event of anomalies.

Strengthening configuration generation safety

This improvement pivots on a ‘shift-left’ strategy where we want to ensure that we catch regression early before they propagate to production. It also includes fixing the latent defects which were the proximate cause of the outage.

Fixing outage specific defects - We have fixed the control-plane defects that could generate incompatible tenant metadata under specific operation sequences. We have also remediated the associated data-plane defects.
Stronger cross-version validation - We are expanding our test and validation suite to account for changes across multiple control plane build versions. This is expected to be fully completed by February 2026.
Fuzz testing - Automated fuzzing and testing of metadata generation contract between the control plane and the data plane. This allows us to generate an expanded set of invalid/unexpected configuration combinations which might not be achievable by traditional test cases alone. This is expected to be fully completed by February 2026.

Preventing incompatible configurations from being propagated

This segment of the resiliency strategy strives to ensure that a potentially dangerous configuration change never propagates beyond canary stage.

Protection system is “always-on” - Enhancements to operational procedures and tooling prevent bypass in all scenarios (including internal cleanup/maintenance), and any cleanup must flow through the same guarded stages and health checks as standard configuration changes. This is completed.
Making rollout behavior more predictable and conservative - Configuration processing in the data plane is now fully synchronous. Every data plane issue due to incompatible meta data can be detected withing 10 seconds at every stage. This is completed.
Enhancement to deployment pipeline - Additional stages during roll-out and extended bake time between stages serve as an additional safeguard during configuration propagation. This is completed.
Recovery tool improvements now make it easier to revert to any previous version of LKG with a single click. This is completed.

These changes significantly improve system safety. Post-outage we have increased the configuration propagation time to approximately 45 minutes. We are working towards reducing configuration propagation time closer to pre-incident levels once additional safeguards covered in the Data plane resiliency section below are completed by mid-January, 2026.

Data plane resiliency

The data plane recovery was the toughest part of recovery efforts during the October incidents. We must ensure fast recovery as well as resilience to configuration processing related issues for the data plane. To address this, we implemented changes that decouple the data plane from incompatible configuration changes. With these enhancements, the data plane continues operating on the last known good configuration—even if the configuration pipeline safeguards fail to protect as intended.

Decoupling data plane from configuration changes

Each server’s data plane consists of a master process which accepts configuration changes and manages lifecycle of multiple worker processes which serve customer traffic. One of the critical reasons for the prolonged outage in October was that due to latent defects in the data plane, when presented with a bad configuration the master process crashed. The master is a critical command-and-control process and when it crashes it takes down the entire data plane, in that node. Recovery of the master process involves reloading hundreds of thousands of configurations from scratch and took approximately 4.5 hours.

We have since made changes to the system to ensure that even in the event of the master process crash due to any reason - including incompatible configuration data being presented - the workers remain healthy and able to serve traffic. During such an event, the workers would not be able to accept new configuration changes but will continue to serve customer traffic using the last known good configuration. This work is completed.

Introducing Food Taster: strengthening config propagation resiliency

In our efforts to further strengthen Azure Front Door’s configuration propagation system, we are introducing an additional configuration safeguard known internally as Food Taster which protects the master and worker processes from any configuration change related incidents, thereby ensuring data plane resiliency.

The principle is simple: every data-plane server will have a redundant and isolated process – the Food Taster – whose only job is to ingest and process new configuration metadata first and then pass validated configuration changes to active data plane. This redundant worker does not accept any customer traffic.

All configuration processing in this Food Taster is fully synchronous. That means we do all parsing, validation, and any expensive or risky work up front, and we do not move on until the Food Taster has either proven the configuration is safe or rejected it. Only when the Food Taster successfully loads the configuration and returns “Config OK” does the master process proceed to load the same config and then instruct the worker processes to do the same. If anything goes wrong in the Food Taster, the failure is contained to that isolated worker; the master and traffic-serving workers never see that invalid configuration.

We expect this safeguard to reach production globally in January 2026 timeframe. Introduction of this component will also allow us to return closer to pre-incident level of configuration propagation while ensuring data plane safety.

Closing

This is the first in a series of planned blogs on Azure Front Door resiliency enhancements. We are continuously improving platform safety and reliability and will transparently share updates through this series. Upcoming posts will cover advancements in tenant isolation and improvements to recovery time objectives (RTO).

We deeply value our customers’ trust in Azure Front Door. The October incidents reinforced how critical configuration resiliency is, and we are committed to exceeding industry expectations for safety, reliability, and transparency. By hardening our configuration pipeline, strengthening safety gates, and reinforcing isolation boundaries, we’re making Azure Front Door even more resilient so your applications can be too.

Azure Networking 2025: Powering cloud innovation and AI at global scale

Sudha_Mahajan — Thu, 18 Dec 2025 23:20:29 GMT

In 2025, Azure’s networking platform proved itself as the invisible engine driving the cloud’s most transformative innovations. Consider the construction of Microsoft’s new Fairwater AI datacenter in Wisconsin – a 315-acre campus housing hundreds of thousands of GPUs. To operate as one giant AI supercomputer, Fairwater required a single flat, ultra-fast network interconnecting every GPU. Azure’s networking team delivered: the facility’s network fabric links GPUs at 800 Gbps speeds in a non-blocking architecture, enabling 10× the performance of the world’s fastest supercomputer. This feat showcases how fundamental networking is to cloud innovation. Whether it’s uniting massive AI clusters or connecting millions of everyday users, Azure’s globally distributed network is the foundation upon which new breakthroughs are built.

In 2025, the surge of AI workloads, data-driven applications, and hybrid cloud adoption put unprecedented demands on this foundation. We responded with bold network investments and innovations. Each new networking feature delivered in 2025, from smarter routing to faster gateways, was not just a technical upgrade but an innovation enabling customers to achieve more. Recapping the year’s major releases across Azure Networking services and key highlights how AI both drive and benefit from these advancements.

Unprecedented connectivity for a hybrid and AI era

Hybrid connectivity at scale: Azure’s network enhancements in 2025 focused on making global and hybrid connectivity faster, simpler, and ready for the next wave of AI-driven traffic. For enterprises extending on-premises infrastructure to Azure, Azure ExpressRoute private connectivity saw a major leap in capacity: Microsoft announced support for 400 Gbps ExpressRoute Direct ports (available in 2026) to meet the needs of AI supercomputing and massive data volumes. These high-speed ports – which can be aggregated into multi-terabit links – ensure that even the largest enterprises or HPC clusters can transfer data to Azure with dedicated, low-latency links. In parallel, Azure VPN Gateway performance reached new highs, with a generally available upgrade that delivers up to 20 Gbps aggregate throughput per gateway and 5 Gbps per individual tunnel. This is a 3× increase over previous limits, enabling branch offices and remote sites to connect to Azure even more seamlessly without bandwidth bottlenecks. Together, the ExpressRoute and VPN improvements give customers a spectrum of high-performance options for hybrid networking – from offices and datacenters to the cloud – supporting scenarios like large-scale data migrations, resilient multi-site architectures, and hybrid AI processing.

Simplified global networking: Azure Virtual WAN (vWAN) continued to mature as the one-stop solution for managing global connectivity. Virtual WAN introduced forced tunneling for Secure Virtual Hubs (now in preview), which allows organizations to route all Internet-bound traffic from branch offices or virtual networks back to a central hub for inspection. This capability simplifies the implementation of a “backhaul to hub” security model – for example, forcing branches to use a central firewall or security appliance – without complex user-defined routing.

Empowering multicloud and NVA integration: Azure recognizes that enterprise networks are diverse. Azure Route Server improvements enhanced interoperability with customer equipment and third-party network virtual appliances (NVAs). Notably, Azure Route Server now supports up to 500 virtual network connections (spokes) per route server, a significant scale boost that enables larger hub-and-spoke topologies and simplified Border Gateway Protocol (BGP) route exchange even in very large environments. This helps customers using SD-WAN appliances or custom firewalls in Azure to seamlessly learn routes from hundreds of VNet spokes – maintaining central routing control without manual configuration. Additionally, Azure Route Server introduced a preview of hub routing preference, giving admins the ability to influence BGP route selection (for example, preferring ExpressRoute over a VPN path, or vice versa). This fine-grained control means hybrid networks can be tuned for optimal performance and cost.

Resilience and reliability by design

Azure’s growth has been underpinned by making the network “resilient by default.”

We shipped tools to help validate and improve network resiliency. ExpressRoute Resiliency Insights was released for general availability – delivering an intelligent assessment of an enterprise’s ExpressRoute setup. This feature evaluates how well your ExpressRoute circuits and gateways are architected for high availability (for example, using dual circuits in diverse locations, zone-redundant gateways, etc.) and assigns a resiliency index score as a percentage. It will highlight suboptimal configurations – such as routes advertised on only one circuit, or a gateway that isn’t zone-redundant – and provide recommendations for improvement. Moreover, Resiliency Insights includes a failover simulation tool that can test circuit redundancy by mimicking failures, so you can verify that your connections will survive real-world incidents. By proactively monitoring and testing resilience, Azure is helping customers achieve “always-on” connectivity even in the face of fiber cuts, hardware faults, or other disruptions.

Security, governance, and trust in the network

As enterprises entrust more core business to Azure, the platform’s networking services advanced on security and governance – helping customers achieve Zero Trust networks and high compliance with minimal complexity. Azure DNS now offers DNS Security Policies with Threat Intelligence feeds (GA). This capability allows organizations to protect their DNS queries from known malicious domains by leveraging continuously updated threat intel. For example, if a known phishing domain or C2 (command-and-control) hostname appears in DNS queries from your environment, Azure DNS can automatically block or redirect those requests. Because DNS is often the first line of detection for malware and phishing activities, this built-in filtering provides a powerful layer of defense that’s fully managed by Azure. It’s essentially a cloud-delivered DNS firewall using Microsoft’s vast threat intelligence – enabling all Azure customers to benefit from enterprise-grade security without deploying additional appliances.

Network traffic governance was another focus. The introduction of forced tunneling in Azure Virtual WAN hubs (preview) shared above is a prime example where networking meets security compliance.

Optimizing cloud-native and edge networks

We previewed DNS intelligent traffic control features – such as filtering DNS queries to prevent data exfiltration and applying flexible recursion policies – which complement the DNS Security offering in safeguarding name resolution. Meanwhile, for load balancing across regions, Azure Traffic Manager’s behind-the-scenes upgrades (as noted earlier) improved reliability, and it’s evolving to integrate with modern container-based apps and edge scenarios.

AI-powered networking: Both enabling and enabled by AI

We are infusing AI into networking to make management and troubleshooting more intelligent. Networking functionality in Azure Copilot accelerates tasks like never before: it outlines the best practices instantly and troubleshooting that once required combing through docs and logs can be conversational. It effectively democratizes networking expertise, helping even smaller IT teams manage sophisticated networks by leveraging AI recommendations.

The future of cloud networking in an AI world

As we close out 2025, one message is clear: networking is strategic. The network is no longer a static utility – it is the adaptive circulatory system of the cloud, determining how far and fast customers can go. By delivering higher speeds, greater reliability, tighter security, and easier management, Azure Networking has empowered businesses to connect everything to anything, anywhere – securely and at scale. These advances unlock new scenarios: global supply chains running in real-time over a trusted network, multi-player AR/VR and gaming experiences delivered without lag, and AI models trained across continents.

Looking ahead, AI-powered networking will become the norm. The convergence of AI and network tech means we will see more self-optimizing networks that can heal, defend, and tune themselves with minimal human intervention.

Network Detection and Response (NDR) in Financial Services

Marc de Droog — Thu, 18 Dec 2025 08:31:59 GMT

Organizations in the Financial Services industry handling sensitive account holder information must comply with the Payment Card Industry Data Security Standard (PCI DSS). The latest version, PCI DSS v4.0.1 of June 2024, reinforces the requirements for network security monitoring.

Traditional network security tools, such as firewalls and Intrusion Detection and Prevention Systems (IDPS), struggle to meet these requirements because they either lack deep visibility or generate too many false positives. This is where Network Detection and Response (NDR) comes in. NDR solutions look at the network traffic within the Cardholder Data Environment (CDE) and use advanced methods (behavioral analytics, machine learning, threat intel) to detect anomalies or attacks in real-time, and facilitate quick responses.

This post explains how NDR supports PCI DSS v4.0.1 compliance, with a focus on deployments in Azure. We will map NDR capabilities to key PCI requirements, describe how Azure’s native tools (Azure Virtual Network TAP, VNET Flow Logs, Traffic Analytics) enable an NDR solution by capturing network data, and discuss third-party NDR tools that analyze this data for threats.

We will also evaluate Microsoft Sentinel’s role as a partial NDR solution, and highlight how Microsoft Defender for Cloud contributes to PCI compliance.

The role of NDR in PCI DSS Compliance

PCI DSS v4.0.1 is organized into 12 main requirement areas. NDR technology primarily supports the control objectives in Requirement 10 (“Log and Monitor All Access to System Components and Cardholder Data”) and Requirement 11 (“Test Security of Systems and Networks Regularly”) , while also aiding in demonstrating compliance with Requirement 4 ("Protect Cardholder Data with Strong Cryptography ...") and Requirement 12 ("Support Information Security with Organizational Policies and Programs").

Below is how NDR aids in building compliance with these controls:

Logging & Monitoring

Organizations are required to “log and monitor all access to system components and cardholder data”. Requirement 10.4.1 calls for automated mechanisms to perform log reviews and detect anomalies at least daily. An NDR solution addresses this by automatically analyzing network traffic logs and generating alerts for suspicious behavior. Every connection or data transfer involving cardholder systems is continuously scrutinized.

For example, if a database containing card data suddenly starts sending large amounts of data to an unfamiliar server, NDR will log that event and flag it for investigation immediately. This satisfies the intent of Requirement 10 by ensuring that not only are network events being recorded, but they are also under active surveillance at all times. NDR essentially serves as an automated network log reviewer, catching things a manual review might miss. This helps meet Requirement 10’s mandate for timely review so that "... incidents can be quickly identified and proactively addresses."

Intrusion Detection

Requirement 11.5 requires the use of intrusion-detection and/or prevention systems (IDS/IPS) at the perimeter of, and at critical points within the CDE. Additionally, 11.5.1.1 requires service providers to employ IDS to detect covert communication attempts, such as malware trying to reach a command-and-control server.

NDR solutions fulfill the role of an IDS by continuously inspecting network traffic for attack signatures and unusual patterns. But NDR goes beyond a legacy IDS: instead of only matching known signatures, it also uses behavior analysis to catch “zero-day” or insider threats (for example, an NDR might detect lateral movement within the network based on abnormal access patterns, even if no signature exists for that behavior). All traffic inbound to, outbound from, and within the CDE is watched and any suspect activity is alerted on.

Showing an NDR solution is in place will satisfy auditors that “Intrusion-detection and/or intrusion prevention techniques are used to detect and/or prevent intrusions into the network”. The high fidelity of NDR alerts (versus older IDS with many false positives) also means the organization is more likely to respond to real incidents – aligning with PCI’s push for effective, risk-based security.

Network Segmentation and Scope Reduction

While not a direct requirement but rather guidance to Requirement 12.5, network segmentation of the Cardholder Data Environment is encouraged to reduce scope - i.e. the sections of the entire environment that are subject to compliance with PCI DSS.

If segmentation is used, it must be monitored. NDR assists here by monitoring the network boundaries of the CDE. It can verify that only allowed communications occur across segments. For instance, if the CDE is isolated such that only a particular jump server should access it, and somehow a developer’s workstation tries to directly communicate with a CDE database, NDR would spot that anomaly. That alert would indicate a segmentation failure or misconfiguration that needs fixing. This continuous oversight helps prove that segmentation is effective (PCI assessors may ask for evidence that the segmentation was tested; NDR alerts or logs showing no unauthorized access attempts over months is strong evidence).

Secure Network Traffic and Encryption

Requirement 4.2 requires that cardholder data sent over networks is encrypted with strong cryptography. NDR tools can help enforce this by detecting unencrypted sensitive traffic or usage of weak protocols. Many NDR solutions will recognize when Primary Account Numbers (PANs) or Sensitive Authentication Data (SAD) appear in plaintext in network traffic and raise an alert. They also often track the SSL/TLS versions and cipher suites used in connections. For example, an NDR can alert if a server in the CDE is accepting TLS 1.0 or if any data is transmitted without encryption where encryption is expected.

Azure VNET Flow Logs provide an “encryption flag” for flows when VNET Encryption is used, which NDR or analytics can use to quickly identify non-encrypted channels. While the primary responsibility for encryption lies in configuration (enabling encryption at the application- (TLS) and infrastructure (VNET Encryption) layers), NDR provides verification. It detects PAN or SAD sent in the clear, and thus supports compliance with Requirement 4. If the NDR never or rarely alerts on cleartext, that’s evidence encryption is consistently applied; if it does alert, it allows quick remediation.

Incident Detection and Response

An incident response plan and processes for reacting to security events must be in place per Requirement 12.5. NDR significantly enhances an organization’s ability to detect and respond to incidents in a timely manner. By providing real-time alerts with rich context (like packet captures or detailed flow info), NDR ensures that when an intrusion or suspicious event happens, the security team is immediately notified with the information needed to act. With NDR integrated to alerting systems, companies can demonstrate that network alerts are automatically generated and investigated as part of their incident response program.

For example, if NDR generates an alert about malware beaconing from a server, the analyst can respond via playbooks (possibly automated, as we’ll discuss with Sentinel) to isolate that server, and later documentation will show the alert and response timeline. This satisfies PCI’s expectation that you not only have monitoring (Req.10/11) but also act on it swiftly (Req. 12.10.5, which expects alerts to trigger the incident response process). Furthermore, the forensic data from NDR (like packet logs) will help in the investigation phase of incident response – determine what data might have been accessed, which systems were affected, etc., which is crucial for PCI breach reporting.

NDR is a key sensor feeding the incident response process, and having it in place with documented procedures closes the loop on several PCI requirements (monitor, detect, respond).

Azure Native Tools for Enabling NDR

In Azure, implementing NDR starts with capturing the right data. Azure provides native tools to mirror network traffic and collect flow information, which NDR systems (or Azure’s own analytics) can then analyze. Key Azure-native components are Azure Virtual Network TAP, Virtual Network (VNET) Flow Logs, and Traffic Analytics.

Azure Virtual Network TAP

Virtual Network TAP (Terminal Access Point) (VTAP) copies network traffic from source Virtual Machines to a collector or traffic analytics tool, running as a Network Virtual Appliance (NVA). VTAP creates a full copy of the traffic, including packet payload content. Traffic collectors and analytics tools are 3rd party partner products, amongst which are the major NDR solutions. VTAP is an agentless, cloud-native traffic tap at the Azure network infrastructure level. It is entirely out-of-band; it has no impact on the source VM's network performance and the source VM is unaware of the tap. Tapped traffic is VXLAN-encapsulated and delivered to the collector NVA, either in the same or a peered VNET as the source VMs.

VTAP is crucial to building a PCI DSS compliant CDE in Azure: full visibility of all network traffic enables implementation of the IDS functionality specified in Requirement 11.5. All traffic involving cardholder data systems can be monitored: not only traffic to/from the internet, but also East-West traffic between VMs. By deploying VTAP on the subnets that make up the CDE, anything suspicious on those networks is seen. This helps meet the requirements of monitoring " ... at the perimeter of the CDE" and “ ... at critical points inside the CDE”, without needing agents on each VM.

Azure VTAP provides the raw network data pipeline needed for NDR. In a PCI audit, citing the use of VTAP plus an NDR appliance as evidence that “all network traffic is being captured and analyzed” is a strong compliance position.

Virtual Network Flow Logs

Virtual Network Flow Logs (VNET Flow Logs) capture IP traffic flow metadata of traffic in a virtual network, which includes source- and destination IP addresses, Layer 4 (transport) protocol and port numbers, flow direction and state, and encryption status. Flow Logs also show whether the flow was allowed or denied by a Network Security Group (NSG) or a Security Admin Rule. VNET Flow Logs do not capture traffic content - they record who talks to whom, with all the details, but not what is said. Log records are written to a storage account in JSON format, and from there can be read for analysis by Azure Traffic Analytics and 3rd party analytics tools such as Security Information and Event Management (SIEM) and NDR solutions.

From a PCI perspective, VNET Flow Logs serve as a comprehensive audit trail of network activity. They show every connection to and from systems in the CDE including source, destination, and whether it was permitted or blocked. This supports Requirement 10 (Log and Monitor All Access) – retain these logs for the required 12 months as evidence all network access attempts are logged.

During daily monitoring, these logs can be queried to find anomalies through Traffic Analytics or a SIEM. For example, a regular query would be: “show any flows from CDE subnet to external IPs not on the whitelist” – any results would indicate a potential policy violation or compromise. If NDR is the heart of real-time detection, flow logs are the record-keeper that ensures nothing slips by unrecorded. Even if an attacker stays under the radar of detection, flow logs later allow forensic analysis to trace their actions. Azure’s VNET flow logs being ubiquitous and simple to enable means organizations have little reason not to log all network traffic. It is a baseline best practice that also satisfies the PCI DSS logging mandate. Enabling flow logs and keeping them in Azure Storage with access controls fulfills PCI requirements around log integrity and access control. They essentially answer, “If an auditor asks to see who communicated with the DB server on June 1 at 3PM, can you show that?” – with flow logs, yes, you can.

Traffic Analytics

Traffic Analytics is an Azure service that processes Flow Logs to provide insights and visualizations. Once VNET Flow Logs are enabled, Traffic Analytics can read the raw logs at 10 minute- or hourly intervals and process them into insightful information. This information is stored in a Azure Log Analytics workspace for further evaluation. Traffic Analytics includes ready-made dashboards in the Azure Portal and its consolidated flow information is available for evaluation through Kusto queries, and tools such as Azure Sentinel and third party SIEMs.

Traffic Analytics will aggregate flows and show things like top talkers (top source/destination IPs), traffic distribution (ports, protocols), and importantly, flagged security issues. For example, it can identify if a VM has an open port to the internet that is unusual, or if there is traffic to an IP address that Microsoft Threat Intelligence marks as malicious (these show up as “MaliciousFlow” in the output). It can also highlight sudden changes in traffic volume or a high number of denied flows (potential port scan or attack attempts).

Think of Traffic Analytics as a built-in basic NDR-lite or network SIEM for Azure: it won’t catch advanced threats, but it will definitely surface misconfigurations and obvious red flags. From a compliance standpoint, Traffic Analytics is useful for demonstrating proactive monitoring. Instead of showing an auditor raw JSON logs, demonstrate Traffic Analytics: e.g., “In the last week, these were the only external connections and all were expected, and no malicious IPs contacted.” It helps satisfy the requirement that logs and network events are reviewed daily, by providing an easy to interpret summary that can be checked at a glance. If something is noted (like an unexpected open port), this can be remediated as part of security operations, showing the auditor that not only do traffic is logged, it is analyzed and acted upon.

It is important to note Traffic Analytics is not a full threat detection system – it is rule-based and limited to what can be inferred from flow records (Layer 3/4 info). It does not inspect payloads (no Layer 7 analysis) and it may not catch subtler anomalies (e.g., data exfiltration hiding in allowed HTTPS traffic might not trigger any obvious threshold in flow stats). Therefore, while it is a great native feature and certainly helps with compliance reporting, for robust NDR one would use Traffic Analytics as a supplement to, not a replacement for, an advanced NDR platform. In Azure, many customers use Traffic Analytics for general network hygiene monitoring and feed its findings into Sentinel or a SIEM for follow-up.

In summary, Azure’s native network monitoring tools lay the groundwork for NDR by capturing the necessary data:
• Azure VTAP provides full-fidelity packet capture (the raw material for deep detection).
• VNET Flow Logs provide broad coverage of who talked to whom and when (excellent for audit and pattern analysis).
• Traffic Analytics provides immediate insights from those logs (great for compliance checks and basic anomaly spotting).

The next step is feeding this data to powerful analytics engines to actually perform the “detection and response” – that’s where third-party NDR or advanced Azure services come into play.

Advanced Analysis with Third-Party NDR Solutions

Azure’s native capabilities collect the raw data, but effective threat detection requires specialized analytics beyond what Azure provides. This is where third-party NDR solutions are indispensable. These solutions ingest the packet or flow data and use their own engines to detect threats like intrusions, malware traffic, or policy violations in real time.

Third-party NDR platforms bring mature, cutting-edge detection algorithms that have been refined on large datasets and numerous environments. They often use a combination of machine learning (for anomaly detection) and signature/threat intelligence (for known threat detection). Azure’s Traffic Analytics or basic Defender for Cloud alerts might report, for instance, that a VM made an outbound connection on an unusual port, but a third-party NDR could dig deeper and say “that connection contained data patterns consistent with credit card numbers in clear text” or “this series of packets matches the behavior of the Cobalt Strike beacon malware.”

NDR solutions do deep packet inspection (DPI) and behavioral analysis to catch subtle threats and minimize false positives. For organizations in Financial Services, targeted by Advanced Persistent Threats and sophisticated attackers, this level of insight is crucial. It is also necessary for meeting the spirit of PCI’s IDS requirement – a smarter IDS means fewer missed intrusions. Many third-party NDRs also come with features like user/device identification, threat chain visualization, and built-in compliance reporting specific to PCI or other standards.

Microsoft has worked with numerous security vendors to ensure their NDR solutions work seamlessly in Azure. The Virtual Network TAP partner list includes vendors in two broad categories: network packet brokers and security analytics/NDR solutions.

Network Packet Brokers (e.g., Gigamon, Keysight): These tools focus on aggregating, filtering, and distributing the tapped traffic. Their strength is handling large volumes of data and directing it efficiently to multiple analysis tools. For instance, Gigamon’s GigaVUE for Azure can take the VTAP stream, filter out irrelevant traffic, and feed it to both an NDR and a performance monitoring tool simultaneously. The advantage is scalability and flexibility; the limitation is that packet brokers themselves typically don’t do deep threat analysis – they are complementary infrastructure.
Security Analytics / NDR Platforms (e.g., Darktrace, Vectra, ExtraHop, Corelight, Fortinet, Netscout, Trend Micro, Arista, etc.): These are the actual brains performing threat detection. Each has its strengths:

o AI-Driven NDR: Solutions like Darktrace and Vectra AI emphasize machine learning to establish a baseline of normal network behavior and then detect anomalies. Darktrace, for example, uses unsupervised AI to identify subtle deviations that could indicate a threat (like a device that suddenly starts connecting to a new domain at odd hours). These are good at catching novel or insider threats. They often come with insightful visualization of network patterns. A possible limitation is they can sometimes produce alerts that require expert interpretation (why did the AI flag this?), but vendors have improved in providing explainability.

o Protocol/Behavioral NDR: ExtraHop Reveal(x), Corelight (Zeek) and Netscout Omnis focus on deep packet and protocol analysis. ExtraHop decodes dozens of protocols (DNS, database protocols, cloud service APIs) in real-time and can detect specific issues like database exfiltration or use of deprecated TLS versions. It was even audited to exceed PCI IDS requirements by using behavior-based detections. Corelight uses the open-source Zeek (Bro) engine to log detailed metadata about traffic, which can be incredibly rich for investigation and custom detection scripts. Strength: very detailed, low false positives when tuned; limitation: may require more tuning or skilled users to get the most out of raw data (Corelight, for example, gives you great data but you might still need a SIEM or Splunk queries to raise the alerts).

o Integrated Ecosystem Solutions: Fortinet and Arista are examples where NDR is part of a broader security ecosystem. Fortinet’s FortiNDR is attractive if you already use Fortinet, as the logs and management tie into FortiAnalyzer and you can leverage FortiGuard threat intel. A FortiGate VM can receive mirrored traffic and apply its IPS/IDS signatures and even ML-based detections. Arista Networks (which acquired Awake Security) offers Arista NDR that’s known for entity-centric threat hunting – it builds profiles of devices on the network and can identify rogue or compromised systems by their traffic patterns. These integrated solutions often pair well with their own hardware or cloud frameworks (Fortinet with its firewalls, Arista with its switches), and can sometimes take automated actions (e.g., instruct a FortiGate to block an IP upon detection).

o Managed NDR Services: eSentire MDR for example provides technology plus a 24/7 human SOC. They deploy sensors that leverage VTAP (they are a VTAP partner) and their analysts review and respond to every alert. The strength is that you get expert eyes on everything (great for meeting PCI’s requirement that alerts are promptly addressed), and the weakness is relying on a third-party – though for many, outsourcing this is prudent.

In practice, an organization in the Financial Services Industry will choose a combination of these based on the specifics of their environment, their compliance needs and existing tools and capabilities. Some might use a packet broker and an NDR together (e.g., Gigamon to optimize traffic flow and ExtraHop to analyze it). Others might use an all-in-one virtual appliance from a vendor like Vectra that directly handles ingestion and analysis.

Sentinel: Complimentary to NDR

Microsoft Sentinel is Azure’s native Security Information and Event Management (SIEM) and orchestration platform. It aggregates logs from many sources runs analytics on them. It is a powerful tool for security monitoring, but it is not specialized for network traffic analysis in the way NDR solutions are.

Let's look at how Sentinel factors into the NDR/compliance equation:

Log Centralization and Correlation: Sentinel ingests Azure VNET Flow Logs and Azure Firewall- and third party firewall logs, as well as alerts from NDR platforms, and any other relevant data (event logs from servers, Azure AD logs, etc.), and aggregates and correlates these events. This is very valuable in daily operations and in investigations - it suppresses the noise, surfacing meaningful, actionable events to security staff. For PCI DSS compliance, having a central SIEM like Sentinel helps satisfy the requirement that logged events are aggregated and reviewed.
Built-in Analytics for Network Events: Sentinel provides default analytics rules and allows custom rule creation using Kusto Query Language (KQL). For network monitoring, one could create rules such as:

o “Alert if more than 50 denied flows hit a CDE server in 5 minutes” (indicating a possible attack).

o “Alert if any flow originates from an IP known in Threat Intelligence to be malicious” (Sentinel integrates threat intel feeds).

o “Alert if a normally internal-only VM initiates outbound traffic” (indicating a potentially compromised server).

These can catch some intrusion attempts. However, setting up robust network anomaly detection in Sentinel requires effort and expertise. Out-of-the-box, Sentinel does not come with a comprehensive library of network threat detection rules – it might have a few (like detecting port scan patterns or spikes in traffic) but not the depth of an NDR product.

No Deep Packet Inspection: Sentinel cannot analyze raw packet payloads or protocol details beyond what is in logs. It relies on sources like flow logs which only have IP/port info, or on other products (like an NDR or IDS) to generate an alert that Sentinel then ingests. This is a fundamental limitation – for example, Sentinel by itself would not be able to detect that an SQL query contained a suspicious UNION SELECT (something an NDR inspecting the SQL protocol might catch). Therefore, Sentinel alone, without something feeding it detailed alerts, would likely miss many attack techniques that do not manifest in log data.
Alert Fatigue and Tuning: If one tried to use Sentinel as the primary IDS by writing custom rules on flow logs, one might end up with a lot of false positives or noise that needs tuning. NDR vendors invest heavily in fine-tuning detections to be as accurate as possible in network context (for example, distinguishing a legitimate network scan by a vulnerability management tool from a malicious scan by an attacker). With Sentinel, that tuning burden falls on the security team. While Sentinel’s analytics can be quite sophisticated, practically, in-house development of NDR logic in Sentinel is reinventing the wheel.
Compliance Sufficiency: Could Sentinel alone satisfy PCI DSS requirements for IDS? In theory, if configured to ingest all relevant logs and set up with analytics to alert on suspicious network events, Sentinel might convince an assessor. For example, using Azure Firewall or a third-party firewall that outputs logs to Sentinel, and Sentinel alerting on those logs (like on intrusion signatures those firewalls catch), might tick the box. However, most auditors expect a dedicated IDS/IPS or NDR technology rather than a custom SIEM query solution. The PCI DSS 4.0.1 guidance explicitly talks about IDS/IPS having up-to-date signatures or detection capabilities for common threats – Sentinel by itself doesn’t maintain a library of network attack signatures; that’s not its function. Moreover, Requirement 11.5.1 basically assumes a distinct IDS/IPS tool. Sentinel would likely be viewed as supplemental – great for aggregating alerts, but not the generator of those alerts.
Incident Response Automation: Sentinel is extremely valuable in orchestrating responses. It can trigger playbooks (with Logic Apps) based on alerts. If an NDR alert comes in (or a custom Sentinel alert triggers), Sentinel can automate actions: isolate a VM by applying a new NSG, disable a user account in Azure AD, send notifications to the team. Sentinel can log the whole process for audit. Having such automation shows you not only detect, but respond swiftly and consistently – aligning with PCI’s incident response testing requirements. Sentinel also retains incident history, which helps in the PCI requirement for reviewing security incidents and responses as part of annual processes.

In conclusion, Sentinel is best seen as complimentary to NDR rather than as a complete NDR solution in itself.

It is excellent in terms of log management and initiating responses (covering aspects of Requirements 10 and 12), but on its own it doesn’t fulfill the technical depth of Requirement 11’s intrusion detection. It’s best thought of as the “nerve center” that an NDR (the “sensory organ”) feeds into. In an ideal Azure PCI deployment, you would use Sentinel alongside an NDR: the NDR detects the nuanced network threats and sends alerts to Sentinel; Sentinel then correlates those with other info and kicks off response actions.

Microsoft Defender for Cloud and PCI DSS Compliance

Defender for Cloud is Azure’s Cloud Security Posture Management (CSPM) and Cloud Workload Protection Platform (CWPP). It continuously assesses your Azure (and multi-cloud) resources against security best practices and provides threat protection via various “Defender” plans (for VMs, databases, storage, etc.). When it comes to PCI DSS compliance, Defender for Cloud is a useful service because it can map your Azure environment to PCI requirements and help automate compliance checking.

Here’s how Defender for Cloud contributes:

Regulatory Compliance Dashboard – Defender for Cloud has a built-in compliance dashboard where you can enable the PCI DSS v4.0.1 standard for your Azure environment, which can span multiple subscriptions. It shows a control-by-control assessment of compliance, based on Azure Policy and Defender scans. For example, it will check that VMs have disk encryption enabled, that Key Vaults have soft delete enabled, that network watchers are enabled, etc., mapping to various PCI controls. It won’t automatically check every single PCI requirement (some require manual processes), but it covers those that can be programmatically assessed. This is extremely helpful for preparing for an audit – you get a compliance score, see where gaps are, then remediate them. The dashboard essentially translates Azure’s security state into PCI language, saving a lot of manual effort. A practical example: for network-specific controls, it might check that NSGs are present on subnets, or that flow logging is enabled (to meet monitoring requirements).
Continuous Configuration Monitoring – Many PCI requirements are about having proper configurations (e.g., secure configurations for systems, firewalls in place, no default passwords). Defender for Cloud continuously monitors Azure resources and generates security recommendations when something deviates from best practice (many of which align with PCI controls). For instance, if a critical VM in the CDE is missing an NSG or has a rule allowing “Any” source, Defender for Cloud will flag that as a recommendation to fix – effectively catching a potential PCI violation early. It also checks for things like missing vulnerability assessments on SQL, or unencrypted traffic on storage, etc. By following Defender’s recommendations, you inherently move toward PCI compliance. This addresses the preventive side of PCI – e.g., Requirement 1 (firewall configuration) and Requirement 2 (secure system configurations) are supported by these continuous assessments.
Threat Detection and Alerts: Defender for Cloud includes Defender plans for various resources that provide threat detection. For example, Defender for Servers monitors VMs for suspicious processes, malware, and also does file integrity monitoring (FIM). FIM is actually a PCI Requirement (11.5) – you must monitor critical system files for changes. By enabling Defender for Servers on your Azure VMs, you fulfill this, as it uses the Defender for Endpoint agent to track file changes and generate alerts if, say, system binaries are modified unexpectedly. Additionally, Defender for Servers and other plans generate network-related security alerts: e.g., “Potential malicious outbound connection from VM” or “Port scanning activity detected from VM.” These come from analyzing the VM’s telemetry and network flows. While not as comprehensive as a dedicated NDR, these built-in alerts offer a baseline IDS capability. For example, if a VM in the CDE starts port scanning others, Defender for Cloud will flag it, which covers the requirement that you should detect internal reconnaissance. There are also alerts like “Suspicious SQL query activity” for Azure SQL or “Anomalous access pattern” for storage accounts. All Defender for Cloud alerts appear in its Security Alerts blade, and can be forwarded to Sentinel. From a PCI perspective, having these alerts means you have multiple layers of monitoring (host-level and network-level), which is ideal. It demonstrates that even if an attack doesn’t trigger an NDR network alert, it might trigger a host alert that you’re also watching.
Adaptive Network Hardening: This feature of Defender for Cloud looks at your NSG rules and actual traffic and recommends hardening (like “these IPs are the only ones seen accessing your VM, consider tightening NSG to only those”). By following these, you reduce your attack surface, which indirectly helps comply with PCI network access restrictions. It is not a mandated control, but it’s a good practice.
Vulnerability Management: Though not directly NDR, Defender for Cloud’s integrated vulnerability scanning (through Qualys or Defender’s scanner) helps satisfy PCI Requirement 11.3 (regular vulnerability scans) and Requirement 6 (address vulnerabilities). This is complementary to NDR – one stops attacks, the other prevents them by patching. It’s worth noting in compliance documentation that you use these Azure-native capabilities to meet various requirements.

In summary, Defender for Cloud acts as a compliance and security safety net. It ensures you have the right security controls configured (so that your NDR has a solid foundation to monitor), and it provides additional threat detection on Azure resources. For PCI DSS 4.0.1, which has many controls beyond just network monitoring, Defender for Cloud helps with:

Requirement 5 (anti-malware) by monitoring for malware on VMs.
Requirement 6/11 (vulnerability management and scanning) via its vulnerability assessments.
Requirement 10 (log retention) by advising enabling of diagnostic logs and storing them; and it keeps its alerts logs.
Requirement 11.5 (change detection) via File Integrity Monitoring on servers.
Requirement 8 (MFA, principle of least privilege) through Azure AD and Identity recommendations.
Requirement 12 (security policy and monitoring) by giving a centralized compliance view and integrating with incident workflows.

However, like Sentinel, Defender for Cloud is not a specialized NDR. Its network alerts are generally simpler (e.g., known malicious IP or basic port scan detection) and should not be solely relied on to meet the IDS requirement. They complement a true NDR or firewall IDS. A strong strategy is: use Defender for Cloud to get your Azure environment in a secure, compliant state (so all baseline controls are green), and use NDR to actively monitor and defend that environment from sophisticated attacks. The synergy of the two covers a vast swath of PCI requirements in a largely automated fashion.

Bringing it all together

The Financial Services Industry operates under intense security scrutiny, and PCI DSS v4.0.1 raises the bar further by requiring proactive and continuous network monitoring. Network Detection and Response (NDR) is a key technology that helps meet these challenges by providing advanced intrusion detection and full visibility into network traffic.

In the context of PCI DSS:

NDR ensures that all access to networks carrying cardholder data is monitored in real-time (fulfilling Requirement 10’s logging/monitoring mandate with automation).
It serves as the IDS/IPS required to detect malicious network activity (addressing Requirement 11’s call for intrusion detection).
It supports network segmentation by verifying that segmentation is not breached and alerting on any anomalous connection (thereby underpinning the isolation that PCI strongly recommends for scope reduction).
It feeds into a robust incident response workflow, enabling the organization to react swiftly to suspected breaches (covering the intent of Requirement 12.10 on incident response).

In Azure, achieving a PCI-compliant NDR setup is very feasible using a combination of Azure’s native capabilities and partner solutions. Azure provides the plumbing (VTAP for full packet mirror, VNET Flow Logs for thorough logging, Traffic Analytics for basic analysis) and the integration points (connecting to Sentinel, Defender for Cloud, etc.). Third-party NDR platforms bring in the sophisticated analysis that can identify threats with high accuracy, something that Azure’s basic tools alone might not catch. By leveraging both, organizations in Financial Services can create a layered defense:

Azure Defender for Cloud to maintain strong security posture and baseline compliance (it makes sure configurations are correct and provides some threat detection and compliance mapping).
Azure vTAP + Flow Logs to ensure no network activity goes unnoticed.
Advanced NDR to actually inspect traffic deeply and spot intrusions or policy violations in real-time.
Microsoft Sentinel to unify the monitoring, correlate across sources, and automate response, serving as the command center that ensures every alert is handled (thus meeting PCI’s expectation of prompt action and thorough monitoring records).

This layered approach not only checks the compliance boxes but also materially improves security. It demonstrates the principle that “compliance is the floor, not the ceiling” – the organization implements NDR not just to satisfy PCI requirements but to actively protect critical data.

For auditors and stakeholders, the combination of evidence from NDR, Azure logs, and Defender for Cloud provides confidence that the network is secure.

In conclusion, NDR is important in the financial sector for both security and compliance. PCI DSS 4.0.1 explicitly or implicitly requires capabilities that an NDR delivers (continuous monitoring, intelligent alerting, quick incident containment). Implementing NDR in Azure using the described architecture enables organizations in Financial Services to meet those requirements effectively. They gain peace of mind that their cloud cardholder data environment is under vigilant watch, and they stand on solid ground when undergoing PCI assessments. As threats are evolving and compliance standards are tightening, this blend of Azure technology and NDR solutions exemplifies best practice: using the full power of cloud and AI-driven security to protect sensitive financial data and maintain customer trust.

Announcing Azure DNS security policy with Threat Intelligence feed general availability

Sergio Figueiredo — Thu, 20 Nov 2025 20:22:48 GMT

A successful protective DNS strategy seamlessly hardens an entire environment without adding friction: it must protect every virtual network and region consistently, apply real-time and highly accurate threat intelligence, deliver clear visibility into what was blocked and why, integrate smoothly with existing DNS infrastructure, and maintain near-zero performance impact.

Above all, it should reduce operational noise—lowering incident volume, SOC workload, and risk—while being easy to deploy, easy to trust, and impossible for attackers to slip past.

If customers do not have visibility into their DNS traffic, reliable detection and mitigation functionality, there is a risk of exposure to attacks which can turn into data theft. This translates into financial and intellectual property loss.

Therefore, it’s key to enable scenarios where Azure DNS customers can mitigate security threats such as data theft, compromised workloads with Zero Day threats, and provide the customers with the right service to detect, visualize, and mitigate this overseen attack vector.

With DNS security policy customers have access to a rich value proposition where not only is it possible to filter DNS traffic with allow/block functionality but also gain visibility of DNS traffic at the virtual network level in all regions whilst integrating with known facilities such as Log Analytics, Event Hubs, or storage accounts to keep their logs.

We are excited to share that Azure DNS security policy with Threat Intelligence is now in general availability.

A quick overview of Azure DNS security policy

DNS security policy was launched recently and offers the ability to filter and log DNS queries at the virtual network (VNET) level. Policy applies to both public and private DNS traffic within a VNET. DNS logs can be sent to a storage account, log analytics workspace, or event hubs. You can choose to allow, alert, or block DNS queries.

With DNS security policy you can:

Create rules to protect against DNS-based attacks by blocking name resolution of known or malicious domains.
Save and view detailed DNS logs to gain insight into your DNS traffic.

A DNS security policy has the following associated elements and properties:

Location: The Azure region where the security policy is created and deployed.
DNS traffic rules: Rules that allow, block, or alert based on priority and domain lists.
Virtual network links: A link that associates the security policy to a virtual vetwork.
DNS domain lists: Location-based lists of DNS domains.

What is being announced today?

Azure DNS security policy with Threat Intelligence feed allows early detection and prevention of security incidents on customer Virtual Networks where known malicious domains sourced by Microsoft’s Security Response Center (MSRC) can be blocked from name resolution.

Azure DNS security policy with Threat Intelligence feed is being announced to all customers and will have regional availability in all public regions.

For more information about the capabilities available, please visit the Azure DNS security policy technical documentation webpage.

What can customers start doing with Azure DNS Threat Intelligence feed today?

Apart from the features which were announced earlier for DNS security policy, the feed will be available as a managed domain list and will enable you to protect your workloads against known malicious domains with Microsoft’s own managed Threat Intelligent feed.

With Threat Intelligence you will benefit from the following:

Smart protection

Almost all attacks begin with a DNS query. Threat Intelligence managed domain list enables you to detect and prevent security incidents early.

Continuous updates

The feed is automatically updated by Microsoft so that you stay protected against newly detected malicious domains.

Monitoring and blocking known malicious domains

You have the flexibility of just observing the activity in Alert only mode or block the suspected activity in blocking mode.
Enabling logging gives you visibility into all DNS traffic in the virtual network.

DNS security policy Threat Intelligence feed in GA is also available to use via PowerShell, CLI, .NET, Java, Python, REST, Typescript, Go, ARM, and Terraform.

Key use cases for this service:

Configure Threat Intelligence as a managed domain list in your DNS security policies for an additional layer of protection against known malicious domains.
Get visibility of compromised hosts which are trying to resolve known malicious domains from your virtual networks.
Log and setup alerts if malicious domains are being resolved in any given virtual network where the Threat Intel feed is configured.
Seamlessly integrate with your virtual networks and other services such as Azure Private DNS Zones, Private Resolver, and other services in the VNET.

Fully managed:

Built-in high availability, zone redundancy, and low latency name resolution.

Cost reduction:

Reduce operating costs and run at a fraction of the price of traditional IaaS solutions. There is no need to provision additional instances of IaaS Virtualization Appliances or VM-based solutions and added operational complexity.

Protect and monitor your DNS traffic:

Capture DNS logs from your virtual networks into Log Analytics, Event Hubs, storage accounts, and apply Threat Intelligence as a managed domain list to your DNS filtering rules for additional protection of your workloads.

DevOps friendly

Build your pipelines with Terraform, ARM, or Bicep.

Get started and share your feedback

You can try Azure DNS security policy with Threat Intelligence feed today. For more information about the capabilities available, please visit the Azure DNS security policy technical documentation webpage. Post your ideas and suggestions on the networking community page.

Announcing the public preview of StandardV2 NAT Gateway and StandardV2 public IPs

aimeelittleton — Tue, 18 Nov 2025 17:30:00 GMT

In today’s rapidly changing digital landscape, organizations are innovating faster and delivering cloud native experiences at global scale. With this acceleration comes higher expectations: applications must remain always available. An outage in a single availability zone can have a ripple effect on application performance, user experience, and business continuity. To safeguard against zone outages, making your cloud architecture zone resilient isn't an option—it’s a necessity.

A key part of any resilient design is ensuring reliable outbound connectivity. Azure NAT Gateway is a fully managed network address translation service that provides highly scalable and secure internet connectivity for resources inside your virtual networks.

We’re excited to announce the public preview of the StandardV2 SKU NAT Gateway, an evolution of Azure NAT Gateway built for the next generation of scale, performance, and resiliency. StandardV2 NAT Gateway delivers zone redundancy, enhanced data processing limits, IPv6 support, and flow logs all at the same price as the Standard SKU NAT Gateway.

This release also marks the public preview of StandardV2 SKU public IP addresses and prefixes. A new SKU of public IPs that must be used with StandardV2 NAT Gateway. In combination, StandardV2 IPs provide high-throughput connectivity to support demanding workloads.

StandardV2 NAT Gateway is zone resilient and provides dual-stack connectivity.

What’s new in StandardV2 NAT Gateway

Zone redundancy

StandardV2 NAT Gateway is zone-redundant by default in regions with availability zones. Deployed as a single resource operating across multiple zones, StandardV2 NAT Gateway ensures outbound connectivity even if one zone becomes unavailable.

StandardV2 NAT Gateway spans across multiple availability zones in a region.

For example, a virtual machine in zone 2 connects outbound from zones 1, 2, or 3 through a StandardV2 NAT Gateway (as shown in the figure). If zone 1 experiences an outage, existing connections in that zone may fail, but new connections will seamlessly flow through zones 2 and 3—keeping your applications online and resilient. All existing connections through zones 2 and 3 will persist. To learn more, see StandardV2 NAT Gateway zone-redundancy.

In zone down scenarios, new connections flow through the remaining healthy zones with StandardV2 NAT Gateway.

This kind of resiliency is essential for any critical workload to ensure high availability. Whether you’re a global SaaS provider that needs to maintain service continuity for your customers during zonal outages or an e-commerce platform that needs to ensure high availability during peak shopping seasons, StandardV2 NAT Gateway can help you achieve greater protection against zonal outages.

Higher performance

StandardV2 doubles the performance of the Standard SKU, supporting up to 100 Gbps throughput and 10 million packets per second. These enhanced data processing limits are ideal for data-intensive and latency-sensitive applications requiring consistent, high-throughput outbound access to the internet. For more information, see StandardV2 NAT Gateway performance.

StandardV2 public IPs

Alongside NAT Gateway, StandardV2 public IP Addresses and Prefixes are now available in public preview. StandardV2 SKU public IPs are a new offering of public IPs that must be used with StandardV2 NAT Gateway to provide outbound connectivity. Standard SKU public IPs are not compatible with StandardV2 NAT Gateway. See how to deploy StandardV2 public IPs.

IPv6 (dual-stack) support

StandardV2 NAT Gateway now supports dual-stack (IPv4 + IPv6) connectivity, enabling organizations to meet regulatory requirements, optimize performance for modern architectures, and future-proof workloads at internet scale. Each NAT Gateway supports up to 16 IPv4 and 16 IPv6 StandardV2 public IP addresses or prefixes. See IPv6 support for StandardV2 NAT gateway for more information.

StandardV2 NAT Gateway support for dual stack connectivity now in public preview.

Flow logs

With StandardV2, you can now enable flow logs to gain deeper visibility into outbound traffic patterns. Flow logs capture detailed IP-level traffic information, helping you:

Troubleshoot connectivity issues more efficiently.

Identify top talkers behind the NAT Gateway (which virtual machines initiate the most connections outbound).

Analyze traffic for compliance and security auditing for your organization

Learn more at Enable flow logs on StandardV2 NAT Gateway.

Deploying StandardV2 NAT Gateway and public IPs

You can deploy StandardV2 NAT Gateway and StandardV2 public IPs using ARM templates, Bicep, PowerShell, or CLI.

Portal and Terraform support is coming soon. For more information on client support, see StandardV2 NAT Gateway SKU.

Learn More

Integrating Azure Application Gateway v2 with Azure API Management for secure and scalable API

ranjsharma — Tue, 18 Nov 2025 17:33:43 GMT

Why Application Gateway v2 + Azure API Management?

- Layer-7 routing with path-based rules, host headers, URL rewrites, and WAF protection (OWASP Core Rule Set).

- Azure API Management provides API abstraction, versioning, throttling, caching, JWT validation, and per-API policies.

- Combined, App Gateway becomes the internet-facing secure entry point and Azure API Management the control plane for API governance.

Scenario:

Internet → App Gateway (WAF) → Azure API Management (External) → Backends

Best when Azure API Management needs to be publicly reachable but protected by WAF and central routing.

[Client] ─HTTPS──> [App Gateway v2 (WAF)] ─HTTPS──> [Azure API Management (External)] ─> [Private/On-prem/Azure Backends]

Pros: Simple, fast to implement, WAF in front, supports CDN/Front Door chaining.

Cons: Azure API Management is public; additional steps required for IP allow-lists and mTLS.

Scenario2:

Internet → App Gateway (WAF) → Azure API Management (Internal) via Private Endpoint

Azure API Management is internal; only App Gateway is public. Zero-trust friendly.

[Client] ─HTTPS──> [App Gateway v2 (WAF, Public)] ─HTTPS──> [Private Endpoint] ─> [Azure API Management (Internal)] ─> [Backends]

Pros: Azure API Management is not exposed to the internet; traffic flows through App Gateway + Private Link.

Cons: Requires vNet planning, DNS, and App Gateway-to-Private Link name resolution.

Scenario3:

Azure API Management (External) → App Gateway (Internal) → Private Backends

Azure API Management is the public front door; App Gateway does L7 routing to internal services.

[Client] ─HTTPS──> [Azure API Management (External)] ─HTTPS──> [App Gateway (Internal/WAF)] ─> [Backends]

Pros: Privileged Identity Management security & governance is your internet front door.

Cons: More Azure API Management policy work; App Gateway must be reachable from Azure API Management.

Network & DNS design checklist:

- Virtual networks & subnets:

- App Gateway-Subnet (required dedicated subnet)

- Azure API Management-Subnet (for internal tier)

- Shared-services-Subnet for Bastion/jumpbox/logging

- Private Link: Enable Azure API Management private endpoint in Azure API Management-Subnet.

- Private DNS Zones: privatelink.azure-api.net for Azure API Management, custom zones for backends.

- Name resolution: App Gateway must resolve Azure API Management private FQDN via vNet DNS or Azure Private DNS.

- Firewall & NSGs: Restrict inbound/outbound; allow only required ports to Azure API Management, Key Vault, Log Analytics.

- Hybrid Connectivity: Site-to-site VPN or ExpressRoute for on-prem backends; consider Azure Firewall or NVA.

Certificates & TLS

- Custom Domains:

- App Gateway: api.contoso.com

- Azure API Management: gateway.contoso.com (with custom hostname on Azure API Management)

- TLS Ports: HTTPS 443 end-to-end; disable TLS 1.0/1.1.

- Cert storage: Use Azure Key Vault for SSL certs; integrate App Gateway & Azure API Management with Key Vault (managed identity).

- mTLS (Client Certs): Enforce on Azure API Management with policies; optionally on App Gateway via mutual auth for selected listeners.

WAF (Web Application Firewall) on App Gateway v2

- Modes: Detection vs prevention (recommend prevention once tuned).

- CRS: Start with 3.2; baseline exclusions for APIs (JSON payloads).

- Managed rules: Enable bot protection, set anomaly scoring, create exclusions for headers like Authorization.

- Logging: Send WAF logs to Log Analytics; build alerts for blocked requests spikes.

Azure API Management Policies – Common patterns

- Inbound: `validate-jwt`, `check-header`, `rate-limit`, `ip-filter`, `set-backend-service`

- Backend: `retry`, `forward-request` with mTLS

- Outbound: `set-header`, `find-and-replace`, `cache`

- Global vs API vs Operation: Keep global minimal; override at API/operation for precision.

- Dev, Test, Prod: Parameterize via named values and Key Vault references.

Example – JWT validation and rate limit:

xml

<validate-jwt header-name="Authorization" failed-validation-httpcode="401">

<openid-config url="https://login.microsoftonline.com/<tenant-id>/v2.0/.well-known/openid-configuration" />

<audience>api://contoso-app-id</audience>

</audiences>

<issuer>https://sts.windows.net/<tenant-id>/</issuer>

</issuers>

</validate-jwt>

<rate-limit calls="100" renewal-period="60" />

</inbound>

<forward-request />

</backend>

</outbound>

</policies>

## Terraform – Core Resources (App Gateway v2 + Azure API Management)

Note: Simplified example; parameterize for prod, add Key Vault integrations, diagnostics, and role assignments.

# Variables (example)

variable "location" { default = "eastus" }

variable "rg_name" { default = "rg-appgw-apim" }

variable "vnet_name" { default = "vnet-core" }

variable "appgw_subnet" { default = "snet-appgw" }

variable "apim_subnet" { default = "snet-apim" }

provider "azurerm" { features {} }

resource "azurerm_resource_group" "rg" {

name = var.rg_name

location = var.location

}

resource "azurerm_virtual_network" "vnet" {

name = var.vnet_name

location = var.location

resource_group_name = azurerm_resource_group.rg.name

address_space = ["10.10.0.0/16"]

}

resource "azurerm_subnet" "snet_appgw" {

name = var.appgw_subnet

resource_group_name = azurerm_resource_group.rg.name

virtual_network_name = azurerm_virtual_network.vnet.name

address_prefixes = ["10.10.1.0/24"]

}

resource "azurerm_subnet" "snet_apim" {

name = var.apim_subnet

resource_group_name = azurerm_resource_group.rg.name

virtual_network_name = azurerm_virtual_network.vnet.name

address_prefixes = ["10.10.2.0/24"]

}

# Public IP for App Gateway

resource "azurerm_public_ip" "appgw_pip" {

name = "pip-appgw"

resource_group_name = azurerm_resource_group.rg.name

location = var.location

allocation_method = "Static"

sku = "Standard"

}

# Application Gateway v2 (WAF)

resource "azurerm_application_gateway" "appgw" {

name = "agw-v2-waf"

resource_group_name = azurerm_resource_group.rg.name

location = var.location

sku {

name = "WAF_v2"

tier = "WAF_v2"

capacity = 2

}

gateway_ip_configuration {

name = "appgw-ipcfg"

subnet_id = azurerm_subnet.snet_appgw.id

}

frontend_port {

name = "https-port"

port = 443

}

frontend_ip_configuration {

name = "appgw-feip"

public_ip_address_id = azurerm_public_ip.appgw_pip.id

}

ssl_certificate {

name = "ssl-agw"

data = filebase64("certs/agw.pfx")

password = var.agw_pfx_password

}

http_listener {

name = "listener-https"

frontend_ip_configuration_name = "appgw-feip"

frontend_port_name = "https-port"

protocol = "Https"

ssl_certificate_name = "ssl-agw"

host_name = "api.contoso.com"

}

backend_address_pool {

name = "apim-bepool"

# For Azure API Management private endpoint, use FQDN via custom probe, or IP when static

fqdns = ["gateway.contoso.internal"]

}

backend_http_settings {

name = "https-settings"

cookie_based_affinity = "Disabled"

port = 443

protocol = "Https"

pick_host_name_from_backend_address = true

request_timeout = 30

probe_name = "apim-probe"

}

probe {

name = "apim-probe"

protocol = "Https"

path = "/status-0123456789abcdef"

pick_host_name_from_backend_http_settings = true

interval = 30

timeout = 30

unhealthy_threshold = 3

}

request_routing_rule {

name = "route-to-apim"

rule_type = "Basic"

http_listener_name = "listener-https"

backend_address_pool_name = "apim-bepool"

backend_http_settings_name = "https-settings"

}

waf_configuration {

enabled = true

firewall_mode = "Prevention"

rule_set_type = "OWASP"

rule_set_version = "3.2"

}

# Azure API Management (Developer SKU for demo – use Premium for prod & VNET integration)

resource "azurerm_api_management" "apim" {

name = "apim-contoso"

location = var.location

resource_group_name = azurerm_resource_group.rg.name

publisher_name = "Contoso"

publisher_email = "admin@contoso.com"

sku_name = "Developer_1"

virtual_network_type = "None" # Use "Internal" for vNet, then add private endpoint

}

## Azure DevOps – CI/CD YAML (App Gateway + Azure API Management via Terraform)

`yaml

trigger:

branches:

include: [ main ]

pool:

vmImage: 'ubuntu-latest'

variables:

TF_VERSION: '1.8.5'

ARM_USE_MSI: true

stages:

- stage: Validate

jobs:

- job: tf_validate

steps:

- task: Bash@3

displayName: 'Install Terraform'

inputs:

targetType: 'inline'

script: |

sudo apt-get update && sudo apt-get install -y unzip

curl -L -o tf.zip https://releases.hashicorp.com/terraform/${TF_VERSION}/terraform_${TF_VERSION}_linux_amd64.zip

unzip tf.zip && sudo mv terraform /usr/local/bin/

terraform -version

- task: AzureCLI@2

displayName: 'Terraform init & validate'

inputs:

azureSubscription: '$(AZURE_SERVICE_CONNECTION)'

scriptType: 'bash'

scriptLocation: 'inlineScript'

inlineScript: |

terraform init -backend-config=backend.hcl

terraform fmt -check

terraform validate

- stage: Plan

jobs:

- job: tf_plan

steps:

- task: AzureCLI@2

inputs:

azureSubscription: '$(AZURE_SERVICE_CONNECTION)'

scriptType: 'bash'

scriptLocation: 'inlineScript'

inlineScript: |

terraform plan -out=tfplan

- publish: tfplan

artifact: tfplan

- stage: Apply

condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/main'))

jobs:

- job: tf_apply

steps:

- download: current

artifact: tfplan

- task: AzureCLI@2

inputs:

azureSubscription: '$(AZURE_SERVICE_CONNECTION)'

scriptType: 'bash'

scriptLocation: 'inlineScript'

inlineScript: |

terraform apply -auto-approve tfplan

##Observability & diagnostics

- Access Logs: App Gateway & WAF logs to Log Analytics; query with KQL.

- Azure API Management Metrics: Requests, backend duration, cache hits; enable diagnostic settings to Log Analytics/Storage/Event Hub.

- End-to-end tracing: Correlate `x-correlation-id` across App Gateway, Azure API Management, and backend logs.

- Alerts: 4xx/5xx thresholds, WAF blocks spike, Azure API Management throttling events, TLS certificate expiry.

## Security hardening

- Enforce TLS 1.2+, disable weak ciphers.

- WAF exclusions tuned minimally; regular rule reviews.

- Azure API Management IP allow-lists for admin endpoints; use RBAC and separate admin vs gateway hostnames.

- Private Endpoints for Azure API Management & backends; deny public network access where possible.

- mTLS from App Gateway→Azure API Management or Client→Azure API Management when required (Key Vault for client certs).

- DDoS Protection on vNet with public exposure; consider Azure Front Door WAF for global edge.

## Cost & Performance

- Right-size App Gateway v2 capacity; enable autoscaling for variable traffic.

- Use Azure API Management Premium only if you need vNet, multi-region, or zone redundancy; otherwise consider Standard/Developer for non-prod.

- Caching policies in Azure API Management reduce backend load; use response compression.

- Health probes optimized for backend responsiveness (avoid tight intervals).

## Troubleshooting

- App Gateway 502/504: Check backend health, probe path, SNI/host header, TLS ciphers, DNS resolution to Azure API Management.

- Azure API Management 401/403: Validate JWT audience/issuer; clock skew; named values; policy order.

- Private Endpoint: DNS record in `privatelink.azure-api.net` exists; App Gateway subnet can resolve; NSG not blocking.

- Cert Issues: PFX password correct; full chain present; key usage supports server auth.

- Performance: Turn on App Gateway autoscaling; review Azure API Management throttling; check backend rate limits.

## Production checklist

- Custom domains & cert rotation via Key Vault

- WAF in Prevention with tuned exclusions

- Azure API Management policies for auth, rate limiting, cache, headers

- Private endpoints + DNS validated end-to-end

- Autoscaling & health probes tuned

- Diagnostics & alerts configured

- CI/CD gated approvals; Terraform state secured

- Runbooks for failover & certificate renewal

Azure Virtual Network Manager + Azure Virtual WAN

SimonaTarantola — Mon, 17 Nov 2025 16:54:52 GMT

Azure continues to expand its networking capabilities, with Azure Virtual Network Manager and Azure Virtual WAN (vWAN) standing out as two of the most transformative services. When deployed together, they offer the best of both worlds: the operational simplicity of a managed hub architecture combined with the ability for spoke VNets to communicate directly, avoiding additional hub hops and minimizing latency

Revisiting the classic hub-and-spoke pattern

Element	Traditional hub-and-spoke role
Hub VNet	Centralized network that hosts shared services including firewalls (e.g., Azure Firewall, NVAs), VPN/ExpressRoute gateways, DNS servers, domain controllers, and central route tables for traffic management. Acts as the connectivity and security anchor for all spoke networks.
Spoke VNets	Host individual application workloads and peer directly to the hub VNet. Traffic flows through the hub for north-south connectivity (to/from on-premises or internet) and cross-spoke communication (east-west traffic between spokes).
Benefits	• Single enforcement point for security policies and network controls • No duplication of shared services across environments • Simplified routing logic and traffic flow management • Clear network segmentation and isolation between workloads • Cost optimization through centralized resources

However, this architecture comes with a trade-off: every spoke-to-spoke packet must route through the hub, introducing additional network hops, increased latency, and potential throughput constraints.

How Virtual WAN modernizes that design

Virtual WAN replaces a do-it-yourself hub VNet with a fully managed hub service:

Managed hubs – Azure owns and operates the hub infrastructure.
Automatic route propagation – routes learned once are usable everywhere.
Integrated add-ons – Firewalls, VPN, and ExpressRoute ports are first-class citizens.

By default, Virtual WAN enables any-to-any routing between spokes. Traffic transits the hub fabric automatically—no configuration required.

Why direct spoke mesh?

Certain patterns require single-hop connectivity

Micro-service meshes that sit in different spokes and exchange chatty RPC calls.
Database replication / backups where throughput counts, and hub bandwidth is precious.
Dev / Test / Prod spokes that need to sync artifacts quickly yet stay isolated from hub services.
Segmentation mandates where a workload must bypass hub inspection for compliance yet still talk to a partner VNet.

Benefits

Lower latency – the hub detour disappears.
Better bandwidth – no hub congestion or firewall throughput cap.
Higher resilience – spoke pairs can keep talking even if the hub is under maintenance.

The peering explosion problem

With pure VNet peering, the math escalates fast:
For n spokes you need n × (n-1)/2 links. Ten spokes? 45 peerings. Add four more? Now 91.

Each extra peering forces you to:

Touch multiple route tables.
Update NSG rules to cover the new paths.
Repeat every time you add or retire a spoke.
Troubleshoot an ever-growing spider web.

Where Azure Virtual Network Manager Steps In?

Azure Virtual Network Manager introduces Network Groups plus a Mesh connectivity policy:

Azure Virtual Network Manager Concept	What it gives you
Network group	A logical container that groups multiple VNets together, allowing you to apply configurations and policies to all members simultaneously
Mesh connectivity	Automated peering between all VNets in the group, ensuring every member can communicate directly with every other member without manual configuration
Declarative config	Intent-based approach where you define the desired network state, and Azure Virtual Network Manager handles the implementation and ongoing maintenance
Dynamic updates	Automatic topology management—when VNets are added to or removed from a group, Azure Virtual Network Manager reconfigures all necessary connections without manual intervention

Operational complexity collapses from O(n²) to O(1)—you manage a group, not 100+ individual peerings.

A complementary model: Azure Virtual Network Manager mesh inside vWAN

Since Azure Virtual Network Manager works on any Azure VNet—including the VNets you already attach to a vWAN hub—you can apply mesh policies on top of your existing managed hub architecture:

Spoke VNets join a vWAN hub for branch connectivity, centralized firewalling, or multi-region reach.
The same spokes are added to an Azure Virtual Network Manager Network Group with a mesh policy.
Azure Virtual Network Manager builds direct peering links between the spokes, while vWAN continues to advertise and learn routes.

Result:

All VNets still benefit from vWAN’s global routing and on-premises integration.
Latency-critical east-west flows now travel the shortest path—one hop—as if the VNets were traditionally peered.
Rather than choosing one over the other, organizations can leverage both vWAN and Azure Virtual Network Manager together, as the combination enhances the strengths of each service.

Performance illustration

Spoke-to-Spoke Communication with Virtual WAN without Azure Virtual Network Manager mesh:

Spoke-to-Spoke Communication with Virtual WAN with Azure Virtual Network Manager mesh:

Observability & protection

NSG flow logs – granular packet logs on every peered VNet.
Azure Virtual Network Manager admin rules – org-wide guardrails that trump local NSGs.
Azure Monitor + SIEM – route flow logs to Log Analytics, Sentinel, or third-party SIEM for threat detection.
Layered design – hub firewalls inspect north-south traffic; NSGs plus admin rules secure east-west flows.

Putting it all together

Virtual WAN offers fully managed global connectivity, simplifying the integration of branch offices and on-premises infrastructure into your Azure environment.
Azure Virtual Network Manager mesh establishes direct communication paths between spoke VNets, making it ideal for workloads requiring high throughput or minimal latency in east-west traffic patterns.
When combined, these services provide architects with granular control over traffic routing. Each flow can be directed through hub services when needed or routed directly between spokes for optimal performance—all without re-architecting your network or creating additional management complexity.

By pairing Azure Virtual Network Manager’s group-based mesh with VWAN’s managed hubs, you get the best of both worlds: worldwide reach, centralized security, and single-hop performance where it counts.

Delivering web applications over IPv6

Marc de Droog — Fri, 14 Nov 2025 13:57:25 GMT

The IPv4 address space pool has been exhausted for some time now, meaning there is no new public address space available for allocation from Internet Registries. The internet continues to run on IPv4 through technical measures such as Network Address Translation (NAT) and Carrier Grade NAT, and reallocation of address space through IPv4 address space trading.

IPv6 will ultimately be the dominant network protocol on the internet, as IPv4 life-support mechanisms used by network operators, hosting providers and ISPs will eventually reach the limits of their scalability. Mobile networks are already changing to IPv6-only APNs; reachability of IPv4-only destinations from these mobile network is through 6-4 NAT gateways, which sometimes causes problems.

Client uptake of IPv6 is progressing steadily. Google reports 49% of clients connecting to its services over IPv6 globally, with France leading at 80%.

IPv6 client access measured by Google:

Meanwhile, countries around the world are requiring IPv6 reachability for public web services. Examples are the United States, European Union member states among which the Netherlands and Norway, and India, and Japan.

IPv6 adoption per country measured by Google:

Entities needing to comply with these mandates are looking at Azure's networking capabilities for solutions. Azure supports IPv6 for both private and public networking, and capabilities have developed and expanded over time.

This article discusses strategies to build and deploy IPv6-enabled public, internet-facing applications that are reachable from IPv6(-only) clients.

Azure Networking IPv6 capabilities

Azure's private networking capabilities center on Virtual Networks (VNETs) and the components that are deployed within. Azure VNETs are IPv4/IPv6 dual stack capable: a VNET must always have IPv4 address space allocated, and can also have IPv6 address space. Virtual machines in a dual stack VNET will have both an IPv4 and an IPv6 address from the VNET range, and can be behind IPv6 capable External- and Internal Load Balancers. VNETs can be connected through VNET peering, which effectively turns the peered VNETs into a single routing domain. It is now possible to peer only the IPv6 address spaces of VNETs, so that the IPv4 space assigned to VNETs can overlap and communication across the peering is over IPv6. The same is true for connectivity to on-premise over ExpressRoute: the Private Peering can be enabled for IPv6 only, so that VNETs in Azure do not have to have unique IPv4 address space assigned, which may be in short supply in an enterprise.

Not all internal networking components are IPv6 capable yet. Most notable exceptions are VPN Gateway, Azure Firewall and Virtual WAN; IPv6 compatibility is on the roadmap for these services, but target availability dates have not been communicated.

But now let's focus on Azure's externally facing, public, network services. Azure is ready to let customers publish their web applications over IPv6.

IPv6 capable externally facing network services include:

- Azure Front Door

- Application Gateway

- External Load Balancer

- Public IP addresses and Public IP address prefixes

- Azure DNS

- Azure DDOS Protection

- Traffic Manager

- App Service (IPv6 support is in public preview)

IPv6 Application Delivery

IPv6 Application Delivery refers to the architectures and services that enable your web application to be accessible via IPv6. The goal is to provide an IPv6 address and connectivity for clients, while often continuing to run your application on IPv4 internally.

Key benefits of adopting IPv6 in Azure include:

✅ Expanded Client Reach: IPv4-only websites risk being unreachable to IPv6-only networks. By enabling IPv6, you expand your reach into growing mobile and IoT markets that use IPv6 by default. Governments and enterprises increasingly mandate IPv6 support for public-facing services.

✅Address Abundance & No NAT: IPv6 provides a virtually unlimited address pool, mitigating IPv4 exhaustion concerns. This abundance means each service can have its own public IPv6 address, often removing the need for complex NAT schemes. End-to-end addressing can simplify connectivity and troubleshooting.

✅ Dual-Stack Compatibility: Azure supports dual-stack deployments where services listen on both IPv4 and IPv6. This allows a single application instance or endpoint to serve both types of clients seamlessly. Dual-stack ensures you don’t lose any existing IPv4 users while adding IPv6 capability.

✅Performance and Future Services: Some networks and clients might experience better performance over IPv6. Also, being IPv6-ready prepares your architecture for future Azure features and services as IPv6 integration deepens across the platform.

General steps to enable IPv6 connectivity for a web application in Azure are:

Plan and Enable IPv6 Addressing in Azure: Define an IPv6 address space in your Azure Virtual Network. Azure allows adding IPv6 address space to existing VNETs, making them dual-stack. A /56 segment for the VNET is recommended, /64 segment for subnets are required (Azure requires /64 subnets). If you have existing infrastructure, you might need to create new subnets or migrate resources, especially since older Application Gateway v1 instances cannot simply be “upgraded” to dual-stack.
Deploy or Update Frontend Services with IPv6: Choose a suitable Azure service (Application Gateway, External / Global Load Balancer, etc.) and configure it with a public IPv6 address on the frontend. This usually means selecting *Dual Stack* configuration so the service gets both an IPv4 and IPv6 public IP. For instance, when creating an Application Gateway v2, you would specify IP address type: DualStack (IPv4 & IPv6). Azure Front Door by default provides dual-stack capabilities with its global endpoints.
Configure Backends and Routing: Usually your backend servers or services will remain on IPv4. At the time of writing this in October 2025, Azure Application Gateway does not support IPv6 for backend pool addresses. This is fine because the frontend terminates the IPv6 network connection from the client, and the backend initiates an IPv4 connection to the backend pool or origin. Ensure that your load balancing rules, listener configurations, and health probes are all set up to route traffic to these backends. Both IPv4 and IPv6 frontend listeners can share the same backend pool. Azure Front Door does support IPv6 origins.
Update DNS Records: Publish a DNS AAAA record for your application’s host name, pointing to the new IPv6 address. This step is critical so that IPv6-only clients can discover the IPv6 address of your service. If your service also has an IPv4 address, you will have both A (IPv4) and AAAA (IPv6) records for the same host name. DNS will thus allow clients of either IP family to connect. (In multi-region scenarios using Traffic Manager or Front Door, DNS configuration might be handled through those services as discussed later).
Test IPv6 Connectivity: Once set up, test from an IPv6-enabled network or use online tools to ensure the site is reachable via IPv6. Azure’s services like Application Gateway and Front Door will handle the dual-stack routing, but it’s good to verify that content loads on an IPv6-only connection and that SSL certificates, etc., work over IPv6 as they do for IPv4.

Next, we explore specific Azure services and architectures for IPv6 web delivery in detail.

External Load Balancer - single region

Azure External Load Balancer (also known as Public Load Balancer) can be deployed in a single region to provide IPv6 access to applications running on virtual machines or VM scale sets. External Load Balancer acts as a Layer 4 entry point for IPv6 traffic, distributing connections across backend instances. This scenario is ideal when you have stateless applications or services that do not require Layer 7 features like SSL termination or path-based routing.

Key IPv6 Features of External Load Balancer:

- Dual-Stack Frontend: Standard Load Balancer supports both IPv4 and IPv6 frontends simultaneously. When configured as dual-stack, the load balancer gets two public IP addresses – one IPv4 and one IPv6 – and can distribute traffic from both IP families to the same backend pool.

- Zone-Redundant by Default: Standard Load Balancer is zone-redundant by default, providing high availability across Azure Availability Zones within a region without additional configuration.

- IPv6 Frontend Availability: IPv6 support in Standard Load Balancer is available in all Azure regions. Basic Load Balancer does not support IPv6, so you must use Standard SKU.

- IPv6 Backend Pool Support: While the frontend accepts IPv6 traffic, the load balancer will not translate IPv6 to IPv4. Backend pool members (VMs) must have private IPv6 addresses. You will need to add private IPv6 addressing to your existing VM IPv4-only infrastructure. This is in contrast to Application Gateway, discussed below, which will terminate inbound IPv6 network sessions and connect to the backend-end over IPv4.

- Protocol Support: Supports TCP and UDP load balancing over IPv6, making it suitable for web applications and APIs, but also for non-web TCP- or UDP-based services accessed by IPv6-only clients.

To set up an IPv6-capable External Load Balancer in one region, follow this high-level process:

Enable IPv6 on the Virtual Network: Ensure the VNET where your backend VMs reside has an IPv6 address space. Add a dual-stack address space to the VNET (e.g., add an IPv6 space like 2001:db8:1234::/56 to complement your existing IPv4 space). Configure subnets that are dual-stack, containing both IPv4 and IPv6 prefixes (/64 for IPv6).
Create Standard Load Balancer with IPv6 Frontend: In the Azure Portal, create a new Standard Load Balancer. During creation, configure the frontend IP with both IPv4 and IPv6 public IP addresses. Create or select existing Standard SKU public IP resources – one for IPv4 and one for IPv6.
Configure Backend Pool: Add your virtual machines or VM scale set instances to the backend pool. Note that your backend instances will need to have private IPv6 addresses, in addition to IPv4 addresses, to receive inbound IPv6 traffic via the load balancer.
Set Up Load Balancing Rules: Create load balancing rules that map frontend ports to backend ports. For web applications, typically map port 80 (HTTP) and 443 (HTTPS) from both the IPv4 and IPv6 frontends to the corresponding backend ports. Configure health probes to ensure only healthy instances receive traffic.
Configure Network Security Groups: Ensure an NSG is present on the backend VM's subnet, allowing inbound traffic from the internet to the port(s) of the web application. Inbound traffic is "secure by default" meaning that inbound connectivity from internet is blocked unless there is an NSG present that explicitly allows it.
DNS Configuration: Create DNS records for your application: an A record pointing to the IPv4 address and an AAAA record pointing to the IPv6 address of the load balancer frontend.

Outcome: In this single-region scenario, IPv6-only clients will resolve your application's hostname to an IPv6 address and connect to the External Load Balancer over IPv6.

Example: Consider a web application running on a VM (or a VM scale set) behind an External Load Balancer in Sweden Central. The VM runs the Azure Region and Client IP Viewer containerized application exposed on port 80, which displays the region the VM is deployed in and the calling client's IP address. The load balancer's front-end IPv6 address has a DNS name of ipv6webapp-elb-swedencentral.swedencentral.cloudapp.azure.com. When called from a client with an IPv6 address, the application shows its region and the client's address.

Limitations & Considerations:

- Standard SKU Required: Basic Load Balancer does not support IPv6. You must use Standard Load Balancer.

- Layer 4 Only: Unlike Application Gateway, External Load Balancer operates at Layer 4 (transport layer). It cannot perform SSL termination, cookie-based session affinity, or path-based routing. If you need these features, consider Application Gateway instead.

- Dual stack IPv4/IPv6 Backend required: Backend pool members must have private IPv6 addresses to receive inbound IPv6 traffic via the load balancer. The load balancer does not translate between the IPv6 frontend and an IPv4 backend.

- Outbound Connectivity: If your backend VMs need outbound internet access over IPv6, you need to configure an IPv6 outbound rule.

Global Load Balancer - multi-region

Azure Global Load Balancer (aka Cross-Region Load Balancer) provides a cloud-native global network load balancing solution for distributing traffic across multiple Azure regions. Unlike DNS-based solutions, Global Load Balancer uses anycast IP addressing to automatically route clients to the nearest healthy regional deployment through Microsoft's global network.

Key Features of Global Load Balancer:
- Static Anycast Global IP: Global Load Balancer provides a single static public IP address (both IPv4 and IPv6 supported) that is advertised from all Microsoft WAN edge nodes globally. This anycast address ensures clients always connect to the nearest available Microsoft edge node without requiring DNS resolution.
- Geo-Proximity Routing: The geo-proximity load-balancing algorithm minimizes latency by directing traffic to the nearest region where the backend is deployed. Unlike DNS-based routing, there's no DNS lookup delay - clients connect directly to the anycast IP and are immediately routed to the best region.
- Layer 4 Pass-Through: Global Load Balancer operates as a Layer 4 pass-through network load balancer, preserving the original client IP address (including IPv6 addresses) for backend applications to use in their logic.
- Regional Redundancy: If one region fails, traffic is automatically routed to the next closest healthy regional load balancer within seconds, providing instant global failover without DNS propagation delays.

Architecture Overview: Global Load Balancer sits in front of multiple regional Standard Load Balancers, each deployed in different Azure regions. Each regional load balancer serves a local deployment of your application with IPv6 frontends. The global load balancer provides a single anycast IP address that clients worldwide can use to access your application, with automatic routing to the nearest healthy region.

Multi-Region Deployment Steps:

Deploy Regional Load Balancers: Create Standard External Load Balancers in multiple Azure regions (e.g. Sweden Central, East US2). Configure each with dual-stack frontends (IPv4 and IPv6 public IPs) and connect them to regional VM deployments or VM scale sets running your application.
Configure Global Frontend IP address: Create a Global tier public IPv6 address for the frontend, in one of the supported Global Load Balancer home regions . This becomes your application's global anycast address.
Create Global Load Balancer: Deploy the Global Load Balancer in the same home region. The home region is where the global load balancer resource is deployed - it doesn't affect traffic routing.
Add Regional Backends: Configure the backend pool of the Global Load Balancer to include your regional Standard Load Balancers. Each regional load balancer becomes an endpoint in the global backend pool. The global load balancer automatically monitors the health of each regional endpoint.
Set Up Load Balancing Rules: Create load balancing rules mapping frontend ports to backend ports. For web applications, typically map port 80 (HTTP) and 443 (HTTPS). The backend port on the global load balancer must match the frontend port of the regional load balancers.
Configure Health Probes: Global Load Balancer automatically monitors the health of regional load balancers every 5 seconds. If a regional load balancer's availability drops to 0, it is automatically removed from rotation, and traffic is redirected to other healthy regions.
DNS Configuration: Create DNS records pointing to the global load balancer's anycast IP addresses. Create both A (IPv4) and AAAA (IPv6) records for your application's hostname pointing to the global load balancer's static IPs.

Outcome: IPv6 clients connecting to your application's hostname will resolve to the global load balancer's anycast IPv6 address. When they connect to this address, the Microsoft global network infrastructure automatically routes their connection to the nearest participating Azure region. The regional load balancer then distributes the traffic across local backend instances. If that region becomes unavailable, subsequent connections are automatically routed to the next nearest healthy region.

Example: Our web application, which displays the region it is in, and the calling client's IP address, now runs on VMs behind External Load Balancers in Sweden Central and East US2. The External Load Balancer's front-ends are in the backend pool of a Global Load Balancer, which has a Global tier front-end IPv6 address. The front-end has an FQDN of `ipv6webapp-glb.eastus2.cloudapp.azure.com` (the region designation `eastus2` in the FQDN refers to the Global Load Balancer's "home region", into which the Global tier public IP must be deployed).

When called from a client in Europe, Global Load Balancer directs the request to the instance deployed in Sweden Central.

When called from a client in the US, Global Load Balancer directs the request to the instance deployed in US East 2.

Features:
- Client IP Preservation: The original IPv6 client address is preserved and available to backend applications, enabling IP-based logic and compliance requirements.
- Floating IP Support: Configure floating IP at the global level for advanced networking scenarios requiring direct server return or high availability clustering.
- Instant Scaling: Add or remove regional deployments behind the global endpoint without service interruption, enabling dynamic scaling for traffic events.
- Multiple Protocol Support: Supports both TCP and UDP traffic distribution across regions, suitable for various application types beyond web services.

Limitations & Considerations:
- Home Region Requirement: Global Load Balancer can only be deployed in specific home regions, though this doesn't affect traffic routing performance.
- Public Frontend Only: Global Load Balancer currently supports only public frontends - internal/private global load balancing is not available.
- Standard Load Balancer Backends: Backend pool can only contain Standard Load Balancers, not Basic Load Balancers or other resource types.
- Same IP Version Requirement: NAT64 translation isn't supported - frontend and backend must use the same IP version (IPv4 or IPv6).
- Port Consistency: Backend port on global load balancer must match the frontend port of regional load balancers for proper traffic flow.
- Health Probe Dependencies: Regional load balancers must have proper health probes configured for the global load balancer to accurately assess regional health.

Comparison with DNS-Based Solutions:
Unlike Traffic Manager or other DNS-based global load balancing solutions, Global Load Balancer provides:
- Instant Failover: No DNS TTL delays - failover happens within seconds at the network level.
- True Anycast: Single IP address that works globally without client-side DNS resolution.
- Consistent Performance: Geo-proximity routing through Microsoft's backbone network ensures optimal paths.
- Simplified Management: No DNS record management or TTL considerations.

This architecture delivers global high availability and optimal performance for IPv6 applications through anycast routing, making it a good solution for latency-sensitive applications requiring worldwide accessibility with near-instant regional failover.

Application Gateway - single region

Azure Application Gateway can be deployed in a single region to provide IPv6 access to applications in that region. Application Gateway acts as the entry point for IPv6 traffic, terminating HTTP/S from IPv6 clients and forwarding to backend servers over IPv4. This scenario works well when your web application is served from one Azure region and you want to enable IPv6 connectivity for it.

Key IPv6 Features of Application Gateway (v2 SKU):
- Dual-Stack Frontend: Application Gateway v2 supports both [IPv4 and IPv6 frontends](https://learn.microsoft.com/en-us/azure/application-gateway/application-gateway-faq). When configured as dual-stack, the gateway gets two IP addresses – one IPv4 and one IPv6 – and can listen on both. (IPv6-only is not supported; IPv4 is always paired). IPv6 support requires Application Gateway v2, v1 does not support IPv6.
- No IPv6 on Backends: The backend pool must use IPv4 addresses. IPv6 addresses for backend servers are currently not supported. This means your web servers can remain on IPv4 internal addresses, simplifying adoption because you only enable IPv6 on the frontend.
- WAF Support: The Application Gateway Web Application Firewall (WAF) will inspect IPv6 client traffic just as it does IPv4.

Single Region Deployment Steps: To set up an IPv6-capable Application Gateway in one region, consider the following high-level process:

Enable IPv6 on the Virtual Network: Ensure the region’s VNET where the Application Gateway will reside has an IPv6 address space. Configure a subnet for the Application Gateway that is dual-stack (contains both an IPv4 subnet prefix and an IPv6 /64 prefix).
Deploy Application Gateway (v2) with Dual Stack Frontend: Create a new Application Gateway using the Standard_v2 or WAF_v2 SKU.
Populate Backend Pool: Ensure your backend pool (the target application servers or service) contains (DNS names pointing to) IPv4 addresses of your actual web servers. IPv6 addresses are not supported for backends.
Configure Listeners and Rules: Set up listeners on the Application Gateway for your site. When creating an HTTP(S) listener, you choose which frontend IP to use – you would create one listener for IPv4 address and one for IPv6. Both listeners can use the same domain name (hostname) and the same underlying routing rule to your backend pool.
Testing and DNS: After the gateway is deployed and configured, note the IPv6 address of the frontend (you can find it in the Gateway’s overview or in the associated Public IP resource). Update your application’s DNS records: create an AAAA record pointing to this IPv6 address (and update the A record to point to the IPv4 if it changed). With DNS in place, test the application by accessing it from an IPv6-enabled client or tool.

Outcome: In this single-region scenario, IPv6-only clients will resolve your website’s hostname to an IPv6 address and connect to the Application Gateway over IPv6. The Application Gateway then handles the traffic and forwards it to your application over IPv4 internally. From the user perspective, the service now appears natively on IPv6. Importantly, this does not require any changes to the web servers, which can continue using IPv4.

Application Gateway will include the source IPv6 address in an X-Forwarded-For header, so that the backend application has visibility of the originating client's address.

Example: Our web application, which displays the region it is deployed in and the calling client's IP address, now runs on a VM behind Application Gateway in Sweden Central. The front-end has an FQDN of `ipv6webapp-appgw-swedencentral.swedencentral.cloudapp.azure.com`.

Application Gateway terminates the IPv6 connection from the client and proxies the traffic to the application over IPv4. The client's IPv6 address is passed in the X-Forwarded-For header, which is read and displayed by the application.

Calling the application's API endpoint at `/api/region` shows additional detail, including the IPv4 address of the Application Gateway instance that initiates the connection to the backend, and the original client IPv6 address (with the source port number appended) preserved in the X-Forwarded-For header.

{
  "region": "SwedenCentral",
  "clientIp": "2001:1c04:3404:9500:fd9b:58f4:1fb2:db21:60769",
  "xForwardedFor": "2001:1c04:3404:9500:fd9b:58f4:1fb2:db21:60769",
  "remoteAddress": "::ffff:10.1.0.4",
  "isPrivateIP": false,
  "expressIp": "2001:1c04:3404:9500:fd9b:58f4:1fb2:db21:60769",
  "connectionInfo": {
    "remoteAddress": "::ffff:10.1.0.4",
    "remoteFamily": "IPv6",
    "localAddress": "::ffff:10.1.1.68",
    "localPort": 80
  },
  "allHeaders": {
    "x-forwarded-for": "2001:1c04:3404:9500:fd9b:58f4:1fb2:db21:60769"
  },
  "deploymentAdvice": "Public IP detected successfully"
}

Limitations & Considerations:

- Application Gateway v1 SKUs are not supported for IPv6. If you have an older deployment on v1, you’ll need to migrate to v2.

- IPv6-only Application Gateway is not allowed. You must have IPv4 alongside IPv6 (the service must be dual-stack). This is usually fine, as dual-stack ensures all clients are covered.

- No IPv6 backend addresses: The backend pool must have IPv4 addresses.

- Management and Monitoring: Application Gateway logs traffic from IPv6 clients to Log Analytics (the client IP field will show IPv6 addresses).

- Security: Azure’s infrastructure provides basic DDoS protection for IPv6 endpoints just as for IPv4. However, it is highly recommended to deploy Azure DDoS Protection Standard: this provides enhanced mitigation tailored to your specific deployment. Consider using the Web Application Firewall function for protection against application layer attacks.

Application Gateway - multi-region

Mission-critical web applications should be deploy in multiple Azure regions, achieving higher availability and lower latency for users worldwide. In a multi-region scenario, you need a mechanism to direct IPv6 client traffic to the “nearest” or healthiest region. Azure Application Gateway by itself is a regional service, so to use it in multiple regions, we use Azure Traffic Manager for global DNS load balancing, or use Azure Front Door (covered in the next section) as an alternative. This section focuses on the Traffic Manager + Application Gateway approach to multi-region IPv6 delivery.

Azure Traffic Manager is a DNS-based load balancer that can distribute traffic across endpoints in different regions. It works by responding to DNS queries with the appropriate endpoint FQDN or IP, based on the routing method (Performance, Priority, Geographic) configured. Traffic Manager is agnostic to the IP version: it either returns CNAMEs, or AAAA records for IPv6 endpoints and A records for IPv4. This makes it suitable for routing IPv6 traffic globally.

Architecture Overview: Each region has its own dual-stack Application Gateway. Traffic Manager is configured with an endpoint entry for each region’s gateway. The application’s FQDN is now a domain name hosted by Traffic Manager such as ipv6webapp.traffimanager.net, or a CNAME that ultimately points to it.

DNS resolution will go through Traffic Manager, which decides which regional gateway’s FQDN to return. The client then connects directly to that Application Gateway’s IPv6 address, as follows:

1. DNS query: Client asks for ipv6webapp.trafficmanager.net, which is hosted in a Traffic Manager profile.
2. Traffic Manager decision: Traffic Manager sees an incoming DNS request and chooses the best endpoint (say, Sweden Central) based on routing rules (e.g., geographic proximity or lowest latency).
3. Traffic Manager response: Traffic Manager returns the FQDN of the Sweden Central Application Gateway to the client.
4. DNS Resolution: The client resolves regional FQDN and receives a AAAA response containing the IPv6 address.
5. Client connects: The client’s browser connects to the West Europe App Gateway IPv6 address directly. The HTTP/S session is established via IPv6 to that regional gateway, which then handles the request.
6. Failover: If that region becomes unavailable, Traffic Manager’s health checks will detect it and subsequent DNS queries will be answered with the FQDN of the secondary region’s gateway.

Deployment Steps for Multi-Region with Traffic Manager:

Set up Dual-Stack Application Gateways in each region: Similar to the single-region case, deploy an Azure Application Gateway v2 in each desired region (e.g., one in North America, one in Europe). Configure the web application in each region, these should be parallel deployments serving the same content.
Configure a Traffic Manager Profile: In Azure Traffic Manager, create a profile and choose a routing method (such as Performance for nearest region routing, or Priority for primary/backup failover). Add endpoints for each region. Since our endpoints are Azure services with IPs, we can either use Azure endpoints (if the Application Gateways have Azure-provided DNS names) or External endpoints using the IP addresses. The simplest way is to use the Public IP resource of each Application Gateway as an Azure endpoint – ensure each App Gateway’s public IP has a DNS label (so it has a FQDN). Traffic Manager will detect those and also be aware of their IPs. Alternatively, use the IPv6 address as an External endpoint directly. Traffic Manager allows IPv6 addresses and will return AAAA records for them.
DNS Setup: Traffic Manager profiles have a FQDN (like ipv6webapp.trafficmanager.net). You can either use that as your service’s CNAME, or you can configure your custom domain to CNAME to the Traffic Manager profile.
Health Probing: Traffic Manager continuously checks the health of endpoints. When endpoints are Azure App Gateways, it uses HTTP/S probes to a specified URI path, to each gateway’s address. Make sure each App Gateway has a listener on the probing endpoint (e.g., a health check page) and that health probes are enabled.
Testing Failover and Distribution: Test the setup by querying DNS from different geographical locations (to see if you get the nearest region’s IP). Also simulate a region down (stop the App Gateway or backend) and observe if Traffic Manager directs traffic to the other region. Because DNS TTLs are involved, failover isn’t instant but typically within a couple of minutes depending on TTL and probe interval.

Considerations in this Architecture:

- Latency vs Failover: Traffic Manager as a DNS load balancer directs users at connect time, but once a client has an answer (IP address), it keeps sending to that address until the DNS record TTL expires and it re-resolves. This is fine for most web apps. Ensure the TTL in the Traffic Manager profile is not too high (the default is 30 seconds).

- IPv6 DNS and Connectivity: Confirm that each region’s IPv6 address is correctly configured and reachable globally. Azure’s public IPv6 addresses are globally routable. Traffic Manager itself is a global service and fully supports IPv6 in its decision-making.

- Cost: Using multiple Application Gateways and Traffic Manager incurs costs for each component (App Gateway is per hour + capacity unit, Traffic Manager per million DNS queries). This is a trade-off for high availability.

- Alternative: Azure Front Door: Azure Front Door is an alternative to the Traffic Manager + Application Gateway combination. Front Door can automatically handle global routing and failover at layer 7 without DNS-based limitations, offering potentially faster failover. Azure Front Door is discussed in the next section.

In summary, a multi-region IPv6 web delivery with Application Gateways uses Traffic Manager for global DNS load balancing. Traffic Manager will seamlessly return IPv6 addresses for IPv6 clients, ensuring that no matter where an IPv6-only client is, they get pointed to the nearest available regional deployment of your app. This design achieves global resiliency (withstand a regional outage) and low latency access, leveraging IPv6 connectivity on each regional endpoint.

Example: The global FQDN of our application is now ipv6webapp.trafficmanager.net and clients will use this FQDN to access the application regardless of their geographical location.
Traffic Manager will return the FQDN of one of the regional deployments, `ipv6webapp-appgw-swedencentral.swedencentral.cloudapp.azure.com` or `ipv6webappr2-appgw-eastus2.eastus2.cloudapp.azure.com` depending on the routing method configured, the health state of the regional endpoints and the client's location. Then the client resolves the regional FQDN through its local DNS server and connects to the regional instance of the application.

DNS resolution from a client in Europe:

Resolve-DnsName ipv6webapp.trafficmanager.net
Name                           Type   TTL   Section    NameHost
----                           ----   ---   -------    --------
ipv6webapp.trafficmanager.net  CNAME  59    Answer     ipv6webapp-appgw-swedencentral.swedencentral.cloudapp.azure.com
Name       : ipv6webapp-appgw-swedencentral.swedencentral.cloudapp.azure.com
QueryType  : AAAA
TTL        : 10
Section    : Answer
IP6Address : 2603:1020:1001:25::168

And from a client in the US:

Resolve-DnsName ipv6webapp.trafficmanager.net
Name Type TTL Section NameHost
---- ---- --- ------- --------
ipv6webapp.trafficmanager.net CNAME 60 Answer ipv6webappr2-appgw-eastus2.eastus2.cloudapp.azure.com
Name : ipv6webappr2-appgw-eastus2.eastus2.cloudapp.azure.com
QueryType : AAAA
TTL : 10
Section : Answer
IP6Address : 2603:1030:403:17::5b0

Azure Front Door

Azure Front Door is an application delivery network with built-in CDN, SSL offload, WAF, and routing capabilities. It provides a single, unified frontend distributed across Microsoft’s edge network. Azure Front Door natively supports IPv6 connectivity.

For applications that have users worldwide, Front Door offers advantages:
- Global Anycast Endpoint: Provides anycast IPv4 and IPv6 addresses, advertised out of all edge locations, with automatic A and AAAA DNS record support.
- IPv4 and IPv6 origin support: Azure Front Door supports both IPv4 and IPv6 origins (i.e. backends), both within Azure and externally (i.e. accessible over the internet).
- Simplified DNS: Custom domains can be mapped using CNAME records.
- Layer-7 Routing: Supports path-based routing and automatic backend health detection.
- Edge Security: Includes DDoS protection and optional WAF integration.

Front Door enables "cross-IP version" scenario's: a client can connect to the Front Door front-end over IPv6, and then Front Door can connect to an IPv4 origin. Conversely, an IPv4-only client can retrieve content from an IPv6 backend via Front Door.

Front Door preserves the client's source IP address in the X-Forwarded-For header.

Note: Front Door provides managed IPv6 addresses that are not customer-owned resources. Custom domains should use CNAME records pointing to the Front Door hostname rather than direct IP address references.

Private Link Integration

Azure Front Door Premium introduces Private Link integration, enabling secure, private connectivity between Front Door and backend resources, without exposing them to the public internet.

When Private Link is enabled, Azure Front Door establishes a private endpoint within a Microsoft-managed virtual network. This endpoint acts as a secure bridge between Front Door’s global edge network and your origin resources, such as Azure App Service, Azure Storage, Application Gateway, or workloads behind an internal load balancer.

Traffic from end users still enters through Front Door’s globally distributed POPs, benefiting from features like SSL offload, caching, and WAF protection. However, instead of routing to your origin over public, internet-facing, endpoints, Front Door uses the private Microsoft backbone to reach the private endpoint. This ensures that all traffic between Front Door and your origin remains isolated from external networks.

The private endpoint connection requires approval from the origin resource owner, adding an extra layer of control. Once approved, the origin can restrict public access entirely, enforcing that all traffic flows through Private Link.

Private Link integration brings following benefits:

- Enhanced Security: By removing public exposure of backend services, Private Link significantly reduces the risk of DDoS attacks, data exfiltration, and unauthorized access.
- Compliance and Governance: Many regulatory frameworks mandate private connectivity for sensitive workloads. Private Link helps meet these requirements without sacrificing global availability.
- Performance and Reliability: Traffic between Front Door and your origin travels over Microsoft’s high-speed backbone network, delivering low latency and consistent performance compared to public internet paths.
- Defense in Depth: Combined with Web Application Firewall (WAF), TLS encryption, and DDoS protection, Private Link strengthens your security posture across multiple layers.
- Isolation and Control: Resource owners maintain control over connection approvals, ensuring that only authorized Front Door profiles can access the origin.
- Integration with Hybrid Architectures: For scenarios involving AKS clusters, custom APIs, or workloads behind internal load balancers, Private Link enables secure connectivity without requiring public IPs or complex VPN setups.

Private Link transforms Azure Front Door from a global entry point into a fully private delivery mechanism for your applications, aligning with modern security principles and enterprise compliance needs.

Example: Our application is now placed behind Azure Front Door. We are combining a public backend endpoint and Private Link integration, to show both in action in a single example. The Sweden Central origin endpoint is the public IPv6 endpoint of the regional External Load Balancers and the origin in US East 2 is connected via Private Link integration

The global FQDN `ipv6webapp-d4f4euhnb8fge4ce.b01.azurefd.net` and clients will use this FQDN to access the application regardless of their geographical location. The FQDN resolves to Front Door's global anycast address, and the internet will route client requests to the nearest Microsoft edge from this address is advertised. Front Door will then transparently route the request to the nearest origin deployment in Azure. Although public endpoints are used in this example, that traffic will be route over the Microsoft network.

From a client in Europe:

Calling the application's api endpoint on `ipv6webapp-d4f4euhnb8fge4ce.b01.azurefd.net/api/region` shows some more detail.

{
  "region": "SwedenCentral",
  "clientIp": "2001:1c04:3404:9500:fd9b:58f4:1fb2:db21",
  "xForwardedFor": "2001:1c04:3404:9500:fd9b:58f4:1fb2:db21",
  "remoteAddress": "2a01:111:2053:d801:0:afd:ad4:1b28",
  "isPrivateIP": false,
  "expressIp": "2001:1c04:3404:9500:fd9b:58f4:1fb2:db21",
  "connectionInfo": {
    "remoteAddress": "2a01:111:2053:d801:0:afd:ad4:1b28",
    "remoteFamily": "IPv6",
    "localAddress": "2001:db8:1:1::4",
    "localPort": 80
  },
  "allHeaders": {
    "x-forwarded-for": "2001:1c04:3404:9500:fd9b:58f4:1fb2:db21",
    "x-azure-clientip": "2001:1c04:3404:9500:fd9b:58f4:1fb2:db21"
  },
  "deploymentAdvice": "Public IP detected successfully"
}

"remoteAddress": "2a01:111:2053:d801:0:afd:ad4:1b28" is the address from which Front Door sources its request to the origin.

From a client in the US:

The detailed view shows that the IP address calling the backend instance now is local VNET address. Private Link sources traffic coming in from a local address taken from the VNET it is in. The original client IP address is again preserved in the X-Forwarded-For header.

{
  "region": "eastus2",
  "clientIp": "2603:1030:501:23::68:55658",
  "xForwardedFor": "2603:1030:501:23::68:55658",
  "remoteAddress": "::ffff:10.2.1.5",
  "isPrivateIP": false,
  "expressIp": "2603:1030:501:23::68:55658",
  "connectionInfo": {
    "remoteAddress": "::ffff:10.2.1.5",
    "remoteFamily": "IPv6",
    "localAddress": "::ffff:10.2.2.68",
    "localPort": 80
  },
  "allHeaders": {
    "x-forwarded-for": "2603:1030:501:23::68:55658"
  },
  "deploymentAdvice": "Public IP detected successfully"
}

Conclusion

IPv6 adoption for web applications is no longer optional. It is essential as public IPv4 address space is depleted, mobile networks increasingly use IPv6 only and governments mandate IPv6 reachability for public services. Azure's comprehensive dual-stack networking capabilities provide a clear path forward, enabling organizations to leverage IPv6 externally without sacrificing IPv4 compatibility or requiring complete infrastructure overhauls.

Azure's externally facing services — including Application Gateway, External Load Balancer, Global Load Balancer, and Front Door — support IPv6 frontends, while Application Gateway and Front Door maintain IPv4 backend connectivity. This architecture allows applications to remain unchanged while instantly becoming accessible to IPv6-only clients.

For single-region deployments, Application Gateway offers layer-7 features like SSL termination and WAF protection. External Load Balancer provides high-performance layer-4 distribution. Multi-region scenarios benefit from Traffic Manager's DNS-based routing combined with regional Application Gateways, or the superior performance and failover capabilities of Global Load Balancer's anycast addressing.

Azure Front Door provides global IPv6 delivery with edge optimization, built-in security, and seamless failover across Microsoft's network. Private Link integration allows secure global IPv6 distribution while maintaining backend isolation.

The transition to IPv6 application delivery on Azure is straightforward: enable dual-stack addressing on virtual networks, configure IPv6 frontends on load balancing services, and update DNS records. With Application Gateway or Front Door, backend applications require no modifications. These Azure services handle the IPv4-to-IPv6 translation seamlessly. This approach ensures both immediate IPv6 accessibility and long-term architectural flexibility as IPv6 adoption accelerates globally.

Extending Layer-2 (VXLAN) networks over Layer-3 IP network

SaravSubramanian — Mon, 10 Nov 2025 17:29:05 GMT

Introduction

Virtual Extensible LAN (VXLAN) is a network virtualization technology that encapsulates Layer-2 Ethernet frames inside Layer-3 UDP/IP packets. In essence, VXLAN creates a logical Layer-2 overlay network on top of an IP network, allowing Ethernet segments (VLANs) or underlay IP packet to be stretched across routed infrastructures. A key advantage is scale: VXLAN uses a 24-bit segment ID (VNI) instead of the 12-bit VLAN ID, supporting around 16 million isolated networks versus the 4,094 VLAN limit. This makes VXLAN ideal for large cloud data centers and multi-tenant environments that demand many distinct network segments.

VXLAN’s Layer-2 overlays bring flexibility and mobility to modern architectures. Because VXLAN tunnels can span multiple Layer-3 domains, organizations can extend VLANs across different sites or subnets – for example, creating a tunnel that extends over two data centers over an IP WAN as long as underlying tunnel IP is reachable. This enables seamless workload mobility and disaster recovery: also helps virtual machines or applications can move between physical locations without changing IP addresses, since they remain in the same virtual L2 network. The overlay approach also decouples the logical network from the physical underlay, meaning you can run your familiar L2 segments over any IP routing infrastructure while leveraging features like equal-cost multi-path (ECMP) load balancing and avoiding large spanning-tree domains. In short, VXLAN combines the best of both worlds – the simplicity of Layer-2 adjacency with the scalability of Layer-3 routing – making it a foundational tool in cloud networking and software-defined data centers.

Layer-2 VXLAN overlay on a Layer-3 IP network allows customers or edge networks to stretch Ethernet (VLAN) segments across geographically distributed sites using an IP backbone. This approach preserves VLAN tags end-to-end and enables flexible segmentation across locations without needing an extend or continuous Layer-2 network in the core. It also helps hide or avoid the underlying IP network complexities.

However, it’s crucial to account for MTU overhead (VXLAN adds ~50 bytes of header) so that the overlay’s VLAN MTU is set smaller than the underlay IP MTU – otherwise fragmentation or packet loss can occur. Additionally, because VXLAN doesn’t inherently signal link status, implementing Bidirectional Forwarding Detection (BFD) on the VXLAN interfaces provides rapid detection of neighbor failures, ensuring quick rerouting or recovery when a tunnel endpoint goes down.

VXLAN overlay use case and benefits

VXLAN is a standard protocol (IETF RFC 7348) that can encapsulate Layer-2 Ethernet frames into Layer-3 UDP/IP packets. By doing so, VXLAN creates an L2 overlay network on top of an L3 underlay. The VXLAN tunnel endpoints (VTEPs), which can be routers, switches, or hosts, wrap the original Ethernet frame (including its VLAN tag) with an IP/UDP header plus a VXLAN header, then send it through the IP network. The default UDP port for VXLAN is 4789. This mechanism offers several key benefits:

Preserves VLAN Tags and L2 Segmentation: The entire Ethernet frame is carried across, so the original VLAN ID (802.1Q tag) is maintained end-to-end through the tunnel. Even if an extra tag is added at the ingress for local tunneling, the customer’s inner VLAN tag remains intact across the overlay. This means a VLAN defined at one site will be recognized at the other site as the same VLAN, enabling seamless L2 adjacency. In practice, VXLAN can transport multiple VLANs transparently by mapping each VLAN or service to a VXLAN Network Identifier (VNI).
Flexible network segmentation at scale: VXLAN uses a 24-bit VNI (VXLAN Network ID), supporting about 16 million distinct segments, far exceeding the 4094 VLAN limit of traditional 802.1Q networks. This gives architects freedom to create many isolated L2 overlay networks (for multi-tenant scenarios, application tiers, etc.) over a shared IP infrastructure. Geographically distributed sites can share the same VLANs and broadcast domain via VXLAN, without the WAN routers needing any VLAN configurations. The IP/MPLS core only sees routed VXLAN packets, not individual VLANs, simplifying the underlay configuration.
No need for end-to-end VLANs in underlay: Traditional solutions to extend L2 might rely on methods like MPLS/VPLS or long ethernet trunk lines, which often require configuring VLANs across the WAN and can’t scale well. In a VXLAN overlay, the intermediate L3 network remains unaware of customer VLANs, and you don’t need to trunk VLANs across the WAN. Each site’s VTEP encapsulates and decapsulates traffic, so the core routers/switches just forward IP/UDP packets. This isolation improves scalability and stability—core devices don’t carry massive MAC address tables or STP domains from all sites. It also means the underlay can use robust IP routing (OSPF, BGP, etc.) with ECMP, rather than extending spanning-tree across sites. In short, VXLAN lets you treat the WAN like an IP cloud while still maintaining Layer-2 connectivity between specific endpoints.
Multi-path and resilience: Since the overlay runs on IP, it naturally leverages IP routing features. ECMP in the underlay, for example, can load-balance VXLAN traffic across multiple links, something not possible with a single bridged VLAN spanning the WAN. The encapsulated traffic’s UDP header even provides entropy (via source port hashing) to help load-sharing on multiple paths. Furthermore, if one underlay path fails, routing protocols can reroute VXLAN packets via alternate paths without disrupting the logical L2 network. This increases reliability and bandwidth usage compared to a Layer-2 only approach.

Diagram: VXLAN Overlay Across a Layer-3 WAN – Below is a simplified illustration of two sites using a VXLAN overlay. “Site A” and “Site B” each have a local VLAN (e.g. VLAN 100) that they want to bridge across an IP WAN. The VTEPs at each site encapsulate the Layer-2 frames into VXLAN/UDP packets and send them over the IP network. Inside the tunnel, the original VLAN tag is preserved. In this example, a BFD session (red dashed line) runs between the VTEPs to monitor the tunnel’s health, as explained later.

Figure 1: Two sites (A and B) extend “VLAN 100” across an IP WAN using a VXLAN tunnel. The inner VLAN tag is preserved over the L3 network. A BFD keepalive (every 900ms) runs between the VXLAN endpoints to detect failures.

The practical effect of this design is that devices in Site A and Site B can be in the same VLAN and IP subnet, broadcast to each other, etc., even though they are connected by a routed network. For example, if Site A has a machine in VLAN 100 with IP 10.1.100.5/24 and Site B has another in VLAN 100 with IP 10.1.100.10/24, they can communicate as if on one LAN – ARP, switches, and VLAN tagging function normally across the tunnel.

MTU and overhead considerations

One critical consideration for deploying VXLAN overlays is handling the increased packet size due to encapsulation. A VXLAN packet includes additional headers on top of the original Ethernet frame: an outer IP header, UDP header, and VXLAN header (plus an outer Ethernet header on the WAN interface). This encapsulation adds approximately 50 bytes of overhead to each packet (for IPv4; about 70 bytes for IPv6).

In practical terms, if your original Ethernet frame was the typical 1500-byte payload (1518 bytes with Ethernet header and CRC, or 1522 with a VLAN tag), the VXLAN-encapsulated version will be ~1550 bytes. **The underlying IP network *must* accommodate these larger frames**, or you’ll get fragmentation or drops. Many network links by default only support 1500-byte MTUs, so without adjustments, a VXLAN carrying a full-sized VLAN packet would exceed that. Though modern networks runs jumbo frames (~9k), if the underlying encapsulated packet frames exceeds 8950 bytes it can create problems like control-plane failure (ex BGP session tear down) or fragmentation for data packet causing out of order packet.

Solution: Either raise the MTU on the underlay network or enforce a lower MTU on the overlay. Network architects generally prefer to increase the IP MTU of the core so the overlay can carry standard 1500-byte Ethernet frames unfragmented. For example, one vendor’s guide recommends configuring at least a 1550-byte MTU on all network segments to account for VXLAN’s ~50B overhead. In enterprise environments, it’s common to use “baby jumbo” frames (e.g. 1600 bytes) or full jumbo (9000 bytes) in the datacenter/WAN to accommodate various tunneling overheads. If increasing the underlay MTU is not possible (say, over an ISP that only supports 1500), then the VLAN MTU on the overlay should be reduced – for instance, set the VLAN interface MTU to 1450 bytes, so that even with the 50B VXLAN overhead the outer packet remains 1500 bytes. This prevents any IP fragmentation.

Why Fragmentation is Undesirable: VXLAN itself doesn’t include any fragmentation mechanism; it relies on the underlay IP to fragment if needed. But IP fragmentation can harm performance and some devices/drop policies might simply drop oversized VXLAN packets instead of fragmenting. In fact, certain implementations don’t support VXLAN fragmentation or Path MTU discovery on tunnels. The safe approach is to ensure no encapsulated packet ever exceeds the physical MTU. That means planning your MTUs end-to-end: make the core links slightly larger than the largest expected overlay packet.

Diagram: VXLAN Encapsulation and MTU Layering – The figure below illustrates the components of a VXLAN-encapsulated frame and how they contribute to packet size. The original Ethernet frame (yellow) with a VLAN tag is wrapped with a new outer Ethernet, IP, UDP, and VXLAN header . The extra headers add ~50 bytes. If the inner (yellow) frame was, say, 1500 bytes of payload plus 18 bytes Ethernet overhead, the outer packet becomes ~1568 bytes (including new headers and FCS). In practice the old FCS is replaced by a new one, so the net growth is ~50 bytes. The key takeaway: the IP transport must handle the total size.

Figure 2: Layered view of a VXLAN-encapsulated packet (not to scale). The original Ethernet frame with VLAN tag (yellow) is encapsulated by outer headers (blue/green/red/gray), resulting in ~50 bytes of overhead for IPv4. The outer packet must fit within the WAN MTU (e.g. 1518B if inner frame is 1468B) to avoid fragmentation.

In summary, ensure the IP underlay’s MTU is configured to accommodate the VXLAN overhead. If using standard 1500-byte MTUs on the WAN, set your overlay interfaces (VLAN SVIs or bridge MTUs) to around 1450 bytes. In many cases if possible, raising the WAN MTU to 1600 or using jumbo frames throughout is the best practice to provide ample headroom. Always test your end-to-end path with ping sweeps (e.g. using the DF-bit and varying sizes) to verify that the encapsulated packets aren’t being dropped due to MTU limits.

Neighbor failure detection with BFD

One challenge with overlays like VXLAN is that the logical link lacks immediate visibility into physical link status. If one end of the VXLAN tunnel goes down or the path fails, the other end’s VXLAN interface may remain “up” (since its own underlay interface is still up), potentially blackholing traffic until higher-level protocols notice. VXLAN itself doesn’t send continuous “link alive” messages to check the remote VTEP’s reachability.

To address this, network engineers deploy BFD on VXLAN endpoints. BFD is a lightweight protocol specifically designed for rapid failure detection independent of media or routing protocol. It works by two endpoints periodically sending very fast, small hello packets to each other (often every 50ms or less). If a few consecutive hellos are missed, BFD declares the peer down – often within <1 second, versus several seconds (or tens of seconds) with conventional detection.

Applying BFD to VXLAN: Many router and switch vendors support running BFD over a VXLAN tunnel or on the VTEP’s loopback adjacencies. When enabled, the two VTEPs will continuously ping each other at the configured interval. If the VXLAN tunnel fails (e.g. one site loses connectivity), BFD on the surviving side will quickly detect the loss of response. This can then trigger corrective actions: for instance, the BFD can generate logs for the logical interface or notify the routing protocol to withdraw routes via that tunnel. In designs with redundant tunnels or redundant VTEPs, BFD helps achieve sub-second failover – traffic can switch to a backup VXLAN tunnel almost immediately upon a primary failure. Even in a single-tunnel scenario, BFD gives an early alert to the network operator or applications that the link is down, rather than quietly dropping packets.

Example: If Site A and Site B have two VXLAN tunnels (primary and backup) connecting them, running BFD on each tunnel interface means that if the primary’s path goes down, BFD at Site A and B will detect it within milliseconds and inform the routing control-plane. The network can then shift traffic to the backup tunnel right away. Without BFD, the network might have to wait for a timeout (e.g. OSPF dead interval or even ARP timeouts) to realize the primary tunnel is dead, causing a noticeable outage.

BFD is protocol-agnostic – it can integrate with any routing protocols. For VXLAN, it’s purely a monitoring mechanism: lightweight and with minimal overhead on the tunnel. Its messages are small UDP packets (often on port 3784/3785) that can be sourced from the VTEP’s IP. The frequency is configurable based on how fast you need detection vs. how much overhead you can afford; common timers are 300ms with 3x multiplier (detect in ~1s) for moderate speeds, or even 50ms with 3x (150ms detection) for high-speed failover requirements.

Bottom line: Implementing BFD dramatically improves the reliability of a VXLAN-based L2 extension. Since VXLAN tunnels don’t automatically signal if a neighbor is unreachable, BFD acts as the heartbeat. Many platforms even allow BFD to directly influence interface state (for example, the VXLAN interface can be tied to go down when BFD fails) so that any higher-level protocols (like VRRP, dynamic routing, etc.) immediately react to the loss. This prevents lengthy outages and ensures the overlay network remains robust even over a complex WAN.

Conclusion

Deploying a Layer-2 VXLAN overlay across a Layer-3 WAN unlocks powerful capabilities: you can keep using familiar VLAN-based segmentation across sites while taking advantage of an IP network’s scalability and resilience. It’s a vendor-neutral solution widely supported in modern networking gear. By preserving VLAN tags over the tunnel, VXLAN makes it possible to stretch subnets and broadcast domains to remote locations for workloads that require Layer-2 adjacency. With the huge VNI address space, segmentation can scale for large enterprises or cloud providers well beyond traditional VLAN limits.

However, to realize these benefits successfully, careful attention must be paid to MTU and link monitoring. Always accommodate the ~50-byte VXLAN overhead by configuring proper MTUs (or adjusting the overlay’s MTU) – this prevents fragmentation and packet loss that can be very hard to troubleshoot after deployment. And since a VXLAN tunnel’s health isn’t apparent to switches/hosts by default, use tools like BFD to add fast failure detection, thereby avoiding black holes and improving convergence times. In doing so, you ensure that your stretched network is not only functional but also resilient and performant.

By following these guidelines – leveraging VXLAN for flexible L2 overlays, minding the MTU, and bolstering with BFD – network engineers can build a robust, wide-area Layer-2 extension that behaves nearly indistinguishably from a local LAN, yet rides on the efficiency and reliability of a Layer-3 IP backbone. Enjoy the best of both worlds: VLANs without borders, and an IP network without unnecessary constraints.

References: VXLAN technical overview and best practices from vendor documentation and industry sources have been used to ensure accuracy in the above explanations. This ensures the blog is grounded in real-world proven knowledge while remaining vendor-neutral and applicable to a broad audience of cloud and network professionals.

Simplify container network metrics filtering in Azure Container Networking Services for AKS

KhushbuP — Mon, 24 Nov 2025 18:49:33 GMT

We’re excited to introduce container network metrics filtering in Azure Container Networking Services for Azure Kubernetes Service (AKS) is now in public preview! This capability transforms how you manage network observability in Kubernetes clusters by giving you control over what metrics matter most.

Why excessive metrics are a problem (and how we’re fixing it)

In today’s large-scale, microservices-driven environments, teams often face metrics bloat, collecting far more data than they need. The result?

High storage & ingestion costs: Paying for data you’ll never use.
Cluttered dashboards: Hunting for critical latency spikes in a sea of irrelevant pod restarts.
Operational overhead: Slower queries, higher maintenance, and fatigue.

Our new filtering capability solves this by letting you define precise filters at the pod level using standard Kubernetes custom resources. You collect only what matters, before it ever reaches your monitoring stack.

Key Benefits: Signal Over Noise

Benefit	Your Gain
Fine-grained control	Filter by namespace or pod label. Target critical services and ignore noise.
Cost optimization	Reduce ingestion costs for Prometheus, Grafana, and other tools.
Improved observability	Cleaner dashboards and faster troubleshooting with relevant metrics only.
Dynamic & zero-downtime	Apply or update filters without restarting Cilium agents or Prometheus.

How it works: Filtering at the source

Unlike traditional sampling or post-processing, filtering happens at the Cilium agent level—inside the kernel’s data plane.

You define filters using the ContainerNetworkMetric custom resource to include or exclude metrics such as:

DNS lookups
TCP connection metrics
Flow metrics
Drop (error) metrics

This reduces data volume before metrics leave the host, ensuring your observability tools receive only curated, high-value data.

Example: Filtering flow metrics to reduce noise

Here’s a sample ContainerNetworkMetric CRD that filters only dropped flows from the traffic/http namespace and excludes flows from traffic/fortio pods:

apiVersion: acn.azure.com/v1alpha1
kind: ContainerNetworkMetric
metadata:
  name: container-network-metric
spec:
  filters:
    - metric: flow
      includeFilters:
        # Include only DROPPED flows from traffic namespace
        verdict:
          - "dropped"
        from:
          namespacedPod:
            - "traffic/http"
      excludeFilters:
        # Exclude traffic/fortio flows to reduce noise
        from:
          namespacedPod:
            - "traffic/fortio"

Before filtering:

After applying filters:

Getting started today

Ready to simplify your network observability?

Enable Advanced Container Networking Services: Make sure Advanced Container Networking Services is enabled on your AKS cluster.
Define Your Filter: Apply the ContainerNetworkMetric CRD with your include/exclude rules.
Validate: Check your settings via ConfigMap and Cilium agent logs.
See the Impact: Watch ingestion costs drop and dashboards become clearer!

👉 Learn more in the Metrics Filtering Guide.

Try the public preview today and take control of your container network metrics.

Layer 7 Network Policies for AKS: Now Generally Available for Production Security and Observability!

KhushbuP — Sat, 08 Nov 2025 03:12:57 GMT

We are thrilled to announce that Layer 7 (L7) Network Policies for Azure Kubernetes Service (AKS), powered by Cilium and Advanced Container Networking Services (ACNS), has reached General Availability (GA)!

The journey from public preview to GA signifies a critical step: L7 Network Policies are now fully supported, highly optimized, and ready for your most demanding, mission-critical production workloads.

A Practical Example: Securing a Multi-Tier Retail Application

Let's walk through a common production scenario. Imagine a standard retail application running on AKS with three core microservices:

frontend-app: Handles user traffic and displays product information.
inventory-api: A backend service that provides product stock levels. It should be read-only for the frontend.
payment-gateway: A highly sensitive service that processes transactions. It should only accept POST requests from the frontend to a specific endpoint.

The Security Challenge: A traditional L4 policy would allow the frontend-app to talk to the inventory-api on its port, but it couldn't prevent a compromised frontend pod from trying to exploit a potential vulnerability by sending a DELETE or POST request to modify inventory data.

The L7 Policy Solution: With GA L7 policies, you can enforce the Principle of Least Privilege at the application layer. Here's how you would protect the inventory-api:

apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: protect-inventory-api
spec:
  endpointSelector:
    matchLabels:
      app: inventory-api
  ingress:
  - fromEndpoints:
    - matchLabels:
        app: frontend-app
    toPorts:
    - ports:
      - port: "8080" # The application port
        protocol: TCP
      rules:
        http:
        - method: "GET"            # ONLY allow the GET method
          path: "/api/inventory/.*"  # For paths under /api/inventory/

The Outcome:

Allowed: A legitimate request from the frontend-app (GET /api/inventory/item123) is seamlessly forwarded.
Blocked: Assuming frontend-app is compromised, any malicious request (like DELETE /api/inventory/item123) originating from it is blocked at the network layer. This Zero Trust approach protects the inventory-api service from the threat, regardless of the security state of the source service.

This same principle can be applied to protect the payment-gateway, ensuring it only accepts POST requests to the /process-payment endpoint, and nothing else.

Beyond L7: Supporting Zero Trust with Enhanced Security

In addition, toL7 application-level policies to ensure Zero Trust, we support Layer 3/4 network security and advanced egress controls like Fully Qualified Domain Name (FQDN) filtering.

This comprehensive approach allows administrators to:

Restrict Outbound Connections (L3/L4 & FQDN): Implement strict egress control by ensuring that workloads can only communicate with approved external services. FQDN filtering is crucial here, allowing pods to connect exclusively to trusted external domains (e.g., www.trusted-partner.com), significantly reducing the risk of data exfiltration and maintaining compliance. To learn more, visit the FQDN Filtering Overview.
Enforce Uniform Policy Across the Cluster (CCNP): Extend protections beyond individual namespaces. By defining security measures as a Cilium Clusterwide Network Policy (CCNP), thanks to its General Availability (GA), administrators can ensure uniform policy enforcement across multiple namespaces or the entire Kubernetes cluster, simplifying management and strengthening the overall security posture of all workloads. To learn

CCNP Example: L4 Egress Policy with FQDN Filtering

This policy ensures that all pods across the cluster (CiliumClusterwideNetworkPolicy) are only allowed to establish outbound connections to the domain *.example.com on the standard web ports (80 and 443).

apiVersion: cilium.io/v2
kind: CiliumClusterwideNetworkPolicy
metadata:
  name: allow-egress-to-example-com
spec:
  endpointSelector: {} # Applies to all pods in the cluster
  egress:
    - toFQDNs:
        - matchPattern: "*.example.com" # Allows access to any subdomain of example.com
      toPorts:
        - ports:
            - port: "443"
              protocol: TCP
            - port: "80"
              protocol: TCP

Operational Excellence: Observability You Can Trust

A secure system must be observable. With GA, the integrated visibility of your L7 traffic is production ready.

In our example above, the blocked DELETE request isn't silent. It is immediately visible in your Azure Managed Grafana dashboards as a "Dropped" flow, attributed directly to the protect-inventory-api policy. This makes security incidents auditable and easy to diagnose, enabling operations teams to detect misconfigurations or threats in real time. Below is a sample dashboard layout screenshot.

Next Steps: Upgrade and Secure Your Production!

We encourage you to enable L7 Network Policies on your AKS clusters and level up your network security controls for containerized workloads. We value your feedback as we continue to develop and improve this feature. Please refer to the Layer 7 Policy Overview for more information and visit How to Apply L7 Policy for an example scenario.

Azure Networking Blog articles

Introducing the Container Network Insights Agent for AKS: Now in Public Preview

The Challenge

The solution: Container Network Insights Agent

What makes the container network insights agent different

Where the container network insights agent fits

Safe by Design

Get Started

Enabling fallback to internet for Azure Private DNS Zones in hybrid architectures

A demonstration of Virtual Network TAP

Connecting an ExpressRoute circuit to Megaport Virtual Edge

Create the Expressroute Circuit

Create MVE and ExpressRoute connections

Configure Private Peering

Configure Cisco IOS

Announcing public preview: Cilium mTLS encryption for Azure Kubernetes Service

Why Cilium mTLS encryption matters

Architecture and design: How Cilium mTLS works

Cilium agent: Transparent traffic interception and wiring

Ztunnel: Node‑level mTLS enforcement

SPIRE: Workload identity and trust

End‑to‑End workflow example

Workload enrolment and scope

Getting started in AKS

Looking ahead

Azure Front Door: Resiliency Series – Part 2: Faster recovery (RTO)

Why recovery at edge scale is deceptively hard

Persisting ‘validated configurations’ across restarts

Making recovery scale with active traffic, not total tenants

Continuous validation through Game Days

Closing

ExpressRoute Gateway Microsoft initiated migration

Objective

Backend migration overview

Backend process details

This image uses an example ExpressRoute gateway named ERGW-A with two connections (Conn-A and LAconn) in the resource group RGA.

Portal walkthrough

Validate stage

Prepare stage

Deployment status

Deployment verification

Gateway tag

Migrate stage

Commit stage

Frequently asked questions

Unlock outbound traffic insights with Azure StandardV2 NAT Gateway flow logs

Recommended Outbound Connectivity

What are flow logs?

Why should I use flow logs?

Workflow of troubleshooting with flow logs

Conclusion

Have more questions?

Data Center Quantized Congestion Notification: Scaling congestion control for RoCE RDMA in Azure

Why congestion control matters in RDMA networks

How DCQCN works

Interoperability challenges across different hardware generations

DCQCN tuning: Achieving fairness and performance

Real-world results

Conclusion

Azure Front Door: Implementing lessons learned following October outages

Abhishek Tiwari, Vice President of Engineering, Azure NetworkingAmit Srivastava, Principal PM Manager, Azure NetworkingVarun Chawla, Partner Director of Engineering

Introduction

October incidents: recap and key learnings

How our configuration propagation works

Customer configuration processing in Azure Front Door data plane stack

Processing flow for each configuration update:

Configuration propagation safeguards

Strengthening configuration generation safety

Preventing incompatible configurations from being propagated

Data plane resiliency

Decoupling data plane from configuration changes

Introducing Food Taster: strengthening config propagation resiliency

Closing

Azure Networking 2025: Powering cloud innovation and AI at global scale

Network Detection and Response (NDR) in Financial Services

The role of NDR in PCI DSS Compliance

Logging & Monitoring

Intrusion Detection

Network Segmentation and Scope Reduction

Secure Network Traffic and Encryption

Abhishek Tiwari, Vice President of Engineering, Azure Networking
Amit Srivastava, Principal PM Manager, Azure Networking
Varun Chawla, Partner Director of Engineering