Inter-Hub Connectivity Using Azure Route Server
By Mays_Algebary shruthi_nair As your Azure footprint grows with a hub-and-spoke topology, managing User-Defined Routes (UDRs) for inter-hub connectivity can quickly become complex and error-prone. In this article, we’ll explore how Azure Route Server (ARS) can help streamline inter-hub routing by dynamically learning and advertising routes between hubs, reducing manual overhead and improving scalability. Baseline Architecture The baseline architecture includes two Hub VNets, each peered with their respective local spoke VNets as well as with the other Hub VNet for inter-hub connectivity. Both hubs are connected to local and remote ExpressRoute circuits in a bowtie configuration to ensure high availability and redundancy, with Weight used to prefer the local ExpressRoute circuit over the remote one. To maintain predictable routing behavior, the VNet-to-VNet configuration on the ExpressRoute Gateway should be disabled. Note: Adding ARS to an existing Hub with Virtual Network Gateway will cause downtime that expect to last 10 minutes. Scenario 1: ARS and NVA Coexist in the Hub Option A: Full Traffic Inspection ARS and NVA Coexist in the Hub In this scenario, ARS is deployed in each Hub VNet, alongside the Network Virtual Appliances (NVAs). NVA1 in Region1 establishes BGP peering with both the local ARS (ARS1) and the remote ARS (ARS2). Similarly, NVA2 in Region2 peers with both ARS2 (local) and ARS1 (remote). Let’s break down what each BGP peering relationship accomplishes. For clarity, we’ll focus on Region1, though the same logic applies to Region2: NVA1 Peering with Local ARS1 Through BGP peering with ARS1, NVA1 dynamically learns the prefixes of Spoke1 and Spoke2 at the OS level, eliminating the need to manually configure these routes. The same applies for NVA2 learning Spoke3 and Spoke4 prefixes via its BGP peering with ARS2. NVA1 Peering with Remote ARS2 When NVA1 peers with ARS2, the Spoke1 and Spoke2 prefixes are propagated to ARS2. ARS2 then injects these prefixes into NVA2 at both the NIC level with NVA1 as the next hop, and at the OS level. This mechanism removes the need for UDRs on the NVA subnets to enable inter-hub routing. Additionally, ARS2 advertises the Spoke1 and Spoke2 prefixes to both ExpressRoute circuits (EXR2 and EXR1 due to bowtie configuration) via GW2, making them reachable from on-premises through either EXR1 or EXR2. 👉Important: To ensure that ARS2 accepts and propagates Spoke1/Spoke2 prefixes received via NVA1, AS-Override must be enabled. Without AS-Override, BGP loop prevention will block these routes at ARS2, since both ARS1 and ARS2 use the default ASN 65515, and ARS2 will consider the route as already originated locally. The same principle applies in reverse for Spoke3 and Spoke4 prefixes being advertised from NVA2 to ARS1. Traffic Flow Inter-Hub Traffic: Spoke VNets are configured with UDRs that contain only a default route (0.0.0.0/0) pointing to the local NVA as the next hop. Additionally, the “Propagate Gateway Routes” setting should be set to False to ensure all traffic, whether East-West (intra-hub/inter-hub) or North-South (to/from internet), is forced through the local NVA for inspection. Local NVAs will have the next hop to the other region spokes injected at the NIC level by local ARS, pointing to the other region NVA, for example NVA2 will have next hop to Spoke1 and Spoke2 as NVA1 (10.0.1.4) and vice versa. Why are UDRs still needed on spokes if ARS handles dynamic routing? Even with ARS in place, UDRs are required to maintain control of the next hop for traffic inspection. For instance, if Spoke1 and Spoke2 do not have UDRs, they will learn the remote spoke prefixes (e.g., Spoke3/Spoke4) injected via ARS1, which received them from NVA2. This results in Spoke1/Spoke2 attempting to route traffic directly to NVA2, a path that is invalid, since the spokes don’t have the path to NVA2. The UDR ensures traffic correctly routes through NVA1 instead. On-Premises Traffic: To explain the on-premises traffic flow, we'll break it down into two directions: Azure to on-premises, and on-premises to Azure. Azure to On-Premises Traffic Flow: As previously noted, Spokes send all traffic, including traffic to on-premises, via NVA1 due to the default route in the UDR. NVA1 then routes traffic to the local ExpressRoute circuit, using Weight to prefer the local path over the remote. Note: While NVA1 learns on-premises prefixes from both local and remote ARSs at the OS level, this doesn’t affect routing decisions. The actual NIC-level route injection determines the next hop, ensuring traffic is sent via the correct path—even if the OS selects a different “best” route internally. The screenshot below from NVA1 shows four next hops to the on-premises network 10.2.0.0/16. These include the local ARS (ARS1: 10.0.2.5 and 10.0.2.4) and the remote ARS (ARS2: 10.1.2.5 and 10.1.2.4). On-Premises to Azure Traffic Flow In a bowtie ExpressRoute configuration, Azure VNet prefixes are advertised to on-premises through both local and remote ExpressRoute circuits. Because of this dual advertisement, the on-premises network must ensure optimal path selection when routing traffic to Azure. From Azure side, to maintain traffic symmetry, add UDRs at the GatewaySubnet (GW1 and GW2) with specific routes to the local Spoke VNets, using the local NVA as the next hop. This ensures return traffic flows back through the same path it entered. 👉How Does the ExpressRoute Edge Router Select the Optimal Path? You might ask: If Spoke prefixes are advertised by both GW1 and GW2, how does the ExpressRoute edge router choose the best path? (e.g., diagram below shows EXR1 learns Region1 prefixes from GW1 and GW2) Here’s how: Edge routers (like EXR1) receive the same Spoke prefixes from both gateways. However, these routes have different AS-Path lengths: - Routes from the local gateway (GW1) have a shorter AS-Path. - Routes from the remote gateway (GW2) have a longer AS-Path because NVA1’s ASN (e.g., 65001) is prepended twice as part of the AS-Override mechanism. As a result, the edge router (EXR1) will prefer the local path from GW1, ensuring efficient and predictable routing. For example: EXR1 receives Spoke1, Spoke2, and Hub1-VNet prefixes from both GW1 and GW2. But because the path via GW1 has a shorter AS-Path, EXR1 will select that as the best route. (Refer to the diagram below for a visual of the AS-Path difference). Final Traffic Flow: Option-A Insights: This design simplifies UDR configuration for inter-hub routing, especially useful when dealing with non-contiguous prefixes or operating across multiple hubs. For simplicity, we used a single NVA in each Hub-VNet while explaining the setup and traffic flow throughout this article. However, a high available (HA) NVA deployment is recommended. To maintain traffic symmetry in an HA setup, you’ll need to enable the next-hop IP feature when peering with Azure Route Server (ARS). When on-premises traffic inspection is required, the UDR setup in the GatewaySubnet becomes more complex as the number of Spokes increases. As your Azure network scales, keep in mind that Azure Route Server supports a maximum of 16 BGP peers per instance (as of the time writing this article). This limit can impact architectures involving multiple NVAs or hubs. Option B: Bypass On-Premises Inspection If on-premises traffic inspection is not required, NVAs can advertise a supernet prefix summarizing the local Spoke VNets to the remote ARS. This approach provides granular control over which traffic is routed through the NVA and eliminates the need for BGP peering between the local NVA and local ARS. All other aspects of the architecture remain the same as described in Option A. For example, NVA2 can advertise the supernet 192.168.2.0/23 (supernet of Spoke3 and Spoke4) to ARS1. As a result, Spoke1 and Spoke2 will learn this route with NVA2 as the next hop. To ensure proper routing (as discussed earlier) and inter-hub inspection, you need apply a UDR in Spoke1 and Spoke2 that overrides this exact supernet prefix, redirecting traffic to NVA1 as the next hop. At the same time, traffic destined for on-premises will follow the system route through the local ExpressRoute gateway, bypassing NVA1 altogether. In this setup: UDRs on the Spokes should have "Propagate Gateway Routes" set to True. No UDRs are needed in the GatewaySubnet. 👉Can NVA2 Still Advertise Specific Spoke Prefixes? You might wonder: Can NVA2 still advertise specific prefixes (e.g., Spoke3 and Spoke4) learned from ARS2 to ARS1 instead of a supernet? Yes, this is technically possible, but it requires maintaining BGP peering between NVA2 and ARS2. However, this introduces UDR complexity in Spoke1 and Spoke2, as you'd need to manually override each specific prefix. This also defeats the purpose of using ARS for simplified route propagation, undermining the efficiency and scalability of the design. Bypass On-Premises Inspection Final Traffic Flow: Option B: Bypass on-premises inspection traffic flow Option-B Insights: This approach reduces the number of BGP peerings per ARS. Instead of maintaining two BGP sessions (local NVA and remote NVA) per Hub, you can limit it to just one, preserving capacity within ARS’s 8-peer limit for additional inter-hub NVA peerings. Each NVA should advertise a supernet prefix to the remote ARS. This can be challenging if your Spokes don’t use contiguous IP address spaces, as described in Option B. Scenario 2: ARS in the Hub and the NVA in Transit VNet In Scenario 1, we highlighted that when on-premises inspection is required, managing UDRs at the GatewaySubnet becomes increasingly complex as the number of Spoke VNets grows. This is due to the need for UDRs to include specific prefixes for each Spoke VNet. In this scenario, we eliminate the need to apply UDRs at the GatewaySubnet altogether. In this design, the NVA will be deployed in Transit VNet, where: Transit-VNet will be peered with local Spoke VNets and with the local Hub-VNet to enable intra-Hub and on-premises connectivity. Transit-VNet also peered with remote Transit VNets (e.g., Transit-VNet1 peered with Transit-VNet2) to handle inter-Hub connectivity through the NVAs. Additionally, Transit-VNets are peered with remote Hub-VNets, to establish BGP peering with the remote ARS. NVAs OS will need to add static routes for the local Spoke VNets prefixes, it can be specific or it can supernet prefix, which will later be advertised to ARSs over BGP Peering, then ARS will advertise it to on-premises via ExpressRoute. NVAs will BGP peer with local ARS and also with the remote ARS. To understand the reasoning behind this design, let’s take a closer look at the setup in Region1, focusing on how ARS and NVA are configured to connect to Region2. This will help illustrate both inter-hub and on-premises connectivity. The same concept applies in reverse from Region2 to Region1. Inetr-Hub: To enable NVA1 in Region1 to learn prefixes from Region2, NVA2 will configure static routes at the OS level for Spoke3 and Spoke4 (or their supernet prefix) and advertise them to ARS1 via remote BGP peering. As a result, these prefixes will be received by NVA1, both at the NIC level, with NVA2 as the next hop, and at the OS level for proper routing. Spoke1 and Spoke2 will have a UDR with a default route pointing to NVA1 as the next hop. For instance, when Spoke1 needs to communicate with Spoke3, the traffic will first route through NVA1. NVA1 will then forward the traffic to NVA2 using VNet peering between the two Hubs. A similar configuration will be applied in Region2, where NVA1 will configure static routes at the OS level for Spoke1 and Spoke2 (or their supernet prefix) and advertise them to ARS2 via remote BGP peering, as a result, these prefixes will be received by NVA2, both at the NIC level (injected by ARS2), with NVA1 as the next hop, and at the OS level for proper routing. Note: At the OS level, NVA1 learns Spoke3 and Spoke4 prefixes from both local and remote ARSs. However, the NIC-level route injection determines the actual next hop, so even if the OS selects a different best route, it won’t affect forwarding behavior. same applies to NVA2. On-Premises Traffic: To explain the on-premises traffic flow, we'll break it down into two directions: Azure to on-premises, and on-premises to Azure. Azure to On-Premises Traffic Flow: Spokes in Region1 route all traffic through NVA1 via a default route defined in their UDRs. Because of BGP peering between NVA1 and ARS1, ARS1 advertises the Spoke1 and Spoke2 (or their supernet prefix) to on-premises through ExpressRoute (EXR1). The Transit-VNet1 (hosting NVA1) is peered with Hub1-VNet, with “Use Remote Gateway” enabled. This allows NVA1 to learn on-premises prefixes from the local ExpressRoute gateway (GW1), and traffic to on-premises is routed through the local ExpressRoute circuit (EXR1) due to higher BGP Weight configuration. Note: At the OS level, NVA1 learns on-prem prefixes from both local and remote ARSs. However, the NIC-level route injection determines the actual next hop, so even if the OS selects a different best route, it won’t affect forwarding behavior. same applies to NVA2. On-Premises to Azure Traffic Flow: Through BGP peering with ARS1, NVA1 enables ARS1 to advertise Spoke1 and Spoke2 (or their supernet prefix) to both EXR1 and EXR2 circuits (due to the ExpressRoute bowtie setup). Additionally, due to BGP peering between NVA1 and ARS2, ARS2 also advertises Spoke1 and Spoke2 (or their supernet prefix) to EXR2 and EXR1 circuits. As a result, both ExpressRoute edge routers in Region1 and Region2 learn the same Spoke prefixes (or their supernet prefix) from both GW1 and GW2, with identical AS-Path lengths, as shown below. EXR1 learns Region1 Spokes's supernet prefixes from GW1 and GW2 This causes non-optimal inbound routing, where traffic from on-premises destined to Region1 Spokes may first land in Region2’s Hub2-VNet before traversing to NVA1 in Region1. However, return traffic from Spoke1 and Spoke2 will always exit through Hub1-VNet. To prevent suboptimal routing, configure NVA1 to prepend the AS path for Spoke1 and Spoke2 (or their supernet prefix) when advertising them to the remote ARS2. Likewise, ensure NVA2 prepends the AS path for Spoke3 and Spoke4 (or their supernet prefix) when advertising to ARS1. This approach helps maintain optimal routing under normal conditions and during ExpressRoute failover scenarios. Below diagram shows NVA1 is setting AS-Prepend for Spoke1 and Spoke2 supernet prefix when BGP peer with remote ARS (ARS1), same will apply for NVA2 when advertising Spoke3 and Spoke4 prefixes to ARS1. Final Traffic Flow: Full Inspection: Traffic flow when NVA in Transit-VNet Insights: This solution is ideal when full traffic inspection is required. Unlike Scenario 1 - Option A, it eliminates the need for UDRs in the GatewaySubnet. When ARS is deployed in a VNet (typically in Hub VNets), the VNet will be limited to 500 VNet peerings (as of the time writing this article). However, in this design, Spokes peer with the Transit-VNet instead of directly with the ARS VNet, allowing you to scale beyond the 500-peer limit by leveraging Azure Virtual Network Manager (AVNM) or submitting a support request. Some enterprise customers may encounter the 1,000-route advertisement limit on the ExpressRoute circuit from the ExpressRoute gateway. In traditional hub-and-Spoke designs, there's no native control over what is advertised to ExpressRoute. With this architecture, NVAs provide greater control over route advertisement to the circuit. For simplicity, we used a single NVA in each Hub-VNet while explaining the setup and traffic flow throughout this article. However, a high available (HA) NVA deployment is recommended. To maintain traffic symmetry in an HA setup, you’ll need to enable the next-hop IP feature when peering with Azure Route Server (ARS). This design does require additional VNet peerings, including: Between Transit-VNets (inter-region), Between Transit-VNets and local Spokes, and Between Transit-VNets and both local and remote Hub-VNets.
Migrating from MSEE Hairpin Routing to AVNM Mesh for Large-Scale VNet-to-VNet Connectivity
Introduction A common pattern in large Azure deployments is to route VNet-to-VNet traffic through Microsoft Enterprise Edge (MSEE) routers. This happens when spoke VNets in a hub-and-spoke topology communicate with each other by hairpinning through the ExpressRoute circuit: traffic exits the Azure data center, traverses MSEE, and re-enters the data center to reach the destination VNet. This pattern works, but it was not designed as the long-term connectivity model for east-west traffic. With Azure Virtual Network Manager (AVNM) mesh connectivity and recent scale improvements — including high-scale mesh up to 5,000 VNets and High-Scale Private Endpoints (HSPE) up to 20,000 Private Endpoints across connected VNets — enterprises can migrate to a direct, in-datacenter routing model that removes MSEE dependency for VNet-to-VNet traffic. This article explains why the migration is useful, what the new scale limits are, how to enable the required features, and how to execute the migration with minimal disruption. Who This Is For This migration is most relevant if you have 50 or more spoke VNets communicating east-west through ExpressRoute hairpin routing, are approaching VNet peering limits, want to reduce ExpressRoute utilization from internal traffic, or need a simpler centrally managed connectivity model. Even if most east-west flows must continue through a hub firewall for inspection, you can still simplify connectivity management for the flows allowed to go direct. Why MSEE Hairpin Routing for VNet-to-VNet Is Not Recommended When spoke VNets communicate through MSEE, traffic follows a suboptimal path: Spoke A → Hub VNet → ExpressRoute Gateway → MSEE → ExpressRoute Gateway → Hub VNet → Spoke B Single point of failure. MSEE becomes a shared dependency for east-west traffic. An MSEE outage or capacity constraint can affect every VNet pair in the topology. Lower-latency path. Traffic no longer needs to leave the data center and return for spoke-to-spoke communication. Direct mesh keeps east-west traffic on an in-datacenter path, which is generally lower latency than hairpinning through MSEE. Bandwidth constraints. MSEE circuits have finite bandwidth. Routing east-west traffic through them competes with north-south on-premises traffic and can saturate the circuit. Operational risk at scale. Large deployments place significant load on MSEE infrastructure, creating scalability, reliability, and operational concerns for environments with thousands of VNets. Why Manual Peering Was Not a Practical Alternative The obvious alternative — creating direct VNet peerings between every spoke pair — solves the MSEE dependency but introduces its own operational complexity: Combinatorial growth. Connecting N spokes requires N × (N - 1) / 2 peering relationships. For 100 spokes, that is 4,950 peerings. For 1,000 spokes, it is nearly 500,000. Management overhead. Each peering must be individually provisioned, monitored, and maintained. This increases drift, audit, and operational overhead. How AVNM Mesh Solves This AVNM mesh provides group-based connectivity. You define a set of VNets as a network group, apply a mesh connectivity configuration, and AVNM establishes bi-directional connectivity across all members. Traffic between meshed VNets stays within the Azure data center: no MSEE traversal, no hub hop, and no manual peering management. Define once, connect all. A single mesh configuration connects every VNet in the group to every other VNet. Centralized management. Add or remove VNets from the group; AVNM reconciles connectivity automatically. Direct spoke-to-spoke paths. Traffic flows directly between VNets, bypassing both the hub and MSEE. Dynamic membership. Use Azure Policy to auto-enroll new VNets based on tags or resource group conditions. How to Migrate Large Scale Topology High-scale mesh lets user migrate larger topology — up to 5,000 VNets and 20,000 Private Endpoints in a mesh. Scale — Standard Mesh vs. High-Scale Mesh Dimension Standard Mesh High-Scale Mesh VNets per mesh Up to 250 (soft limit, can request increase) Up to 5,000 Private Endpoints per mesh Up to 2,000 Up to 20,000 (with HSPE enabled) Private Endpoints per VNet Up to 1,000 Up to 5,000 (with HSPE enabled) Enabling High-Scale Private Endpoints (HSPE) As a mesh footprint grows, so does the number of Private Endpoints deployed across the connected VNets. The default platform limits — 1,000 Private Endpoints per VNet and 2,000 across connected VNets and mesh — can be reached quickly in large environments. Enabling HSPE raises these limits to 5,000 and 20,000, respectively. For large-scale mesh migrations, enable HSPE proactively if the environment is expected to grow toward standard Private Endpoint limits by following the steps below. Step 1 — Prepare Each VNet Ensure Private Endpoint Network Policies are set to Enabled or RouteTableEnabled on all subnets containing Private Endpoints. This is a prerequisite. Set the VNet-level property PrivateEndpointVNetPolicies to Basic. This activates HSPE on the VNet. Step 2 — Enable HSPE on the Mesh Configuration In the AVNM mesh connectivity configuration, enable the option for high-scale private endpoints. AVNM validates that all VNets in the mesh are HSPE-enabled. If any VNet is missing the configuration, the deployment is blocked with a clear error. Step 3 — Deploy Deploy the connectivity configuration. AVNM applies HSPE across the mesh. Behavior Changes When Enabling HSPE Brief connection reset. Enabling or disabling HSPE triggers a one-time, approximately 1-second connection reset for existing Private Endpoint connections in the VNet. Plan this during a maintenance window. Per-PE Bytes In / Out monitoring is no longer available. HSPE treats each Private Endpoint IP like any other IP in the VNet, which removes per-PE traffic counters. If you depend on per-PE metrics, evaluate alternatives before enabling. On-premises PE traffic billing changes. PE traffic originating from on-premises appears as an aggregate bill on the gateway VNet, not on the individual Private Endpoint resource. The total bill does not change. How to Avoid Downtime Mesh coexists with existing peerings. AVNM does not delete manually created peerings unless explicitly configured to do so. Traffic shifts automatically. Once mesh is deployed, spoke-to-spoke traffic routes directly. The MSEE hairpin path remains available if mesh is removed. No reconfiguration of hub components. Firewalls, gateways, and NVAs in the hub continue to function. North-south on-premises traffic still flows through the hub gateway. Rollback is simple while the legacy path remains. Keep the MSEE hairpin path in place during validation so affected VNets can fall back by removing them from the network group or undeploying the mesh configuration. Security and East-West Traffic Inspection A common concern is whether direct mesh connectivity bypasses hub firewalls or network virtual appliances. The answer depends on how inspection is enforced today. Mesh provides connectivity, not routing policy. If spoke subnets have UDRs that direct traffic through a hub NVA or firewall, those UDRs continue to apply and can keep inspected flows on the firewall path. Security Admin Rules provide centralized segmentation. For flows that do not require firewall inspection, AVNM Security Admin Rules can enforce network-level allow or deny policies across network groups. Use both where appropriate. Mesh can provide direct connectivity for approved flows while Security Admin Rules enforce segmentation boundaries where required. Recommendation: Before migrating, inventory which spoke-to-spoke flows currently traverses the firewall. Decide per flow whether to maintain inspection by keeping UDRs in place or allowing a direct mesh path by removing the UDR for that flow pair. Migration Order and Process The migration from MSEE hairpin routing to AVNM mesh is non-disruptive by design. Mesh connectivity overlays on top of existing peerings and takes routing precedence for east-west traffic. You do not need to tear down the existing hub-and-spoke topology first. Recommended Steps Design the mesh topology. Group VNets by region into mesh groups. If you expect more than 250 VNets per mesh, register the AllowHighScaleConnectedGroup feature in advance. Create a Network Manager and define Network Groups. Ensure that the AVNM scope covers all relevant subscriptions. Use static membership for initial migration or dynamic membership through Azure Policy for ongoing enrollment. Enable HSPE on every VNet in the mesh. Follow the HSPE enablement steps if you need to have more than 2,000 Pes in a mesh. Schedule the change during a maintenance window to account for the brief connection reset. Create the mesh connectivity configuration in AVNM. Select the network groups, enable mesh topology, enable high-scale private endpoints, and enable global mesh if cross-region connectivity is required. Deploy incrementally. Start with a pilot region or non-critical environment. Validate effective routes, spoke-to-spoke connectivity, spoke-to-hub-to-on-premises connectivity, Private Endpoint reachability, and expected VNet flow log patterns before expanding to production regions. Route Behavior During and After Migration During migration, mesh and MSEE can coexist. Mesh-connected VNets receive direct routes for mesh-connected destinations, while existing ExpressRoute gateway routes continue to serve on-premises destinations. UDRs still override system routes, so forced-tunneling and firewall inspection patterns remain in effect when UDRs are present. Mesh destinations. Traffic between mesh-connected VNets goes directly instead of hairpinning through MSEE when no UDR overrides the route. On-premises destinations. ExpressRoute continues to provide north-south connectivity to on-premises networks. Gateway transit. Spokes can continue to reach on-premises through the hub gateway when the design uses gateway transit. Infrastructure-as-Code Considerations If VNet peerings are managed through Terraform, Bicep, or ARM templates, treat AVNM mesh as the new source of truth only after validation. Deploy mesh first. AVNM mesh can coexist with existing peerings, so do not remove peering resources from IaC before validating the mesh path. Validate the traffic path. Use effective routes, Connection Monitor, and flow logs to confirm traffic is using mesh where expected. Guard against drift. Review pipeline state and lifecycle settings before decommissioning old peerings, especially in environments where multiple teams manage network resources. Codify AVNM. Manage the Network Manager, network groups, configuration, and deployment through IaC so mesh becomes the governed connectivity model. DNS Resolution Across Mesh Mesh connectivity does not change DNS resolution behavior by itself. If spoke VNets are already linked to Private DNS Zones hosted or managed through the hub, those links continue to determine name resolution. If spokes use custom DNS servers in the hub, verify that any UDR changes made during migration do not unintentionally alter the DNS traffic path. Migration Example — Two Hub-and-Spoke Topologies into a Single Mesh This example shows how an enterprise can migrate two regional hub-and-spoke environments into one centrally managed AVNM mesh while preserving the existing MSEE path during validation. Current State — Two Hub-and-Spoke Topologies with MSEE Hairpin Contoso Corp operates a large Azure environment in the East US region with two hub-and-spoke topologies: Topology A Topology B Hub VNet Hub-A Hub-B Spoke VNets 500 500 ExpressRoute Gateway ER-GW-A in Hub-A ER-GW-B in Hub-B ExpressRoute Circuit Shared circuit, connected to both gateways Same shared circuit Avg. Private Endpoints per spoke ~8 (4,000 total) ~12 (6,000 total) Total Private Endpoints 10,000 across both topologies How traffic flows today: Spoke-to-spoke within Topology A: Spoke-A-01 → Hub-A → ER-GW-A → MSEE → ER-GW-A → Hub-A → Spoke-A-02 Spoke-to-spoke across topologies: Spoke-A-01 → Hub-A → ER-GW-A → MSEE → ER-GW-B → Hub-B → Spoke-B-01 Every spoke-to-spoke packet — whether within the same topology or across topologies — exits the data center, traverses MSEE, and re-enters. With 1,000 spokes generating east-west traffic, MSEE becomes a shared single point of failure and adds latency to every flow. Target State — A Single AVNM Mesh with 1,000 VNets Contoso's goal is to consolidate all 1,000 spoke VNets into a single AVNM mesh, removing MSEE from the east-west traffic path. Spoke-to-spoke traffic, any pair: Spoke-A-01 → directly→ Spoke-B-01. Traffic stays in the data center and uses the direct mesh path. MSEE role: MSEE carries north-south on-premises traffic. East-west load is removed from the ExpressRoute hairpin path. Migration Execution Phase 0 — Pre-Work Register the feature. Since the mesh contains 1,000 VNets, above the 250 standard limit, Contoso registers the AllowHighScaleConnectedGroup feature flag on the subscription. This enables high-scale mesh support for up to 5,000 VNets. Inventory Private Endpoints. With 10,000 Private Endpoints across 1,000 VNets, Contoso exceeds the standard mesh Private Endpoint limit of 2,000. HSPE must be enabled. Phase 1 — Enable HSPE on All 1,000 Spoke VNets Batch 1: Enable HSPE on all 500 spoke VNets in Topology A during the maintenance window by following the instructions. Batch 2: Apply the same configuration to all 500 spokes in Topology B. Expected impact: Each VNet may experience a brief connection reset for existing Private Endpoint connections when HSPE is enabled. Schedule the change during a maintenance window. Phase 2 — Create the AVNM Mesh Create a Network Manager scoped to the management group containing all 1,000 spoke VNets. Define a single network group called eastus-mesh-all-spokes with dynamic membership using an Azure Policy and the tags. Create a mesh connectivity configuration. Set topology to Mesh, network group to eastus-mesh-all-spokes, high-scale private endpoints to Enabled, and global mesh to Not needed because all VNets are in the same region. Save the configuration as a draft and do not deploy yet. Phase 3 — Incremental Deployment Wave 1 — Pilot: Contoso deploys the mesh configuration to 50 dev/test spokes, either by using a temporary network group or by tagging only those VNets initially. Validation includes effective routes showing ConnectedGroup as the next-hop type for meshed spoke prefixes, spoke-to-spoke connectivity through the direct mesh path, Private Endpoint reachability across meshed spokes, unchanged on-premises connectivity through the hub and MSEE, and VNet flow logs that confirm expected direct spoke-to-spoke flows. Wave 2 — Light production traffic VNets: After a successful pilot, Contoso tags light production traffic VNets. The dynamic network group picks them up automatically. Contoso redeploys the connectivity configuration, runs the same validation checklist, and monitors traffic to confirm that east-west traffic from these VNets is moving away from the ExpressRoute hairpin path. Wave 3 — All remaining production VNets: Contoso tags all remaining spokes and redeploys the configuration. At this point, all 1,000 spokes are in the mesh. No downtime migration: During Waves 2 and 3, existing MSEE hairpin routing remains functional. VNets not yet in the mesh continue to communicate through MSEE. VNets already in the mesh communicate directly through the mesh. This avoids a planned connectivity gap during migration. Phase 4 — Post-Migration Validation After deployment, confirm that mesh is active and traffic is taking the expected path before decommissioning the legacy route. Effective routes. Verify spoke subnets show direct routes to peer VNet prefixes instead of routing through the gateway or MSEE. Connection Monitor. Track representative spoke-to-spoke flows and compare latency and reachability before and after migration. VNet flow logs. Confirm east-west traffic matches the expected mesh path and is not still traversing the ExpressRoute gateway path. Network Watcher topology. Visualize the resulting connectivity model and identify any VNets not enrolled in the target network group. If traffic is still hairpinning after mesh deployment, check for UDRs overriding system routes, spokes missing from the network group, deployment not committed to the target region, or IaC pipelines recreating legacy peerings. Rollback Quick rollback while MSEE remains in place. Remove affected VNets from the network group or undeploy the mesh connectivity configuration. AVNM removes only the connectivity it created, and traffic can fall back to the existing MSEE hairpin path. Rollback after decommissioning legacy paths. If old peerings or route dependencies have already been removed, rollback may require reprovisioning those resources and should be treated as a longer change window. Recommendation. Keep the MSEE hairpin path available for at least two weeks after mesh deployment, monitor traffic patterns, and only then remove the legacy path. Before and After Summary Metric Before (MSEE Hairpin) After (AVNM HSPE Mesh) Spoke-to-spoke latency in the same region Higher due to the MSEE hairpin path Lower-latency direct in-datacenter path; actual latency depends on workload, region, and network conditions Traffic path for east-west Spoke → Hub → MSEE → Hub → Spoke Spoke → Spoke directly through the Mesh MSEE dependency for east-west Yes, shared dependency No MSEE dependency for east-west traffic Manual peerings required 0 when using hairpin routing, but 499,500 if built manually for 1,000 spokes 0 manual spoke-to-spoke peerings; AVNM manages connectivity Private Endpoints supported 2,000 per mesh under standard limits 20,000 per mesh with HSPE Rollback complexity Not applicable to the current hairpin model Remove VNets from the network group or undeploy the connectivity configuration Migration downtime Not applicable Designed for no planned downtime when deployed incrementally and validated carefully Closing Notes Migrating to AVNM mesh does not require tearing down your existing network. The hub gateway, firewalls, and NVAs continue to function as they do today. What changes is that east-west spoke-to-spoke traffic stops leaving the data center unnecessarily. MSEE is not the right tool for the east-west fabric. Removing internal traffic from the ExpressRoute circuit is a reliability and capacity improvement. AVNM mesh replaces combinatorial complexity with group-based intent. The operational model scales with the number of groups, not the number of VNets. High-scale mesh and HSPE remove the ceiling — up to 5,000 VNets and 20,000 Private Endpoints per mesh. The migration is incremental and reversible. Mesh coexists with existing paths, and you can validate wave by wave before decommissioning the legacy route. Start with a pilot mesh in a non-critical environment, validate the traffic shift, and expand from there. Resources Connectivity configurations in Azure Virtual Network Manager Increase Private Endpoint virtual network limits Create a mesh topology with Azure Virtual Network Manager Security admin rules in Azure Virtual Network Manager Create a mesh network topology with Azure Virtual Network Manager using Terraform
ICMP Support for Azure StandardV2 NAT Gateway
Our customers, across all industries, depend on consistent outbound connectivity and observability to operate and troubleshoot their cloud workloads at scale. As Azure environments grow in size and complexity, teams increasingly rely on standard network diagnostic tools such as ping to quickly validate reachability and identify connectivity issues. When customers route outbound traffic through Azure NAT Gateway, they benefit from centralized and scalable outbound connectivity designed to prevent SNAT port exhaustion, while simplifying management by removing the need for per‑resource public IPs. Historically, outbound connectivity through Azure NAT Gateway supported TCP and UDP traffic only for Azure workloads. We are pleased to announce that Azure StandardV2 NAT Gateway now supports outbound ping through ICMP Echo Request and Echo Reply traffic. Workloads that rely on NAT Gateway for internet egress can now use ping to validate reachability, improving operational visibility and simplifying network troubleshooting while preserving the scalability and resiliency characteristics of NAT Gateway. What’s now supported? Azure StandardV2 NAT Gateway supports outbound ICMP Echo Request and Echo Reply traffic (ping) for IPv4 and IPv6. Other ICMP message types are not currently supported. With this update, ICMP traffic sent from workloads behind a NAT Gateway can be translated to a public IP address and reliably receive responses from external endpoints. This brings ICMP behavior in line with existing TCP and UDP outbound flows, enabling consistent network diagnostics across all common transport types. This capability is available automatically when workloads egress through a StandardV2 NAT Gateway - no additional configuration or rule changes are required. Scenarios enabled by ICMP support Support for ICMP through StandardV2 NAT Gateway unlocks several common operational scenarios: Basic reachability testing – Operators can use ping from virtual machines, VM scale sets, or AKS nodes behind NAT Gateway to verify outbound connectivity to internet endpoints. Health validation and troubleshooting – Teams can perform quick checks during incident triage without deploying jump hosts, public IPs per VM, or temporary firewall exceptions. These scenarios are especially valuable in large‑scale environments where workloads rely on centralized outbound connectivity via NAT Gateway. How ICMP works through StandardV2 NAT Gateway Unlike TCP or UDP, ICMP does not use ports to differentiate flows. Instead, ICMP messages include an identifier field, which is used by diagnostic tools to correlate requests and responses. A common troubleshooting scenario is when workloads deployed behind an Azure StandardV2 NAT Gateway are unable to reach external endpoints. Since these workloads typically do not have public IP addresses, it can be difficult to quickly determine whether the issue is related to basic network reachability or higher-layer configuration. In such cases, ICMP‑based diagnostics such as ping can be used to validate outbound connectivity and help isolate the source of the issue. When a workload sends an outbound ICMP packet through a NAT Gateway: NAT Gateway translates the source private IP to one of its associated public IP addresses. The platform tracks the ICMP flow using the ICMP identifier and other packet attributes. Return ICMP responses from the external destination are matched to the originating flow. The response is translated back to the original private IP and delivered to the workload. This stateful handling allows replies to be routed correctly even when multiple workloads are sending ICMP traffic concurrently through the same NAT Gateway. Using ICMP for end-to-end outbound troubleshooting To validate outbound connectivity using ICMP from workloads deployed behind StandardV2 NAT Gateway: Go to your virtual network in the Azure portal and select the subnet hosting your workload. Verify that a StandardV2 NAT Gateway is associated with the subnet. The workload does not require a public IP address. Connect to the workload (for example, a virtual machine) running in the subnet. Run an ICMP ping to an external destination to validate outbound reachability. Successful ping from a workload behind an Azure StandardV2 NAT Gateway demonstrating outbound connectivity and ICMP Echo reply. Review the results. Successful ICMP echo replies confirm that outbound connectivity through StandardV2 NAT Gateway is functioning and that ICMP traffic is translated and returned correctly Summary With ICMP Echo support, Azure StandardV2 NAT Gateway provides a more complete outbound networking experience by enabling essential diagnostic tools to function reliably through NAT. Customers can validate reachability, perform basic troubleshooting, and diagnose connectivity issues while continuing to benefit from the simplicity and scale of centralized outbound connectivity. Learn More Azure NAT Gateway overview Troubleshoot Azure NAT GatewaySimplify Virtual WAN Spoke Connectivity at Scale with Azure Virtual Network Manager
With Azure Virtual Network Manager (AVNM) integration, organizations using Virtual WAN for transitive connectivity can simplify spoke connectivity and policy management across large-scale hub-and-spoke deployments. By using a Virtual WAN hub as the hub in an AVNM hub-and-spoke topology, organizations can define connectivity and routing intent once at the network group level and apply it consistently across large numbers of spoke VNets. This reduces repetitive per-spoke connection and routing configuration, helps maintain operational consistency as deployments expand, and makes it easier to manage hub-and-spoke environments at scale. Together, AVNM’s centralized, group-based orchestration and Virtual WAN’s managed routing, security integration, and hybrid connectivity provide a more streamlined way to simplify operations and scale with confidence. What is Azure Virtual Network Manager? Azure Virtual Network Manager is a management service that lets you group, configure, and deploy network connectivity and security policies across virtual networks at scale. Instead of configuring VNet peering and access rules on each virtual network individually, you define network groups — logical collections of virtual networks based on static selection or dynamic Azure Policy conditions — and apply connectivity configurations and security admin rules to those groups. Key capabilities include: Hub-and-spoke and mesh topologies — Define how virtual networks in a network group connect to a central hub or to each other. Network groups — Group VNets statically or dynamically (using tags, subscriptions, resource group names, or other Azure Policy conditions). Security admin rules — Author and enforce access control lists across all VNets in a network group, providing a centralized layer of defense that complements NSGs and firewalls. Region-scoped deployment — Deploy configurations to specific Azure regions, enabling incremental rollout and controlled blast radius. AVNM operates as an overlay management layer — it orchestrates VNet peering, connectivity, and security rules without replacing the underlying networking primitives. What is Azure Virtual WAN? Azure Virtual WAN as a service brings together routing, security, VPN, ExpressRoute, and transitive connectivity in a hub-and-spoke architecture. A Virtual WAN hub is a managed regional resource that acts as a central transit point for branch connectivity, remote users, private enterprise connectivity, spoke virtual networks, and private traffic routing through security services. Site-to-site VPN connectivity (branch offices, SD-WAN devices) Point-to-site VPN connectivity (remote users) ExpressRoute private connectivity (on-premises datacenters) VNet-to-VNet transitive connectivity (spoke virtual networks) Routing, firewall, and encryption for private traffic All hubs in a Standard Virtual WAN are connected in a full mesh over the Microsoft backbone, enabling any-to-any connectivity between spokes, branches, and remote users across regions. Virtual WAN removes the need to manually manage complex route tables and transit VNets — routing is handled by the hub's built-in router. What this integration enables When you select a Virtual WAN hub as the hub in an AVNM connectivity configuration, AVNM handles the spoke-to-hub wiring for you. For each virtual network in your selected network groups: If the VNet is not yet connected to the Virtual WAN hub, AVNM creates the Virtual Network connection to Virtual WAN hub and applies a consistent routing configuration with Virtual WAN connection policy. If the VNet is already connected, AVNM updates the existing Virtual Network connection to utilize the routing properties in the Virtual WAN connection policy. A connection policy is a hub-level Virtual WAN resource that defines shared routing behavior for the virtual network connections it governs, including route table association and propagation, route maps, internet security settings, and propagated labels. Because the policy applies these settings consistently across governed connections, it helps standardize routing and overrides conflicting settings configured directly on individual connections. How it works The setup follows AVNM's standard workflow: Create a network group. Add virtual networks as members — either statically (by selecting specific VNets) or dynamically (using Azure Policy conditions such as tags or resource group names). Create a connectivity configuration. Choose hub-and-spoke topology, select your Virtual WAN hub as the hub, and select or create a connection policy. Deploy. Commit the configuration to your target regions. AVNM connects all VNets in the network groups to the Virtual WAN hub and applies the connection policy in parallel. You can also enable direct connectivity within a spoke network group. When enabled, VNet-to-VNet traffic within that group routes directly between virtual networks instead of transiting the Virtual WAN hub — useful for latency-sensitive or high-throughput east-west workloads. By default, direct connectivity is regional; enable global mesh to extend it across Azure regions. Key use cases Bulk spoke onboarding Connect many virtual networks to a Virtual WAN hub in one operation. All connections are orchestrated in parallel by AVNM, and the pre-defined routing configuration is automatically applied. Policy-based dynamic onboarding Use Azure Policy to define network group membership conditions. When a new virtual network matches those conditions—for example, a VNet tagged env:prod—it is automatically added to the network group. On the next deployment, AVNM connects it to the Virtual WAN hub with the correct routing configuration, reducing manual onboarding effort. Batch routing configuration updates Push routing changes to all virtual networks in a network group as a single, fully parallelized operation. This significantly reduces maintenance window duration for network-wide changes and makes rollback straightforward. Incremental deployment Segment your network into precise update domains by creating separate network groups — for example, by environment (staging, dev, production) or by region. Deploy connection policies to each group or region independently. This lets you test changes on a smaller subset before applying them broadly, minimizing blast radius. Mesh for selective inspection bypass If you use routing intent to send all private traffic through a firewall in the Virtual WAN hub, certain high-throughput or latency-sensitive flows (such as database replication) may benefit from bypassing that inspection. Enable direct connectivity in AVNM to create a mesh between selected spokes, allowing VNet-to-VNet traffic to route directly while all other traffic continues through the hub firewall. Security admin rules at scale Define network groups for your Virtual WAN spokes, then use AVNM security admin rules to author and deploy access control lists across those spokes. This provides an additional layer of defense alongside next-generation firewalls in the Virtual WAN hub. Getting started Prerequisites: An existing Azure Virtual Network Manager instance An existing Azure Virtual WAN and Virtual WAN hub One or more virtual networks to use as spoke members To configure: Go to your Network Manager instance in the Azure portal. Create a network group and add your spoke VNets. Create a connectivity configuration → select hub-and-spoke → select your Virtual WAN hub → select or create a connection policy → add spoke network groups. Deploy the configuration to your target regions. In your Virtual WAN resource, verify that the expected spoke VNet connections are in a connected state. Review effective routes in the virtual hub to confirm routing behavior matches the selected connection policy. For detailed step-by-step instructions, see Configure Azure Virtual WAN hub for Azure Virtual Network Manager. For more on connection policy, see Connection policy in Azure Virtual WAN. Learn more Azure Virtual Network Manager documentation Virtual WAN and Virtual Network Manager integration overview Azure Virtual WAN documentationSummarized Gateway Prefixes for Route Advertisement in Azure Virtual Networks
Background Many Azure deployments follow a hub-and-spoke topology: one VNet is designated as the hub and holds the connection to on-premises (via ExpressRoute Gateway, VPN Gateway, or both), and workload VNets — the spokes — peer to the hub to reach on-premises and shared services. This centralizes gateway connectivity so many workloads can share a single ExpressRoute or VPN Gateway. However, in large hub-and-spoke topologies, ExpressRoute and VPN Gateway limits on advertised prefixes (for example, 1,000 IPv4 and 100 IPv6 prefixes) can be reached. Because each spoke adds its own address prefixes to that count, these limits are approached quickly, constraining how far the topology can scale. What's New With Summarized Gateway Prefixes, customers can now advertise a single covering prefix (for example, 10.0.0.0/16) instead of many smaller CIDRs (for example, multiple /24s) – dramatically reducing advertised route count and enabling larger-scale Azure environments. A new property, summarizedGatewayPrefixes, is now available on the Virtual Network resource in public preview. When configured on a hub VNet, it controls what your ExpressRoute Gateway and VPN Gateway advertise to on-premises, replacing the default behavior of advertising all individual hub and spoke VNet CIDRs with a set of aggregated prefixes you define. For example, instead of advertising 10.0.1.0/24, 10.0.2.0/24, 10.0.3.0/24, and so on for each spoke, you can advertise a single 10.0.0.0/16. Key Benefits Fewer advertised routes — Replace hundreds of individual spoke CIDRs with a small set of summarized prefixes. Scales with your topology — Supports deployments with 500+ spokes without requiring address plan redesigns or VNet splits. IPv4 and IPv6 — Summarize both address families. Works with both gateway types — Supported on ExpressRoute Gateway and VPN Gateway. Simple configuration — A single property on the VNet resource. No additional services or dependencies. Backward compatible — If the property is left empty, behavior is unchanged: all hub and peered spoke address spaces are advertised as before. How It Works Default behavior ExpressRoute Gateway and VPN Gateway advertise all address spaces of the hub VNet and all address spaces of peered spoke VNets to on-premises. With summarizedGatewayPrefixes configured The gateways advertise the summarized prefixes instead of the hub VNet's individual address spaces. For each peered spoke, if the spoke's address space falls within a summarized prefix, the spoke's individual CIDRs are suppressed from advertisement. Spoke address spaces not covered by a summarized prefix continue to be advertised individually. Example: Without Summarization With Summarization 10.0.1.0/24, 10.0.2.0/24, 10.0.3.0/24, … 10.0.0.0/16 Hundreds of prefixes One prefix Getting Started Open the hub VNet (the VNet containing your GatewaySubnet) in the Azure portal. Go to Address space → Advertised gateway prefixes. Add one or more IPv4 or IPv6 CIDR prefixes that cover the address spaces you want to summarize. Navigate to your virtual network and verify that the summarized prefixes appear. Things to Know The property is set on the hub VNet (the VNet with the GatewaySubnet). The summarized prefixes list can include prefixes outside the VNet's own address space. Avoid overlap among prefixes within the list, but overlap with peered VNet address spaces is expected in hub-and-spoke designs. For dual-stack (IPv4 + IPv6) VNets, define both IPv4 and IPv6 summarized prefixes explicitly.
Azure virtual network terminal access point (TAP) public preview announcement
What is virtual network TAP? Virtual network TAP allows customers continuously stream virtual machine network traffic to a network packet collector or analytics tool. Many security and performance monitoring tools rely on packet-level insights that are difficult to access in cloud environments. Virtual network TAP bridges this gap by integrating with our industry partners to offer: Enhanced security and threat detection: Security teams can inspect full packet data in real-time to detect and respond to potential threats. Performance monitoring and troubleshooting: Operations teams can analyze live traffic patterns to identify bottlenecks, troubleshoot latency issues, and optimize application performance. Regulatory compliance: Organizations subject to compliance frameworks such as Health Insurance Portability and Accountability Act (HIPAA), and General Data Protection Regulation (GDPR) can use virtual network TAP to capture network activity for auditing and forensic investigations. Why use virtual network TAP? Unlike traditional packet capture solutions that require deploying additional agents or network appliances, virtual network TAP leverages Azure's native infrastructure to enable seamless traffic mirroring without complex configurations and without impacting the performance of the virtual machine. A key advantage is that mirrored traffic does not count towards virtual machine’s network limits, ensuring complete visibility without compromising application performance. Additionally, virtual network TAP supports all Azure virtual machine SKU. Deploying virtual network TAP The portal is a convenient way to get started with Azure virtual network TAP. However, if you have a lot of Azure resources and want to automate the setup you may want to use a PowerShell, CLI, or REST API. Add a TAP configuration on a network interface that is attached to a virtual machine deployed in your virtual network. The destination is a virtual network IP address in the same virtual network as the monitored network interface or a peered virtual network. The collector solution for virtual network TAP can be deployed behind an Azure Internal Load balancer for high availability. You can use the same virtual network TAP resource to aggregate traffic from multiple network interfaces in the same or different subscriptions. If the monitored network interfaces are in different subscriptions, the subscriptions must be associated to the same Microsoft Entra tenant. Additionally, the monitored network interfaces and the destination endpoint for aggregating the TAP traffic can be in peered virtual networks in the same region. Partnering with industry leaders to enhance network monitoring in Azure To maximize the value of virtual network TAP, we are proud to collaborate with industry-leading security and network visibility partners. Our partners provide deep packet inspection, analytics, threat detection, and monitoring solutions that seamlessly integrate with virtual network TAP: Network packet brokers Partner Product Gigamon GigaVUE Cloud Suite for Azure Keysight CloudLens Security analytics, network/application performance management Partner Product Darktrace Darktrace /NETWORK Netscout Omnis Cyber Intelligence NDR Corelight Corelight Open NDR Platform LinkShadow LinkShadow NDR Fortinet FortiNDR Cloud FortiGate VM cPacket cPacket Cloud Suite TrendMicro Trend Vision One™ Network Security Extrahop RevealX Bitdefender GravityZone Extended Detection and Response for Network eSentire eSentire MDR Vectra Vectra NDR AttackFence AttackFence NDR Arista Networks Arista NDR See our partner blogs: Bitdefender + Microsoft Virtual Network TAP: Deepening Visibility, Strengthening Security Streamline Traffic Mirroring in the Cloud with Azure Virtual Network Terminal Access Point (TAP) and Keysight Visibility | Keysight Blogs eSentire | Unlocking New Possibilities for Network Monitoring and… LinkShadow Unified Identity, Data, and Network Platform Integrated with Microsoft Virtual Network TAP Extrahop and Microsoft Extend Coverage for Azure Workloads Resources | Announcing cPacket Partnership with Azure virtual network terminal access point (TAP) Gain Network Traffic Visibility with FortiGate and Azure virtual network TAP Get started with virtual network TAP To learn more and get started, visit our website. We look forward to seeing how you leverage virtual network TAP to enhance security, performance, and compliance in your cloud environment. Stay tuned for more updates as we continue to refine and expand on our feature set! If you have any questions please reach out to us at azurevnettap@microsoft.com.
Consistent DNS resolution in a hybrid hub spoke network topology
DNS is one of the most essential networking services, next to IP routing. A modern hybrid cloud network may have various sources of DNS: Azure Private DNS Zones, public DNS, domain controllers, etc. Some organizations may also prefer to route their public Internet DNS queries through a specific DNS provider. Therefore, it is crucial to ensure consistent DNS resolution across the whole (hybrid) network. This article describes how DNS Private Resolver can be leveraged to build such architecture.
Private subnets by default in Azure Virtual Networks: What changed and how to use NAT Gateway
Azure is evolving to better support secure‑by‑default cloud architectures. Starting with API version 2025‑07‑01 (released after March 31, 2026), newly created virtual networks now default to using private subnets. This change removes the long‑standing platform behavior of automatically enabling outbound internet access through implicit public IPs, also known as default outbound access (DOA). As a result: newly deployed virtual machines will not have public outbound connectivity unless explicitly configured. What changed? Previously, Azure automatically assigned a hidden Microsoft‑owned public IP to virtual machines deployed without an explicit outbound method (such as NAT Gateway, Load Balancer outbound rules, or instance‑level public IPs). This allowed public outbound connectivity without requiring customer configuration. While convenient, this model introduced challenges: Security – Implicit internet access conflicts with Zero Trust principles. Reliability – Platform‑managed outbound IPs can change unexpectedly. Operational consistency – VMSS instances or multi‑NIC VMs may egress using different default outbound IPs. With API version 2025‑07‑01 and later: Subnets in newly created VNets are private by default. The subnet property `defaultOutboundAccess` is set to false. Azure no longer assigns implicit outbound public IPs. This applies across deployment methods including Portal, ARM/Bicep, CLI, and PowerShell. Portal has started using the new model as of April 1, 2026. Note: This change has not yet applied to Terraform. Am I impacted by this change? Deployment scenario Behavior Existing VNets or VMs using DOA ✅ Unchanged New VMs in existing VNets ✅ Unchanged Subnets already using explicit outbound ✅ Continue using configured outbound method New VMs in new VNets (with subnets created using API 07-01-2025 or later) 🔒 Subnets private by default New VMs in private subnets without explicit outbound configured ❌ No public outbound connectivity Existing workloads are not impacted. If required, you can still create new subnets without the private setting by choosing the appropriate configuration option during creation. See the FAQ section of this blog for more information. However, we strongly recommend transitioning to an explicit outbound method so that: Your workloads won’t be affected by public IP address changes. You have greater control over how your VMs connect to public endpoints. Your VMs use traceable IP resources that you own. When is outbound connectivity required? If your virtual network contains virtual machines, you must configure explicit outbound connectivity. Here are common scenarios that require it: Virtual machine operating system activation and updates, such as Windows or Linux. Pulling container images from public registries (Docker Hub or Microsoft Container Registry). Accessing 3rd party SaaS or public APIs Virtual machine scale sets using flexible orchestration mode are always secure by default and therefore require an explicit outbound method. Private subnets don’t apply to delegated or managed subnets that host PaaS services. In these cases, the service handles outbound connectivity—see the service-specific documentation for details. Recommended outbound connectivity method: StandardV2 NAT Gateway Azure now recommends using an explicit outbound connectivity method such as: NAT Gateway Load Balancer outbound rules Public IP assigned to the VM Network Virtual Appliance (NVA) / Firewall Among these, Azure StandardV2 NAT Gateway is the recommended method for outbound connectivity for scalable and resilient outbound connectivity. StandardV2 NAT Gateway: Provides zone‑redundancy by default in supported regions Supports up to 100 Gbps throughput Provides dual-stack support with IPv4 and IPv6 public IPs Uses customer‑owned static public IPs Enables outbound connectivity without allowing inbound internet access Requires no route table configuration when associated to a subnet When configured, NAT Gateway automatically becomes the subnet’s default outbound path and takes precedence over: Load Balancer outbound rules VM instance‑level public IPs Note: UDRs for 0.0.0.0/0 traffic directed to virtual appliances/Firewall takes precedence over NAT gateway. Migrate from Default Outbound Access to NAT Gateway To transition from DOA to Azure’s recommended method of outbound, StandardV2 NAT Gateway: Go to your virtual network in the portal, and select the subnet you want to modify. In the Edit subnet menu, select the ‘Enable private subnet’ checkbox under the Private subnet section Enabling private subnet can also be done through other supported clients, below is an example for CLI, in which the default-outbound parameter is set to false: az network vnet subnet update \ --resource-group rgname \ --name subnetname \ --vnet-name vnetname \ --default-outbound false 3. Deploy a StandardV2 NAT gateway resource. 4. Associate one or more StandardV2 public IP addresses or prefixes. 5. Attach the NAT gateway to the target subnet. Once associated: All new outbound traffic from that subnet uses NAT Gateway automatically VM‑level public IPs are no longer required Existing outbound connections are not interrupted Note: Enabling private subnet on an existing subnet will not affect any VMs already using default outbound IPs. Private subnet ensures that only new VMs don’t receive a default outbound public IP. For step-by-step guidance, see migrate default outbound access to NAT Gateway. FAQ 1. Will my existing workloads lose outbound connectivity? No. Workloads currently using default outbound IPs are not impacted by this change. The private subnet by default update only affects: Newly created VNets New subnets created using the updated API, 2025-07-01 New virtual machines deployed into those subnets using the updated API VMs and subnets using an explicit outbound connectivity method like a NAT gateway, NVA / Firewall, a VM instance level public IP or Load balancer outbound rules is not impacted by this change. 2. Why can’t my new VM reach the internet or other public endpoints within Microsoft (e.g. VM activation, updates)? New subnets are private by default. If your deployment does not include an explicit outbound method — such as a NAT Gateway, Public IP, Load Balancer outbound rule, or NVA/Firewall— outbound connectivity is not automatically enabled. 3. My workload has a dependency on default outbound IPs and isn’t ready to move to private subnets, what should I do? You can opt-out of the default private subnet setting by disabling the private subnet feature. You can do this in the portal by unselecting the private subnet checkbox: Disabling private subnet can also be done through other supported clients, below is an example for CLI, in which the default-outbound parameter is set to true: az network vnet subnet update \ --resource-group rgname \ --name subnetname \ --vnet-name vnetname \ --default-outbound true 4. Why do I see an alert showing that I have a default outbound IP on my VM? There's a NIC-level parameter `defaultOutboundConnectivityEnabled` that tracks whether a default outbound IP is allocated to a VM/Virtual Machine Scale Set instance. If detected, the Azure portal displays a notification banner and will generate Azure Advisor recommendations about disabling default outbound connectivity for your VMs / VMSS. 5. How do I clear this alert? To remove the default outbound IP and clear the alert: Configure a StandardV2 NAT gateway (or other explicit outbound method). Set your subnet to be private or by setting the subnet property defaultOutboundAccess = false using one of the supported clients. Stop and deallocate any applicable virtual machines (this will remove the default outbound IP currently associated with the VM). 6. I have a NAT gateway (or UDR pointing to an NVA) configured for my private subnet, why do I still see this alert? In some cases, a default outbound IP is still assigned to virtual machines in a non-private subnet, even when an explicit outbound method—such as a NAT gateway or a UDR directing traffic to an NVA/firewall—is configured. This does not mean that the default outbound IP is used for egress traffic. To fully remove the assignment (and clear the alert): Set the subnet to private Stop and deallocate the affected virtual machines Summary The move to private subnets by default improves the security posture of Azure networking deployments by removing implicit outbound internet access. Customers deploying new workloads must now explicitly configure outbound connectivity. StandardV2 NAT Gateway provides a scalable, resilient method for enabling outbound internet access without exposing workloads to inbound connections or relying on platform‑managed IPs. Learn more Default Outbound Access StandardV2 NAT Gateway Migrate Default Outbound Access to StandardV2 NAT GatewayAzure VNet Data Gateway for Secure Power BI & Power Platform Access in Enterprises
What Is a VNet data gateway? The VNet data gateway is a Microsoft‑managed gateway service that runs inside a delegated subnet of an Azure Virtual Network. It allows supported Microsoft cloud services—such as Power BI, Power Platform dataflows, and Microsoft Fabric workloads—to securely connect to data sources that are protected using private networking. Key characteristics: No customer‑managed VM or container No OS, patching, or gateway software upgrades Gateway lifecycle fully managed by Microsoft Traffic stays on the Azure backbone network Works seamlessly with Private Endpoints This makes it ideal for enterprise and regulated environments where security and operational efficiency are equally important. Why Enterprises need VNet data gateway Eliminates gateway infrastructure management Traditional gateways require: Virtual machines High availability setup OS patching and scaling Monitoring and troubleshooting With the VNet data gateway: Microsoft manages compute lifecycle No VM or gateway software to maintain No HA or load balancer design needed ✅ Result: Significant reduction in operational and maintenance overhead for platform and infrastructure teams. Secure access to private Azure resources Most enterprise Azure environments use: Private Endpoints NSGs and route tables Firewalls blocking public access The VNet data gateway: Is injected into a delegated subnet in your VNet Uses private IP addressing Enforces NSG and UDR rules Communicates with Microsoft services over a Microsoft‑managed internal tunnel ✅ Result: Data sources remain fully private—no public endpoints or inbound ports required. Designed for Power Platform & Power BI at Scale The gateway supports secure access for: Power BI semantic models Power BI paginated reports Microsoft Fabric Dataflow Gen2 Fabric pipelines and copy jobs Because it’s cloud‑native and centrally managed, the VNet data gateway scales well in large enterprises standardizing on Power Platform and Fabric. High‑level architecture overview At runtime, the VNet data gateway works as follows: A query is initiated from Power BI / Power Platform Query details and credentials are sent to the Microsoft Power Platform VNet service A containerized gateway instance is injected into the delegated subnet The gateway connects to the private data source using private networking Results are sent back to Power BI or Power Platform via a Microsoft‑managed internal tunnel Key security highlights: No inbound connectivity No public IP exposure Traffic remains on Azure backbone Full enforcement of NSGs and routing rules Key Enterprise benefits Least management overhead – no gateway servers Zero Trust aligned – private-only connectivity Fully managed by Microsoft Enterprise-grade security & governance Works with Azure Private Endpoint architectures When to Use VNet Data Gateway Scenario Recommendation Azure private PaaS services ✅ VNet data gateway Private Endpoint–only access ✅ VNet data gateway Zero Trust network model ✅ VNet data gateway Minimal ops & maintenance ✅ VNet data gateway On‑prem only, no Azure ❌ Traditional gateway Step‑by‑step configuration: VNet data gateway (Enterprise setup) High‑level flow (What you will configure) Register required Azure resource provider Prepare Azure Virtual Network and subnet Configure private connectivity to data source Create the VNet data gateway Create and bind data source connections Validate with Power BI / Power Platform workloads Step 1: Register Microsoft.PowerPlatform resource provider Why this step is required The VNet data gateway is a Microsoft‑managed service that is injected into your Azure VNet. Azure must explicitly allow Power Platform to deploy managed infrastructure into your subscription. Configuration steps Sign in to Azure portal Navigate to Subscriptions Select the subscription that hosts the target VNet Go to Resource providers Search for Microsoft.PowerPlatform Click Register ✅ Status must show Registered This step enables subnet delegation to Power Platform services. Step 2: Prepare the Azure Virtual Network Why this step is required The gateway runs inside your VNet. It must be placed in a dedicated, delegated subnet to maintain isolation and security boundaries. Requirements VNet can be in any Azure region Subnet must be exclusive to VNet data gateway Subnet must have outbound connectivity to the data source Configuration steps Go to Azure portal → virtual networks Select your existing VNet (or create one) Navigate to Subnets → + Subnet Configure: Subnet name: snet-vnet-datagateway Address range: /27 or larger (recommended) Subnet delegation: Microsoft.PowerPlatform/vnetaccesslinks Save the subnet ⚠️ Do not place any VMs, app gateway, or other workloads in this subnet. Step 3: Configure private connectivity to the data source Why this step is required Enterprises typically block public access to PaaS services. The VNet data gateway is designed to work natively with private endpoints. Example: Azure SQL / SQL Managed Instance Create Private Endpoint for the data service Attach it to the same VNet (can be different subnet) Create or link a Private DNS Zone, for example: privatelink.database.windows.net Link the Private DNS Zone to the VNet Ensure DNS resolution from the delegated subnet resolves to private IP ✅ This ensures all traffic remains private and internal. Step 4: Create the VNet data gateway Why this step is required This is where the actual Microsoft‑managed gateway is logically created and associated with your VNet. Configuration steps You can do this from either Power BI Service or Power Platform Admin Center. Using Power Platform Admin Center Go to https://admin.powerplatform.microsoft.com Select Data → Gateways Click + New → Virtual network data gateway Provide: Gateway name Azure subscription Resource group Virtual network Delegated subnet Click Create 📌 Notes: Gateway metadata is stored in Power BI tenant home region Gateway runtime executes in the VNet region No VM or scale settings are required Step 5: Create and configure data source connections Why this step is required The gateway exists, but Power BI / Power Platform must know which data sources can be accessed via it. Configuration steps (Power BI example) Go to Power BI Service Navigate to Settings → Manage connections and gateways Select the newly created VNet data gateway Click + New connection Provide: Data source type (Azure SQL, Storage, Databricks, etc.) Server / endpoint name (private DNS name) Authentication (SQL / Entra ID) Save the connection Assign users or security groups ✅ This step enables governance and access control. Step 6: Use the gateway in Power BI / Power Platform Power BI Open dataset or semantic model settings Under Gateway connection, select: Use a data gateway Choose the VNet data gateway Apply changes Refresh or run queries Power Platform / Fabric Select the same connection when configuring: Dataflows Gen2 Fabric pipelines Copy jobs Step 7: Validate and test Validation Checklist ✅ DNS resolves to private IP ✅ No public endpoint access enabled ✅ NSGs allow outbound traffic to data source ✅ Dataset refresh succeeds ✅ No gateway VM exists in subscription Optional: Enable logging and auditing from Power BI / Fabric Monitor gateway health in Admin Center Key Enterprise design guidance (Best practices) Use one gateway per environment tier (Prod / Non‑Prod) Use dedicated VNets for data access where possible Use Private Endpoint only (avoid service endpoints) Control access via AAD groups, not individuals Avoid mixing gateway subnet with other workloads Conclusion: For enterprises looking to consume Power Platform, Power BI, and Microsoft Fabric securely while keeping operational overhead close to zero, the VNet data gateway is the recommended approach. It removes gateway infrastructure complexity, strengthens security posture, and aligns perfectly with modern Azure landing zone and Zero Trust architectures.Custom DHCP support in Azure
Discover the intricacies of Dynamic Host Configuration Protocol (DHCP), a network protocol used for assigning IP addresses and network parameters. Learn about the DORA process, lease renewals, and the role of DHCP Relay in large enterprises. Gain insight into how DHCP operates in Azure natively and how its support in Azure has evolved over time, including the removal of rate-limiting for relayed traffic. This comprehensive guide also covers the limitations and potential workarounds of DHCP in Azure. Ideal for network administrators and IT professionals.
