reliability and resiliency in azure
24 TopicsProactive Reliability Series — Article 2: Regional Distribution Patterns for Azure Workloads
Introduction Public cloud platforms — including Microsoft Azure — are built on three foundational principles that distinguish them from traditional on-premises infrastructure: Elasticity: The platform can automatically expand and contract resource capacity in response to demand. Capacity is not statically provisioned; it is drawn from a shared pool and released when no longer needed. Scaling: Workloads can scale horizontally (adding more instances) or vertically (increasing instance size) on demand, without pre-procurement of physical hardware. Consumption-based billing: Customers pay for what they use, when they use it. Cost is proportional to resource consumption, not to physical capacity reserved in advance. These principles are properties of the platform, not of any single location. Microsoft Azure Cloud is not a single region — it is a globally distributed platform comprising dozens of regions across every major geography, interconnected by a private backbone network. When an organisation deploys to Azure, it is deploying into this global system; the choice of which region or regions to use is based on an organizational strategy and architectural decision. Using a single Azure region is a valid choice in many scenarios, but it has to be a deliberate architecture decision, not an omission. Why Multi-Region? Microsoft Azure CTO Mark Russinovich summarises the case for multi-region in Achieve agility and scale in a dynamic cloud world: organisations that span multiple regions gain scalability and flexibility (choosing from the full Azure region portfolio, including differentiated pricing, AI capabilities, and deployment options), resilience and availability (reducing the impact of regional disruptions through multiple backup and recovery options), and performance and reduced latency (serving users from infrastructure that is geographically closer to them). The post's closing recommendation — "leverage Azure as a cloud platform, not a datacenter region" — makes explicit what the multi-region decision ultimately is: a choice to treat the platform's global footprint as an asset, not a constraint. Mark Russinovich — Achieve agility and scale in a dynamic cloud world (Microsoft Azure, September 2024) This article also does not argue whether to adopt a multi-region strategy — that is a business and risk decision. It describes what the options are: the available regional distribution patterns, the forces each resolves, and the trade-offs each accepts. Regional workload distribution is not simply an application-level decision — it is an organisational one. It shapes how a company scales its cloud presence, manages cost and operational complexity across a growing portfolio, meets data residency and regulatory obligations, and positions itself to respond to changing conditions. Multi-region is often a necessity, not a free choice: growth ambitions, compliance requirements, or risk obligations may demand it. But necessity does not determine form. These patterns define the decision space: whether operating across multiple regions is warranted at all, and if so, which structural arrangement fits the organisation's scale, objectives, and operational capability. Several patterns are adapted from Gregor Hohpe's multi-cloud strategy patterns, originally described in Multi Cloud Architecture: Decisions and Options and further elaborated in Multi-cloud: From Buzzword to Decision Model. Important This is an unofficial guide to regional distribution patterns for Azure workloads. It is not an official Microsoft publication and is not officially supported, endorsed, or maintained by Microsoft. All descriptions and recommendations are based on publicly available Azure documentation and general distributed systems principles. Always refer to official Azure documentation and the Azure Well-Architected Framework for authoritative guidance. The Risks and Opportunities Pattern selection is a direct response to specific risks or solution quality requirements. Before evaluating patterns (options), it is necessary to understand what risks are actually relevant — infrastructure faults, capacity constraints, service coverage gaps, and compliance or business obligations — and what their scope of impact is. Azure infrastructure is complex and distributed. While Microsoft invests heavily in reliability, faults can and do occur across a wide range of blast radii — from a single compute instance at the narrowest end, through Availability Zone and regional failures, to global service disruptions at the widest. The appropriate pattern is the one that reduces unacceptable risks to a tolerable level. The Azure Well-Architected Framework — Reliability pillar recommends Failure Mode Analysis (FMA) as the structured technique for enumerating failure modes, assessing their impact, and identifying mitigations before they are needed in production. The fault types below are the infrastructure-layer inputs to that analysis. For a detailed breakdown of each fault type — including likelihood analysis, real-world incident examples, and detection guidance — see Proactive Reliability Series — Article 1: Fault Types in Azure. Risk Catalog (Sample) The following is a representative sample of risks relevant to regional workload distribution decisions, not an exhaustive catalogue. Not all entries are infrastructure faults — some represent business and compliance obligations. The Category column identifies the type of each risk. Likelihood values are relative planning heuristics to help prioritise resilience investments — they are not statistical probabilities and do not represent Azure SLA commitments. RiskCategoryBlast RadiusLikelihoodPrimary MitigationService Fault (Region)Infrastructure FaultSingle service within a regionMediumRegion redundancyRegion FaultInfrastructure FaultRegional degradation or full regional loss (partial-to-full region impact)LowRegion redundancy; cross-region failoverNetwork POP Location FaultInfrastructure FaultNetwork colocation site (affects connectivity, not compute)LowExpressRoute Metro (dual peering locations); network path redundancyService Fault (Global)Infrastructure FaultWorldwide or multiple regions simultaneouslyVery LowAccept risk; use alternative service if downtime is intolerableRegional Service Capacity ConstraintInfrastructure CapacitySingle region (required capacity unavailable — whole service or specific SKU — at failover time or sustained shortage)LowRegion redundancy; Capacity Reservations; Hot Standby; alternative regionService Regional UnavailabilityInfrastructure FaultSingle region (desired service not offered in that region)LowDeploy to a region where the service is availableGeopolitical RiskComplianceOne or more regions (regulatory or political mandate to relocate)LowPortable pattern; pre-validated alternative regionSustainability ConstraintComplianceOne or more regions (sustainability targets unachievable in current region)LowPortable pattern; relocate to region with required sustainability profile The Patterns While there could be many ways to distribute workloads across Azure regions, the following patterns represent the most common and widely applicable approaches. Each pattern is a structural topology that defines how workloads are deployed and how they respond to Risks and Quality Requirements. The patterns are not mutually exclusive — they can be combined or layered to meet specific requirements. The following patterns describe the principal ways workloads can be distributed across Azure regions. Patterns 1-3 are described and found very often, while patterns 4-5 are less common but still important to consider. The table below summarises the patterns, their intent, and the primary driver for their adoption. #PatternIntentPrimary Driver1SingleAll resources in one Azure region; AZ redundancy optionalSimplicity; cost; data residency constraints2FailoverPrimary region serves traffic; secondary region is a cold or hot standby for DRBusiness continuity3ParallelSame workload deployed to multiple regions simultaneously; all activeContinuous availability; zero-downtime failover4SegmentedServices or service portfolio distributed across regions by BU, LOB, tenant, or data residencyIsolation; sovereignty; independent release cadence5PortableFull portability; workloads can be relocated between regions without application changesOperational flexibility; on-demand relocation 1. Single Region Pattern Pattern Name: Single Region Workload Distribution Classification: Regional workload distribution Scope: Workload — applies to a single application or service deployment, Workload (Application) Portfolio Intent: Deploy all workload resources in one Azure region; Context: A workload is being deployed to Azure and must decide how many regions to use. The business impact of a regional outage has been assessed — either as tolerable within the workload’s criticality tier, or as not applicable because data sovereignty constraints prohibit cross-region replication. The team needs to treat Single Region as an explicitly chosen architecture, not an omission. Problem: Every additional Azure region adds infrastructure cost and workload integration complexity due to the introduced network latency. Not all workloads justify this overhead. The question is not “should I always use multiple regions?” but “when is a single region the correct and explicitly chosen answer, and when does adding a second region produce risk-reduction that justifies the cost?” Forces: The workload’s risk profile does not justify cross-region redundancy: the business impact of a regional outage is tolerable, data sovereignty rules prohibit cross-region replication, or reliability requirements are fully met within a single region with Availability Zone redundancy. The cost of multi-region infrastructure produces no corresponding risk-reduction return for this workload. Solution Place all compute, data, and networking resources in a single Azure region. Apply Availability Zone redundancy within that region for protection against datacenter-level failures. Formally accept region-level risk as within tolerance for this workload’s criticality tier — this is a deliberate architecture decision, not an omission. Structure: Implementation: Enable Availability Zones for all production resources where supported. Document the risk-acceptance decision explicitly at the application level. Consequences Benefits: Lowest cost and operational footprint of all patterns. No cross-region routing, replication lag, or failover coordination complexity. Simplest deployment pipeline, observability surface, and incident response. Liabilities: No mitigation for any region-level fault — full workload loss on regional failure. Data concentrated in one geography; no cross-region durability without explicit configuration. No pre-deployed capacity in an alternative region. Risk posture: RiskAssessmentRegion Fault❌ Primary unaddressed risk — partial degradation or full regional failure has no cross-region recovery path; accept or upgrade patternService Fault (Region)❌ Regional service failures have no cross-region alternativeRegional Service Capacity Constraint❌ No alternative region available; both on-demand failover provisioning and sustained SKU shortages have no mitigation pathNetwork POP Location Fault✅ Addressable within this pattern via ExpressRoute Metro (dual peering locations in the same metro); does not require a multi-region distribution change Known Uses Development and test environments; Bronze- or Non-Critical-tier workloads; workloads with strict data residency constraints that prohibit cross-region replication; proof-of-concept and time-limited deployments. See Azure Services - Sample Multi-Region Capabilities Pattern Mapping for how individual Azure services implement this pattern. References Azure reliability documentation overview Availability zones overview Azure Well-Architected Framework — Reliability pillar 2. Failover Pattern (Primary + Standby Region) Pattern Name: Failover Workload Distribution Also Known As: Active-Passive, Disaster Recovery (DR), Business Continuity and Disaster Recovery (BCDR) Classification: Regional workload distribution, Application Design, Platform/Infrastructure Design, Disaster Recovery Scope: Workload — an application and infrastructure design pattern that requires both layers to work in tandem; Workload (Application) Portfolio Intent: Recover from a regional disaster or other longer duration region outage by designating a primary region to carry all production traffic and a standby region to absorb that traffic upon primary region failure. The typical quality attribute metrics that govern its design are RTO (Recovery Time Objective — maximum tolerable downtime), RPO (Recovery Point Objective — maximum tolerable data loss), and MTTR (Mean Time To Recovery — the observed average recovery time, measured through drills and real incidents, that validates whether the RTO target is achievable in practice). Context: A workload must survive region-level failures, but the architectural complexity of running two fully active deployments simultaneously — with multi-region write-conflict resolution — is not justified. The business can tolerate a bounded recovery time and, depending on the sub-variant chosen, a bounded data loss window. Problem: A fully active multi-region deployment (Parallel pattern) introduces multi-region write-conflict complexity that the application cannot or need not absorb. The Failover pattern trades continuous availability for single-writer simplicity: one region is active, one is standby, and recovery is bounded by RTO/RPO targets. The sub-variant choice (Cold / Warm / Hot) then determines how much cost is invested in standby readiness — from minimal (~1.1×) to near-full duplication (~2×) — based on how fast recovery must be. Forces: A single active write region is required — multi-region write-conflict resolution adds unacceptable consistency risk or development complexity. RTO and RPO targets must be met, but budget constrains how pre-warmed the standby region can be, driving the Cold / Warm / Hot sub-variant selection. Regional Service Capacity Constraint in the standby region is a residual risk for Cold and Warm sub-variants — the standby may fail to scale at failover time unless capacity is pre-reserved. Solution Designate one region as primary (all production traffic under normal conditions) and a second as standby (no production traffic until failover). The standby’s readiness level — the sub-variant choice — is determined by the RTO/RPO requirements and cost envelope. Replication from primary to standby is continuous; failover is triggered manually or automatically when the primary becomes unavailable. Structure: Note on "Active-Passive": This pattern is often called Active-Passive in Microsoft documentation. That framing is accurate at the traffic level (one region active, one passive), but it can obscure the architectural intent. The Failover label here emphasises the capability being purchased: the ability to redirect the entire workload to a pre-designated region when the primary is unavailable. Implementation: The key design decision is how ready the standby is at the moment it is needed. Three sub-variants define this readiness spectrum: Sub-variantSecondary stateCost multiplierCold StandbyNo running compute; data either in scheduled backups or continuously replicated~1.1–1.5×Warm StandbyReduced-scale compute running; data continuously replicated~1.5–1.8×Hot StandbyFull-scale compute running; data continuously replicated~2× Cold Standby — No compute is running in the standby region under normal conditions. Cold Standby covers two positions within this state, differing in how the data layer is protected: Backup/Restore: Data is backed up or geo-replicated on a scheduled basis. On failover, infrastructure must be deployed from scratch and data restored before traffic can be redirected. RTO is measured in hours; RPO equals the interval between the last backup cycle and the failure event. Data-layer live, compute stopped: Core infrastructure (networking, identity, data tier) is kept running with continuous replication to the standby region; compute is stopped or scaled to zero. On failover, compute is started and scaled up to meet full load. RTO is typically 15–60 minutes; RPO is bounded by async replication lag rather than backup interval. Both positions share the defining characteristic of Cold Standby: no production-equivalent compute running in the secondary region under normal conditions. The difference is the investment in keeping the data layer live, which reduces both RTO and the data loss window at a modestly higher steady-state cost. Warm Standby — The standby region runs a scaled-down but functionally complete version of the workload. Traffic is not routed there under normal conditions. On failover, the secondary scales up and traffic is redirected. A brief scale-out lag occurs before the secondary absorbs full traffic; the running environment eliminates cold-start delay. Hot Standby — The standby region runs a full, production-equivalent deployment — same compute capacity, same configuration — but receives no traffic under normal conditions. Data is continuously and near-synchronously replicated. Failover is fast and often automated because no scale-up is required. Consequences Benefits: Enables recovery from region-level faults at a fraction of Parallel pattern cost. Flexible cost-vs-RTO trade-off across Cold / Warm / Hot sub-variants. No multi-region write-conflict complexity; single active write region throughout normal and recovery operation. Liabilities: Cold and Warm standby introduce meaningful RTO (minutes to hours). Replication lag creates a data loss window (RPO > 0) at the moment of failover. The failover path is the least-exercised code path — untested recovery inflates actual RTO. Failover is often neglected and not properly and regularly tested - this leads toward a fear to execute failover when needed (and it is not only full region disaster). Risk posture: RiskAssessmentRegion Fault⚠️ Primary driver; standby region absorbs traffic on full regional loss, but partial regional degradation may not trigger automated failover. RTO depends on sub-variantRegional Service Capacity Constraint⚠️ Cold/Warm Standby are exposed to both on-demand provisioning failure and sustained SKU shortages — mitigated by Capacity Reservations or Hot StandbyService Fault (Region)✅ Standby region provides an alternative deployment for regional service failuresService Regional Unavailability⚠️ Secondary region must be verified for full service parity at design time — absent services block failover regardless of compute readiness Known Uses Business-critical workloads with defined RTO/RPO targets that cannot accept region-level risk but do not require continuous multi-region availability; workloads with single-writer data models where multi-region write-conflict resolution is unacceptable; regulatory environments where a designated recovery region must be pre-approved. See Azure Services - Sample Multi-Region Capabilities Pattern Mapping. References Business continuity and disaster recovery — Azure reliability WAF Reliability — Disaster recovery Azure Site Recovery overview 3. Parallel Workload Distribution (Simultaneous Active Deployment) Pattern Name: Parallel Workload Distribution (Simultaneous Active Deployment) Also Known As: Active-Active Classification: Regional workload distribution, Application Design Scope: Workload — an application-level design pattern; requires the application and its data layer to be explicitly designed for concurrent multi-region operation, Workload (Application) Portfolio Intent: Deploy the same workload simultaneously to two or more Azure regions, all serving production traffic. Context: Two independent drivers lead to this pattern, often simultaneously: the workload serves geographically distributed users who require regional proximity to meet latency targets, and the availability tier demands zero-downtime through region-level failures. A single deployment point cannot satisfy both. Manual failover timelines, standby region promotion, and RPO windows are incompatible with the required availability tier. This pattern is the mandated baseline for Mission-Critical workloads in the Azure Well-Architected Framework. Problem: Passive standby and failover mechanisms introduce recovery time and data loss windows that are incompatible with high-availability targets (e.g. 99.99%+). Meeting both demands requires all regions to be equal active participants — not a primary and a standby. But this demands that the data layer either supports concurrent writes across regions, relies on continuous cross-region replication with a defined consistency model, or is predominantly read-heavy — and that pre-provisioned capacity is maintained in every active region at all times. Forces: Regional failure must produce zero downtime — manual failover timelines cannot satisfy the availability target. Traffic originates from geographically distributed users who require regional proximity to meet latency SLAs. Availability targets (e.g. 99.99%+) eliminate passive standby as a viable option. The application can tolerate eventual consistency or is read-heavy enough that multi-region write complexity is manageable. Once multi-region data access is solved, every region's compute actively serves production traffic — the capacity that Failover Hot Standby holds idle is fully utilised; the cost multiplier buys active production capacity, not idle insurance. Solution Deploy the workload identically to two or more Azure regions. Route production traffic to all active deployments simultaneously via a global load balancer — under normal conditions, each user is directed to the nearest active region, minimising latency. On regional failure, the load balancer automatically rebalances traffic to the remaining healthy regions — no manual promotion, no scale-up delay. All regions are equal peers; there is no concept of primary and secondary. Structure: Implementation: Deployment Stamps are commonly used to implement Parallel at scale — multiple active regional instances behind global routing. See Deployment Stamps pattern — Azure Architecture Center. Stamps are not exclusive to Parallel: the same approach can also support Segmented and, in some designs, Failover. Regions active: 2+ (all serving production traffic simultaneously). Typical cost multiplier: ~2–3×. Microsoft guidance — Mission-Critical workloads: The Azure Well-Architected Framework’s Mission-Critical design methodology explicitly advises active-active multi-region deployment as the baseline for workloads targeting 99.99% availability or higher. The application design guidance states: “The application must be able to withstand regional and zone failures. It must be deployed in an active/active model so that the load is distributed among all regions.” The regions and availability zones guide reinforces this: “Mission-critical workloads should use both multiple availability zones and multiple regions.” The WAF Reliability pillar describes active-active as the mechanism to achieve zero downtime, noting it is “ideal for mission-critical workloads that require uninterrupted availability.” Consequences Benefits: Regional failure triggers automatic traffic rebalancing — no manual failover, no service interruption. Lowest RTO of all patterns; zero-downtime regional failure recovery. On regional failure, remaining regions absorb redirected traffic immediately — if at capacity, the workload degrades under load rather than failing completely; degraded performance is a fundamentally better failure mode than an unavailability window. Serves geographically distributed users within latency bounds simultaneously from the nearest active region. Compute deployed per region actively generates production value under normal conditions — the nominal cost multiplier buys utilised capacity, not idle standby insurance. Liabilities: Highest nominal cost (~2–3×) — though compared to Failover Hot Standby (~2×), the effective cost of resiliency is lower: every unit of deployed capacity actively serves production traffic rather than sitting idle as insurance. Requires the application to support multi-region writes or be predominantly read-heavy; write-conflict resolution is an application responsibility. Multi-region CI/CD, distributed observability, and write-conflict handling add steady-state operational overhead — but eliminate the failure-event burden: no failover procedure, no drill schedule, no risk of untested recovery paths inflating actual RTO. Risk posture: RiskAssessmentRegion Fault✅ Primary driver; automatic traffic rebalancing handles both partial regional degradation and full regional loss — no promotion or manual steps requiredService Fault (Global)⚠️ No regional workload distribution pattern mitigates a truly global service disruption — but the impact is often partial: only specific SKUs, tiers, or versions of a service may be affected, leaving workloads on unaffected variants operational. Where the risk is intolerable, the mitigation is service substitution: switching to an alternative Azure service with equivalent functionality, or a third-party / self-hosted equivalentService Fault (Region)✅ Automatic rebalancing redirects traffic away from the affected region without manual failoverRegional Service Capacity Constraint✅ All regions are pre-deployed and running; no on-demand capacity provisioning required at failover time — if a region fails and remaining regions reach capacity limits, the result is degraded performance under load, not complete unavailability Known Uses Mission-Critical workloads targeting 99.99%+ availability per WAF guidance; globally distributed consumer applications where regional proximity is a primary SLA requirement; financial trading and payment platforms where any recovery window is commercially unacceptable; real-time communication and streaming services where failover lag degrades the user experience. See Azure Services - Sample Multi-Region Capabilities Pattern Mapping. References Mission-critical baseline architecture on Azure WAF Reliability — Mission-critical workloads Azure Cosmos DB — Distribute data globally Azure Front Door — Multi-region load balancing 4. Segmented Workload Distribution (Regional Distribution by Boundary) Pattern Name: Segmented Workload Distribution Also Known As: Deployment Boundaries, Regional Portfolio Allocation Classification: Regional workload distribution (portfolio scope) Scope: Portfolio / Organisation — a structural pattern for distributing a portfolio of workloads, tenants, or business units across regions; individual workloads within a segment may independently apply any other pattern Intent: Assign each Azure region a distinct, non-overlapping responsibility boundary so that regions are differentiated by ownership and isolation rather than by redundancy. Context: A portfolio of workloads, a multi-tenant application, or a multi-LOB organisation must distribute services or data across regions. The drivers include regulatory boundaries, tenant isolation, blast-radius containment, or operational independence — not simply increasing redundancy. Individual boundaries within the portfolio have materially different criticality tiers, release cadences, and recovery requirements. Problem: Running a second Azure region purely as a Failover standby generates ongoing cost without delivering business value under normal conditions. How can an organisation operate multiple regions so that every region carries real production workload, the cost is justified by utilisation rather than insurance alone, and each region's scope is independent enough that faults and changes in one area do not propagate to others? Forces: Regulatory, sovereignty, or compliance obligations drive geographic boundary placement — but strict data residency that prohibits cross-boundary replication also prevents the cross-region recovery that makes Segmented cost-efficient; a boundary whose data cannot leave its region can only recover within that region (Single Region posture), regardless of what neighbouring segments deploy. Independent release cadences, lifecycle autonomy, and scaling requirements across services, tenants, or business units conflict with coupled shared-infrastructure deployments. Blast-radius containment requirements prevent a single fault or bad deployment from affecting the entire portfolio. This pattern does not answer how each boundary recovers; that choice is made independently per boundary using Single, Failover, or Parallel. Solution Define non-overlapping responsibility boundaries and assign each boundary to a region. Each region owns its boundary exclusively — no region is a replica of another. Each boundary independently selects its own recovery posture (Single, Failover, or Parallel) based on its own criticality and requirements. The key structural advantage is region reuse: because multiple regions are already deployed and carrying real production load, each region can simultaneously serve as the Failover standby or Parallel peer for a neighbouring boundary — the same infrastructure investment delivers both production utilisation and recovery capability. This dual-purpose reuse is only available where cross-boundary data replication is permitted; where strict data residency prohibits it, each boundary must treat itself as isolated and plan recovery within its own region. Structure: Implementation: The boundary can be defined at different scopes: Within one application: tenants, markets, release rings, or data partitions assigned to different regions. Across a service portfolio: different applications, domains, or business capabilities intentionally placed in different regions. Segmentation axisExampleGeography / data residencyEU services and data in West Europe, US services and data in East USBusiness unit / LOBFinance portfolio in Region A, HR portfolio in Region BCustomer tierPremium customer workloads in dedicated region(s), standard in shared region(s)Release ringRing 0 workloads in Region A, Ring 1 workloads in Region BScale tierHigh-volume service groups in larger regions, low-volume groups in smaller regions Segmented is orthogonal to the Single / Failover / Parallel dimension. It answers what is each region responsible for?, not how does each responsibility recover? A global organisation might run some boundaries as Single, some as Failover (warm standby), and selected mission-critical boundaries as Parallel — all within one Segmented topology. Consequences Benefits: A fault in one boundary is contained to that region and does not propagate to adjacent boundaries. Each boundary independently selects its own recovery posture, cost level, and compliance configuration. Supports independent release cadences, scaling policies, and lifecycle management per boundary. Region reuse: already-deployed regions carrying production load can simultaneously serve as Failover standby or Parallel peer for neighbouring boundaries — the infrastructure investment delivers both production utilisation and recovery capability without paying for idle standby capacity. Liabilities: Cross-boundary dependencies — shared identity, shared data stores — undermine isolation guarantees and must be minimised by design. Governance overhead scales with the number of active boundaries; requires a formal boundary ownership model to remain manageable. Risk posture: RiskAssessmentService Fault (Region)⚠️ Fault is contained to the affected boundary; adjacent boundaries continue operating — within-boundary recovery depends on that boundary’s postureRegion Fault⚠️ Only the boundary hosted in the affected region is impacted — RTO/RPO is determined by that boundary's individual recovery postureRegional Service Capacity Constraint⚠️ Only the boundary in the capacity-constrained region is affected; other boundaries continue operating — mitigation depends on the boundary's own topology (Failover or Parallel provides alternatives; Single does not)Geopolitical Risk✅ Boundaries can be relocated independently; the rest of the estate continues operating while the affected boundary is relocatedService Regional Unavailability✅ Each boundary can be independently placed in a region where all required services are available Known Uses Geo-distributed enterprise application portfolios; organisations with a federated business model where autonomous business units operate independently with their own release cadence, cost accountability, and compliance obligations; SaaS platforms with tenant-per-region isolation; regulated financial and healthcare services with strict data residency by jurisdiction. See Azure Services - Sample Multi-Region Capabilities Pattern Mapping. 5. Portable Workload Distribution (Full Abstraction) Name: Portable Workload Distribution (Full Abstraction) Also Known As: Region-Agnostic Deployment, Cloud-Neutral Deployment Classification: Operational property (applicable to any structural pattern) Scope: Workload — a design property of an individual workload; composable with any structural pattern at either workload or portfolio scope Intent: Fully abstract the workload from the underlying Azure environment so it can be relocated to any Azure region at any time without modifying application code or configuration. Context: A workload is subject to compliance obligations — regulatory, geopolitical, or sustainability — that may require region relocation on short notice. Or the workload's operational requirements include the ability to optimise cost, respond to capacity constraints, or avoid service unavailability across regions. Portability is not the default outcome — it requires an explicit design decision and sustained engineering investment. Without that intent, the default is a workload that is structurally bound to its current region. Problem: Without a portability investment at design time, a relocation trigger forces significant rearchitecting under pressure rather than as a controlled migration. Forces: Compliance obligations — regulatory, geopolitical, or sustainability — may require relocation on short notice; deferring the portability decision converts a design choice into a forced rearchitecting event at the worst possible time. Data portability is the hardest dimension: scheduled backup/restore, continuous replication, and live migration each introduce cost, complexity, and consistency trade-offs that must be accepted at design time. Relocation may be temporary or permanent; the architecture must support both without distinguishing between them at deploy time. A port may be partial (subset of workloads) or full (entire estate); partial porting creates transient cross-region dependencies that must be explicitly designed for and eliminated as the migration progresses. Solution Solution: Select and implement a data portability mechanism — continuous replication, backup/restore, or live migration — whose cost, RPO, and operational model are explicitly accepted at design time. The target region is not fixed at design time — it is chosen when a trigger event occurs, and can be any eligible region. The workload stays in its current region under normal conditions and relocates only when a trigger event warrants it. Structure: Implementation: Portable is a layered property, not a separate structural topology. The underlying structural topology (Single, Failover, Parallel, or Segmented) determines traffic routing and redundancy; Portable governs whether that topology can be instantiated in a different region without application changes. The cost multiplier adds toolchain and abstraction overhead on top of the chosen structural topology. Data portability — the critical path: Compute portability is straightforward — container images and environment-agnostic configuration are standard practice. Data portability is the harder problem. Two mechanisms make it achievable: Continuous replication: The data layer replicates to the target region at all times, so the data is already present when a relocation is triggered. Azure Cosmos DB with multi-region writes, Azure SQL Database geo-replication, and Azure Storage geo-redundancy (RA-GRS/RA-GZRS) are common implementations. Continuous replication minimises RPO but adds steady-state cost. Automated data migration: A codified and continuously tested migration pipeline moves data to the target region at relocation time. Appropriate when continuous replication cost is not justified, or when the data tier does not support native geo-replication. The migration must be automatable, testable in isolation, and fast enough to satisfy the workload’s RTO for the trigger event. Port modes: Partial vs. Full A port — the act of relocating a workload using the Portable pattern — can be scoped in two ways: Partial port: A subset of workloads is relocated to the target region while others remain in the source region. This creates transient cross-region dependencies — service calls, data access, shared identity — between moved and not-yet-moved workloads. These dependencies must be explicitly designed for, monitored for latency and failure, and eliminated progressively as the migration advances. Partial porting is the natural execution mode for large estates where simultaneous full relocation is operationally infeasible. Full port: All workloads are relocated to the target region, either simultaneously or in a planned sequence that keeps cross-region dependencies only for the duration of each step. A full port is a long-term, intentional change of primary region. It is fundamentally different from a Failover event: DimensionFailoverFull Port (Portable)IntentQuick recovery and return to primary region when resolvedPermanent change of the regionDurationTemporary — primary is restored after the eventLong-term or permanent — target becomes the new primaryReturnExpected — traffic and workloads revert to original regionNot expected — no return is plannedDriverRegion fault, outage, or transient unavailabilityCompliance, cost, sustainability, or strategic decisionTransition TimeAiming RTOTransition can take weeks or months (but it can be designed to serve the Failover purpose and meet RTO and RPO requirements)Transition disruptionMinimal — automated or semi-automated failoverManaged — gradual migration with a cross-region dependency period The Portable pattern must support both modes and both scopes. A workload that can only be relocated as an atomic all-or-nothing operation has limited practical utility; a workload designed for incremental partial porting is far more executable at scale. Consequences Benefits: Workload can relocate to any Azure region without application changes — eliminates region lock-in. Target region is determined at the time of the trigger, not at design time — unlike Failover's fixed designated standby, the destination can be any eligible region and can change between port events as requirements evolve. Primary architectural mitigation for compliance-driven relocation risks (Geopolitical Risk, Sustainability Constraint). Relocation can be temporary (workload returns after trigger resolves) or permanent — the architecture supports both without distinguishing between them. Liabilities: Portability is costly to establish and maintain; data portability and the abstraction layer add ongoing engineering and toolchain overhead. Data portability is the hardest and most underestimated engineering challenge — relocating compute is straightforward; relocating live data at acceptable cost, latency, and consistency is not. Introduces dependency on the abstraction toolchain — portability conventions must be actively enforced as engineering standards; without governance, individual implementation decisions erode them over time. Risk posture: RiskAssessmentGeopolitical Risk✅ Primary driver — region-agnostic workload relocates to a compliant region; unportable workloads face forced migration under time pressureSustainability Constraint✅ Workload moves to a region with the required sustainability profile without application-level changesRegional Service Capacity Constraint✅ Workload can be relocated to an alternative region with available capacity — addresses both on-demand provisioning failure and sustained shortagesRegion Fault✅ Workload can be relocated to an alternative region; recovery speed depends on data portability readinessService Regional Unavailability✅ Workload can be redirected to any region where required services are available — portability removes the fixed-region constraint Known Uses Workloads subject to data sovereignty or geopolitical obligations that may require region relocation on regulatory notice; sustainability-committed workloads that may need to move to regions with a lower carbon intensity; workloads in rapidly expanding organisations that need to follow business growth into new geographies without rearchitecting. Several patterns are adapted from Gregor Hohpe's cloud strategy patterns: Multi Cloud Architecture: Decisions and Options — Architect Elevator. See Azure Services - Sample Multi-Region Capabilities Pattern Mapping. Related Patterns Portable vs. Failover — Both result in a workload running in a different region, and a full port and a Hot Standby activation look nearly identical at execution time. The distinctions are fundamental: Failover is a topology (one region is permanently designated as standby); Portable is a property that can be layered on top of any topology. Failover requires a fixed secondary region to be designated at design time — the target is known, pre-provisioned, and not interchangeable. Portable has no fixed target: the destination region is chosen at the time of the trigger, can be any eligible region, and can differ between port events as compliance, cost, or operational requirements change. Failover is triggered by an unplanned disruption and is temporary — the expectation is to return to the original primary when the event resolves. A Portable port is triggered by a deliberate decision and is permanent — the destination becomes the new primary with no planned return. A Failover workload can also be Portable, applying both patterns simultaneously: Failover handles unplanned disruptions; Portable handles deliberate relocation decisions. They are orthogonal — not alternatives. From Patterns to a Cloud Growth Strategy The patterns in this article show that regional distribution is not a single decision — it is a spectrum of options, each with a different cost, complexity, and risk mitigation profile. The patterns are not mutually exclusive: a portfolio — or even a single workload — may combine multiple patterns to achieve its reliability, compliance, and operational goals. Having the full range of patterns available does not answer the harder question: which patterns are right for your organisation, which applications need them, and what will it take to get there? That is a strategy question — and it requires a deliberate answer. Most organisations today operate their workloads in the Single Region or Failover pattern. Transitioning from that baseline to Parallel, Segmented, or Portable distribution is not an infrastructure change — it is a programme of work that requires investment justification, architectural readiness, and a governed execution plan. A Cloud Growth Strategy based on regional workload distribution starts by defining the organisation's objectives: what reliability targets must be met, which compliance or sovereignty constraints apply, what operational scale is planned, and where the current estate falls short. From those objectives it derives what the regional footprint should look like for this organisation — not as a generic best practice, but as a concrete commitment about which patterns apply to which parts of the portfolio, at what pace, and at what cost. Conclusion Azure's global footprint makes regional workload distribution a choice for every workload — but not a requirement for all of them. The decision starts with risk: the Risk Catalog identifies which fault types and capacity or compliance constraints are actually relevant to the workload, and what blast radius each carries. From that foundation, five patterns emerge — Single Region, Failover, Parallel, Segmented, and Portable — each resolving a distinct set of forces at a different cost and complexity point. Real Azure services rarely fit one pattern cleanly; the service examples in this article illustrate how capability gaps, consistency models, and replication architectures constrain which patterns are structurally achievable. Translating this into organisational practice requires a deliberate Cloud Growth Strategy: classify applications by criticality, assess their suitability for each pattern against their current state, and produce a governed distribution map that is maintained as the portfolio and platform evolve.Choosing two-zone and three-zone patterns for zone-resilient Azure workloads
This article complements our Advancing Reliability series post, Two zones or three? A design framework for zone-resilient Azure workloads. Together, they provide guidance on both the considerations and deployment patterns for zone-resilient Azure workloads. Purpose This article helps customers evaluate Azure workload components and choose zone-resilient patterns that meet their requirements during a single-zone failure. The goal is to identify where two zones can meet workload requirements, where three zones are required, and where service-managed zone redundancy is the right approach. Important: This guidance is a decision framework, not a service support matrix. Availability zone behavior varies by Azure service, SKU, tier, region, and configuration. Always validate the selected design against the relevant Azure reliability guidance and test the workload's actual failure behavior before finalizing the architecture. Zone resiliency helps protect against the loss of a single availability zone. It does not protect against a full-region outage. For mission-critical workloads or workloads with strict disaster recovery requirements, evaluate whether the workload also requires a multi-region design. Availability zones are separate groups of datacenters within an Azure region. Each availability zone has independent power, cooling, and networking infrastructure. Azure services that support availability zones generally expose support through zone-redundant or zonal deployment models. Key takeaways Evaluate zone patterns at the workload component level, not only at the whole-workload level. Many workloads can meet single-zone failure objectives by using two zones for some components and three zones only where component requirements demand them. For the single-zone failure mode, two-zone and three-zone patterns can both meet resource availability objectives when remaining capacity and failover behavior are validated. Use service-managed zone redundancy when it meets workload requirements. Use two zones when the component can meet its availability, durability, capacity, performance, and operational requirements across two zones. Use three zones when a component requires the additional zone for post-failure capacity, data durability, replication topology, quorum, leader election, or operational margin. Balance cost and operational complexity after defining the workload's resiliency objective. Do not assume that two zones are always cheaper or that three zones are always required. Zone-redundant and zonal deployment models Zone-resilient designs depend on the deployment model used by each Azure service. Zone-redundant resources are distributed or replicated across multiple availability zones. Azure manages replication, request distribution, and failover behavior for the service. Where available and aligned to workload requirements, zone-redundant resources should be preferred, especially for production deployments. Zonal resources are pinned to a specific availability zone. A zonal resource is isolated from failures in other zones, but it is not automatically resilient to a failure in its own zone. To make a zonal service resilient, customers need to deploy separate instances across multiple zones and design the workload to route traffic, replicate data, detect failures, and recover. Some Azure services offer both zonal and zone-redundant deployment options, while others may be zone-resilient by default. In some cases, availability zone support requires additional configuration, service modification, or redeployment. Support can also vary by region, SKU, tier, or service configuration. Before choosing a deployment model, review the service-specific reliability guidance and availability zone support matrix to understand the options and requirements for your scenario. Primary decision dimensions Current Azure Well-Architected guidance emphasizes deploying production workloads across two or more failure domains to improve resiliency to failures within a single failure domain. In Azure regions that support availability zones, those failure domains are exposed as availability zone boundaries. When deciding whether a component should use two zones, three zones, or service-managed zone redundancy, evaluate three dimensions: Resource availability: For the single-zone failure mode, two-zone and three-zone patterns can both meet resource availability objectives when the remaining zone or zones can support the required operating state and failover behavior. The additional zone in a three-zone design does not by itself make the component resilient to more than one zone failure within the same region. Data consistency and durability: Stateful components might require three zones when data durability, replication topology, quorum, consensus, leader election, or split-brain prevention depends on a third failure domain or third replica placement. Cost and capacity: For the same post-failure performance target, a two-zone design can require more recovery capacity than a three-zone design. Cost optimization should be evaluated after the resiliency objective is defined. If more than one availability zone is unavailable in the same region, the concern can become regional rather than only workload-specific because foundational regional services require at least two surviving availability zones for continued regional availability. Workloads with requirements beyond a single-zone failure should evaluate disaster recovery or multi-region design separately. Common workload components can be grouped as follows: Component category Typical zone decision Stateless resources that support networking or application code and do not store persistent data Two-zone or three-zone flexibility, based on remaining capacity, routing, latency, and operational requirements. Stateful resources that use node-based quorum, consensus, or leader election Three zones, a third failure domain, or a product-specific witness pattern is commonly required to avoid split-brain or quorum-loss scenarios. Critical data stores that require three replicas for the highest durability targets Three-zone replication might be required to support the intended durability level, such as eleven-nines-style durability targets. Validate service-specific claims. Other stateful resources Two-zone, three-zone, or service-managed patterns can be valid depending on service behavior, recovery time objective (RTO), recovery point objective (RPO), durability, failover, and recovery requirements. Component-level decision framework Walk through the workload by critical flow and component. For each component, determine whether service-managed zone redundancy applies, whether a two-zone pattern can meet the component's single-zone failure objective, or whether three zones are required. Component question What to evaluate Decision guidance Is zone resiliency managed by the Azure service? Confirm whether the service provides zone-redundant or zone-resilient behavior through a supported configuration, SKU, tier, or replication mode. Use service-managed zone redundancy when it meets workload requirements. Do not force a two-zone or three-zone customer-managed design onto services where Azure manages placement and failover internally. Is the component stateless or easily replaceable? Evaluate routing, health probes, scale-out behavior, post-failure capacity, deployment automation, monitoring, and recovery steps. Two zones can often meet requirements for stateless or easily replaceable components when the remaining zone can support the required degraded or full operating state. Use three zones when capacity distribution or operational requirements justify the added design and testing scope. Does the component store durable state? Evaluate replication mode, data durability, consistency, RTO, RPO, failover behavior, recovery behavior, and service-specific support. Use the pattern that satisfies the data protection and recovery requirements. Three zones might be required when the storage or data service requires an additional zone for durability, triple-replica placement, replication topology, or recovery behavior. Does the component use quorum, consensus, or leader election? Validate replica placement, majority behavior, witness or tie-breaker design, leader election, split-brain prevention, recovery, and failback. Do not assume that three replicas across two zones are sufficient. Three zones, a third failure domain, or a product-specific witness pattern might be required to tolerate a zone loss safely. Is the component latency-sensitive? Define latency and throughput thresholds, test candidate zone pairs with realistic protocols and configuration, and identify which paths are actually latency-sensitive. Two zones can be appropriate for latency-sensitive synchronous paths that use tested placement. Logical zone numbers can map to different physical zones across subscriptions, so validate zone mapping when selected zone pairs matter. What capacity must remain after one zone fails? Define the required post-failure operating state for the component: degraded but acceptable, full baseline, or standby recovery. Choose the zone pattern that can satisfy the required post-failure operating state. Two zones can meet requirements when each remaining zone has enough capacity for the target state. Three zones can reduce the capacity each zone must carry for the same post-failure target. How will the design be operated and reviewed? Assign owners for monitoring, testing, incident response, failover, failback, and periodic reassessment. Confirm that identity, network security, encryption, secrets management, policy, monitoring, and data protection requirements are preserved. Document why each component uses service-managed zone redundancy, two zones, or three zones. If three zones are required for a component, state the requirement that drives that decision. Where two zones can meet requirements A two-zone pattern can be a practical way to meet single-zone failure objectives when the component's requirements are satisfied across two zones. Common examples include: Stateless compute, application, or network components where traffic can be routed to the remaining zone. Pairwise active-passive or active-active designs that are simpler to deploy, test, and operate across two zones. Latency-sensitive synchronous paths where a tested zone pair meets performance requirements. Customer-managed zonal resources where the remaining capacity, failover process, monitoring, recovery, and failback behavior are validated. Components whose acceptable degraded operating state can be supported after one zone is unavailable. The design should define what happens after one zone is unavailable, including remaining capacity, acceptable degradation, data consistency, failover behavior, recovery steps, observability, and operational ownership. Where three zones are required Three zones are required when two zones cannot meet the component's requirements during or after a single-zone failure. Examples include: A required post-failure operating state that cannot be supported by the remaining zone in a two-zone design. Data durability or replication requirements that depend on placement across three zones, such as triple-replica placement for the highest durability targets. Quorum, consensus, or leader-election designs that require a third failure domain or witness placement to avoid losing quorum or creating split-brain risk. Three zones still primarily help a workload tolerate a single availability-zone failure within a region. When this guidance refers to an additional failure domain, it means an additional Azure availability-zone placement option or an application-level failure domain needed by a specific component design, such as quorum, witness, or leader-election behavior. Where three zones may add value Some components do not strictly require three zones but can benefit from the additional placement option. In these cases, the value is usually related to capacity distribution, maintenance flexibility, or operational margin, not a different single-zone availability failure mode. Use three zones when the additional zone helps the component meet the required post-failure operating state, reduces how much capacity each zone must carry, or improves the ability to maintain and recover the component without violating workload requirements. Data consistency and durability considerations Stateful components should be evaluated based on consistency, replication, quorum, durability, RTO, and RPO requirements. Some distributed systems use quorum, leader election, or consensus-based coordination to maintain consistent state. Three zones are not universally required for every quorum-based architecture, but quorum-based systems require careful validation. Replica count alone is not enough. Replica placement, majority behavior, witness or tie-breaker placement, leader election, write consistency, split-brain prevention, recovery behavior, and failure-domain assumptions all affect whether a two-zone or three-zone deployment is appropriate. Some systems can place three or more replicas across two zones, and some managed services provide zone-resilient behavior without exposing direct customer control over zone placement. These patterns should not be generalized. In customer-managed majority-quorum systems, placing replicas across only two failure domains can still lose quorum if the majority-holding zone is unavailable. A third failure domain, witness, or tie-breaker might be required to tolerate a zone loss safely, depending on the product architecture. For customer-managed stateful systems, validate replica count and placement, quorum behavior, leader election behavior, synchronous and asynchronous replication, split-brain prevention, failure recovery behavior, data durability requirements, and RTO and RPO objectives. For managed data services, understand how the service implements zone resiliency, which configuration choices are available, and how the service behaves during zone-down scenarios. Capacity planning for a single-zone failure Capacity planning should start with the component's required operating state after the loss of one availability zone. Determine the capacity required to support the component in an approved degraded resiliency state, then use that post-failure target to compare zone patterns. For active-active deployments, use: Total Capacity = Target Remaining Capacity x Z / (Z - 1) In this formula: Target Remaining Capacity is the capacity the component must have after one zone is unavailable. Z is the number of zones used by the component. For example, if a component requires 100 units of baseline capacity and can operate at an approved degraded level of at least 80 units after losing one zone, the model estimates: Deployment model Formula Total provisioned capacity Capacity per zone Remaining after one zone loss Two availability zones 80 x 2 / (2 - 1) 160 units 80 units 80 units Three availability zones 80 x 3 / (3 - 1) 120 units 40 units 80 units This example compares designs against the same post-failure operating objective. The acceptable degraded state should be explicitly defined, tested, and approved. It should include the minimum remaining capacity, expected throttling or prioritization behavior, scale-out assumptions, and how long the component can remain in the degraded state. If the component must maintain full baseline capacity after one zone is unavailable, use the full baseline capacity as the target remaining capacity. Capacity requirements depend on workload architecture, scaling behavior, service limits, failover behavior, and acceptable degradation. Cost and operational considerations Cost and operational complexity should be evaluated after the resiliency objective is defined. They should not be used to dismiss two-zone designs that can meet component requirements, and they should not be used to justify two-zone designs that do not meet requirements. For the same post-failure capacity target, a three-zone design can require less total provisioned capacity than a two-zone design because the recovery capacity is distributed across more zones. Use the capacity model to understand that tradeoff before optimizing costs. For workloads with predictable usage, evaluate commitment-based discounts such as Azure savings plans or Azure reservations where they apply to the selected services. Operationally, consider whether the selected pattern can be deployed, monitored, tested, failed over, recovered, and reviewed consistently. A two-zone pattern can be simpler for pairwise designs. A three-zone pattern can provide more operational margin for components that benefit from additional placement options. Component classification workflow Before finalizing the zone pattern, confirm that: Each critical flow is decomposed into the components that support it. Each component is evaluated across resource availability, data consistency and durability, and cost or capacity impact. Each component is classified as service-managed zone-redundant, two-zone customer-managed, or three-zone required. The reason for requiring three zones is documented when two zones do not meet the component objective. Each Azure service's zone support, SKU, tier, region, and configuration requirements are validated. Customer-managed zonal resources have validated routing, load balancing, replication, failover, monitoring, recovery, and failback. Remaining capacity and acceptable degradation after one zone loss are documented. Stateful or quorum-based components have validated replica placement, witness or tie-breaker behavior, leader election, and split-brain prevention. Latency-sensitive paths have been tested across the selected zone placement. Security, identity, monitoring, and data protection requirements are preserved. Ownership for testing, incident response, failover, failback, and periodic reassessment is assigned. Related public guidance Azure services that support availability zones Enable zone resiliency for Azure workloads Zonal resources and zone resiliency What are Azure availability zones Architecture strategies for availability zones and regionsAnnouncing the resiliency agent in Azure Copilot, now in public preview
The distance between knowing and doing Most teams already know that zonal resiliency matters. What slows them down is everything in between: assessing posture across subscriptions and regions, deciding which gaps should be prioritized, writing the templates, and confirming the change actually rolled out. That is four separate workstreams, in four separate places, usually picked up long after the application is shipped. The resiliency agent closes that distance. We’re excited to announce that it is now available in public preview as part of the agents in Azure Copilot. The agent brings resiliency assessment, prioritized recommendations, and deployment-ready code into a single conversation, so resiliency becomes something your team does continuously rather than something you audit once a year. Learn more about Azure Copilot agents here. What the resiliency agent does The Resiliency agent is the conversational, action-oriented layer of Azure Infrastructure Resiliency Manager. The platform gives you a goal-driven view of your resiliency posture, and the agent lets you act on that posture in plain language. It brings together the signals you would otherwise chase manually across availability zones, Azure Backup, Azure Site Recovery, service level indicators, and Azure health models, then turns them into workload-aware guidance you can execute. Starting with zonal resiliency, it is designed to complement the resiliency capabilities you already rely on rather than replace them. The agent is built on the same foundational belief as Azure Infrastructure Resiliency Manager: application resiliency is a continuous journey, not a one-time task. That journey runs in three phases: Start Resilient Get Resilient Stay Resilient. This public preview addresses the activities associated with first two phases, frontloading the capabilities customers asked for most: direct remediation, Infrastructure as Code integration, and cost visibility. Stay Resilient comes next. What you can do in public preview Generate deployment-ready Bicep and Terraform with zone-redundant settings already enabled, before your first deployment Find the service groups and resources that are not zone resilient across your subscriptions and regions Prioritize remediation with cost-aware indicators that flag whether a fix needs extra spend, downtime, or redeployment Generate ready-to-deploy scripts to configure zone resiliency for supported resource types Create and update service groups so your applications are modeled correctly from the start Start resilient: build it in from the first line of code The most durable resiliency is the kind you never have to retrofit. Starting resilient means treating zone resiliency as an architectural requirement on day one, and the Resiliency Agent makes that the path of least resistance. Describe the application you are building, and the agent helps you generate deployment-ready Infrastructure as Code with the recommended resiliency configurations already enabled at creation time. Ask for a Bicep or Terraform template for a new workload and the agent returns one with zone-redundant settings pre-built for your databases, VMs, and load balancers, aligned to the goals you set. For example, try prompts like: “Generate a Bicep template for a zonally resilient VM setup.” “Create Terraform templates for a resilient architecture with VMs, AKS, and Storage.” The payoff: new services reach production already meeting your high availability standards, and the expensive redesign never has to happen. Get resilient: close the gaps in what you already run Applications change. The configuration that was right at first deployment is often not the one a business-critical workload needs two years later. This is where the resiliency agent turns posture insight into prioritized, executable action. Ask the agent to assess your environment, and it surfaces the resources and service groups that are not zone resilient, that are exposed to a datacenter outage, or that have key alerts waiting on attention. Every recommendation carries cost-aware decision support, qualitative indicators that flag whether a fix requires additional spend, downtime, or redeployment, so your team can sequence remediation against business value instead of guesswork. When you are ready to act, the agent generates ready-to-deploy scripts to configure zone resiliency for supported resource types, or targeted IaC snippets to close one specific gap. Try prompts like: “Assign resiliency goals for my application.” “Give me the posture of my service group: Contoso Payments.” “Enable zonal resiliency for my PostgreSQL server.” “Generate scripts to create a zonally resilient storage account.” This is a human-in-the-loop model by design. The agent brings the analysis, the recommendations, and the code. You keep full control over execution. Remediation gets faster without giving up governance. Stay resilient: what comes next Resiliency is not only about withstanding a zone outage. It is about protecting your data and knowing you can recover when something does go wrong. Recovery orchestration plans and zone drills are available today in Azure Infrastructure Resiliency Manager, and we are bringing both into the agent experience next. That closes the loop: assess, remediate, and validate recovery readiness in the same conversation. Get started with the public preview Open Azure Copilot and select the Resiliency Agent from the dropdown or the side panel. Three actions will get you value in the first session. Model your application. The agent works from your application, not a flat list of resources, so make sure it is represented as a service group that reflects your latest resource discovery. You can create service groups natively through the agent. Try: “Create a service group for my application.” Find your gaps. Ask which of your service groups and resources are not zone resilient. Most teams uncover exposure that was previously invisible. See Set resiliency goals and track compliance. Design it in. Ask for a Bicep template with zone-redundant defaults for a workload you plan to deploy and see how much retrofit effort disappears. See Generate resilient Bicep templates. What to expect in the preview Ready-to-deploy zone resiliency scripts are supported for Azure services that have zonal resiliency support in Infrastructure Resiliency Manager. See the support matrix for current coverage. Some advanced configurations, including multi-user authorization and Azure Site Recovery, may require manual steps, and the agent provides guidance to help you complete them. Where we are going This public preview is a starting line, not a finish. We are building toward end-to-end, agent-led journeys across Start, Get, and Stay Resilient, with cross-agent orchestration that reaches into migration, deployment, and observability, and multi-surface delivery so resiliency validation shows up in your IDEs, APIs, and CI/CD pipelines, not only in Azure Copilot. Service coverage will keep expanding, and the guidance will keep getting richer. The goal is simple and ambitious: make resilient-by-design the path of least resistance for every application on Azure. Preview is where that gets shaped, so tell us what is working and what you need next at azureresiliency@microsoft.com. Start here: open Azure Copilot, select the resiliency agent, and ask which of your applications are not zone resilient yet. Resources Announcing Azure Infrastructure Resiliency Manager public preview Agents in Azure Copilot public preview Resiliency capabilities in Agents (preview) in Azure Copilot, Azure Infrastructure Resiliency Manager overview Get started with service groups Microsoft Learn Questions or feedback: azureresiliency@microsoft.comSimpler, private connectivity between Azure and AWS with Azure Multicloud Interconnect
Multicloud is no longer the exception — it is how most enterprises operate. Teams run analytics in one cloud and applications in another, place workloads to meet data-residency requirements, and increasingly move large volumes of data between clouds to train and serve AI models. Yet the network that connects these environments has remained one of the hardest parts of a multicloud strategy to get right. Connecting Azure and AWS privately has traditionally meant stitching together Azure ExpressRoute, AWS Direct Connect, a connectivity provider or colocation footprint, customer-managed routers, BGP sessions, and link-layer encryption — then owning the resiliency design and day-to-day operations across all of it. The result is often slow to deliver, difficult to troubleshoot, and inconsistent in performance. Today, Microsoft and AWS are together introducing Azure Multicloud Interconnect, a jointly engineered, fully managed service that delivers private, high-throughput connectivity between Azure and AWS through a single logical resource. Both clouds coordinate provisioning, resiliency, encryption, and lifecycle operations, so the connection between your environments simply works — end to end. The challenge with connecting clouds today For most organizations, cross-cloud connectivity has been a build-it-yourself exercise. Establishing a private link between Azure and AWS typically requires provisioning an ExpressRoute circuit on one side and a Direct Connect circuit on the other, engaging a connectivity provider or securing space in a colocation facility to bridge them, and then configuring and maintaining the routers, BGP peering, and encryption that tie the two clouds together. Each of those pieces is owned by a different team or vendor, which makes the end-to-end path only as reliable as its least-managed component. When something breaks, isolating the root cause means coordinating across Microsoft, AWS, a network provider, and your own operations team — and mean time to resolution suffers as a result. Capacity planning is equally difficult: bandwidth is often provisioned for peak demand and left underused, while scaling up to meet a new AI or data initiative can take weeks. The outcome is a connectivity layer that is slow to stand up, expensive to operate, and hard to reason about — exactly the opposite of what a multicloud strategy is meant to deliver. Azure Multicloud Interconnect removes that complexity by turning cross-cloud connectivity into a managed service. What is Azure Multicloud Interconnect? Azure Multicloud Interconnect is a provider-managed, private intercloud connectivity service built on the proven foundations of Azure ExpressRoute and AWS Direct Connect. Instead of assembling and operating the underlying components yourself, you establish one interconnect resource and consume a private, dedicated path between your Azure and AWS environments. Under the covers, Microsoft and AWS coordinate the circuits, routing, and encryption on your behalf. You get the outcome you want — a reliable private connection between clouds — without becoming the systems integrator for it. Because the service builds on ExpressRoute and Direct Connect, it fits naturally into the connectivity models, tooling, and operational practices your teams already use in each cloud. One managed resource replaces a stack of circuits, routers, BGP sessions, and encryption you would otherwise build and operate yourself. Figure 1. Azure Multicloud Interconnect provides a single, managed private path between Azure and AWS, built on ExpressRoute and Direct Connect with a quad-redundant, 400G-class design and MACsec encryption. How it works Azure Multicloud Interconnect is engineered for the performance and availability that production and AI-scale workloads demand: A single logical connection. You provision and manage one interconnect resource. There are no customer-owned routers to rack, configure, or patch between the clouds. Quad-redundant, multi-site design. The data path is built across redundant devices and diverse sites — four Azure Microsoft Enterprise Edge (MSEE) routers and four AWS routers — so there is no single point of failure. High bandwidth backbone. A high-capacity LAG-based design provides the headroom needed for large-scale data movement and distributed AI traffic. Elastic bandwidth. Scale capacity up or down as demand changes, rather than provisioning for peak and paying for it year-round. Encryption by default. MACsec link-layer encryption is enabled automatically, so traffic between clouds is protected without extra configuration. Enterprise-ready networking. The service is IPv6-ready, supports APIPA addressing, and targets a 99.99% availability SLA. Routing and provisioning are coordinated by the platform, which removes the most common sources of cross-cloud connectivity errors — mismatched BGP configuration, inconsistent encryption settings, and asymmetric or fragile failover paths. A foundation you can trust Azure Multicloud Interconnect is not a new, unproven path — it extends two connectivity services that enterprises already rely on: Azure ExpressRoute and AWS Direct Connect. Both are private connectivity backbones trusted for mission-critical hybrid and cloud workloads, and the interconnect inherits their carrier-grade capacity, global edge presence, and operational maturity from day one. It also means your teams do not have to learn a separate model. The interconnect appears as a first-class resource governed by the identity, access, and monitoring controls each cloud already provides, and it can be automated with the tooling you use today. Extending trusted services, rather than introducing a parallel one, is what allows Microsoft and AWS to offer intercloud connectivity as a managed experience with confidence. Why this matters for your team Azure Multicloud Interconnect is designed to change the economics and the experience of running across clouds. Taken together, these benefits shift cross-cloud networking from a specialized, high-effort project to a repeatable, on-demand capability. Instead of dedicating senior engineers to build and babysit intercloud links, your team can direct that expertise toward the applications and data platforms that differentiate your business — while trusting that the connective tissue between clouds is reliable, secure, and ready to scale. Faster time to value. Turn up private Azure–AWS connectivity through a guided, managed workflow instead of a multi-week integration project spanning several vendors. Lower operational burden. Microsoft manages the infrastructure, resiliency, and lifecycle, so your networking team is freed from patching routers and diagnosing cross-cloud faults. Predictable, high performance. A dedicated, private path with high capacity that delivers the consistent throughput and low latency that public-internet or VPN paths cannot guarantee. Security you don’t have to assemble. Private connectivity plus default MACsec encryption keeps intercloud traffic off the public internet and protected in transit. A consistent experience in both clouds. Provision, monitor, and manage the interconnect using the native constructs and tooling your teams already rely on in Azure and AWS alike. Built for the way enterprises use multicloud “I've heard a very consistent message across many years of customer engagements - we are multi-cloud enterprise by design. With this announcement we are taking a burden on connecting clouds away from the customers." Igor Sakhnov, CVP Azure Networking Customers have told us where dedicated, managed intercloud connectivity makes the biggest difference. Azure Multicloud Interconnect is designed for scenarios such as: Distributed AI workloads. Move training data and model outputs between clouds at high throughput to feed pipelines wherever the compute lives. Large-scale data movement. Replicate datasets, back up across clouds, and support analytics that span Azure and AWS. Cross-cloud disaster recovery. Use a second cloud as a resilient recovery target over a private, reliable link. Hybrid and best-of-breed architectures. Run each application on the cloud that suits it best while keeping the connection between them private and performant. Regulated and sovereign workloads. Keep intercloud traffic on a private path to help meet data-residency and compliance requirements. Workload migration. Rehost or rebalance workloads between clouds without re-engineering connectivity for each move. Availability and what’s next Azure Multicloud Interconnect is launching first for Azure and AWS connectivity, the pairing customers ask about most often. Microsoft and AWS are starting with a preview so that you can validate the experience against your own architectures, with general availability to follow. This is the beginning of a broader journey. Building on existing multicloud connectivity capabilities, we plan to extend the managed interconnect model to additional clouds — including Google Cloud (coming soon)— so that a consistent, provider-managed experience can span your entire multicloud estate. As always, the roadmap will be guided by customer demand and real-world use. Our shared goal is simple: make the network the easiest part of your multicloud strategy, not the hardest. By delivering intercloud connectivity as a managed service, Microsoft and AWS want every organization — from a team moving its first dataset between clouds to an enterprise operating at AI scale — to connect Azure and AWS privately, securely, and with confidence, and to do it in minutes rather than months. Get started Learn more about Azure Multicloud Interconnect, Azure ExpressRoute, and AWS Direct Connect from Microsoft and AWS documentation, talk with your Microsoft or AWS account team about joining the preview, and tell us which cloud pairings and scenarios matter most for your organization. We are building this with your feedback. Azure Multicloud Interconnect is jointly delivered by Microsoft and AWS, built on Azure ExpressRoute and AWS Direct Connect. Feature availability, performance targets, and timelines may evolve as the service moves from preview to general availability.4.4KViews2likes0CommentsAzure Incident Retrospective - Please register! Session 2 - Tracking ID: ZJV6-SGG-BW8
Join our upcoming live webcast for a transparent discussion about this recent Azure service incident led by our engineering teams. Network connectivity issues in West US Tracking ID: ZJV6-SGG | Impacted: 23 July 2026 Same content presented in both sessions: pick the one that works best for your timezone! What to expect 📚 Understand What happened, how we responded, and what we learned 💬 Ask Live Q&A with our engineering experts throughout the session 🛠 Learn The fixes we've put in place and guidance for workload resiliency Choose your session Same content presented at both times: pick the one that works best for your timezone: Session 1 17:30 UTC Thursday, 27 Aug 2026 Register now → Session 2 05:30 UTC Friday, 28 Aug 2026 Register now → 10:30 AM US Pacific (PDT) 1:30 PM US Eastern (EDT) 6:30 PM London (BST) 1:30 AM +1 Beijing (CST) 3:30 AM +1 Sydney (AEDT) 5:30 AM +1 Auckland (NZDT) 10:30 PM -1 US Pacific (PDT) 1:30 AM US Eastern (EDT) 6:30 AM London (BST) 1:30 PM Beijing (CST) 3:30 PM Sydney (AEDT) 5:30 PM Auckland (NZDT) Our engineering leaders Jamie Gaudette Vice President Azure Networking Cloud+AI Engineering LinkedIn ↗ ⚠️ Prepare before the livestream Read the Post Incident Review (PIR) ahead of time so you can ask any follow up questions during the live Q&A Helpful resources 🔔 Azure Service Health Alerts Get alerts for relevant incidents by setting up notifications via email, SMS, or webhook 🎥 Past Retrospective Recordings Watch recordings of previous retrospective livestreams 📄 Azure Post Incident Reviews Learn more about PIRs and the retrospective program184Views0likes0CommentsAzure Incident Retrospective - Please register! Session 1 - Tracking ID: ZJV6-SGG-BW8
Join our upcoming live webcast for a transparent discussion about this recent Azure service incident led by our engineering teams. Network connectivity issues in West US Tracking ID: ZJV6-SGG | Impacted: 23 July 2026 Same content presented in both sessions: pick the one that works best for your timezone! What to expect 📚 Understand What happened, how we responded, and what we learned 💬 Ask Live Q&A with our engineering experts throughout the session 🛠 Learn The fixes we've put in place and guidance for workload resiliency Choose your session Same content presented at both times: pick the one that works best for your timezone: Session 1 17:30 UTC Thursday, 27 Aug 2026 Register now → Session 2 05:30 UTC Friday, 28 Aug 2026 Register now → 10:30 AM US Pacific (PDT) 1:30 PM US Eastern (EDT) 6:30 PM London (BST) 1:30 AM +1 Beijing (CST) 3:30 AM +1 Sydney (AEDT) 5:30 AM +1 Auckland (NZDT) 10:30 PM -1 US Pacific (PDT) 1:30 AM US Eastern (EDT) 6:30 AM London (BST) 1:30 PM Beijing (CST) 3:30 PM Sydney (AEDT) 5:30 PM Auckland (NZDT) Our engineering leaders Jamie Gaudette Vice President Azure Networking Cloud+AI Engineering LinkedIn ↗ ⚠️ Prepare before the livestream Read the Post Incident Review (PIR) ahead of time so you can ask any follow up questions during the live Q&A Helpful resources 🔔 Azure Service Health Alerts Get alerts for relevant incidents by setting up notifications via email, SMS, or webhook 🎥 Past Retrospective Recordings Watch recordings of previous retrospective livestreams 📄 Azure Post Incident Reviews Learn more about PIRs and the retrospective program188Views1like0CommentsAzure Virtual Network routing appliance is now generally available
Modern cloud networks are evolving faster than ever. Organizations are building larger AI platforms, connecting more services through private connectivity, adopting IPv6, and expanding applications across regions and business units. As these environments grow, the network becomes a critical foundation for delivering performance, resiliency, and operational simplicity. Today, we're excited to announce the general availability of Azure Virtual Network routing appliance, a managed, platform-native routing service designed to provide high-performance connectivity across Azure virtual networks at cloud scale. Virtual Network routing appliance brings together Azure-native operations, specialized networking infrastructure, built-in resiliency, and high-bandwidth forwarding to help organizations build the next generation of cloud network architectures. Built for the era of AI infrastructure AI is changing the scale at which networks operate. Training clusters, inference services, analytics platforms, data processing pipelines, and distributed application environments generate unprecedented volumes of east-west traffic. These workloads require high-performance connectivity between services, networks, and regions while maintaining operational simplicity. Virtual Network routing appliance provides a managed routing foundation for these environments, enabling organizations to scale network connectivity alongside their AI investments. Instead of building and operating custom routing infrastructure, teams can focus on accelerating innovation, deploying new services, and delivering business outcomes. Scale hub-and-spoke architectures Hub-and-spoke remains one of the most widely adopted network architectures in Azure because it provides centralized governance, simplified operations, and efficient connectivity. As organizations expand, however, these architectures often grow from a handful of virtual networks into hundreds or even thousands of connected environments. Virtual Network routing appliance enables customers to scale these architectures while maintaining a consistent operational model. By providing a dedicated routing layer within the hub, Virtual Network routing appliance simplifies connectivity between applications, shared services, and business units while supporting the scale required by modern enterprise environments. The result is a network architecture that remains manageable even as organizational growth accelerates. Unlock large-scale private connectivity Private connectivity has become the default connectivity model for modern cloud deployments. Applications, databases, platforms, shared services, and partner solutions increasingly depend on private communication patterns across Azure environments. Virtual Network routing appliance provides a centralized routing foundation that helps customers build and scale these architectures while maintaining a consistent private networking experience across their environments. Virtual Network routing appliance can also help scale Private Endpoint connectivity beyond the current 20,000-endpoint HSPE boundary. Looking ahead, it establishes a foundation for further accelerating private connectivity to on-premises environments, without introducing additional architectural specifics. As organizations continue consolidating services onto private connectivity models, Virtual Network routing appliance provides the performance and scale needed to support long-term growth. Accelerate your IPv6 journey IPv6 adoption continues to grow across enterprise, telecommunications, and cloud environments. Organizations increasingly need network architectures capable of supporting IPv4, IPv6, and dual-stack deployments while maintaining operational consistency. Virtual Network routing appliance supports IPv4, IPv6, and dual-stack virtual networks, enabling customers to modernize network architectures and expand address space without introducing new operational complexity. Whether organizations are beginning their IPv6 transition or building IPv6-first architectures, Virtual Network routing appliance provides a consistent routing foundation across both address families. Simplify multi-region architectures Modern applications rarely live within a single region. Organizations increasingly deploy workloads globally to improve performance, resiliency, business continuity, and regulatory compliance. These architectures require a networking foundation capable of supporting connectivity across regions while remaining simple to operate and govern. Virtual Network routing appliance helps customers build scalable multi-region network architectures by providing a centralized, high-performance routing layer that integrates naturally into Azure networking designs. This allows teams to focus on application architecture and customer experience rather than operational management of routing infrastructure. Built for enterprise scale As organizations continue to grow, networking teams face a common challenge: supporting increasing scale without increasing operational complexity. Virtual Network routing appliance was designed to meet this challenge by combining: High-performance routing using specialized Azure networking infrastructure Built-in resiliency and availability zone support Native Azure management and governance integration Support for IPv4, IPv6, and dual-stack deployments Integrated monitoring and observability through Azure Monitor metrics available Configurable bandwidth tiers for production workloads Future support for scaling Private Endpoints beyond 20,000 These capabilities allow customers to build large-scale networking architectures while maintaining a familiar Azure-native operational experience. Learn more Azure Virtual Network routing appliance is more than a new networking resource. It is a foundational building block for the next generation of Azure networking. Organizations are continuing to build larger AI platforms, expand private connectivity, increase multi-region deployments, and modernize network architectures. These transformations require a routing foundation that can scale alongside them. Virtual Network routing appliance provides that foundation, delivering the performance, scale, resiliency, and operational simplicity required for modern cloud networks. Whether you're building an AI platform, expanding a hub-and-spoke architecture, scaling private connectivity, enabling IPv6, or designing a global application footprint, Azure Virtual Network routing appliance helps simplify networking so you can focus on what matters most: delivering innovation faster. Overview of Routing Appliances - Azure Virtual Network | Microsoft Learn4.4KViews1like0CommentsSimplify secure, zone-resilient outbound connectivity with Azure Firewall and StandardV2 NAT Gateway
As organizations modernize their applications in Azure, secure and resilient outbound connectivity has become just as critical as inbound security. Workloads need reliable access to external APIs, SaaS services, operating system updates, and partner endpoints, while still meeting strong security controls, predictable egress IPs, and high availability. Achieving all of this consistently requires using the right networking services together. To make this easier, we’ve updated the Azure Firewall create experience in the Azure portal to include StandardV2 NAT Gateway directly in the deployment flow. This new experience makes it quick and seamless to adopt a secure, scalable, and zone‑resilient outbound architecture from day one by using Azure Firewall and Azure NAT Gateway together. In this post, we’ll cover: Why pairing Azure Firewall with StandardV2 NAT Gateway is a recommended design How this combination simplifies secure and resilient outbound connectivity What’s new in the Azure Firewall portal experience and how to get started Why Azure Firewall and StandardV2 NAT Gateway? Azure Firewall and Azure NAT Gateway are designed to complement each other, each focusing on what they do best: Azure Firewall provides centralized traffic inspection and policy enforcement, including IP address and FQDN filtering, threat intelligence, and logging. StandardV2 NAT Gateway delivers high‑scale outbound SNAT, static egress IPs, and built‑in zone redundancy. StandardV2 NAT Gateway is zone‑redundant by default, automatically spanning availability zones within a region. This means outbound connectivity remains available even during a zonal failure without requiring multiple zonal NAT gateways or additional routing configurations. Together, this pairing cleanly separates: Security policy and inspection handled by Azure Firewall Outbound scale, resiliency, and IP predictability handled by NAT Gateway This separation is key for modern, large‑scale cloud workloads. A recommended outbound architecture In a typical hub‑and‑spoke design: Workloads in spoke virtual networks route outbound traffic to Azure Firewall in the hub Firewall policies inspect and allow the traffic StandardV2 NAT Gateway is attached to the AzureFirewallSubnet in the Hub Approved traffic flows through StandardV2 NAT Gateway for SNAT Traffic exits Azure using static, predictable public IPs This approach provides several important benefits: Scalable SNAT capacity for high‑connection workloads Static outbound IPs for partner allow‑listing and compliance Zone‑resilient outbound connectivity by default For step-by-step architectural guidance, see Integrate NAT gateway with Azure Firewall in a hub and spoke architecture. Built for secure and resilient Azure environments As customers increasingly adopt availability zones, large‑scale VMSS or AKS deployments, and zero‑trust network models, outbound connectivity must be secure, predictable, and resilient. By pairing Azure Firewall with StandardV2 NAT Gateway—and now surfacing this pairing directly in the portal create experience—customers can start with a production‑ready outbound architecture that scales with their environment. What's new in the Azure Firewall create experience in the portal When creating a new Azure Firewall in the portal, customers can now: Select and associate a StandardV2 NAT Gateway during firewall deployment. Reduce post‑deployment configuration and manual touches. Start with a recommended zone-resilient outbound architecture by default. By bringing NAT Gateway directly into the Firewall create flow, the portal helps guide customers toward a more secure and scalable outbound setup—without requiring them to stitch services together after the fact. Get started You can try the updated experience today by creating a new Azure Firewall in the Azure portal and selecting StandardV2 NAT Gateway during deployment. With just a couple clicks of a button: In the Basics tab, configure your Firewall settings (ex., SKU, policy, virtual network). In the *new* Advanced tab, create a new or add an existing StandardV2 NAT gateway and associate StandardV2 public IP addresses or prefixes. The StandardV2 NAT gateway is automatically attached to the Firewall subnet—no additional routing or configuration required. Review and Create. Note: StandardV2 NAT Gateway is not yet available in all regions. If your selected region does not support StandardV2 NAT Gateway, the option to enable StandardV2 NAT gateway will not appear during Firewall creation. Refer to StandardV2 NAT Gateway limitations for more information. For more details, see: Integrate StandardV2 NAT Gateway with Azure Firewall Integrate NAT Gateway with Azure Firewall in a hub‑and‑spoke network Azure NAT Gateway SKUs286Views0likes0CommentsAzure Front Door edge actions: programmable compute for a secure, resilient, AI-ready edge
The need for secure edge programmability As modern web applications increasingly move decision-making closer to users, programmable compute at the edge is becoming a foundational capability for delivering low-latency, personalized, and intelligent experiences. Azure Front Door edge actions introduces lightweight customer-defined logic that executes close to users at Microsoft's global edge (https://aka.ms/edgeactionsblog). The engineering challenge extends well beyond moving code closer to the request path. It is about enabling edge programmability while preserving the core guarantees customers expect from a global edge platform: hyperscale performance and acceleration, strong security and tenant isolation, resiliency, and fast, controlled recovery. That sets up a much higher engineering bar than simply bringing a serverless runtime to the edge. Programmability introduces customer code, new execution paths, runtime dependencies, and additional failure modes directly into the critical request path. Architecture therefore must make flexibility a first-class capability without compromising the operational characteristics of a hyperscale edge platform. Preserving performance at hyperscale The first architectural challenge was preserving the performance characteristics of Azure Front Door while introducing programmable execution into the request path. Every additional execution step has the potential to increase latency, amplify failures, or reduce throughput at global scale. Edge actions was therefore designed to add programmability without changing the fundamental performance profile customers already expect from Azure Front Door. At request time, Azure Front Door evaluates the request, determines whether an edge action should be executed based on the associated rule, invokes the edge actions runtime, and applies the result inline. Because the runtime sits directly in the request path, every design decision was guided by a common principle: keep execution local whenever possible, bound latency when dependencies degrade, and ensure optional compute never becomes a platform-wide latency amplifier. Performance design principles Node-local execution keeps request processing on the same machine whenever possible, minimizing cross-node communications and preserving low latency. Minimized inter-node hops keep the common path compact while still enabling cluster-level fallback when local dependencies deteriorate. Connection reuse through Edge Action Agent reduces gRPC invocation overhead and improves hot path efficiency. Lightweight Hyperlight isolation provides strong tenant isolation with an execution model suitable for latency-sensitive edge workloads. Fast-fail and circuit-breaker protects latency by bounding waits on degraded dependencies and preventing cascading pressure. Together, these architectural choices introduce programmable compute without turning the Azure Front Door data plane into a distributed orchestration layer. The hot path remains local, predictable, and bounded, with fallback used only when necessary to preserve performance across the global edge. Security and tenant isolation by design Running customer-defined code on a shared global edge fundamentally changes the security model. Unlike traditional request processing, programmable execution introduces untrusted customer code directly into the request path, making strong isolation a foundational architectural requirement rather than an operational safeguard. For Azure Front Door edge actions, every execution is designed to run within a dedicated Hyperlight micro-VM, providing hardware-enforced isolation between customer workloads, the Azure Front Door data plane, and the underlying host environment. Security design principles Hypervisor-backed isolation ensures customer code executes within dedicated Hyperlight micro-VM boundaries rather than shared execution environments. Data plane separation isolates edge actions execution from Azure Front Door's core traffic-processing path. Minimal host surface area reduces the attack surface and limits privileged interactions. Restricted execution context exposes only the request information required to process a request. Reduced operational blast radius helps contain compromised or misbehaving workloads. These architectural boundaries extend beyond workload isolation. Azure Front Door's data plane remains physically separated from the edge actions orchestration service, while each execution receives only the minimum context required to perform its task. This defense-in-depth approach reduces both security risk and operational blast radius without compromising performance. Hyperlight: Security without sacrificing performance A key differentiator of Azure Front Door edge actions is its use of Hyperlight micro-VMs to provide hardware-backed isolation without introducing the traditional performance penalties associated with virtual machines. Hyperlight was designed to make VM-level protection practical for high-throughput function execution, enabling strong tenant isolation while remaining suitable for latency-sensitive edge workloads. Edge actions builds this foundation through the edge action orchestrator, which maintains a pool of warm Hyperlight sandboxes ready to serve requests. By reusing pre-initialized sandboxes instead of creating a new execution environment for every request, edge actions minimizes initialization overhead, reduces request latency, and sustains higher throughput under load. The result is a security model based on VM isolation that remains compatible with the performance expectations of a hyperscale edge platform. Critically, performance optimizations do not weaken isolation guarantees. After each execution, sandbox state is cleaned before reuse, ensuring that subsequent invocations cannot access data from prior executions while preserving the efficiency benefits of warm sandboxing. In internal benchmarking, lightweight edge actions executed in less than 2 ms inside Hyperlight, with approximately 1.27 ms of total sandbox overhead, demonstrating that strong isolation and high-performance edge execution can coexist. Security enables resiliency Security and resiliency are closely related architectural goals. Isolation helps contain malformed inputs, unexpected behavior, and execution failures, preventing individual workloads from affecting the broader platform. In a multitenant edge service, isolation is not only a security requirement; it is also a key resiliency mechanism. Resiliency built into the platform Strong isolation is not only a security property, but also a foundational resiliency mechanism. By containing malformed inputs, unexpected behavior, and execution failures within dedicated execution boundaries, the platform prevents individual workloads from affecting neighboring tenants or the broader service. At hyperscale, robust isolation is essential for maintaining customer trust and predictable platform reliability. Building on that foundation, Azure Front Door edge actions was designed around a simple operating principle: failures are inevitable, but their impact must be predictable, bounded, and recoverable. Because programmable compute introduces additional execution paths and runtime dependencies into the request path, resiliency must be built into the control points that determine when to execute, stop waiting, or fall back. The platform incorporates lessons learned from operating Azure services at a global scale, with a focus on minimizing blast radius, maintaining service continuity, and enabling controlled recovery when dependencies fail, overload, or time out. Resiliency principles Bound failure impact through isolation and containment. Recover predictably using health-aware routing and fallback paths. Protect customer availability first through graceful degradation. Fail fast rather than fail slowly to avoid latency amplification. Continuously validate assumptions through Game Days and fault injections. These principles translate into request-time behavior through deadlines, circuit breakers, fail-open behavior, and health-based fallback. Together, they ensure that optional programmable execution enhances application capabilities without compromising the stability of Azure Front Door's core request-processing pipeline. Continuous validation of resiliency assumptions Resilient architecture is credible only when validation becomes part of the operating model. For edge actions, Game Days and Fault Injections provide recurring opportunities to verify that architectural assumptions continue to hold under production-like stress. Validation includes chaos and failure injections, timeout and dependency-loss exercises, overload and queue-growth scenarios, mixed-workload testing, and interface fuzzing. These exercises answer practical production questions: Does fail-open behavior protect the request path? Do circuit breakers engage early enough? Does fallback routing preserve service continuity? Do malformed inputs remain contained? Repeated validation also strengthens operations. Detection improves, mitigation becomes more predictable, and recovery evolves from architectural intent into demonstrated operational capability. Built for future intelligent & modern workloads Edge actions is designed for lightweight programmable execution today, but the underlying architecture is intended to support increasingly intelligent decision-making over time. The engineering requirement remains unchanged: future intelligence workloads must operate within the same architectural constraints that govern today's request processing - bounded execution, strong isolation, predictable fallback, and protection of the common request path. Architectural implications for intelligence workloads Real-time AI inferencing for request classification and policy evaluation. Intelligent bot, abuse, and fraud detection closer to users. AI-assisted origin selection and traffic-routing decisions. Application-specific SLM-powered decision making at the edge. In that model, the objective is not simply to introduce more intelligence at the edge, but to ensure that intelligence inherits the same platform guarantees as every other component of the request path. Closing thoughts Programmable edge execution is becoming a foundational capability for modern distributed applications. The engineering challenge, however, extends far beyond running customer code closer to users. It is about preserving the system properties that customers already depend on while introducing a new execution surface into the critical request path. Edge actions demonstrates that edge programmability, performance, security, tenant isolation, and resiliency are not independent design goals - they are a single architectural problem that must be solved together. By keeping the common path protected, failures bounded, tenants strongly isolated, and recovery predictable, Azure Front Door edge actions extends the platform's capabilities without compromising the engineering principles that underpin a global hyperscale edge service. Learn more Introducing Hyperlight Edge actions samples: JavaScript request context597Views1like0CommentsAzure Front Door: Resiliency Series – Part 2: Faster recovery (RTO)
Abhishek Tiwari, Vice President of Engineering, Azure Networking Amit Srivastava, Partner Director of PM, Azure Networking Varun Chawla, Partner Director of Engineering, Azure Networking Karthik Uthaman, Principal Engineer, Azure Networking In Part 1 of this blog series, we outlined our four‑pillar strategy for resiliency in Azure Front Door: configuration resiliency, data plane resiliency, tenant isolation, and accelerated Recovery Time Objective (RTO). Together, these pillars help Azure Front Door remain continuously available and resilient at global scale. Part 1 focused on the first two pillars: configuration and data plane resiliency. Our goal is to make configuration propagation safer, so incompatible changes never escape pre‑production environments. We discussed how incompatible configurations are blocked early, and how data plane resiliency ensures the system continues serving traffic from a last‑known‑good (LKG) configuration even if a bad change manages to propagate. We also introduced ‘Food Taster’, a dedicated sacrificial process running in each edge server’s data plane, that pretests every configuration change in isolation, before it ever reaches the live data plane. In this post, we turn to the recovery pillar. We describe how we have made key enhancements to the Azure Front Door recovery path so the system can return to full operation in a predictable and bounded timeframe. For a global service like Azure Front Door, serving hundreds of thousands of tenants across 210+ edge sites worldwide, we set an explicit target: to be able to recover any edge site – or all edge sites – within approximately 10 minutes, even in worst‑case scenarios. In typical data plane crash scenarios, we expect recovery in under a second. Repair status The first blog post in this series mentioned the two Azure Front Door incidents from October 2025 – learn more by watching our Azure Incident Retrospective session recordings for the October 9 th incident and/or the October 29 th incident. Before diving into our platform investments for improving our Recovery Time Objectives (RTO), we wanted to provide a quick update on the overall repair items from these incidents. We are pleased to report that the work on configuration propagation and data plane resiliency is now complete and fully deployed across the platform (in the table below, “Completed” means broadly deployed in production). With this, we have reduced configuration propagation latency from ~45 minutes to ~20 minutes. We anticipate reducing this even further – to ~15 minutes by the end of April 2026, while ensuring that platform stability remains our top priority. Learning category Goal Repairs Status Safe customer configuration deployment Incompatible configuration never propagates beyond ‘EUAP or canary regions’ Control plane and data plane defect fixes Forced synchronous configuration processing Additional stages with extended bake time Early detection of crash state Completed Data plane resiliency Configuration processing cannot impact data plane availability Manage data-plane lifecycle to prevent outages caused by configuration-processing defects. Completed Isolated work-process in every data plane server to process and load the configuration. Completed 100% Azure Front Door resiliency posture for Microsoft internal services Microsoft operates an isolated, independent Active/Active fleet with automatic failover for critical Azure services Phase 1: Onboarded critical services batch impacted on Oct 29 th outage running on a day old configuration Completed Phase 2: Automation & hardening of operations, auto-failover and self-management of Azure Front Door onboarding for additional services March 2026 Recovery improvements Data plane crash recovery in under 10 minutes Data plane boot-up time optimized via local cache (~1 hour) Completed Accelerate recovery time < 10 minutes April 2026 Tenant isolation No configuration or traffic regression can impact other tenants Micro cellular Azure Front Door with ingress layered shards June 2026 Why recovery at edge scale is deceptively hard To understand why recovery took as long as it did, it helps to first understand how the Azure Front Door data plane processes configuration. Azure Front Door operates in 210+ edge sites with multiple servers per site. The data plane of each edge server hosts multiple processes. A master process orchestrates the lifecycle of multiple worker processes, that serve customer traffic. A separate configuration translator process runs alongside the data plane processes, and is responsible for converting customer configuration bundles from the control plane into optimized binary FlatBuffer files. This translation step, covering hundreds of thousands of tenants, represents hours of cumulative computation. A per edge server cache is kept locally at each server level – to enable a fast recovery of the data plane, if needed. Once the configuration translator process produces these FlatBuffer files, each worker processes them independently and memory-maps them for zero-copy access. Configuration updates flow through a two-phase commit: new FlatBuffers are first loaded into a staging area and validated, then atomically swapped into production maps. In-flight requests continue using the old configuration, until the last request referencing them completes. The data process recovery is designed to be resilient to different failure modes. A failure or crash at the worker process level has a typical recovery time of less than one second. Since each server has multiple such worker processes which serve customer traffic, this type of crash has no impact on the data plane. In the case of a master process crash, the system automatically tries to recover using the local cache. When the local cache is reused, the system is able to recover quickly – in approximately 60 minutes – since most of the configurations in the cache were already loaded into the data plane before the crash. However, in certain cases if the cache becomes unavailable or must be invalidated because of corruption, the recovery time increases significantly. During the October 29 th incident, a data plane crash triggered a complete recovery sequence that took approximately 4.5 hours. This was not because restarting a process is slow, it is because a defect in the recovery process invalidated the local cache, which meant that “restart” meant rebuilding everything from scratch. The configuration translator process then had to re-fetch and re-translate every one of the hundreds of thousands of customer configurations, before workers could memory-map them and begin serving traffic. This experience has crystallized three fundamental learnings related to our recovery path: Expensive rework: A subset of crashes discarded all previously translated FlatBuffer artifacts, forcing the configuration translator process to repeat hours of conversion work that had already been validated and stored. High restart costs: Every worker on every node had to wait for the configuration translator process to complete the full translation, before it could memory-map any configuration and begin serving requests. Unbounded recovery time: Recovery time grew linearly with total tenant footprint rather than with active traffic, creating a ‘scale penalty’ as more tenants onboarded to the system. Separately and together, the insight was clear: recovery must stop being proportional to the total configuration size. Persisting ‘validated configurations’ across restarts One of the key recovery improvements was strengthening how validated customer configurations are cached and reused across failures, rather than rebuilding configuration states from scratch during recovery. Azure Front Door already cached customer configurations on host‑mounted storage prior to the October incident. The platform enhancements post outage focused on making the local configuration cache resilient to crashes, partial failures, and bad tenant inputs. Our goal was to ensure that recovery behavior is dominated by serving traffic safely, not by reconstructing configuration state. This led us to two explicit design goals… Design goals No category of crash should invalidate the configuration cache: Configuration cache invalidation must never be the default response to failures. Whether the failure is a worker crash, master crash, data plane restart, or coordinated recovery action, previously validated customer configurations should remain usable—unless there is a proven reason to discard it. Bad tenant configuration must not poison the entire cache: A single faulty or incompatible tenant configuration should result in targeted eviction of that tenant’s configuration only—not wholesale cache invalidation across all tenants. Platform enhancements Previously, customer configurations persisted to host‑mounted storage, but certain failure paths treated the cache as unsafe and invalidated it entirely. In those cases, recovery implicitly meant reloading and reprocessing configuration for hundreds of thousands of tenants before traffic could resume, even though the vast majority of cached data was still valid. We changed the recovery model to avoid invalidating customer configurations, with strict scoping around when and how cached entries are discarded: Cached configurations are no longer invalidated based on crash type. Failures are assumed to be orthogonal to configuration correctness unless explicitly proven otherwise. Cache eviction is granular and tenant‑scoped. If a cached configuration fails validation or load checks, only that tenant’s configuration is discarded and reloaded. All other tenant configurations remain available. This ensures that recovery does not regress into a fleet‑wide rebuild due to localized or unrelated faults. Safety and correctness Durability is paired with strong correctness controls, to prevent unsafe configurations from being served: Per‑tenant validation on load: Each cached tenant configuration is validated during the ‘load and verification’ phase, before being promoted for traffic serving. Therefore, failures are contained to that tenant. Targeted re‑translation: When validation fails, only the affected tenant’s configuration is reloaded or reprocessed. Therefore, the cache for other tenants is left untouched. Operational escape hatch: Operators retain the ability to explicitly instruct a clean rebuild of the configuration cache (with proper authorization), preserving control without compromising the default fast‑recovery path. Resulting behavior With these changes, recovery behavior now aligns with real‑world traffic patterns - configuration defects impact tenants locally and predictably, rather than globally. The system now prefers isolated tenant impact, and continued service using last-known-good over aggressive invalidation, both of which are critical for predictable recovery at the scale of Azure Front Door. Making recovery scale with active traffic, not total tenants Reusing configuration cache solves the problem of rebuilding configuration in its entirety, but even with a warm cache, the original startup path had a second bottleneck: eagerly loading a large volume of tenant configurations into memory before serving any traffic. At our scale, memory-mapping, parsing hundreds of thousands of FlatBuffers, constructing internal lookup maps, adding Transport Layer Security (TLS) certificates and configuration blocks for each tenant, collectively added almost an hour to startup time. This was the case even when a majority of those tenants had no active traffic at that moment. We addressed this by fundamentally changing when configuration is loaded into workers. Rather than eagerly loading most of the tenants at startup across all edge locations, Azure Front Door now uses an Machine Learning (ML)-optimized lazy loading model. In the new architecture, instead of loading a large number of tenant configurations, we only load a small subset of tenants that are known to be historically active in a given site, we call this the “warm tenants” list. The warm tenants list per edge site is created through a sophisticated traffic analysis pipeline that leverages ML. However, loading the warm tenants is not good enough, because when a request arrives and we don’t have the configuration in memory, we need to know two things. Firstly, is this a request from a real Azure Front Door tenant – and, if it is, where can I find the configuration? To answer these questions, each worker maintains a hostmap that tracks the state of each tenant’s configuration. This hostmap is constructed during startup, as we process each tenant configuration – if the tenant is in the warm list, we will process and load their configuration fully; if not, then we will just add an entry into the hostmap where all their domain names are mapped to the configuration path location. When a request arrives for one of these tenants, the worker loads and validates that tenant’s configuration on demand, and immediately begins serving traffic. This allows a node to start serving its busiest tenants within a few minutes of startup, while additional tenants are loaded incrementally only when traffic actually arrives—allowing the system to progressively absorb cold tenants as demand increases. The effect on recovery is transformative. Instead of recovery time scaling with the total number of tenants configured on a server, it scales with the number of tenants actively receiving traffic. In practice, even at our busiest edge sites, the active tenant set is a small fraction of the total. Just as importantly, this modified form of lazy loading provides a natural failure isolation boundary. Most Edge sites won’t ever load a faulty configuration of an inactive tenant. When a request for an inactive tenant with an incompatible configuration arrives, impact is contained to a single worker. The configuration load architecture now prefers serving as many customers as quickly as possible, rather than waiting until everything is ready before serving anyone. The above changes are slated to complete in April 2026 and will bring our RTO from the current ~1 hour to under 10 minutes – for complete recovery from a worst case scenario. Continuous validation through Game Days A critical element of our recovery confidence comes from GameDay fault-injection testing. We don’t simply design recovery mechanisms and assume they work—we break the system deliberately and observe how it responds. Since late 2025, we have conducted recurring GameDay drills that simulate the exact failure scenarios we are defending against: Food Taster crash scenarios: Injecting deliberately faulty tenant configurations, to verify that they are caught and isolated with zero impact on live traffic. In our January 2026 GameDay, the Food Taster process crashed as expected, the system halted the update within approximately 5 seconds, and no customer traffic was affected. Master process crash scenarios: Triggering master process crashes across test environments to verify that workers continue serving traffic, that the Local Config Shield engages within 10 seconds, and that the coordinated recovery tool restores full operation within the expected timeframe. Multi-region failure drills: Simulating simultaneous failures across multiple regions to validate that global Config Shield mechanisms engage correctly, and that recovery procedures scale without requiring manual per-region intervention. Fallback test drills for critical Azure services running behind Azure Front Door: In our February 2026 GameDay, we simulated the complete unavailability of Azure Front Door, and successfully validated failover for critical Azure services with no impact to traffic. These drills have both surfaced corner cases and built operational confidence. They have transformed recovery from a theoretical plan into tested, repeatable muscle memory. As we noted in an internal communication to our team: “Game day testing is a deliberate shift from assuming resilience to actively proving it—turning reliability into an observed and repeatable outcome.” Closing Part 1 of this series emphasized preventing unsafe configurations from reaching the data plane, and data plane resiliency in case an incompatible configuration reaches production. This post has shown that prevention alone is not enough—when failures do occur, recovery must be fast, predictable, and bounded. By ensuring that the FlatBuffer cache is never invalidated, by loading only active tenants, and by building safe coordinated recovery tooling, we have transformed failure handling from a fleet-wide crisis into a controlled operation. These recovery investments work in concert with the prevention mechanisms described in Part 1. Together, they ensure that the path from incident detection to full service restoration is measured in minutes, with customer traffic protected at every step. In the next post of this series, we will cover the third pillar of our resiliency strategy: tenant isolation—how micro-cellular architecture and ingress-layered sharding can reduce the blast radius of any failure to a small subset, ensuring that one customer’s configuration or traffic anomaly never becomes everyone’s problem. We deeply value our customers’ trust in Azure Front Door. We are committed to transparently sharing our progress on these resiliency investments, and to exceed expectations for safety, reliability, and operational readiness.2.9KViews5likes0Comments