Designing, Migrating and Managing a 15+1-Node SAP BW Scale-Out Landscape on Microsoft Azure

This blog outlines the implementation of SAP BW Scale-Out with 15+1 nodes using virtual machines on the Azure platform, representing one of the early and pioneering examples of SAP BW at this scale on a hyperscale public cloud.

It also highlights the technical considerations and work carried out by the Microsoft Customer & Partners team to understand and validate the performance characteristics of SAP BW, both on-premises and on Azure.

The underlying platform used memory-optimised Mv2-Series virtual machines to support large in-memory databases and demanding workloads. Specifically, the landscape comprised 16 × M416s_v2 (416 vCPU / 5.7 GiB memory), architected across:

• Database nodes
• (A)SCS nodes
• Application server nodes

The customer had previously operated SAP HANA on-premises with 20 nodes [18+2 scale-out] and decided to move critical business systems—including SAP BW, SAP Warehouse Management and SAP IQ (Near-Line Storage)—to Azure as part of a data centre exit.

As part of this migration, the partner proposed the modernisation of selected business processes to take advantage of Azure-native architecture components and improve the end-user experience, including:

• Increased system availability by deploying the SAP system across Azure zones within the region to enhance the availability SLA.
• Automated failover of access points using Azure Standard Load Balancer.
• Optimisation of the scale-out setup using Azure NetApp Files.

The move to Azure not only delivered high availability, it also improved how database connectivity is managed, removing the dependency on DNS for routing to the HANA database.

Implementing SAP BW Scale-Out for Very Large Databases (VLD) can be time-consuming, especially where there is limited prior experience at this scale. The architecture required careful review of critical design aspects, and the target design was tested in a lower environment to validate all technical tests. During this process, important insights were gained, particularly around Load Balancer configuration and the fine-tuning needed to align with customer business expectations.

1. Source Landscape: On-Premises (AS-IS)

The on-premises SAP BW landscape operated with 20 nodes [18+2] to support OLAP workloads.

For any migration programme, it is essential to understand the database growth pattern to inform a 3–5-year growth plan. Unlike on-premises environments, the cloud does not require pre-allocation of infrastructure for a fixed growth trajectory. However, understanding growth trends and resource usage remains vital to:

• Design an appropriate target solution.
• Avoid subsequent minor or major projects just to handle unexpected growth.

For this system:

• The average peak consumption was assumed to be approximately 800,000 SAPS.
• The overall measured peak reached around 993,480 SAPS.

Workspace is another critical factor when projecting memory usage and growth for SAP systems. Typically, customers estimate workspace at 50% or 70% on top of the data and code footprint.

2. Key AS-IS Information

2.1 User Load

Concurrent users must be taken into account, along with any expected changes as part of the migration roadmap. Additional capacity should be considered if significant changes in user load are anticipated on the target platform.

The User Activity view (diagram not shown here) illustrates how users interact with the system over time:

• Total Users – total number of users who logged on during one week.
• Active Users – users who performed more than 400 transaction steps in one week.

2.2 System Performance

Monitoring the size and growth of the HANA database is crucial for ongoing system stability and performance.

2.3 Log Throughput Requirement / Usage on the On-Premises System

One of the critical design considerations is the performance of the on-premises system, as this becomes the baseline for designing the target environment.

3. Target Architecture: 15+1 Scale-Out on Azure (To-Be)

The target architecture is based on SAP BW Scale-Out with 15+1 nodes in each Azure zone:

• SAP BW Scale-Out 15+1 using M416s_v2 in Zone 1.
• SAP BW Scale-Out 15+1 using M416s_v2 in Zone 2.

To provide a consistent access experience across zones:

• A zone-redundant Standard Load Balancer is used to maintain a single point of access.
• Two frontend IP addresses are configured, with two backend pools for Production and DR nodes respectively.

4. Key Target Design Consideration for Scale-Out

4.1 SAP HANA Cloud Measurement Tool (HCMT)

SAP mandates execution of the HANA Cloud Measurement Tool (HCMT) as part of validating the configuration.

4.2 Compute – SAP on Azure Certification

The Azure compute platform (hardware) is required to have a valid SAP HANA hardware certification at the point of deployment. The selected SKU must be listed in the Certified and Supported SAP HANA® Hardware Directory.

Microsoft Cloud offers multiple SKUs that are certified by SAP to run SAP and HANA workloads. The same directory maintains certified hardware SKUs across all providers. SAP HANA, as an in-memory database, must meet specific certification criteria to be supported by SAP. The Microsoft engineering team works closely with SAP to bring new SKUs into the list of Microsoft Cloud hardware supported for HANA workloads.

For HANA workloads, M-Series (Mv1 & Mv2) are preferred SKUs, though Microsoft also offers E-Series and HLI [HANA Large Instances], which are supported for HANA workloads.

In this design:

• Compute: 16 × M416s_v2 in Zone 1 and 16 × M416s_v2 in Zone 2.

4.3 Network Considerations

Network configuration at both the OS and virtual network (VNET) layers plays a critical role in achieving the required throughput and latency between components. For Scale-Out, there must be additional focus on host-based routing and the selection of the right NIC for HSR (HANA System Replication).

The Load Balancer configuration is designed to maintain the same logical hostname for applications and third parties connecting to the database, regardless of the database location across zones.

There are key differences between Scale-Up and Scale-Out:

• In Scale-Up, the entire database runs on a single VM/SKU.
• In Scale-Out, the database is split and distributed across multiple VMs/SKUs.

The communication between VMs/SKUs in a scale-out configuration directly affects database performance. It is therefore highly recommended to have a dedicated NIC and subnet to support internode traffic.

Another key consideration is compute-to-storage communication. To ensure direct connectivity from compute to storage, host-based routing is recommended and is one of the design aspects to meet SAP HANA KPI targets during HCMT execution.

Different connection types can be summarised as:

• Internode traffic (scale-out communication) – recommended to use a dedicated NIC and subnet.
• Compute–storage traffic – should use host-based routing to reach storage directly.
• Client connection / user traffic.
• Additional traffic, depending on configuration, which can be merged with either client or internode traffic based on system requirements during peak periods and should be reviewed during performance and stress testing before go-live.

4.3.1 Network layout

Three subnets created:

• One for Client Network (NIC)
• One for Inter-Node communication & HSR
• One for Storage Network (NIC)

Additionally, a delegated Azure NetApp Files (ANF) subnet.

4.4 Azure Standard Load Balancer with Scale-Out Design

A special requirement from the customer was to manage third-party connections to the HANA database across zones.

To ensure seamless connectivity from third-party systems regardless of the active zone:

• An Azure Standard Load Balancer is configured in front of the scale-out nodes across zones.

This Load Balancer:

• Handles connections to the HANA database.
• Supports the DR failover scenario, maintaining connectivity across zones.

4.5 Storage Considerations

Storage selection is simplified by the fact that only Azure NetApp Files is supported for scale-out configurations with a standby node. Scale-out without a standby node provides more options.

In all cases, storage must be configured to achieve the required IOPS and throughput without driving cost up unnecessarily.

For scale-out with a standby node, Azure NetApp Files is the only supported storage as of June 2022. Alongside this, Azure NetApp Files provides several features that should be carefully evaluated in the target design.

4.5.1 ANF Storage Tier

Ultra / Premium tiers are used as appropriate.

4.5.2 ANF Features

Application Volume Group (AVG): The Application Volume Group for SAP HANA enables customers and partners to deploy all volumes required to install and operate an SAP HANA database according to best practices in a single, optimised workflow. It includes the use of Proximity Placement Group (PPG) with VMs to achieve automated, low-latency deployments.

Manual QoS: With manual QoS volumes, customers and partners do not need to overprovision volume quota to achieve higher throughput, because throughput can be assigned to each volume independently. Total available throughput is defined at the capacity pool level and depends on the size and type of storage.

Dynamic Tiering: Azure NetApp Files provides three performance tiers: Standard, Premium and Ultra. Dynamic Tiering allows customers and partners to use a higher service level for better performance or a lower service level for cost optimisation without waiting time.

ANF Files Backup: The ANF backup feature allows customers and partners to offload Azure NetApp Files snapshots to Azure Blob Storage in a fast and cost-effective way, further protecting data from accidental deletion.

4.5.3 Selected Layout

• ANF Ultra tier selected for HANA data, log and shared mount points.
• ANF Premium selected to host offline transaction log backups.
• Azure Blob Storage used to store AzAcSnap HANA data snapshots and offline transaction log backups.

4.6 Backup and Restore Approach

Backup and restore runtime can be a critical blocker if it does not meet business RPO/RTO requirements.

With Azure NetApp Files, most customers implement the AzAcSnap tool to manage HANA database snapshots, followed by AzCopy to transfer snapshots to Blob Storage for long-term retention.

In this design:

• AzAcSnap is defined as the primary backup, running every 6 hours.
• The first backup is transferred to Blob Storage for long-term retention.
• Transaction log backups run every 15 minutes and are written to an ANF Premium mount.
• AzCopy jobs then transfer these backups to Blob for long-term retention.
• A dedicated server is used to manage AzCopy transfers from ANF volumes to Blob Storage as a temporary measure until ANF Files backup is available in the required region.

4.6 Run Operations and Monitoring

Run, or Business As Usual (BAU), marks the transition from the migration programme to the support project. It is critical that monitoring and configuration are in place to capture alerts and collect sufficient logs for investigation during issue resolution.

Key elements include:

• Proper configuration of monitoring and kdump to ensure logs and dumps are available to analyse unforeseen issues related to the OS or SKU.
• Use of Zabbix together with Azure Monitor for Virtual Machines for ongoing monitoring.
• Kdump configured and enabled on all VMs to capture critical information for troubleshooting unexpected issues.

5. New: Azure NetApp Files Flexible service level (2025 update)

Since this project was originally designed, Azure NetApp Files has introduced a new Flexible service level, which is particularly relevant for SAP BW and SAP HANA workloads on Azure.

5.1 What is the Flexible service level?

The Flexible service level is a new Azure NetApp Files throughput service level that decouples throughput from capacity. It is available for new manual QoS capacity pools. You configure pool throughput (MiB/s) and capacity (TiB) independently instead of being bound to a fixed MiB/s-per-TiB ratio.

This makes it easier to right-size storage for high-throughput, low-capacity workloads (for example, HANA log volumes) and high-capacity, moderate-throughput workloads (for example, BW cold data or shared file systems).

5.2 128 MiB/s baseline throughput at no extra charge

A key benefit of the Flexible service level is the included baseline throughput:

• The minimum throughput you can assign to a Flexible capacity pool is 128 MiB/s, regardless of pool size.
• The first 128 MiB/s of throughput is included in the service level—often referred to as the baseline throughput—and is available at no additional performance surcharge.

In practice, this means every Flexible capacity pool you create automatically includes 128 MiB/s of throughput, and you only pay for any additional throughput configured beyond that baseline.

5.3 Throughput scaling for demanding workloads

With Flexible service level, throughput can scale significantly. The maximum throughput is documented as up to 640 MiB/s per TiB per pool, with an upper bound defined as 5 × 128 MiB/s × pool size (TiB). Throughput can be increased when needed (for example, during peak loads or migration cutovers) and reduced later, subject to a documented cool-down period between downward adjustments.

This flexibility is especially useful for SAP BW and SAP HANA systems with variable peak and off-peak windows, and for migration phases where temporary higher throughput is required for data loads, initial syncs or cutovers, with the option to optimise cost afterwards.

6. Where to learn more

Service levels for Azure NetApp Files – detailed description of Standard, Premium, Ultra and Flexible service levels, including throughput formulas and examples

What’s new in Azure NetApp Files – latest feature announcements and regional availability updates

Service levels for Azure NetApp Files | Microsoft Learn

Azure NetApp Files solution architectures for SAP HANA – reference architectures and best practices for SAP HANA on ANF