Introducing the SAP Testing Automation Framework: Elevating SAP System Testing on Azure

Microsoft

May 19, 2025

Introducing SAP Testing Automation Framework (STAF), a tool designed to automate the testing of high availability and failover mechanisms for SAP systems on Azure. It addresses the challenges of manual testing and configuration error to improve the reliability and efficiency of SAP deployments.

In today’s fast-paced digital landscape, ensuring that enterprise systems perform flawlessly is non-negotiable. As businesses increasingly rely on SAP systems to run their critical operations, testing becomes an essential pillar of operational excellence. Traditionally, SAP system testing has been manual, time-consuming, and prone to gaps. Addressing these critical aspects, Microsoft has introduced the SAP Testing Automation Framework (STAF), an open-source orchestration tool developed to validate SAP deployments on Microsoft Azure. It enables you to assess system configurations against SAP on Azure best practices and to automate various testing scenarios, with an initial focus on high availability (HA) testing.

What is the SAP Testing Automation Framework?

The SAP Testing Automation Framework is an open-source orchestration tool engineered to validate SAP deployments on the Microsoft Azure platform. Its core purpose is to help customers ensure their SAP systems run smoothly by proactively identifying potential issues. It achieves this by simulating system failures, verifying that configurations adhere to best practices, and automating the entire testing process to save time and improve accuracy.

The framework is built on a modular, configuration‑as‑code model using standard tools. The tests are defined in version‑controlled Ansible playbooks, and custom Python modules handle in‑depth checks of both your SAP systems and Azure resources.

Key Features:

Configuration Validation: It checks whether the configurations of SAP HANA scale-up or SAP Central Services align with established SAP on Azure best practices and guidelines.
High Availability Functional Testing: It simulates multiple test cases to ensure that the failover mechanisms are effective. With SAP HANA databases and SAP Central Services as prime examples, this testing validates that when a component fails, the system can gracefully recover without disruption. This helps identify potential issues during new system deployments.
Detailed Reporting: The framework collects detailed telemetry from SAP components and test execution where it captures event sequences, detection timings, failover durations, and system responses. It compiles this data into a comprehensive HTML report with clear pass/fail outcomes and timestamps. Optionally, you can stream logs to Azure Log Analytics or Data Explorer.
Extensible and Pipeline-ready: All framework operations, Ansible playbooks, and custom python modules are defined as code, making them ideal for integration with CI/CD pipelines. You can invoke STAF immediately after your deployment and installation step via SAP Deployment Automation Framework, running comprehensive HA tests before promoting changes.

SAP System High Availability Functional Testing

The initial and most prominent capability of the SAP Testing Automation Framework is its comprehensive High Availability (HA) functional testing for critical SAP components hosted on Microsoft Azure. The framework targets scenarios involving SAP HANA scale-up database and SAP Central Services (ASCS/SCS) deployed in a two-node cluster on SUSE Linux Enterprise Server (SLES) or Red Hat Enterprise Linux (RHEL), providing a cross-distribution solution for our diverse SAP on Azure customer base. For supported configuration, see support matrix.

The SAP Testing Automation Framework employs a systematic approach to validate the robustness of SAP system HA setup. It verifies the configuration and captures the entire sequence of events in an HA scenario: from initial failure detection, through isolation of the faulty component (including fencing), to resource migration, service recovery on the standby node, and implicitly, the consistency of data upon successful recovery. Upon completion of the tests run, STAF compiles results into a clear, HTML‑based report that details configuration compliance checks and functional test outcomes, complete with timestamps and pass/fail statuses. The report also includes logs from /var/log/messages to provide context.

HTML Report after completion of the run

This comprehensive validation of the HA process flow is fundamental to building confidence in the resilience of the SAP system on Azure.

Getting Started with the SAP Testing Automation Framework

The SAP Testing Automation Framework is available as an open-source project on GitHub for the community to use and contribute. You can find the code and documentation in the official repository: Azure/sap-automation-qa: This is the repository supporting the quality assurance for SAP systems running on Azure.. The project is currently in a public preview stage. So, feedback and contributions are welcome to help improve its capabilities.

To start using the framework, you have a couple of options depending on your environment:

SAP system manually deployed (not using SDAF): If you want to validate manually configured high availability of SAP system, you can run the framework in a standalone mode. This involves deploying the management server (for example, an Ubuntu VM that will orchestrate the tests), configuring it with details of your SAP landscape (cluster nodes, IPs, etc.), and then executing the provided playbooks or scripts to run the HA tests. The repository provides guidance on how to configure the necessary variables and run the test scenarios for a Pacemaker cluster environment.
Integration with Deployment Pipelines: For those who already use automated deployment tools like the SAP Deployment Automation Framework (SDAF) for Azure, the testing framework can integrate directly into those pipelines. The framework is designed as a natural extension to SDAF, so it can leverage the same configuration context and Azure resources defined during deployment. This allows you to embed HA testing into your continuous delivery process, every time you deploy or update an SAP environment, the pipeline can automatically run the HA tests and surface any issues before you even hand the system over to end-users or application teams.

Call to Action & Community Engagement

The SAP Testing Automation Framework, in public preview stage, is a significant step forward in reducing misconfigurations and manual effort in high availability deployment of SAP system on Azure. We encourage you to explore the framework, share your feedback, or contribute. During this public preview, we recommend using framework for new greenfield production high availability deployments that are not yet live, or on non-production environments.