SAP Content Server 7.54 on Windows with FileServer as Document Store

1.    Introduction

This blog provides the deployment procedure for setting up High Availability (HA) and Disaster Recovery (DR) for SAP Content Server 7.54 on Windows Server. The SAP Content Server offers document storage either in an SAP MaxDB database or in the file system. When opting to store documents in the file system, the installation of SAP MaxDB is not required. In this guide, we will use FileServer as the document store and configure it using Azure Shared Disk.

 NOTE: According to SAP Note 719971, following the end of maintenance for SAP MaxDB, SAP Content Server will solely use the file system for document storage. There are no plans to support other database systems for document storage in the future.

2.    System Design

Following is the overall high-level design of the SAP Content Server environment with HA & DR which is described in this blog.

The above design is based on Availability zones and provide VM level availability SLA of 99.99% for the HA pair in the Azure Primary region.  For Disaster Recovery setup, we can choose the desired secondary Azure region and setup the Azure Site Recovery to continuously replicate the primary region HA setup of SAP Content Server with Shared disk as File Server.

Following are the details for each of the components of the setup:

2.1  SAP Content Server

SAP Content Server 7.54 is the latest version and used for this setup. It is recommended to always use the latest released version and patch level of SAP Content Server for new installations. Refer to SAP Note 719971 to get latest information on SAP Content Server release strategy and product version.

2.1.1        HA Setup – This is achieved by installing SAP Content Server in Windows WSFC Clustered environment between two VMs across Availability Zones. Virtual IP for the SAP Content Server Instance is maintained as frontend IP in Azure Load balancer. SAP Content Server will have one premium data disk for local ‘usr_sap’ files and one premium shared disk for ‘sapmnt’ and ‘global’ shares. To configure quorum for WSFC failover cluster, Cloud Witness should be created in ZRS Storage account. In the Windows firewall of both VMs, open the load balancer probe ports and SAP content server ports. Refer to TCP/IP ports of All SAP Products.

2.1.2        DR Setup – Azure Site Recovery is used to continuously replicate the VMs, all the configurations and the data in it, to another region.  To replicate VMs using ASR for DR, review supported regions. Its recommended to build the azure landing zone in the DR region including Vnets, subnets, IP ranges before configuration of ASR replication. Azure load balancer and cloud witness must be pre-deployed for DR failover.

As of time this blog was written , ASR for VMs with azure shared disk on Windows Server is in public preview. Please refer to the link for updates on GA. Using Azure Site Recovery for Azure shared disks, you can replicate and recover your WSFC-clusters as a single unit throughout the disaster recovery lifecycle, while creating cluster-consistent recovery points that are consistent across all the disks (including the shared disk) of the cluster. For more information, see Shared disks in Azure Site Recovery.  

ASR create recovery points (both application and crash) that are consistent across all the virtual machines and disks within the cluster. However, during public preview only crash consistent and the latest recovery processed recovery points are supported. For more details on some of common concerns, see FAQs - Shared disks in Azure Site Recovery 

Also, default settings for Defender including Realtime scan should work well with Content Server.  If high CPU utilization due to Defender is observed, open a support case with Microsoft via the Defender Portal.

2.2  Fileserver for Document Store

This setup is using Azure Shared disk – Premium SSD ZRS as Fileserver to store documents. Zone redundant storage (ZRS) replicates managed disk across all the three availability zones.  It is advised to choose the right size of the disk as Azure Shared disk resizing requires shutting down of associated VMs.  Also refer to limitations of premium Azure shared disks .

2.2.1        HA Setup – We will be creating File server role in WSFC Cluster for this shared disk so that its highly available as Windows Fileserver. Fileserver name will be mapped to Load balancer virtual IP, to make it available on either of the VMs. File Server ports do not need to be exposed outside the SAP Content Server cluster nodes. SAP Content Server must be able to access highly available FileServer using port 445.

2.2.2        DR Setup – ASR for Azure shared disk will be used for continuously replicating the documents to the DR region and available as highly available fileserver during DR failover. ASR will also perform the reverse copy from DR region to primary region and then failback the environment once primary region is available. Azure loadbalancer in the DR region is to be used for configuring the virtual IP.

3.    Overview of Deployment Steps

Following are the high-level steps which are described in the blog.

• Preparations
• WSFC Cluster setup and configuration
• SAP Content Server Setup
• WFSC Fileserver setup
• Connect SAP Content Server with S/4 HANA or Business Suite 7
• High Availability Test
• Disaster Recovery Setup with ASR and
• DR failover and failback

4.    Preparations

• Read the required Installation Guide, SAP Notes, SAP on Azure docs and download the installation media.
• Deploy the VMs (of desired SKU) in Availability Zones and Choose Operating System as Windows Server 2022.
• Add data disk of required size to each of the VMs.
• Create 2 Azure Shared disks and attach it to both VMs.
• Disk 1 for SAP Content Server
• Disk 2 for FileServer as document store
• Join the SAP Content Server VMs to the Domain.
• Define Page File in Temp Disk (D Drive).
• Check that necessary Ports (including ILB Probe ports) are open in Windows firewall.
• Disable the Continuous Availability Feature in Windows using the instructions in this link.
• Create an Azure Storage account ZRS for cloud witness.

5.    WSFC Cluster Setup and Configuration

• Create load balancer for virtual IPs of SAP Content Server Instance and File Server. Create A-record in DNS for virtual IPs and virtual hostnames.

Refer to the section Create Azure Internal Load Balancer

• Add the KeepAlive registry entries

Refer to the section Add registry entries on both cluster nodes

• Deploy WSFC Cluster and add cloud witness

Refer to the section Install and Configure Windows Failover Cluster

• Specify the SID of Content Server, VM names in the PowerShell script to Install and Configure cluster.
• Specify the Azure Storage account and Access Key to Configure Cluster cloud quorum
• Alternatively, we can perform deploy WSFC Cluster and add cloud witness from Failover Cluster Manager.
• Update cluster parameters

Adjust the following WSFC Cluster parameters

• • SameSubNetDelay = 2000
  • SameSubNetThreshold = 15
  • RouteHistoryLength = 30

Refer to section Tune the Windows Failover Cluster thresholds.

• Verify that Azure Shared disk is added to the cluster and status is online.

If the Azure Shared disk are not added to the WSFC Cluster then we can add them either by ‘Add Disk’ option in Failover Cluster Manager or by using PowerShell commands as described in Format the shared disk with Powershell.

6.    SAP Content Server Setup

• Run SWPM on 1^st VM

Start the installation of SAP Content Server on First Cluster Node.

• Choose the local disk’s drive letter

 

• Enter the virtual hostname of the content server as defined in the DNS with the virtual IP. Apart from entering SID for SAP Content Server, also select the azure shared disk for SAP Content Server which is already part of WSFC cluster disk.

Continue with other SWPM input parameters to install SAP Content Server on First Cluster Node.

• Choose SAP Content Server to be installed.

• Provide the instance number of SAP Content Server

• Specify the SAP Content Server basic parameter values

And complete the installation on First Node.

• Add probe port to the cluster parameter

Probe port defined in the Azure Load balancer for SAP Content Server need to be updated for SAP Content server SID Role in the WSFC Cluster configuration. This will enable virtual IP connection to the SAP Content Server. Follow the instructions in section “Add a probe port” in the link Install SAP NetWeaver HA on a Windows failover cluster and shared disk for an SAP ASCS/SCS instance in Azure

Validate the probe port in the cluster configuration (as shown in the below example screenshot)

Stop and Start the SAP Content Server Role in WSFC cluster.

• In case not performed earlier, Add windows firewall rule to allow inbound connection to probe port on both VMs.
• SWPM setup for SAP Content Server second Node

• Choose the local disk’s drive letter

And continue with SWPM installation steps.

7.    WSFC File Server Setup

• Create the File Server in WSFC Cluster
• In the ‘Actions’ section in ‘Roles’ tab, click on ‘Configure Role’.

Choose ‘File Server’ as Role and click ‘Next’.

• In next screen for File Server Type, choose ‘File Server for general Use’ and click ‘Next’.
• Enter the name of the fileserver as defined in the DNS (along with Virtual IP). Click Next.
• In the ‘select storage’ screen, choose the cluster disk(azure shared disk) for FileServer. Click ‘Next’ and continue with the setup.
• Update the cluster settings in the File Server Role.

Change the Name to ‘<File Server Role Name> IP’. Choose the Network from drop-down. Update the static IP address field with virtual IP defined for File Server.

Stop and start the Role.

• Update the probe port parameter in the Cluster settings. Follow the probe port changing steps in the ‘Add a probe port’ section in the ‘SAP Content Server Setup’ in this blog.

Make sure that in this step we change the probe port for FileServer Role/IP in the cluster and not for SAP Content Server Role/IP.

Stop and Start the File Server Role.

• Validate that File Server role is online.

8.    Connect the SAP Content Server with S/4 HANA System

• Update the connection details in content repository.

In the S/4 HANA or Business Suite 7 system, run the tr. Code OAC0. Select the content repository name for SAP Content Server.

Update the HTTP Server, port numbers.

Ensure content server related inbound ports are open in windows firewall.

For HTTPS setup, export the PSE of content server from ‘SAP Content Server Administration’ portal and import it in S/4 HANA system under certificate list in ‘System PSE’ and ‘SSL Server Standard’ section.

• Update the repository settings

Click on ‘CS Admin’ as in the previous screenshot. Add/update the parameters (as below) to specify the document storage location as File Server.

• Check that connection tests are successful.
• Refer to SAP Note 2457912 – How to create Content Server Repository in OAC0.

9.    High Availability Tests

We can perform following WSFC failover tests for SAP Content Server and File Server for document storage.

For all the below tests, we should validate that

• • Cluster Roles are online after failover,
  • SAP Content server is running using the link

https://<virtual hostname>:1091/sapcs?serverInfo

• • In S/4HANA tr code OAC0, connections test is working after failover.

Test Scenarios:

• SAP Content Server system and File Server can successfully fail over from node 1 to node 2 and vice-versa. We can use Failover Cluster Manager to perform the failover (move the role to another node) and check the status after failover.
• Restart the Windows guest operating system in which SAP Content Server is active. This initiates an automatic failover of the SAP Content Server cluster group from active node to second node. Perform the same action for active node for File Server.
• Restart the VM from Azure portal and using Azure PowerShell, in which SAP Content Server is active. This initiates an automatic failover of the SAP Content Server cluster group from active node to second node. Perform the same action for active node for File Server.

10.       Disaster Recovery Setup with ASR

To configure ASR for VMs running SAP Content Server with Azure shared disk, you need to follow below steps :

• Set up Resource Group, Virtual Network, Subnet and Recovery Service Vault in the secondary site that you would use in setting up your DR. To learn more about networking, see prepare networking for Azure VM disaster recovery.
• Before enabling ASR on SAP Content Server VMs, it is essential that WSFC is configured, and Azure shared disks are managed by the cluster.
• Configure ASR for SAP Content Server with Azure shared disk by following the steps in the Shared disks in Azure Site Recovery document. Follow Configure replication for Azure VMs in Azure Site Recovery to configure ASR for SAP application servers.
• When you use ASR to set up DR for VMs, the VM’s OS, data disks, and Azure shared disk (for SAP Content Server VMs) are copied to the DR site.
  NOTE: With Azure shared disk, SAP Content Server VMs will be grouped together in ASR. This way, the VMs in the group will replicate together to have consistent recovery snapshot. In the event of a failover, the VMs will fail over as a group.
• After the VMs are replicated, the status of protected cluster and SAP Content Server VMs would turn into “Protected” and the replication health would be “Healthy”.
• Deploy Azure Load Balancer without backend pools as SAP Content Servers VMs are not available in DR region. Keep the health probe ports same as primary site health probe ports.

• Deploy Cloud Witness Storage account of ZRS type in the DR region.
• When VMs without public IP addresses are placed in the backend pool of the internal standard load balancer, there would not be any outbound connectivity from these VMs, unless additional configuration is performed to allow routing to public end point. For details on how to achieve outbound connectivity see public endpoint connectivity for Azure VMs & Standard ILB in SAP HA scenarios.

Tip: Based on your DR strategy, you can either execute the step to deploy load balancer and cloud witness step when you are preparing your DR site like setting up ASR or you can execute at the time of the DR failover process.

11.     Disaster Recovery Failover and Failback

We will perform the failover to the DR region of the SAP Content Server and File Server, start the cluster services for SAP Content Server and File Server, verify its configuration and connection. Then we will ‘commit’ the Failover in ASR.

Afterwards we will re-protect the VMs to start the ASR replication process from DR region to primary region, Failover (failback) VMs from DR region to primary region, start the cluster services for SAP Content Server and File Server and verify its configuration and connection.

Finally, with ASR, we will re-protect the VMs in primary region to continue with the ASR replication process from primary to DR region.

11.1      Failover to DR Region

• Before triggering the failover of SAP Content server, ensure the Domain Controller could be accessed from the DR region.
• Initiate the Failover for recovery group of the SAP Content Server.
• If the VMs were online on the primary site then ASR will show same consistent recovery point for both VMs. Select the Recovery point for the cluster/recovery group and continue with the failover. Refer to the documentation for Recovery point is consistent across all the virtual machines.
• If one of the VM in the recovery group is unavailable for some time, then Select an individual recovery point for the VM that are not part of the cluster recovery point. Refer to the documentation for Recovery point is consistent only for a few virtual machines
• After the failover, the status of replicated items will be ‘Failover Completed’. VMs will be online on the DR region and primary site VMs will be shutdown (if the shutdown checkbox is selected during failover).

 

• Update the backend pool of DR region’s Azure Load Balancer with the VM details.
• Update the DNS entries for SAP Content Server VMs and virtual hostname for SAP SID and File Server, with the new IP address in the DR region.

 

• Change the Cluster quorum to the cloud witness storage account created on the DR site. Use Failover Cluster Manager tool to update the Cloud Witness details.
• Update the virtual network and virtual IP of the SAP Content SID and File Server in the Failover cluster Manager.

 

• Restart the Cluster Roles.
• Update the SAP Content Server virtual IP into the DNS used S/4 HANA or ECC system.

• Perform the WSFC Cluster Failover tests and connect tests in tr code OAC0. Refer to SAP Note 2457912 for testing connections to SAP content server in tr code OAC0.
• Once all checks are successfully completed, ‘commit’ the failover in replicated items in recovery service vault.

11.2      Failback to Primary Region

• Once the former Primary Region is available, we can ‘Re-protect’ the recovery group in the recovery services vault. This step will start replication of all the components of ASR recovery group from DR region to former Primary region.
• Once status of the replicated items is ‘protected’, we can plan to failback the SAP Content Server. Follow the same steps as described in the previous section to ‘failover’ to former primary region like checking the DNS settings and update the cluster setting.

Once the failover to former primary region is successfully completed, ‘re-protect’ the replicated items in the recovery services vault to restart the replication from primary to DR region.