Table of Contents
Oracle Control File Locations (default locations)
Oracle pfile and spifle locations (default locations)
AzAcSnap command to orchestrate the online Oracle backup
AzAcSnap Configuration (GR1azacsnap.json)
Recover to time of backup (snapshot)
Recover to the last Transaction
Version
This article is for SAP/Oracle 19c using Microsoft Azure NetApp Files and AzAcSnap version 5.1 preview or later.
Authors
Scott McCullough, SAP Solutions Architect
Nils Bauer, Technical Marketing Engineer
Overview
Corporations today require their SAP applications to be available 24 hours a day, 7 days a week. Consistent levels of performance are expected regardless of the ever-increasing data volumes and need for routine maintenance tasks such as system backups. Performing backups of SAP Oracle databases are a critical task and can have a significant performance effect on the production SAP systems. With backup windows shrinking and the amount of data that needs to be backed up still increasing, it is difficult to define a time when backups can be performed with minimal impact on the business processes. The time needed to restore and recover SAP systems is of particular concern. That is because the downtime of SAP production and nonproduction systems must be minimized to minimize both the data loss and the cost to the business.
The following summarizes the SAP Oracle backup and recovery challenges:
- Performance impact on production SAP systems. Conventional backups typically lead to a significant performance impact on the production SAP system. That is because there is a heavy load on the database server, the storage system, and the storage network during traditional copy-based backups.
- Shrinking backup windows. Conventional backups can be taken only during times when little dialog or batch activities take place on the SAP system. The scheduling of backups becomes more and more difficult to define when the SAP systems are in use 24/7.
- Rapid data growth. Rapid data growth, together with shrinking backup windows, results in ongoing investments into the backup infrastructure. Incremental or differential backups can address these issues, but this option results in a very slow, cumbersome, and complex restoration process that is harder to verify. The option also usually leads to increased or elongated recovery time objective (RTO) or recovery point objective (RPO) times that are not acceptable to the business.
- Increasing cost of downtime. Unplanned downtime of an SAP system always has a financial impact on the business. A significant part of the unplanned downtime is the time that is needed to restore and recover the SAP system after a failure. The backup and recovery architecture must be designed based on an acceptable RTO.
- Backup and recovery time included in SAP upgrade projects. The project plan for an SAP upgrade always includes at least three backups of the SAP database. The time needed to perform these backups dramatically cuts down the total available time for the upgrade process. The go/no-go decision is generally based on the amount of time required to restore and recover the database from the backup that was previously created. The option to restore very quickly allows more time to solve problems that might occur during the upgrade process rather than just restore the system back to its previous state.
Azure NetApp Files Snapshot technology can be used to create online database backups within minutes. Because a snapshot does not move any physical data blocks on the storage platform, the time needed to create a snapshot is independent of the size of the database. The use of snapshot technology also has no performance impact on the live SAP system. That is because the Azure NetApp Files snapshots do not move or copy data blocks when the snapshot is created or when data in the active file system is changed. Therefore, the creation of snapshots can be scheduled without having to consider peak dialog or batch activity periods. Azure SAP customers leveraging Azure NetApp Files typically schedule multiple online snapshots during the day; for example, scheduling snapshots every four or six hours is common. These snapshots are typically kept for three to five days. For long-term retention older snapshots are then typically vaulted using Azure NetApp Files backup to Azure storage account.
Snapshots also provide key advantages for the restore and recovery operation. Azure NetApp Files ‘Revert Volume’ functionality allows restoration of the entire database to any point in time based on the available snapshots. This restore process is performed near-instantaneously, independent of the size of the database. Because several online snapshots are created during the day, the actual time needed for the recovery process is dramatically reduced, as opposed to a traditional backup approach. A restore operation can be performed using a snapshot that is only a few hours old (rather than up to 24 hours old); therefore, fewer transaction logs need to be applied. As a result, the mean time to recover or RTO, which is the time needed for restore and recovery operations, is reduced to just several minutes compared to multiple hours with conventional single-cycle tape backups.
Azure NetApp Files snapshots are stored on the same volume as the active online data. Therefore, it is recommended to use Azure NetApp Files backup or Azure NetApp Files Cross Region Replication (CRR) to a secondary Azure NetApp File paired region to safeguard the data from accidental deletions.
This article describes how to recover an SAP/Oracle database when using Azure NetApp Files with AzAcSnap orchestration tool to protect the database. The two scenarios that are covered are:
- Recover to the time of backup
- Recover to the latest transaction
|
(!) NOTE: |
Disclaimer
As with anything especially in the IT field, there are multiple ways to accomplish a task. This article is by no means the only way to accomplish this task, it should be used as a guide and can be adjusted per individual circumstances.
Assumptions
The setup of AzAcSnap is not covered within this article. Setup instructions for AzAcSnap with Oracle can be found here.
The management of the Oracle archives contained within /oracle/SID/saparch:oraarch is not covered here. Multiple solutions can be used to manage the archives via BRTools, RMAN or 3rd-party (backint) solutions.
|
(!) NOTE:
You must ensure the Oracle archives that were generated during the time the database was in hot-backup mode are available on the file system. The Oracle recovery will be dependent on these files for a successful recovery of the database. |
In this article the recovery of the SAP Oracle system is being performed on the same system (not a secondary copy).
Lab System Overview
Host: gr1ora
Host OS: Oracle Linux Server 8.5
Oracle version: 19.13.0.0
SAP NW: 7.5
Azure NetApp Files (ANF) NFSv3 volumes.
Mount points
ANF volume/ANF Sub-Directory mount point/File System mount point
|
ANF Volume |
ANF Sub-Directory |
File System |
|
gr1ora-orashared |
oracle |
/oracle |
|
gr1ora-orashared |
usr-sap |
/usr/sap |
|
gr1ora-orashared |
sapmnt |
/sapmnt |
|
gr1ora-orashared |
trans |
/usr/sap/trans |
|
gr1ora-orashared |
oracle_GR1 |
/oracle/GR1 |
|
gr1ora-orashared |
19.0.0 |
/oracle/GR1/19.0.0 |
|
gr1ora-sapdata |
sapdata1 |
/oracle/GR1/sapdata1 |
|
gr1ora-sapdata |
sapdata2 |
/oracle/GR1/sapdata2 |
|
gr1ora-sapdata |
sapdata3 |
/oracle/GR1/sapdata3 |
|
gr1ora-sapdata |
sapdata4 |
/oracle/GR1/sapdata4 |
|
gr1ora-oraarch |
GR1 |
/oracle/GR1/oraarch |
|
gr1ora-origlog |
origlogA |
/oracle/GR1/origlogA |
|
gr1ora-origlog |
origlogB |
/oracle/GR1/origlogB |
|
gr1ora-mirrlog |
mirrlogA |
/oracle/GR1/mirrlogA |
|
gr1ora-mirrlog |
mirrlogB |
/oracle/GR1/mirrlogB |
/etc/fstab:
# Oracle SAP Mount points
10.1.9.4:/gr1ora-orashared/sapmnt /sapmnt nfs rw,vers=3,hard,timeo=600,rsize=262144,wsize=262144,bg,noatime,nointr,lock 0 0
10.1.9.4:/gr1ora-orashared/usr_sap /usr/sap nfs rw,vers=3,hard,timeo=600,rsize=262144,wsize=262144,bg,noatime,nointr,lock 0 0
10.1.9.4:/gr1ora-orashared/trans /usr/sap/trans nfs rw,vers=3,hard,timeo=600,rsize=262144,wsize=262144,bg,noatime,nointr,lock 0 0
10.1.9.4:/gr1ora-orashared/oracle /oracle nfs rw,vers=3,hard,timeo=600,rsize=262144,wsize=262144,bg,noatime,nointr,lock 0 0
10.1.9.4:/gr1ora-orashared/oracle_GR1 /oracle/GR1 nfs rw,vers=3,hard,timeo=600,rsize=262144,wsize=262144,bg,noatime,nointr,lock 0 0
10.1.9.4:/gr1ora-orashared/19.0.0 /oracle/GR1/19.0.0 nfs rw,vers=3,hard,timeo=600,rsize=262144,wsize=262144,bg,noatime,nointr,lock 0 0
10.1.9.4:/gr1ora-oraarch/GR1 /oracle/GR1/oraarch nfs rw,vers=3,hard,timeo=600,rsize=262144,wsize=262144,bg,noatime,nointr,lock 0 0
10.1.9.4:/gr1ora-mirrlog/mirrlogA /oracle/GR1/mirrlogA nfs rw,vers=3,hard,timeo=600,rsize=262144,wsize=262144,bg,noatime,nointr,lock 0 0
10.1.9.4:/gr1ora-mirrlog/mirrlogB /oracle/GR1/mirrlogB nfs rw,vers=3,hard,timeo=600,rsize=262144,wsize=262144,bg,noatime,nointr,lock 0 0
10.1.9.4:/gr1ora-origlog/origlogA /oracle/GR1/origlogA nfs rw,vers=3,hard,timeo=600,rsize=262144,wsize=262144,bg,noatime,nointr,lock 0 0
10.1.9.4:/gr1ora-origlog/origlogB /oracle/GR1/origlogB nfs rw,vers=3,hard,timeo=600,rsize=262144,wsize=262144,bg,noatime,nointr,lock 0 0
10.1.9.4:/gr1ora-sapdata/sapdata1 /oracle/GR1/sapdata1 nfs rw,vers=3,hard,timeo=600,rsize=262144,wsize=262144,bg,noatime,nointr,lock 0 0
10.1.9.4:/gr1ora-sapdata/sapdata2 /oracle/GR1/sapdata2 nfs rw,vers=3,hard,timeo=600,rsize=262144,wsize=262144,bg,noatime,nointr,lock 0 0
10.1.9.4:/gr1ora-sapdata/sapdata3 /oracle/GR1/sapdata3 nfs rw,vers=3,hard,timeo=600,rsize=262144,wsize=262144,bg,noatime,nointr,lock 0 0
10.1.9.4:/gr1ora-sapdata/sapdata4 /oracle/GR1/sapdata4 nfs rw,vers=3,hard,timeo=600,rsize=262144,wsize=262144,bg,noatime,nointr,lock 0 0
Oracle Control File Locations (default locations)
|
/oracle/GR1/sapdata1/cntrl |
cntrlGR1.dbf |
|
/oracle/GR1/origlogA/cntrl |
cntrlGR1.dbf |
|
/oracle/GR1/origlogB/cntrl |
cntrlGR1.dbf |
Oracle pfile and spifle locations (default locations)
|
spfile |
/oracle/GR1/19.0.0/dbs |
spfileGR1.ora |
|
pfile |
/oracle/GR1/19.0.0/dbs |
initGR1.ora |
AzAcSnap backups
This section provides a brief overview of the use of AzAcSnap. More detailed information in available in the AzAcSnap documentation.
AzAcSnap command to orchestrate the online Oracle backup
azacsnap --configfile=/sapcd/snapit/home/GR1/GR1azacsnap.json -c backup --volume data --prefix GR1_hourly --retention 3
- Option definitions can be found in the AzAcSnap documentation.
AzAcSnap Configuration (GR1azacsnap.json)
These ANF volumes will be captured within an Azure NetApp Files storage snapshot while Oracle is in hot backup mode. You must ensure all Azure NetApp Files volumes containing sapdata1-X as well as Oracle control file locations are included.
|
dataVolume |
gr1ora-sapdata |
|
dataVolume |
gr1ora-origlog |
GR1azacsnap.json file
{
"version": "5.1 Preview",
"logPath": "/sapcd/snapit/home/GR1/logs",
"securityPath": "/sapcd/snapit/home/GR1/security",
"comments": [
"GR1 Oracle"
],
"database": [
{
"hana": null,
"oracle": {
"serverAddress": "gr1ora",
"sid": "GR1",
"connectString": "/@AZACSNAP",
"backupAbortWaitSeconds": -1,
"hliStorage": [],
"anfStorage": [
{
"dataVolume": [
{
"resourceId": "/subscriptions/zzzzzzzz-999999999 /resourceGroups/rg-mcscott/providers/Microsoft.NetApp/netAppAccounts/sap-eastus/capacityPools/sap-premium-mqos/volumes/gr1ora-origlog",
"authFile": "/sapcd/snapit/home/GR1/azureauth.json",
"subscription": "zzzzzzzz-999999999999999999",
"resourceGroupName": "rg-mcscott",
"accountName": "sap-eastus",
"poolName": "sap-premium-mqos",
"volume": "gr1ora-origlog"
},
{
"resourceId": "/subscriptions/zzzzzzzz-999999999 /resourceGroups/rg-mcscott/providers/Microsoft.NetApp/netAppAccounts/sap-eastus/capacityPools/sap-premium-mqos/volumes/gr1ora-sapdata",
"authFile": "/sapcd/snapit/home/GR1/azureauth.json",
"subscription": "zzzzzzzz-99999999999999999",
"resourceGroupName": "rg-mcscott",
"accountName": "sap-eastus",
"poolName": "sap-premium-mqos",
"volume": "gr1ora-sapdata"
}
],
"otherVolume": []
}
],
"amdStorage": []
}
}
]
}
Recover to time of backup (snapshot)
This example will recover the SAP Oracle database to the time of backup (no forward recovery). The example will use printer entries within SAP NW transaction SPAD to demonstrate recovery.
Current list of output devices
Show AzAcSnap backup result
Create new spool entry “ZZZZ”
Recover the database to the time of backup that will show only device entries “LP01, URD1”.
Shutdown R3 and the database.
|
(!) NOTE: |
ANF volume/Snapshot
|
gr1ora-sapdata |
gr1_hourly__2022-02-28t170108-2702409z |
You must select the Azure NetApp Files snapshot name you wish to use. Optionally, you can leverage AzAcSnap to revert the Azure NetApp Files volume.
Within your Azure NetApp Files subscription select the capacity pool and volume(s) that contains the sapdata volume(s). Select ‘Snapshots’ and then the appropriate snapshot name.
After confirming the volume name within the portal, the Azure NetApp Files sapdata volume has been reverted to the time of the backup. This operation regardless of size takes only minutes.
Copy the Oracle control file from the recently reverted Azure NetApp Files sapdata volume to the two other control file locations. This will ensure the control file from the time of the backup is consistent amongst the three copies.
As oraSID, copy the /oracle/GR1/sapdata1/cntrl/cntrlGR1.dbf to the other two locations:
- /oracle/GR1/origlogA/cntrl
- /oracle/GR1/origlogB/cntrl
The three Oracle control files are now consistent from the time of the backup.
Recover the database to the time of the backup. Ensure you have the Oracle archive(s) that were generated during the time AzAcSnap had the database in hot backup mode. This is not a forward recovery; this is simply applying any changes to the data files that occurred during hot backup mode.
Recover the database until cancel.
Archive GR1arch1_13_1097689630.dbf was generated during the time AzAcSnap had the database in hot backup mode. Ensure this file exists in the Oracle archive location.
Hit Enter to use suggested.
|
OPTIONAL: You can elect to continue applying Oracle archive files. In this sense, you would be recovering to a later time than the backup. |
Enter “cancel” to end recovery.
Database files are now in a consistent state. To recover the database to this backup we must initialize the online redo logs. This operation will start the database and re-init the redo logs.
Start SAP R/3 and verify printer entries within SPAD.
Successfully completed recovery to time of backup.
Recover to the last Transaction
This section describes how to recover an SAP/Oracle database when using Azure NetApp Files with AzAcSnap orchestration tool to protect the database to the last transaction located in the Oracle redo logs.
The example will use printer entries within SAP NW transaction SPAD to demonstrate recovery.
Current list of output devices
Show AzAcSnap backup result
Create new spool entry “YYYY”
Recover the database with forward recovery to show newly created spool device “YYYY”.
Shutdown R3 and the database.
| (!) NOTE: The following operations are destructive as we will be reverting the ANF volume back in time to the snapshot created via AzAcSnap. Optionally, you could create new a volume from the snapshot as to keep the data currently in the live file systems. |
ANF volume/Snapshot
|
gr1ora-sapdata |
gr1_hourly__2022-02-28t171932-6058388z |
The name of the Azure NetApp Files snapshot can be found within the AzAcSnap log or for this example we are using the most recent snapshot. Optionally, you can leverage AzAcSnap to revert the Azure NetApp Files volume.
Within your ANF subscription select the capacity pool and volume(s) that contains the sapdata volume(s). Select ‘Snapshots’ and then the appropriate snapshot name.
At this point the Oracle datafiles located in the Azure NetApp Files sapdata volume (gr1ora-sapdata) have been reverted to the time of the snapshot.
The Oracle control file in /oracle/GR1/sapdata1/cntrl has also been reverted to the time of the snapshot. For this reason, we must use the latest Oracle control file located in one of the other two locations:
- /oracle/GR1/origlogA/cntrl/cntrlGR1.dbf
- /oracle/GR1/origlogB/cntrl/cntrlGR1.dbf
In this example, as oraSID, copy the /oracle/GR1/origlogA/cntrl/cntrlGR1.dbf to the other two locations. This will ensure all three Oracle control files are the latest and consistent across all three copies.
Recover the database to the last transactions (still located within the redo logs). Ensure you have the Oracle archive(s) that were generated during the time AzAcSnap had the database in hot backup mode plus all the archives generated after the backup was completed.
Recover the database.
Database files are now in a consistent state.
Open the database.
Start SAP and verify printer entries.
Successfully completed recovery to latest transactions.