Oracle 19c on Azure NetApp Files for SAP Workloads
PoC and Validation
Oracle Installation Guide
Introduction
This blog article describes how you can install Oracle 19c on Azure for SAP Applications using Azure NetApp Files (ANF) as storage platform for data and log areas. The support of ANF is mentioned in the two SAP notes:
2015553 - SAP on Microsoft Azure: Support prerequisites - SAP ONE Support Launchpad
This article is written for experienced technical administrators. Not all aspects and considerations are covered because we assume a profound technical knowledge and expertise on all the related technologies like Linux, Oracle DB and NFS.
The minimum supported components/releases are Oracle Linux 8.2, Oracle 19c, and NFS 4.1 on ANF. Always check SAP note #2039619 - SAP Applications on Microsoft Azure using the Oracle Database: Supported Products and Vers...
for precise and exact support information.
Thank you from the awesome support to:
Jan Klokkers Oracle
Christoph Kurucz Oracle
Geert van Teylingen Azure NetApp Files alliance
Jeffrey Steiner NetApp
Juergen Thomas Microsoft
So, let's go and start.
Installation
Create a resource Group.
Create the VM
Creating the Jump Box
Oracle Linux
Check the Kernel version
uname -a
Linux ralforavm01 5.4.17-2011.6.2.el8uek.x86_64 #2 SMP Thu Sep 3 13:38:27 PDT 2020 x86_64 x86_64 x86_64 GNU/Linux
Check the OS Release
cat /etc/os-release
NAME="Oracle Linux Server"
VERSION="8.2"
ID="ol"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="8.2"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Oracle Linux Server 8.2"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:oracle:linux:8:2:server"
HOME_URL="https://linux.oracle.com/"
BUG_REPORT_URL="https://bugzilla.oracle.com/"
List the number and type of CPU'S of the VM
lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 32
On-line CPU(s) list: 0-31
Thread(s) per core: 2
Core(s) per socket: 16
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 63
Model name: Intel(R) Xeon(R) CPU E7-8890 v3 @ 2.50GHz
Stepping: 4
CPU MHz: 2493.986
BogoMIPS: 4987.97
Virtualization: VT-x
Hypervisor vendor: Microsoft
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 46080K
NUMA node0 CPU(s): 0-31
Create a storage Account
Create a capacity Pool
Add a capacity Pool
Depending on the database size and expected performance decide which ANF SKU does fit best for the target database. In our case it is Ultra.
Finished
Volume design
Create the delegated subnet for ANF
Create a Capacity Pool
Select the protocol for the volumes
Add a Capacity Pool
After deploying the above mentioned design it looks like this:
Configure the /etc/hosts
vi /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
172.17.4.9 ralforavm01.localdomain ralforavm01
172.17.10.4 anf01
Configure the hostname as FQDN
vi /etc/hostname
ralforavm01.localdomain
Create the mountpoints for Oracle and SAP
mkdir -p /oracle/C11/sapdata1 /oracle/C11/sapdata2 /oracle/C11/sapdata3 /oracle/C11/sapdata4 /oracle/C11/origlogA /oracle/C11/origlogB /oracle/C11/mirrlogA /oracle/C11/mirrlogB /oracle/C11/saparch /oracle/C11/sapbackup /oracle/C11/sapreorg /oracle/C11/orainstall /oracle/C11/saptrace /oracle/C11/oraarch /oracle/C11/19.0.0 /oracle/stage /oracle/client
mkdir -p /usr/sap/C11 /sapmnt/C11 /usr/sap/trans
mount the root volume and create the mountpoints.
mount anf01:/ralforasapdatav3 /mnt
mkdir /mnt/sapdata1 /mnt/sapdata2 /mnt/sapdata3 /mnt/sapdata4 ; umount /mnt
mount anf01:/ralforaclemirrlog /mnt
mkdir /mnt/mirrlogA /mnt/mirrlogB; umount /mnt
mount anf01:/ralforacleoriglog /mnt
mkdir /mnt/origlogA /mnt/origlogB; umount /mnt
mount anf01:/ralforacleshared /mnt
mkdir /mnt/oraarch /mnt/saparch /mnt/sapbackup /mnt/sapreorg /mnt/saptrace /mnt/Software /mnt/usr_sap /mnt/sapmnt /mnt/19.0.0 /mnt/trans /mnt/ora_client /mnt/orainstall /mnt/ora_stage ;umount /mnt
Manage the idmapd for ANF
vi /etc/idmapd.conf
# Example
[General]
Domain = defaultv4iddomain.com
modify the /etc/fstab
#
# Oracle
#
anf01:/ralforacledatav4/sapdata1 /oracle/C11/sapdata1 nfs rw,hard,timeo=600,vers=4.1,rsize=262144,wsize=262144,noatime,nointr,lock 0 0
anf01:/ralforacledatav4/sapdata2 /oracle/C11/sapdata2 nfs rw,hard,timeo=600,vers=4.1,rsize=262144,wsize=262144,noatime,nointr,lock 0 0
anf01:/ralforacledatav4/sapdata3 /oracle/C11/sapdata3 nfs rw,hard,timeo=600,vers=4.1,rsize=262144,wsize=262144,noatime,nointr,lock 0 0
anf01:/ralforacledatav4/sapdata4 /oracle/C11/sapdata4 nfs rw,hard,timeo=600,vers=4.1,rsize=262144,wsize=262144,noatime,nointr,lock 0 0
anf01:/ralforaclemirrlog/mirrlogA /oracle/C11/mirrlogA nfs rw,hard,timeo=600,vers=4.1,rsize=262144,wsize=262144,noatime,nointr,lock 0 0
anf01:/ralforaclemirrlog/mirrlogB /oracle/C11/mirrlogB nfs rw,hard,timeo=600,vers=4.1,rsize=262144,wsize=262144,noatime,nointr,lock 0 0
##
anf01:/ralforacleoriglog/origlogA /oracle/C11/origlogA nfs rw,hard,timeo=600,vers=4.1,rsize=262144,wsize=262144,noatime,nointr,lock 0 0
anf01:/ralforacleoriglog/origlogB /oracle/C11/origlogB nfs rw,hard,timeo=600,vers=4.1,rsize=262144,wsize=262144,noatime,nointr,lock 0 0
##
anf01:/ralforacleshared/19.0.0 /oracle/C11/19.0.0 nfs rw,hard,timeo=600,vers=4.1,rsize=262144,wsize=262144,noatime,nointr,lock 0 0
anf01:/ralforacleshared/oraarch /oracle/C11/oraarch nfs rw,hard,timeo=600,vers=4.1,rsize=262144,wsize=262144,noatime,nointr,lock 0 0
anf01:/ralforacleshared/saparch /oracle/C11/saparch nfs rw,hard,timeo=600,vers=4.1,rsize=262144,wsize=262144,noatime,nointr,lock 0 0
anf01:/ralforacleshared/sapbackup /oracle/C11/sapbackup nfs rw,hard,timeo=600,vers=4.1,rsize=262144,wsize=262144,noatime,nointr,lock 0 0
anf01:/ralforacleshared/sapreorg /oracle/C11/sapreorg nfs rw,hard,timeo=600,vers=4.1,rsize=262144,wsize=262144,noatime,nointr,lock 0 0
anf01:/ralforacleshared/saptrace /oracle/C11/saptrace nfs rw,hard,timeo=600,vers=4.1,rsize=262144,wsize=262144,noatime,nointr,lock 0 0
anf01:/ralforacleshared/Software /oracle/C11/Software nfs rw,hard,timeo=600,vers=4.1,rsize=262144,wsize=262144,noatime,nointr,lock 0 0
anf01:/ralforacleshared/ora_client /oracle/client nfs rw,hard,timeo=600,vers=4.1,rsize=262144,wsize=262144,noatime,nointr,lock 0 0
anf01:/ralforacleshared/ora_stage /oracle/stage nfs rw,hard,timeo=600,vers=4.1,rsize=262144,wsize=262144,noatime,nointr,lock 0 0
anf01:/ralforacleshared/usr_sap /usr/sap/C11 nfs rw,hard,timeo=600,vers=4.1,rsize=262144,wsize=262144,noatime,nointr,lock 0 0
anf01:/ralforacleshared/trans /usr/sap/trans nfs rw,hard,timeo=600,vers=4.1,rsize=262144,wsize=262144,noatime,nointr,lock 0 0
anf01:/ralforacleshared/sapmnt /sapmnt nfs rw,hard,timeo=600,vers=4.1,rsize=262144,wsize=262144,noatime,nointr,lock 0 0
anf01:/ralforacleshared/orainst /oracle/C11/orainstall nfs rw,hard,timeo=600,vers=4.1,rsize=262144,wsize=262144,noatime,nointr,lock 0 0
Stop the firewalld
systemctl disable firewalld
systemctl stop firewalld
Update the VM’s OS image (no registration required)
yum update
Last metadata expiration check: 1:24:42 ago on Tue 03 Nov 2020 10:04:53 AM UTC.
…
Install 5 Packages
Upgrade 25 Packages
Total download size: 159 M
Is this ok [y/N]:y
Install the required libraries.
dnf install -y oracle-database-preinstall-19c
dnf install -y binutils
dnf install -y compat-libstdc++-33
dnf install -y elfutils-libelf
dnf install -y elfutils-libelf-devel
dnf install -y fontconfig-devel
dnf install -y glibc
dnf install -y glibc-devel
dnf install -y ksh
dnf install -y libaio
dnf install -y libaio-devel
dnf install -y libXrender
dnf install -y libXrender-devel
dnf install -y libX11
dnf install -y libXau
dnf install -y libXi
dnf install -y libXtst
dnf install -y libgcc
dnf install -y librdmacm-devel
dnf install -y libstdc++
dnf install -y libstdc++-devel
dnf install -y libxcbdnf install -y unixODBC
# New for OL8
dnf install -y ipmiutil
dnf install -y libnsl2
dnf install -y libnsl2-devel
dnf install -y net-tools
dnf install -y nfs-utils
dnf install -y libnsl
dnf install -y make
dnf install -y python # ACFS
dnf install -y python-configshell # ACFS
dnf install -y python-rtslib # ACFS
dnf install -y python-six # ACFS
dnf install -y targetcli # ACFS
dnf install -y smartmontools
dnf install -y sysstat
modify the /etc/sysctl.conf
Change the nessesary kernel parameter in /etc/sysctl.conf
kernel.sem = 250 32000 100 128
kernel.shmmni = 4096
kernel.shmall = 1073741824
kernel.shmmax = 4398046511104
kernel.panic_on_oops = 1
net.ipv6.conf.all.disable_ipv6 = 1
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.core.optmem_max = 16777216
net.core.somaxconn = 4096
net.core.netdev_max_backlog = 300000
net.ipv4.ip_local_port_range = 9000 65500
net.ipv4.conf.all.rp_filter = 2
net.ipv4.conf.default.rp_filter = 2
net.ipv4.conf.default.rp_filter = 2
net.ipv4.tcp_rmem = 65536 16777216 16777216
net.ipv4.tcp_wmem = 65536 16777216 16777216
net.ipv4.tcp_max_syn_backlog = 16348
net.ipv4.tcp_slow_start_after_idle = 0
net.ipv4.tcp_dsack = 1
net.ipv4.tcp_sack = 1
net.ipv4.tcp_no_metrics_save = 1
net.ipv4.tcp_moderate_rcvbuf = 1
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_syn_retries = 8
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_timestamps = 1
vm.swappiness = 10
vm.dirty_bytes = 629145600
vm.dirty_background_bytes = 314572800
vm.max_map_count = 2147483647
vm.memory_failure_early_kill = 1
fs.file-max = 820448174
fs.aio-max-nr = 18446744073709551615
Download the latest Oracle 19c DB and the Patch-Set (as this document was written it was) 19.15.0.0 (fMay 2022) from the SAP Software download Center.
Create the SAP and Oracle user
groupadd sapsys -g 79
groupadd dba -g 80
groupadd oper -g 85
groupadd oinstall -g 90
Create the required oracle user
useradd c11adm -u 1001 -d /usr/sap/C11 -g 79 -s /bin/csh -c "SAP Administrator C11"
useradd orac11 -u 1002 -d /oracle/C11 -g 80 -G oper,dba,oinstall -s /bin/bash -c "Oracle Administrator C11"
useradd oracle -u 1003 -d /oracle/C11 -g 90 -G oper,dba -s /bin/bash -c "Oracle Installation"
usermod -a -G oper orac11
disable selinux
[root@ralforavm01 ~]# sestatus
SELinux status: enabled
SELinuxfs mount: /sys/fs/selinux
SELinux root directory: /etc/selinux
Loaded policy name: targeted
Current mode: permissive
Mode from config file: permissive
Policy MLS status: enabled
Policy deny_unknown status: allowed
Memory protection checking: actual (secure)
Max kernel policy version: 31
Disable selinux
vi /etc/selinux/config
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these three values:
# targeted - Targeted processes are protected,
# minimum - Modification of targeted policy. Only selected processes are protected.
# mls - Multi Level Security protection.
SELINUXTYPE=targeted change the user Permission to the sap and oracle directories.
Reboot the server
Check if selinux is disabled
sestatus
SELinux status: disabled
Change the nfs mountpoint permissions.
chown -R c11adm:sapsys /usr/sap/
chown -R orac11:dba /oracle
Unzip the compressed files into “your” Software directory and execute the install process under the directory SAP.
su – orac11
cd /Software/Oracle_19c_Inst/LINUX_X86_64/db_home/SAP
export CV_ASSUME_DISTID=OEL8.1
export DB_SID=C11
export ORACLE_STAGE=/oracle/stage
export ORACLE_BASE=/oracle/C11
Call the Oracle installer for 19c
./19cinstall.sh -silent -db C11 -oracle_stage /oracle/C11/Software/Oracle_19c_Inst/LINUX_X86_64/db_home/
As a root user, execute the following script(s):
- 1. /oracle/oraInventory/orainstRoot.sh
- 2. /oracle/C11/19.0.0/root.sh
[root@ralforavm01 SAP]# /oracle/oraInventory/orainstRoot.sh
[root@ralforavm01 SAP]# /oracle/C11/19.0.0/root.sh
Test connect to Oracle
su – orac11
sqlplus /nolog
SQL> CONNECT / SYS AS SYSDBA
SQL*Plus: Release 19.0.0.0.0 - Production on Wed Nov 4 11:51:28 2020
Version 19.3.0.0.0
Copyright (c) 1982, 2019, Oracle. All rights reserved.
Connected to an idle instance.
SQL> exit
Patch Installation
Install the OPatch utility first.
https://updates.oracle.com/ARULink/PatchDetails/process_form?patch_num=6880880
or download it from SAP swdc --> databases Oracle19c
chmod 777 OPATCH19P_2008-70004508.ZIP
su – orac11
/oracle/C11/19.0.0/OPatch/opatch version
OPatch Version: 12.2.0.1.17
OPatch succeeded.
cd /oracle/C11/19.0.0/
unzip /Software/OPATCH19P_2008-70004508.ZIP
/oracle/C11/19.0.0/OPatch/opatch version
========================================================
GENERIC OPATCH VERSION - FOR USE IN SAP ENVIRONMENT ONLY
========================================================
OPatch Version: 12.2.0.1.21
OPatch succeeded.
Validate the Oracle Inventory
/oracle/C11/19.0.0/OPatch/opatch lsinventory -detail -oh /oracle/C11/19.0.0
…..
...
OPatch succeeded.
Apply the OPatch 19.14.0.0 to the DB
Apply the patches
cd /Software/Oracle_Patch_19.8/31305339/31281355
/oracle/C11/19.0.0/OPatch/opatch apply
========================================================
GENERIC OPATCH VERSION - FOR USE IN SAP ENVIRONMENT ONLY
========================================================
Oracle Interim Patch Installer version 12.2.0.1.21
Copyright (c) 2020, Oracle Corporation. All rights reserved.
Oracle Home : /oracle/C11/19.0.0
Central Inventory : /oracle/oraInventory
from : /oracle/C11/19.0.0/oraInst.loc
OPatch version : 12.2.0.1.21 FOR USE IN SAP ENVIRONMENT ONLY
OUI version : 12.2.0.7.0
Log file location : /oracle/C11/19.0.0/cfgtoollogs/opatch/opatch2020-11-04_16-10-50PM_1.log
Verifying environment and performing prerequisite checks...
OPatch continues with these patches: 31281355
Do you want to proceed? [y|n] Y
...
..
cd ../31304218
/oracle/C11/19.0.0/OPatch/opatch apply
cd ../31305087
/oracle/C11/19.0.0/OPatch/opatch apply
cd ../31335188
/oracle/C11/19.0.0/OPatch/opatch apply
cd /oracle/C11/19/OPatch/
datapatch -verbose
Any databases that have invalid objects after the execution of datapatch should have utlrp.sql run to revalidate those objects.
For example:
cd $ORACLE_HOME/rdbms/admin
sqlplus /nolog
SQL> CONNECT / AS SYSDBA
SQL> @utlrp.sql
Apply the SAP Bundle Patch SAP19P_2008-70004508.ZIP (also downloaded from SWDC
dnf install bc.x86_64
cd /Software/SAP19P/SBP_19800200714_202008
ls -l /oracle/C11/19/MOpatch
ls: cannot access '/oracle/C11/19/MOpatch': No such file or directory
cp -r MOPatch /oracle/C11/19/
$ORACLE_HOME/MOPatch/mopatch.sh -hv
MOPatch - Install Multiple Oracle Patches in One Run - 2.5.1.
Copyright (c) 2007, 2020, Oracle and/or its affiliates. All rights reserved.
cd /Software/SAP19P
ls -l
-rw-r--r--. 1 root root 14532539 Nov 4 15:52 SAP19P_2008-70004508.ZIP
drwxr-xr-x. 24 oracle oinstall 4096 Aug 18 17:02 SBP_19800200714_202008
-rw-r--r--. 1 oracle oinstall 76390 Aug 21 15:21 SIGNATURE.SMF
export IHRDBMS=/oracle/C11/19.0.0
env |grep ORA
ORACLE_UNQNAME=C11
ORACLE_SID=C11
ORACLE_BASE=/oracle/C11
ORACLE_HOME=/oracle/C11/19.0.0
ORA_INVENTORY=/oracle/oraInventory
ORACLE_HOSTNAME=ralforavm01.localdomain
$IHRDBMS/MOPatch/mopatch.sh -v -s SAP19P_2008-70004508.ZIP
Create a Database
https://docs.oracle.com/cd/B28359_01/server.111/b28310/create003.htm#ADMIN11073
vi /oracle/C11/19.0.0/dbs/initC11.ora
db_name='C11'
memory_target=1G
processes = 150
db_block_size=8192
open_cursors=300
compatible ='11.2.0'
sqlplus /nolog
SQL*Plus: Release 19.0.0.0.0 - Production on Thu Nov 5 14:57:34 2020
Version 19.8.0.0.0
Copyright (c) 1982, 2020, Oracle. All rights reserved.
SQL> connect / as sysdba
Connected to an idle instance.
SQL> CREATE SPFILE FROM PFILE;
File created.
SQL> startup nomount
ORACLE instance started.
Total System Global Area 713031464 bytes
Fixed Size 8900392 bytes
Variable Size 557842432 bytes
Database Buffers 134217728 bytes
Redo Buffers 12070912 bytes
SQL>
Create C11 Database (only for a Test! The DB will get created during the SAP installation)
vi create_db.sql
CREATE DATABASE C11
USER SYS IDENTIFIED BY manager
USER SYSTEM IDENTIFIED BY manager
LOGFILE GROUP 1 ('/oracle/C11/origlogA/redo01.log') SIZE 100M,
GROUP 2 ('/oracle/C11/origlogB/redo02.log') SIZE 100M,
GROUP 3 ('/oracle/C11/origlogA/redo03.log') SIZE 100M,
GROUP 4 ('/oracle/C11/origlogB/redo04.log') SIZE 100M
MAXLOGFILES 5
MAXLOGMEMBERS 5
MAXLOGHISTORY 1
MAXDATAFILES 100
CHARACTER SET US7ASCII
NATIONAL CHARACTER SET AL16UTF16
EXTENT MANAGEMENT LOCAL
DATAFILE '/oracle/C11/sapdata1/system01.dbf' SIZE 500M REUSE
SYSAUX DATAFILE '/oracle/C11/sapdata2/sysaux01.dbf' SIZE 325M REUSE
DEFAULT TABLESPACE users
DATAFILE '/oracle/C11/sapdata3/users01.dbf'
SIZE 500M REUSE AUTOEXTEND ON MAXSIZE UNLIMITED
DEFAULT TEMPORARY TABLESPACE tempts1
TEMPFILE '/oracle/C11/sapdata1/temp01.dbf'
SIZE 20M REUSE
UNDO TABLESPACE undotbs
DATAFILE '/oracle/C11/sapdata4/undotbs01.dbf'
SIZE 200M REUSE AUTOEXTEND ON MAXSIZE UNLIMITED;
Create the DB
SQL> @create_db.sql
Database created.
Run the required post processing scripts
SQL> @$ORACLE_HOME/rdbms/admin/catalog.sql;
SQL> @$ORACLE_HOME/rdbms/admin/catproc.sql;
SQL> @$ORACLE_HOME/sqlplus/admin/pupbld.sql;
Test if the DB is open
SQL> select status from v$instance;
STATUS
------------
OPEN
1 row selected.
DROP DATABASE:
SQL> shutdown abort
SQL> startup force mount restrict;
ORACLE instance started.
Total System Global Area 243268208 bytes
Fixed Size 8895088 bytes
Variable Size 180355072 bytes
Database Buffers 50331648 bytes
Redo Buffers 3686400 bytes
Database mounted.
SQL> drop database;
Database dropped.
Enable dNFS
To use Oracle dNFS MultiPath with Azure NetApp Files you will need an oranfstab file, Oracle will look in the following order and locations for the configuration file.
- /etc/oranfstab – Server wide
- $ORACLE_HOME/dbs/oranfstab – Oracle Home specific
You need to specify ONLY those volumes in the oranfstab which are used by Oracle. If you specify not used volumes this construct will end up in an non working dNFS scenario. Theoretically oracle should look into the /etc/mtab to automatically enable dNFS volumes.
Very Important!!
There is very important information available regarding setup, configuration and patch levels this will ensure an optimal operation and support. Please read this section under https://docs.microsoft.com/azure/azure-netapp-files/faq-nfs#oracle-dnfs
Below is a copy of my $ORACLE_HOME/dbs/oranfstab
cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
172.17.4.9 ralforavm01.localdomain ralforavm01
172.17.5.4 anf01
dNFS oranfstab for NFSv4.1
[root@ralforavm01 dbs]# cat oranfstab
server: anf01
path: 172.17.10.4
nfs_version: NFSv4.1
security_default: sys
export: /ralforacledatav4/sapdata1 mount: /oracle/C11/sapdata1
export: /ralforacledatav4/sapdata2 mount: /oracle/C11/sapdata2
export: /ralforacledatav4/sapdata3 mount: /oracle/C11/sapdata3
export: /ralforacledatav4/sapdata4 mount: /oracle/C11/sapdata4
export: /ralforacleoriglog/origlogA mount: /oracle/C11/origlogA
export: /ralforacleoriglog/origlogB mount: /oracle/C11/origlogB
export: /ralforacleshared mount: /oracle/C11/19.0.0
export: /ralforaoraarch mount: /oracle/C11/oraarch
export: /ralforasapbackup mount: /oracle/C11/sapbackup
Enabling dNFS
Change dir to $ORACLE_HOME/rdbms/lib and use the make command e.g
cd $ORACLE_HOME/rdbms/lib
make -f ins_rdbms.mk dnfs_on
rm -f /oracle/C11/19.0.0/rdbms/lib/odm/libnfsodm19.so; \
cp /oracle/C11/19.0.0/lib/libnfsodm19.so /oracle/C11/19.0.0/rdbms/lib/odm/libnfsodm19.so
restart oracle
Verify Oracle dNFS Usage
If dNFS is enabled correctly there should be no errors in the alert files. Here we see dNFS issues in the trace directory:
pwd
/oracle/C11/diag/rdbms/c11/C11/trace
The output here shows dNFS issues.... If everything is configured correctly there should be no output.
grep -i kgnfs *.trc
....
…
C11_lgwr_99210.trc: [53257270973] kgnfs_mntrsp: KGNFS_NFSPROC3_MNT FAIL 13
C11_lgwr_99210.trc:[53257271002] kgnfs_dump_hex_data: 0x7ffe492523bc len 120 lt 30
C11_lgwr_99210.trc: [53257271026] kgnfs_dump_hex_data: 0x7ffe492547e8 len 528 lt 132
C11_lgwr_99210.trc: [53257283545] kgnfs_mntrsp: KGNFS_NFSPROC3_MNT FAIL 13
C11_lgwr_99210.trc:[53257283571] kgnfs_dump_hex_data: 0x7ffe492523bc len 120 lt 30
C11_lgwr_99210.trc: [53257283595] kgnfs_dump_hex_data: 0x7ffe492547e8 len 528 lt 132
C11_lgwr_99210.trc: [53257296302] kgnfs_mntrsp: KGNFS_NFSPROC3_MNT FAIL 13
C11_lgwr_99210.trc:[53257296328] kgnfs_dump_hex_data: 0x7ffe492523bc len 120 lt 30
C11_lgwr_99210.trc: [53257296359] kgnfs_dump_hex_data: 0x7ffe492547e8 len 528 lt 132
C11_ora_99256.trc:[53257072683] kgnfs_mntrsp: KGNFS_NFSPROC3_MNT FAIL 13
C11_ora_99256.trc:[53257072983] kgnfs_dump_hex_data: 0x7ffd811d920c len 120 lt 30
C11_ora_99256.trc: [53257073007] kgnfs_dump_hex_data: 0x7ffd811db638 len 528 lt 132
C11_ora_99256.trc: [53257102434] kgnfs_mntrsp: KGNFS_NFSPROC3_MNT FAIL 13
C11_ora_99256.trc:[53257102459] kgnfs_dump_hex_data: 0x7ffd811d920c len 120 lt 30
C11_ora_99256.trc: [53257102485] kgnfs_dump_hex_data: 0x7ffd811db638 len 528 lt 132
C11_ora_99256.trc:[53257522107] kgnfs_mntrsp: KGNFS_NFSPROC3_MNT FAIL 13
C11_ora_99256.trc:[53257522142] kgnfs_dump_hex_data: 0x7ffd811e39ec len 120 lt 30
C11_ora_99256.trc: [53257522167] kgnfs_dump_hex_data: 0x7ffd811e5e18 len 528 lt 132
In this example dNFS is not enabled correctly.
On NFSv4.1
col SVRNAME format a10
col DIRNAME format a40
col NFSVERSION format a10
col SECURITY format a10
SQL> select SVRNAME,DIRNAME,NFSVERSION,SECURITY from v$dnfs_servers;
SVRNAME DIRNAME NFSVERSION SECURITY
---------- ---------------------------------------- ---------- ----------
anf01 /ralforacleshared NFSv4.1 sys
anf01 /ralforacledatav4/sapdata1 NFSv4.1 sys
anf01 /ralforacledatav4/sapdata2 NFSv4.1 sys
anf01 /ralforacledatav4/sapdata4 NFSv4.1 sys
anf01 /ralforacledatav4/sapdata3 NFSv4.1 sys
anf01 /ralforacleoriglog/origlogB NFSv4.1 sys
anf01 /ralforacleoriglog/origlogA NFSv4.1 sys
anf01 /ralforaoraarch NFSv4.1 sys
8 rows selected.
After switching the redo-logfile also the oraarch should appear
SQL> alter system switch logfile;
System altered.
SQL> select SVRNAME,DIRNAME,NFSVERSION,SECURITY from v$dnfs_servers;
SVRNAME DIRNAME NFSVERSION SECURITY
---------- ---------------------------------------- ---------- ----------
anf01 /ralforacleshared NFSv4.1 sys
anf01 /ralforacledatav4/sapdata1 NFSv4.1 sys
anf01 /ralforacledatav4/sapdata2 NFSv4.1 sys
anf01 /ralforacledatav4/sapdata4 NFSv4.1 sys
anf01 /ralforacledatav4/sapdata3 NFSv4.1 sys
anf01 /ralforacleoriglog/origlogB NFSv4.1 sys
anf01 /ralforacleoriglog/origlogA NFSv4.1 sys
anf01 /ralforaoraarch NFSv4.1 sys
anf01 /ralforaoraarch NFSv4.1 sys
9 rows selected.
Disable dNFS
For test reasons it can be necessary to disable the dNFS cleint.
Change dir to $ORACLE_HOME/rdbms/lib and use the make command e.g
cd $ORACLE_HOME/rdbms/lib
make -f ins_rdbms.mk dnfs_off
rm -f /oracle/C11/19.0.0/rdbms/lib/odm/libnfsodm19.so
restart oracle
Performance Test
Before we start the performance testing we configure the system accordingly.
Reference: I/O Configuration and Design (oracle.com)
DB_FILE_MULTIBLOCK_READ_COUNT parameter is used to specifies the maximum number of blocks read in one I/O operation during a sequential scan.
SQL> show parameter DB_FILE_MULTIBLOCK_READ_COUNT
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
db_file_multiblock_read_count integer 128
SQL> alter system set DB_FILE_MULTIBLOCK_READ_COUNT=256 scope=both;
Setting the dNFS_BATCH_SIZE parameter to 128 (restart of the DB is required to activate)
SQL> alter system set DNFS_BATCH_SIZE=128 scope=spfile;
SQL> shutdown
SQL> startup
SQL> show parameter DNFS_BATCH_SIZE
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
dnfs_batch_size integer 128
For now we keep the default for:
|
SORT_AREA_SIZE |
Determines I/O sizes and concurrency for sort operations. |
|
HASH_AREA_SIZE |
Determines the I/O size for hash operations. |
SQL> show parameter SORT_AREA_SIZE
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
sort_area_size integer 65536
SQL> show parameter HASH_AREA_SIZE
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
hash_area_size integer 131072
Running I/O Calibration
The I/O calibration feature of Oracle Database is accessed using the DBMS_RESOURCE_MANAGER.CALIBRATE_IO procedure. This procedure issues an I/O intensive read-only workload, made up of one megabyte of random of I/Os, to the database files to determine the maximum IOPS (I/O requests per second) and MBPS (megabytes of I/O per second) that can be sustained by the storage subsystem.
Preparation:
SQL> show parameter TIMED_STATISTICS
NAME TYPE VALUE
----------------------------------- ----------- ------------------------------
timed_statistics boolean TRUESQL> show parameter STATISTICS_LEVEL
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
client_statistics_level string TYPICAL
statistics_level string TYPICAL
We did set a performance quota for the ANF volumes. Depending on the size of the ANF volume it is either (Premium) 64MB/s / TB or for (Ultra) 128MB/s / TB. For the maximum troughput you would require large volumes.
Her in this test we used; Data Volume = 4*2TB Ultra = 4*256MB/s
NFSv4.1 dNFS disabled:
SQL> SET SERVEROUTPUT ON
DECLARE
lat INTEGER;
iops INTEGER;
mbps INTEGER;
BEGIN
-- DBMS_RESOURCE_MANAGER.CALIBRATE_IO (<DISKS>, <MAX_LATENCY>, iops, mbps, lat);
DBMS_RESOURCE_MANAGER.CALIBRATE_IO (2, 10, iops, mbps, lat);
DBMS_OUTPUT.PUT_LINE ('max_iops = ' || iops);
DBMS_OUTPUT.PUT_LINE ('latency = ' || lat);
DBMS_OUTPUT.PUT_LINE ('max_mbps = ' || mbps);
end;
/
max_iops = 38049
latency = 1.208
max_mbps = 297
max_iops = 38049
latency = 1
max_mbps = 297
PL/SQL procedure successfully completed.
Data Volume = 4*2TB Ultra = 4*256MB/s
NFSv4.1 dNFS enabled:
SQL> SET SERVEROUTPUT ON
DECLARE
lat INTEGER;
iops INTEGER;
mbps INTEGER;
BEGIN
-- DBMS_RESOURCE_MANAGER.CALIBRATE_IO (<DISKS>, <MAX_LATENCY>, iops, mbps, lat);
DBMS_RESOURCE_MANAGER.CALIBRATE_IO (2, 10, iops, mbps, lat);
DBMS_OUTPUT.PUT_LINE ('max_iops = ' || iops);
DBMS_OUTPUT.PUT_LINE ('latency = ' || lat);
DBMS_OUTPUT.PUT_LINE ('max_mbps = ' || mbps);
end;
/
max_iops = 38083
latency = .497
max_mbps = 296
max_iops = 38083
latency = 0
max_mbps = 296
PL/SQL procedure successfully completed.
For SAP workloads only filesystemio_options=setall is supported from Oracle.
SQL> ALTER SYSTEM SET FILESYSTEMIO_OPTIONS=SETALL SCOPE=SPFILE;
SQL> shutdown
SQL> startup
SQL> SHOW PARAMETER FILESYSTEMIO_OPTIONS
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
filesystemio_options string SETALL
Preparing the Orion Performance check tool
1 ANF Data-Volume 6TB with Ultra = 768MB/s troughput
Creating the orion files for the test
[orac11@ralforavm01 ~]$ dd if=/dev/zero of=/oracle/C11/sapdata1/orion1 bs=1024k count=10000
10000+0 records in
10000+0 records out
10485760000 bytes (10 GB, 9.8 GiB) copied, 12.7701 s, 821 MB/s
[orac11@ralforavm01 ~]$ dd if=/dev/zero of=/oracle/C11/sapdata2/orion2 bs=1024k count=10000
10000+0 records in
10000+0 records out
10485760000 bytes (10 GB, 9.8 GiB) copied, 12.744 s, 823 MB/s
[orac11@ralforavm01 ~]$ dd if=/dev/zero of=/oracle/C11/sapdata3/orion3 bs=1024k count=10000
10000+0 records in
10000+0 records out
10485760000 bytes (10 GB, 9.8 GiB) copied, 12.7365 s, 823 MB/s
[orac11@ralforavm01 ~]$ dd if=/dev/zero of=/oracle/C11/sapdata4/orion4 bs=1024k count=10000
10000+0 records in
10000+0 records out
10485760000 bytes (10 GB, 9.8 GiB) copied, 12.7354 s, 823 MB/s
Preparing the orion “lun” declaration file
vi orion_nfs.lun
/oracle/C11/sapdata1/orion1
/oracle/C11/sapdata2/orion2
/oracle/C11/sapdata3/orion3
/oracle/C11/sapdata4/orion4
Run the test (NFSv4.1)
~/19.0.0/bin/orion -run oltp -testname orion_nfs -hugenotneeded
ORION: ORacle IO Numbers -- Version RDBMS_19.3.0.0.0DBRU_LINUX.X64_190417
orion_nfs_20201125_1351
Calibration will take approximately 25 minutes.
[orac11@ralforavm01 ~]$ ~/19.0.0/bin/orion -run normal -testname orion_nfs -hugenotneeded
ORION: ORacle IO Numbers -- Version RDBMS_19.3.0.0.0DBRU_LINUX.X64_190417
orion_nfs_20201125_1425
Calibration will take approximately 190 minutes.
Using a large value for -cache_size may take longer.
Setting ftype=0
Maximum Large MBPS=771.28 @ Small=5 and Large=8
Maximum Small IOPS=34449 @ Small=19 and Large=0
Small Read Latency: avg=550.983 us, min=322.815 us, max=11369.239 us, std dev=134.209 us @ Small=19 and Large=0
Minimum Small Latency=419.288 usecs @ Small=1 and Large=0
Small Read Latency: avg=419.288 us, min=311.915 us, max=43826.080 us, std dev=153.365 us @ Small=1 and Large=0
Small Read / Write Latency Histogram @ Small=1 and Large=0
Latency: # of IOs (read) # of IOs (write)
0 - 256 us: 0 ( 0.00%) 0 ( 0.00%)
256 - 512 us: 138100 ( 96.71%) 0 ( 0.00%)
512 - 1024 us: 4268 ( 99.70%) 0 ( 0.00%)
1024 - 2048 us: 366 ( 99.95%) 0 ( 0.00%)
2048 - 4096 us: 52 ( 99.99%) 0 ( 0.00%)
4096 - 8192 us: 9 (100.00%) 0 ( 0.00%)
8192 - 16384 us: 3 (100.00%) 0 ( 0.00%)
16384 - 32768 us: 0 (100.00%) 0 ( 0.00%)
32768 - 65536 us: 1 (100.00%) 0 ( 0.00%)
65536 - 268435456 us: 0 (100.00%) 0 ( 0.00%)
Configuration of the Listener
Create the tnsnames.ora file
pwd
/oracle/C11/19/network/admin
vi tnsnames.ora
C11 =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = 172.17.4.9)(PORT = 1521))
)
(CONNECT_DATA =
(SERVICE_NAME = C11)
)
)
Start the listener
[orac11@ralforavm01 admin]$ lsnrctl start
LSNRCTL for Linux: Version 19.0.0.0.0 - Production on 01-DEC-2020 14:07:00
Copyright (c) 1991, 2020, Oracle. All rights reserved.
Starting /oracle/C11/19.0.0/bin/tnslsnr: please wait...
TNSLSNR for Linux: Version 19.0.0.0.0 - Production
Log messages written to /oracle/C11/diag/tnslsnr/ralforavm01/listener/alert/log.xml
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=ralforavm01.localdomain)(PORT=1521)))
Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))
STATUS of the LISTENER
------------------------
Alias LISTENER
Version TNSLSNR for Linux: Version 19.0.0.0.0 - Production
Start Date 01-DEC-2020 14:07:01
Uptime 0 days 0 hr. 0 min. 0 sec
Trace Level off
Security ON: Local OS Authentication
SNMP OFF
Listener Log File /oracle/C11/diag/tnslsnr/ralforavm01/listener/alert/log.xml
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=ralforavm01.localdomain)(PORT=1521)))
The listener supports no services
The command completed successfully
RMAN Configuration
Check the DB logmode:
SQL> select log_mode from v$database;
LOG_MODE
------------
NOARCHIVELOG
Configure database in ARCHIVELOG mode.
SQL> shutdown
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup mount;
ORACLE instance started.
Total System Global Area 1073737800 bytes
Fixed Size 8904776 bytes
Variable Size 864026624 bytes
Database Buffers 188743680 bytes
Redo Buffers 12062720 bytes
Database mounted.
SQL> alter database archivelog;
Database altered.
SQL> alter database open;
Database altered.
Ensure ARCHIVELOG destination.
SQL> select open_mode from v$database;
OPEN_MODE
--------------------
READ WRITE
SQL> archive log list
Database log mode Archive Mode
Automatic archival Enabled
Archive destination /oracle/C11/19.0.0/dbs/arch
Oldest online log sequence 30
Next log sequence to archive 33
Current log sequence 33
Change the oracle archive destination.
SQL> alter system set log_archive_dest_1='location=/oracle/C11/oraarch' scope=both;
SQL> archive log list
Database log mode Archive Mode
Automatic archival Enabled
Archive destination /oracle/C11/oraarch
Oldest online log sequence 30
Next log sequence to archive 33
Current log sequence 33
Ensure the flash/fast recovery area location.
SQL> show parameter db_recovery_file_dest
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
db_recovery_file_dest string
db_recovery_file_dest_size big integer 0
Connect to RMAN prompt with target database.
[orac11@ralforavm01 ~]$ rman target /
Recovery Manager: Release 19.0.0.0.0 - Production on Mon Nov 30 16:47:40 2020
Version 19.8.0.0.0
Copyright (c) 1982, 2019, Oracle and/or its affiliates. All rights reserved.
connected to target database: C11 (DBID=365226479)
RMAN>
Configure RMAN with control file auto-backup feature that will be auto-backup control file in case of major changes done in database.
RMAN> configure controlfile autobackup on;
using target database control file instead of recovery catalog
new RMAN configuration parameters:
CONFIGURE CONTROLFILE AUTOBACKUP ON;
new RMAN configuration parameters are successfully stored
Enable the optimization
RMAN> configure backup optimization on;
rew RMAN configuration parameters:
CONFIGURE BACKUP OPTIMIZATION ON;
new RMAN configuration parameters are successfully stored
Configure the retention policy
RMAN> CONFIGURE RETENTION POLICY TO RECOVERY WINDOW OF 7 DAYS;
new RMAN configuration parameters:
CONFIGURE RETENTION POLICY TO RECOVERY WINDOW OF 7 DAYS;
new RMAN configuration parameters are successfully stored
Connect to the recovery catalog database (RMAN Repository) & Create a tablespace to store RMAN catalog database objects.
SQL> show parameter db_recovery_file_dest
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
db_recovery_file_dest string
db_recovery_file_dest_size big integer 0
SQL> select global_name from global_name;
GLOBAL_NAME
--------------------------------------------------------------------------------
C11
Create a tablespace to be used for RMAN and the temp tablespace
SQL> create tablespace catalogtbs datafile '/oracle/C11/sapdata1/catalogtbs1.dbf' size 100M autoextend on maxsize unlimited;
SQL> create tablespace temporary tablespace temp_01 tempfile '/oracle/C11/sapdata1/temp_01.dbf' size 5M autoextend ON;
Create a RMAN user, assign RMAN tablespace to RMAN user as a default & grant recovery catalog owner,connect & resource privileges to RMAN user.
SQL> create user recoveryman identified by recoveryman;
SQL> alter user recoveryman default tablespace catalogtbs temporary tablespace temp_01 ;
SQL> grant recovery_catalog_owner to recoveryman;
SQL> grant connect,resource to recoveryman;
alter user recoveryman DEFAULT TABLESPACE catalogtbs quota unlimited on catalogtbs;
Connect to RMAN on target and recovery catalog database.
rman target / catalog recoveryman/recoveryman@C11
Recovery Manager: Release 19.0.0.0.0 - Production on Tue Dec 1 14:13:27 2020
Version 19.8.0.0.0
Copyright (c) 1982, 2019, Oracle and/or its affiliates. All rights reserved.
connected to target database: C11 (DBID=365226479)
connected to recovery catalog database
RMAN>
create catalog by issuing the following command in RMAN prompt and register the DB
RMAN> create catalog;
recovery catalog created
RMAN> register database;
database registered in recovery catalog
starting full resync of recovery catalog
full resync complete
Check whether registration was successful.
RMAN> report schema;
Report of database schema for database with db_unique_name C11
List of Permanent Datafiles
===========================
File Size(MB) Tablespace RB segs Datafile Name
---- -------- -------------------- ------- ------------------------
1 500 SYSTEM YES /oracle/C11/sapdata1/system01.dbf
2 325 SYSAUX NO /oracle/C11/sapdata2/sysaux01.dbf
3 305 UNDOTBS YES /oracle/C11/sapdata4/undotbs01.dbf
4 500 USERS NO /oracle/C11/sapdata3/users01.dbf
5 100 CATALOGTBS NO /oracle/C11/sapdata1/catalogtbs1.dbf
List of Temporary Files
=======================
File Size(MB) Tablespace Maxsize(MB) Tempfile Name
---- -------- -------------------- ----------- --------------------
1 20 TEMPTS1 20 /oracle/C11/sapdata1/temp01.dbf
2 5 TEMP_01 32767 /oracle/C11/sapdata1/temp_01.dbf
Or
RMAN> LIST INCARNATION OF DATABASE;
List of Database Incarnations
DB Key Inc Key DB Name DB ID STATUS Reset SCN Reset Time
------ ------- -------- ---------------- --- ---------- ----------
1 2 C11 365226479 CURRENT 1 23-NOV-20
Creating a system backup
[orac11@ralforavm01 ~]$ ls -l /oracle/C11/sapbackup/
total 0
Starting an RMAN session and configure the backup path.
rman target / catalog recoveryman/recoveryman@C11
Recovery Manager: Release 19.0.0.0.0 - Production on Tue Dec 1 14:42:22 2020
Version 19.8.0.0.0
Copyright (c) 1982, 2019, Oracle and/or its affiliates. All rights reserved.
connected to target database: C11 (DBID=365226479)
connected to recovery catalog database
RMAN> CONFIGURE CHANNEL DEVICE TYPE DISK FORMAT '/oracle/C11/sapbackup/full_%u_%s_%p';
new RMAN configuration parameters:
CONFIGURE CHANNEL DEVICE TYPE DISK FORMAT '/oracle/C11/sapbackup/full_%u_%s_%p';
new RMAN configuration parameters are successfully stored
starting full resync of recovery catalog
full resync complete
Next, you should specify how long you want to retain the backup for. When RMAN takes a backup, it automatically deletes all the old backups that are older than the retention period. In the following example, I’m setting the retention period as 7 days, which will keep the DB backup for a week.
RMAN> CONFIGURE RETENTION POLICY TO RECOVERY WINDOW OF 7 DAYS;
old RMAN configuration parameters:
CONFIGURE RETENTION POLICY TO RECOVERY WINDOW OF 7 DAYS;
new RMAN configuration parameters:
CONFIGURE RETENTION POLICY TO RECOVERY WINDOW OF 7 DAYS;
new RMAN configuration parameters are successfully stored
starting full resync of recovery catalog
full resync complete
Typically we’ll use “BACKUP AS BACKUPSET” to backup a database. So, to take a full backup of the database without the archive logs, do the following.
RMAN> BACKUP AS BACKUPSET DATABASE
To take a full backup of the database with the archive logs, do the following:
RMAN> BACKUP AS BACKUPSET DATABASE PLUS ARCHIVELOG;
You can also take a backup of only a specific table space. The following example takes backup of only PRD01 tablespace.
RMAN> BACKUP AS BACKUPSET TABLESPACE PRD01;
Example output:
RMAN> BACKUP AS BACKUPSET DATABASE PLUS ARCHIVELOG;
Starting backup at 01-DEC-20
current log archived
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=44 device type=DISK
channel ORA_DISK_1: starting archived log backup set
channel ORA_DISK_1: specifying archived log(s) in backup set
input archived log thread=1 sequence=33 RECID=1 STAMP=1057971633
input archived log thread=1 sequence=34 RECID=2 STAMP=1058020317
input archived log thread=1 sequence=35 RECID=3 STAMP=1058021321
channel ORA_DISK_1: starting piece 1 at 01-DEC-20
channel ORA_DISK_1: finished piece 1 at 01-DEC-20
piece handle=/oracle/C11/sapbackup/full_02vh07ub_2_1 tag=TAG20201201T144843 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:03
Finished backup at 01-DEC-20
Starting backup at 01-DEC-20
using channel ORA_DISK_1
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
input datafile file number=00001 name=/oracle/C11/sapdata1/system01.dbf
input datafile file number=00004 name=/oracle/C11/sapdata3/users01.dbf
input datafile file number=00002 name=/oracle/C11/sapdata2/sysaux01.dbf
input datafile file number=00003 name=/oracle/C11/sapdata4/undotbs01.dbf
input datafile file number=00005 name=/oracle/C11/sapdata1/catalogtbs1.dbf
channel ORA_DISK_1: starting piece 1 at 01-DEC-20
channel ORA_DISK_1: finished piece 1 at 01-DEC-20
piece handle=/oracle/C11/sapbackup/full_03vh07ug_3_1 tag=TAG20201201T144846 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:07
Finished backup at 01-DEC-20
Starting backup at 01-DEC-20
current log archived
using channel ORA_DISK_1
channel ORA_DISK_1: starting archived log backup set
channel ORA_DISK_1: specifying archived log(s) in backup set
input archived log thread=1 sequence=36 RECID=4 STAMP=1058021335
channel ORA_DISK_1: starting piece 1 at 01-DEC-20
channel ORA_DISK_1: finished piece 1 at 01-DEC-20
piece handle=/oracle/C11/sapbackup/full_04vh07uo_4_1 tag=TAG20201201T144856 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:01
Finished backup at 01-DEC-20
Starting Control File and SPFILE Autobackup at 01-DEC-20
piece handle=/oracle/C11/19.0.0/dbs/c-365226479-20201201-00 comment=NONE
Finished Control File and SPFILE Autobackup at 01-DEC-20
RMAN>
Check the backup location:
[orac11@ralforavm01 admin]$ ls -l /oracle/C11/sapbackup/
total 782612
-rw-r----- 1 orac11 dba 169256448 Dec 1 14:48 full_02vh07ub_2_1
-rw-r----- 1 orac11 dba 628776960 Dec 1 14:48 full_03vh07ug_3_1
-rw-r----- 1 orac11 dba 204800 Dec 1 14:48 full_04vh07uo_4_1
References
Microsoft
Solution architectures using Azure NetApp Files - Oracle
Solution architectures using Azure NetApp Files – SAP on Azure
SAP deployment on Azure using an Oracle DB - Azure Architecture Center | Microsoft Docs
Oracle:
https://oracle-base.com/articles/19c/oracle-db-19c-installation-on-oracle-linux-8
SAP
2015553 - SAP on Microsoft Azure: Support prerequisites - SAP ONE Support Launchpad
NetApp
Oracle Databases on Microsoft Azure Using Azure NetApp Files | TR-4780 | NetApp
TR-3633: Oracle Databases on NetApp ONTAP | NetApp