Fresh Oracle 11.2.0.2 Grid Infrastructure Installation PRVF-5150 PRVF-5184
Fresh Oracle 11.2.0.2 Grid Infrastructure Installation PRVF-5150 PRVF-5184
Oracle 11gR2 Grid Infrastructure installation version 11.2.0.1 is covered here. Linux installation is covered here. The prerequisites for the Oracle 11gR2 installation are described here. Setting up a VMware clusters for Oracle 11gR2 installation is covered here.
The article will not cover a detail step-by-step installation of 11.2.0.2 GI installation as it is similar to 11.2.0.1. Using runcluvfy confirmed that the prerequisites for Oracle GI and RAC are met. Using the OUI from GI 11.2.0.1 also confirmed that the installation prerequisites checks. For the output of the runcluvfy output see the Annex.
However, during a fresh installation of 11.2.0.2 Oracle GI following the 11.2.0.1 document the following errors were encountered, that is, PRVF-5150 and PRVF-5184 while executing the prerequisites checks.
If I use /dev/oracleasm/DISK* instead of ORCL:DISK* as a search patch I get the following error PRVF-5184.
Apparently there is something with the OUI/cluvfy in regards to Udev that is buggy or not documented.
The Oracle® Database Readme 11g Release 2 (11.2) suggests a bug 10044507.
I skipped the prerequisite checks selecting Ignore All check box and pressing Install button. Following the concepts for the 11.2.0.1 install the Oracle GI 11.2.0.2 installation succeeded.
MOS shed some light as well: PRVF-5449 : Check of Voting Disk location “ORCL:(ORCL:)” failed [ID 1267569.1] See Johan Westerduin comment.
Annex
[oracle@raclinux1 grid]$ ./runcluvfy.sh stage -pre crsinst -n raclinux1,raclinux2 Performing pre-checks for cluster services setup Checking node reachability... Node reachability check passed from node "raclinux1" Checking user equivalence... User equivalence check passed for user "oracle" Checking node connectivity... Checking hosts config file... Verification of the hosts config file successful Node connectivity passed for subnet "10.0.2.0" with node(s) raclinux2,raclinux1 ERROR: PRVF-7617 : Node connectivity between "raclinux1 : 10.0.2.15" and "raclinux2 : 10.0.2.15" failed TCP connectivity check failed for subnet "10.0.2.0" Node connectivity passed for subnet "192.168.56.0" with node(s) raclinux2,raclinux1 TCP connectivity check passed for subnet "192.168.56.0" Node connectivity passed for subnet "10.10.20.0" with node(s) raclinux2,raclinux1 TCP connectivity check passed for subnet "10.10.20.0" Node connectivity passed for subnet "192.168.20.0" with node(s) raclinux2,raclinux1 TCP connectivity check passed for subnet "192.168.20.0" Interfaces found on subnet "10.0.2.0" that are likely candidates for VIP are: raclinux2 eth0:10.0.2.15 raclinux1 eth0:10.0.2.15 Interfaces found on subnet "192.168.56.0" that are likely candidates for a private interconnect are: raclinux2 eth1:192.168.56.102 raclinux1 eth1:192.168.56.101 Node connectivity check failed Checking ASMLib configuration. Check for ASMLib configuration passed. Total memory check passed Available memory check passed Swap space check passed Free disk space check passed for "raclinux2:/tmp" Free disk space check passed for "raclinux1:/tmp" Check for multiple users with UID value 1100 passed User existence check passed for "oracle" Group existence check passed for "oinstall" Group existence check passed for "dba" Membership check for user "oracle" in group "oinstall" [as Primary] passed Membership check for user "oracle" in group "dba" passed Run level check passed Hard limits check passed for "maximum open file descriptors" Soft limits check passed for "maximum open file descriptors" Hard limits check passed for "maximum user processes" Soft limits check passed for "maximum user processes" System architecture check passed Kernel version check passed Kernel parameter check passed for "semmsl" Kernel parameter check passed for "semmns" Kernel parameter check passed for "semopm" Kernel parameter check passed for "semmni" Kernel parameter check passed for "shmmax" Kernel parameter check passed for "shmmni" Kernel parameter check passed for "shmall" Kernel parameter check passed for "file-max" Kernel parameter check passed for "ip_local_port_range" Kernel parameter check passed for "rmem_default" Kernel parameter check passed for "rmem_max" Kernel parameter check passed for "wmem_default" Kernel parameter check passed for "wmem_max" Kernel parameter check passed for "aio-max-nr" Package existence check passed for "make-3.81( x86_64)" Package existence check passed for "binutils-2.17.50.0.6( x86_64)" Package existence check passed for "gcc-4.1.2 (x86_64)( x86_64)" Package existence check passed for "libaio-0.3.106 (x86_64)( x86_64)" Package existence check passed for "glibc-2.5-24 (x86_64)( x86_64)" Package existence check passed for "compat-libstdc++-33-3.2.3 (x86_64)( x86_64)" Package existence check passed for "elfutils-libelf-0.125 (x86_64)( x86_64)" WARNING: PRVF-7584 : Multiple versions of package "elfutils-libelf-devel" found on node raclinux1: elfutils-libelf-devel-0.137-3.el5 (x86_64),elfutils-libelf-devel-0.137-3.el5 (i386) Package existence check passed for "elfutils-libelf-devel-0.125( x86_64)" Package existence check passed for "glibc-common-2.5( x86_64)" Package existence check passed for "glibc-devel-2.5 (x86_64)( x86_64)" Package existence check passed for "glibc-headers-2.5( x86_64)" Package existence check passed for "gcc-c++-4.1.2 (x86_64)( x86_64)" Package existence check passed for "libaio-devel-0.3.106 (x86_64)( x86_64)" Package existence check passed for "libgcc-4.1.2 (x86_64)( x86_64)" Package existence check passed for "libstdc++-4.1.2 (x86_64)( x86_64)" Package existence check passed for "libstdc++-devel-4.1.2 (x86_64)( x86_64)" Package existence check passed for "sysstat-7.0.2( x86_64)" Package existence check passed for "ksh-20060214( x86_64)" Check for multiple users with UID value 0 passed Current group ID check passed Starting Clock synchronization checks using Network Time Protocol(NTP)... NTP Configuration file check started... No NTP Daemons or Services were found to be running Clock synchronization check using Network Time Protocol(NTP) passed Core file name pattern consistency check passed. User "oracle" is not part of "root" group. Check passed Default user file creation mask check passed Checking consistency of file "/etc/resolv.conf" across nodes File "/etc/resolv.conf" does not have both domain and search entries defined domain entry in file "/etc/resolv.conf" is consistent across nodes search entry in file "/etc/resolv.conf" is consistent across nodes The DNS response time for an unreachable node is within acceptable limit on all nodes File "/etc/resolv.conf" is consistent across nodes Time zone consistency check passed Starting check for Huge Pages Existence ... Check for Huge Pages Existence passed Starting check for Hardware Clock synchronization at shutdown ... Check for Hardware Clock synchronization at shutdown passed Pre-check for cluster services setup was unsuccessful. Checks did not pass for the following node(s): raclinux1 : 10.0.2.15 [oracle@raclinux1 grid]$ [oracle@raclinux1 grid]$ ./runcluvfy.sh stage -pre crsinst -n raclinux1,raclinux2 -verbose Performing pre-checks for cluster services setup Checking node reachability... Check: Node reachability from node "raclinux1" Destination Node Reachable? ------------------------------------ ------------------------ raclinux1 yes raclinux2 yes Result: Node reachability check passed from node "raclinux1" Checking user equivalence... Check: User equivalence for user "oracle" Node Name Comment ------------------------------------ ------------------------ raclinux2 passed raclinux1 passed Result: User equivalence check passed for user "oracle" Checking node connectivity... Checking hosts config file... Node Name Status Comment ------------ ------------------------ ------------------------ raclinux2 passed raclinux1 passed Verification of the hosts config file successful Interface information for node "raclinux2" Name IP Address Subnet Gateway Def. Gateway HW Address MTU ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0 10.0.2.15 10.0.2.0 0.0.0.0 10.0.2.2 08:00:27:9C:41:1A 1500 eth1 192.168.56.102 192.168.56.0 0.0.0.0 10.0.2.2 08:00:27:3A:50:B2 1500 eth2 10.10.20.22 10.10.20.0 0.0.0.0 10.0.2.2 08:00:27:CA:35:14 1500 eth3 192.168.20.22 192.168.20.0 0.0.0.0 10.0.2.2 08:00:27:B1:72:31 1500 Interface information for node "raclinux1" Name IP Address Subnet Gateway Def. Gateway HW Address MTU ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0 10.0.2.15 10.0.2.0 0.0.0.0 10.0.2.2 08:00:27:97:73:42 1500 eth1 192.168.56.101 192.168.56.0 0.0.0.0 10.0.2.2 08:00:27:A8:70:2A 1500 eth2 192.168.20.21 192.168.20.0 0.0.0.0 10.0.2.2 08:00:27:BF:C3:12 1500 eth3 10.10.20.21 10.10.20.0 0.0.0.0 10.0.2.2 08:00:27:68:35:9F 1500 Interface information for node "raclinux1" Name IP Address Subnet Gateway Def. Gateway HW Address MTU ------ --------------- --------------- --------------- --------------- ----------------- ------ eth0 10.0.2.15 10.0.2.0 0.0.0.0 10.0.2.2 08:00:27:97:73:42 1500 eth1 192.168.56.101 192.168.56.0 0.0.0.0 10.0.2.2 08:00:27:A8:70:2A 1500 eth2 192.168.20.21 192.168.20.0 0.0.0.0 10.0.2.2 08:00:27:BF:C3:12 1500 eth3 10.10.20.21 10.10.20.0 0.0.0.0 10.0.2.2 08:00:27:68:35:9F 1500 Check: Node connectivity of subnet "10.0.2.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- raclinux2[10.0.2.15] raclinux1[10.0.2.15] yes Result: Node connectivity passed for subnet "10.0.2.0" with node(s) raclinux2,raclinux1 Check: TCP connectivity of subnet "10.0.2.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- raclinux1:10.0.2.15 raclinux2:10.0.2.15 failed ERROR: PRVF-7617 : Node connectivity between "raclinux1 : 10.0.2.15" and "raclinux2 : 10.0.2.15" failed Result: TCP connectivity check failed for subnet "10.0.2.0" Check: Node connectivity of subnet "192.168.56.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- raclinux2[192.168.56.102] raclinux1[192.168.56.101] yes Result: Node connectivity passed for subnet "192.168.56.0" with node(s) raclinux2,raclinux1 Check: TCP connectivity of subnet "192.168.56.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- raclinux1:192.168.56.101 raclinux2:192.168.56.102 passed Result: TCP connectivity check passed for subnet "192.168.56.0" Check: Node connectivity of subnet "10.10.20.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- raclinux2[10.10.20.22] raclinux1[10.10.20.21] yes Result: Node connectivity passed for subnet "10.10.20.0" with node(s) raclinux2,raclinux1 Check: TCP connectivity of subnet "10.10.20.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- raclinux1:10.10.20.21 raclinux2:10.10.20.22 passed Result: TCP connectivity check passed for subnet "10.10.20.0" Check: Node connectivity of subnet "192.168.20.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- raclinux2[192.168.20.22] raclinux1[192.168.20.21] yes Result: Node connectivity passed for subnet "192.168.20.0" with node(s) raclinux2,raclinux1 Check: TCP connectivity of subnet "192.168.20.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- raclinux1:192.168.20.21 raclinux2:192.168.20.22 passed Result: TCP connectivity check passed for subnet "192.168.20.0" Interfaces found on subnet "10.0.2.0" that are likely candidates for VIP are: raclinux2 eth0:10.0.2.15 raclinux1 eth0:10.0.2.15 Interfaces found on subnet "192.168.56.0" that are likely candidates for a private interconnect are: raclinux2 eth1:192.168.56.102 raclinux1 eth1:192.168.56.101 Result: Node connectivity check failed Checking ASMLib configuration. Node Name Comment ------------------------------------ ------------------------ raclinux2 passed raclinux1 passed Result: Check for ASMLib configuration passed. Check: Total memory Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 2.8773GB (3017016.0KB) 1.5GB (1572864.0KB) passed raclinux1 2.8773GB (3017016.0KB) 1.5GB (1572864.0KB) passed Result: Total memory check passed Check: Available memory Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 2.71GB (2841676.0KB) 50MB (51200.0KB) passed raclinux1 2.6299GB (2757680.0KB) 50MB (51200.0KB) passed Result: Available memory check passed Check: Swap space Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 4.8437GB (5079032.0KB) 2.8773GB (3017016.0KB) passed raclinux1 4.8437GB (5079032.0KB) 2.8773GB (3017016.0KB) passed Result: Swap space check passed Check: Free disk space for "raclinux2:/tmp" Path Node Name Mount point Available Required Comment ---------------- ------------ ------------ ------------ ------------ ------------ /tmp raclinux2 / 366.0488GB 1GB passed Result: Free disk space check passed for "raclinux2:/tmp" Check: Free disk space for "raclinux1:/tmp" Path Node Name Mount point Available Required Comment ---------------- ------------ ------------ ------------ ------------ ------------ /tmp raclinux1 / 356.672GB 1GB passed Result: Free disk space check passed for "raclinux1:/tmp" Check: User existence for "oracle" Node Name Status Comment ------------ ------------------------ ------------------------ raclinux2 exists(1100) passed raclinux1 exists(1100) passed Checking for multiple users with UID value 1100 Result: Check for multiple users with UID value 1100 passed Result: User existence check passed for "oracle" Check: Group existence for "oinstall" Node Name Status Comment ------------ ------------------------ ------------------------ raclinux2 exists passed raclinux1 exists passed Result: Group existence check passed for "oinstall" Check: Group existence for "dba" Node Name Status Comment ------------ ------------------------ ------------------------ raclinux2 exists passed raclinux1 exists passed Result: Group existence check passed for "dba" Check: Membership of user "oracle" in group "oinstall" [as Primary] Node Name User Exists Group Exists User in Group Primary Comment ---------------- ------------ ------------ ------------ ------------ ------------ raclinux2 yes yes yes yes passed raclinux1 yes yes yes yes passed Result: Membership check for user "oracle" in group "oinstall" [as Primary] passed Check: Membership of user "oracle" in group "dba" Node Name User Exists Group Exists User in Group Comment ---------------- ------------ ------------ ------------ ---------------- raclinux2 yes yes yes passed raclinux1 yes yes yes passed Result: Membership check for user "oracle" in group "dba" passed Check: Run level Node Name run level Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 5 3,5 passed raclinux1 5 3,5 passed Result: Run level check passed Check: Hard limits for "maximum open file descriptors" Node Name Type Available Required Comment ---------------- ------------ ------------ ------------ ---------------- raclinux2 hard 65536 65536 passed raclinux1 hard 65536 65536 passed Result: Hard limits check passed for "maximum open file descriptors" Check: Soft limits for "maximum open file descriptors" Node Name Type Available Required Comment ---------------- ------------ ------------ ------------ ---------------- raclinux2 soft 65536 1024 passed raclinux1 soft 65536 1024 passed Result: Soft limits check passed for "maximum open file descriptors" Check: Hard limits for "maximum user processes" Node Name Type Available Required Comment ---------------- ------------ ------------ ------------ ---------------- raclinux2 hard 16384 16384 passed raclinux1 hard 16384 16384 passed Result: Hard limits check passed for "maximum user processes" Check: Soft limits for "maximum user processes" Node Name Type Available Required Comment ---------------- ------------ ------------ ------------ ---------------- raclinux2 soft 16384 2047 passed raclinux1 soft 16384 2047 passed Result: Soft limits check passed for "maximum user processes" Check: System architecture Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 x86_64 x86_64 passed raclinux1 x86_64 x86_64 passed Result: System architecture check passed Check: Kernel version Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 2.6.18-194.el5 2.6.18 passed raclinux1 2.6.18-194.el5 2.6.18 passed Result: Kernel version check passed Check: Kernel parameter for "semmsl" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 250 250 passed raclinux1 250 250 passed Result: Kernel parameter check passed for "semmsl" Check: Kernel parameter for "semmns" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 32000 32000 passed raclinux1 32000 32000 passed Result: Kernel parameter check passed for "semmns" Check: Kernel parameter for "semopm" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 100 100 passed raclinux1 100 100 passed Result: Kernel parameter check passed for "semopm" Check: Kernel parameter for "semmni" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 128 128 passed raclinux1 128 128 passed Result: Kernel parameter check passed for "semmni" Check: Kernel parameter for "shmmax" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 1544712192 1544712192 passed raclinux1 1544712192 1544712192 passed Result: Kernel parameter check passed for "shmmax" Check: Kernel parameter for "shmmni" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 4096 4096 passed raclinux1 4096 4096 passed Result: Kernel parameter check passed for "shmmni" Check: Kernel parameter for "shmall" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 2097152 2097152 passed raclinux1 2097152 2097152 passed Result: Kernel parameter check passed for "shmall" Check: Kernel parameter for "file-max" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 6815744 6815744 passed raclinux1 6815744 6815744 passed Result: Kernel parameter check passed for "file-max" Check: Kernel parameter for "ip_local_port_range" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 between 9000 & 65500 between 9000 & 65500 passed raclinux1 between 9000 & 65500 between 9000 & 65500 passed Result: Kernel parameter check passed for "ip_local_port_range" Check: Kernel parameter for "rmem_default" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 262144 262144 passed raclinux1 262144 262144 passed Result: Kernel parameter check passed for "rmem_default" Check: Kernel parameter for "rmem_max" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 4194304 4194304 passed raclinux1 4194304 4194304 passed Result: Kernel parameter check passed for "rmem_max" Check: Kernel parameter for "wmem_default" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 262144 262144 passed raclinux1 262144 262144 passed Result: Kernel parameter check passed for "wmem_default" Check: Kernel parameter for "wmem_max" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 1048586 1048576 passed raclinux1 1048586 1048576 passed Result: Kernel parameter check passed for "wmem_max" Check: Kernel parameter for "aio-max-nr" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 1048576 1048576 passed raclinux1 1048576 1048576 passed Result: Kernel parameter check passed for "aio-max-nr" Check: Package existence for "make-3.81( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 make-3.81-3.el5 make-3.81( x86_64) passed raclinux1 make-3.81-3.el5 make-3.81( x86_64) passed Result: Package existence check passed for "make-3.81( x86_64)" Check: Package existence for "binutils-2.17.50.0.6( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 binutils-2.17.50.0.6-14.el5 binutils-2.17.50.0.6( x86_64) passed raclinux1 binutils-2.17.50.0.6-14.el5 binutils-2.17.50.0.6( x86_64) passed Result: Package existence check passed for "binutils-2.17.50.0.6( x86_64)" Check: Package existence for "gcc-4.1.2 (x86_64)( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 gcc-4.1.2-48.el5 (x86_64) gcc-4.1.2 (x86_64)( x86_64) passed raclinux1 gcc-4.1.2-48.el5 (x86_64) gcc-4.1.2 (x86_64)( x86_64) passed Result: Package existence check passed for "gcc-4.1.2 (x86_64)( x86_64)" Check: Package existence for "libaio-0.3.106 (x86_64)( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 libaio-0.3.106-5 (x86_64) libaio-0.3.106 (x86_64)( x86_64) passed raclinux1 libaio-0.3.106-5 (x86_64) libaio-0.3.106 (x86_64)( x86_64) passed Result: Package existence check passed for "libaio-0.3.106 (x86_64)( x86_64)" Check: Package existence for "glibc-2.5-24 (x86_64)( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 glibc-2.5-49 (x86_64) glibc-2.5-24 (x86_64)( x86_64) passed raclinux1 glibc-2.5-49 (x86_64) glibc-2.5-24 (x86_64)( x86_64) passed Result: Package existence check passed for "glibc-2.5-24 (x86_64)( x86_64)" Check: Package existence for "compat-libstdc++-33-3.2.3 (x86_64)( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 compat-libstdc++-33-3.2.3-61 (x86_64) compat-libstdc++-33-3.2.3 (x86_64)( x86_64) passed raclinux1 compat-libstdc++-33-3.2.3-61 (x86_64) compat-libstdc++-33-3.2.3 (x86_64)( x86_64) passed Result: Package existence check passed for "compat-libstdc++-33-3.2.3 (x86_64)( x86_64)" Check: Package existence for "elfutils-libelf-0.125 (x86_64)( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 elfutils-libelf-0.137-3.el5 (x86_64) elfutils-libelf-0.125 (x86_64)( x86_64) passed raclinux1 elfutils-libelf-0.137-3.el5 (x86_64) elfutils-libelf-0.125 (x86_64)( x86_64) passed Result: Package existence check passed for "elfutils-libelf-0.125 (x86_64)( x86_64)" Check: Package existence for "elfutils-libelf-devel-0.125( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 elfutils-libelf-devel-0.137-3.el5 elfutils-libelf-devel-0.125( x86_64) passed raclinux1 elfutils-libelf-devel-0.137-3.el5 elfutils-libelf-devel-0.125( x86_64) passed WARNING: PRVF-7584 : Multiple versions of package "elfutils-libelf-devel" found on node raclinux1: elfutils-libelf-devel-0.137-3.el5 (x86_64),elfutils-libelf-devel-0.137-3.el5 (i386) Result: Package existence check passed for "elfutils-libelf-devel-0.125( x86_64)" Check: Package existence for "glibc-common-2.5( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 glibc-common-2.5-49 glibc-common-2.5( x86_64) passed raclinux1 glibc-common-2.5-49 glibc-common-2.5( x86_64) passed Result: Package existence check passed for "glibc-common-2.5( x86_64)" Check: Package existence for "glibc-devel-2.5 (x86_64)( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 glibc-devel-2.5-49 (x86_64) glibc-devel-2.5 (x86_64)( x86_64) passed raclinux1 glibc-devel-2.5-49 (x86_64) glibc-devel-2.5 (x86_64)( x86_64) passed Result: Package existence check passed for "glibc-devel-2.5 (x86_64)( x86_64)" Check: Package existence for "glibc-headers-2.5( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 glibc-headers-2.5-49 glibc-headers-2.5( x86_64) passed raclinux1 glibc-headers-2.5-49 glibc-headers-2.5( x86_64) passed Result: Package existence check passed for "glibc-headers-2.5( x86_64)" Check: Package existence for "gcc-c++-4.1.2 (x86_64)( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 gcc-c++-4.1.2-48.el5 (x86_64) gcc-c++-4.1.2 (x86_64)( x86_64) passed raclinux1 gcc-c++-4.1.2-48.el5 (x86_64) gcc-c++-4.1.2 (x86_64)( x86_64) passed Result: Package existence check passed for "gcc-c++-4.1.2 (x86_64)( x86_64)" Check: Package existence for "libaio-devel-0.3.106 (x86_64)( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 libaio-devel-0.3.106-5 (x86_64) libaio-devel-0.3.106 (x86_64)( x86_64) passed raclinux1 libaio-devel-0.3.106-5 (x86_64) libaio-devel-0.3.106 (x86_64)( x86_64) passed Result: Package existence check passed for "libaio-devel-0.3.106 (x86_64)( x86_64)" Check: Package existence for "libgcc-4.1.2 (x86_64)( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 libgcc-4.1.2-48.el5 (x86_64) libgcc-4.1.2 (x86_64)( x86_64) passed raclinux1 libgcc-4.1.2-48.el5 (x86_64) libgcc-4.1.2 (x86_64)( x86_64) passed Result: Package existence check passed for "libgcc-4.1.2 (x86_64)( x86_64)" Check: Package existence for "libstdc++-4.1.2 (x86_64)( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 libstdc++-4.1.2-48.el5 (x86_64) libstdc++-4.1.2 (x86_64)( x86_64) passed raclinux1 libstdc++-4.1.2-48.el5 (x86_64) libstdc++-4.1.2 (x86_64)( x86_64) passed Result: Package existence check passed for "libstdc++-4.1.2 (x86_64)( x86_64)" Check: Package existence for "libstdc++-devel-4.1.2 (x86_64)( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 libstdc++-devel-4.1.2-48.el5 (x86_64) libstdc++-devel-4.1.2 (x86_64)( x86_64) passed raclinux1 libstdc++-devel-4.1.2-48.el5 (x86_64) libstdc++-devel-4.1.2 (x86_64)( x86_64) passed Result: Package existence check passed for "libstdc++-devel-4.1.2 (x86_64)( x86_64)" Check: Package existence for "sysstat-7.0.2( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 sysstat-7.0.2-3.el5 sysstat-7.0.2( x86_64) passed raclinux1 sysstat-7.0.2-3.el5 sysstat-7.0.2( x86_64) passed Result: Package existence check passed for "sysstat-7.0.2( x86_64)" Check: Package existence for "ksh-20060214( x86_64)" Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 ksh-20100202-1.el5 ksh-20060214( x86_64) passed raclinux1 ksh-20100202-1.el5 ksh-20060214( x86_64) passed Result: Package existence check passed for "ksh-20060214( x86_64)" Checking for multiple users with UID value 0 Result: Check for multiple users with UID value 0 passed Check: Current group ID Result: Current group ID check passed Starting Clock synchronization checks using Network Time Protocol(NTP)... NTP Configuration file check started... Network Time Protocol(NTP) configuration file not found on any of the nodes. Oracle Cluster Time Synchronization Service(CTSS) can be used instead of NTP for time synchronization on the cluster nodes No NTP Daemons or Services were found to be running Result: Clock synchronization check using Network Time Protocol(NTP) passed Checking Core file name pattern consistency... Core file name pattern consistency check passed. Checking to make sure user "oracle" is not in "root" group Node Name Status Comment ------------ ------------------------ ------------------------ raclinux2 does not exist passed raclinux1 does not exist passed Result: User "oracle" is not part of "root" group. Check passed Check default user file creation mask Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- raclinux2 0022 0022 passed raclinux1 0022 0022 passed Result: Default user file creation mask check passed Checking consistency of file "/etc/resolv.conf" across nodes Checking the file "/etc/resolv.conf" to make sure only one of domain and search entries is defined File "/etc/resolv.conf" does not have both domain and search entries defined Checking if domain entry in file "/etc/resolv.conf" is consistent across the nodes... domain entry in file "/etc/resolv.conf" is consistent across nodes Checking if search entry in file "/etc/resolv.conf" is consistent across the nodes... search entry in file "/etc/resolv.conf" is consistent across nodes Checking DNS response time for an unreachable node Node Name Status ------------------------------------ ------------------------ raclinux2 passed raclinux1 passed The DNS response time for an unreachable node is within acceptable limit on all nodes File "/etc/resolv.conf" is consistent across nodes Check: Time zone consistency Result: Time zone consistency check passed Starting check for Huge Pages Existence ... Check for Huge Pages Existence passed Starting check for Hardware Clock synchronization at shutdown ... Check for Hardware Clock synchronization at shutdown passed Pre-check for cluster services setup was unsuccessful. Checks did not pass for the following node(s): raclinux1 : 10.0.2.15 [oracle@raclinux1 grid]$
Oracle RAC and Oracle ONE NODE RAC 11.2.0 and Transparent Application Failover (TAF)
Oracle RAC and Oracle ONE NODE RAC 11.2.0 and Transparent Application Failover (TAF)
In this article we will look at Transparent Application Failover (TAF) setup with RAC. The TAF concept is tested with Oracle RAC and will use it to test with Oracle ONE NODE RAC. We will setup a tnsnames.ora entry for TAF and verify the TAF failover while migrating with Omotion the instance to another node. Managing Oracle ONE NODE RAC was a subject of an earlier article. The tnanames.ora entry RUPTAF is in the ANNEX. We will run a query as shown in the ANNEX in testtaf.sql and monitor the fail over while the ONE NODE RAC instance is moved to another node of the cluster. Before and after the query execution we will check the failover_method , failover_type and failed_over from v$session. We will connect to RUP database (instance RUP_1 on raclinux2) and will use Omotion to move it to raclinux1 node.
- How is defined RUPTAF
RUPTAF= (DESCRIPTION = (ENABLE=BROKEN) (LOAD_BALANCE = OFF) (FAILOVER = ON) (ADDRESS = (PROTOCOL = TCP)(HOST = rac-scan)(PORT = 1521)) (ADDRESS = (PROTOCOL = TCP)(HOST = raclinux1-vip)(PORT = 1521)) (ADDRESS = (PROTOCOL = TCP)(HOST = raclinux2-vip)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = RUP) (FAILOVER_MODE = (TYPE=SELECT) (METHOD=BASIC) (BACKUP=RUP) ) ) )
2. Setup RAC ONE NODE
We looked at managing Oracle ONE NODE RAC here. We will setup Oracle RAC ONE NODE database and will verify it.
[oracle@raclinux2 ~]$ raconeinit Candidate Databases on this cluster: # Database RAC One Node Fix Required === ======== ============ ============ [1] RAC0 NO N/A [2] RONE NO N/A [3] RUP NO N/A Enter the database to initialize [1]: 3 Database RUP is now running on server raclinux2 Candidate servers that may be used for this DB: raclinux1 Enter the names of additional candidate servers where this DB may run (space delimited): raclinux1 Please wait, this may take a few minutes to finish....... Database configuration modified. [oracle@raclinux2 ~]$ [oracle@raclinux2 ~]$ raconestatus RAC One Node databases on this cluster: Database UP Fix Required Current Server Candidate Server Names ======== == ============ ============================== ======================================== RUP Y N raclinux2 raclinux2 raclinux1 Available Free Servers: [oracle@raclinux2 ~]$
3. Start a SQL statement and migrate the database with Omotion as in step 4
[oracle@raclinux2 admin]$ sqlplus system/sys1@ruptaf SQL*Plus: Release 11.2.0.1.0 Production on Thu Sep 30 13:47:32 2010 Copyright (c) 1982, 2009, Oracle. All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP, Data Mining and Real Application Testing options SQL> @/u03/testtaf.sql SID SERIAL# FAILOVER_TYPE FAILOVER_METHOD FAILED_OVER ---- --------- ------------- --------------- ----------- 49 685 SELECT BASIC NO INSTANCE_NAME ---------------- RUP_1 COUNT(*) ---------- 623183 SID SERIAL# FAILOVER_TYPE FAILOVER_METHOD FAILED_OVER ---- --------- ------------- --------------- ----------- 49 2 SELECT BASIC YES INSTANCE_NAME ---------------- RUP_2 COUNT(*) ---------- 623183 SQL>
4. Migrate with Omotion the RUP database while the SQL query in step 3 is running
[oracle@raclinux2 ~]$ Omotion -v RAC One Node databases on this cluster: # Database Server Fix Required === ======== ============================== ============ [1] RUP raclinux2 N Enter number of the database to migrate [1]: 1 Specify maximum time in minutes for migration to complete (max 30) [30]: RUP Database is administrator managed . RUP database is running in RUP server pool. Current Running instance: RUP_1 Current Active Server : raclinux2 Available Target Server(s) : # Server Available === ================== ========= [1] raclinux1 Y Enter number of the target node [1]: 1 Omotion Started... Starting target instance on raclinux1... Migrating sessions... Stopping source instance on raclinux2... Omotion Completed... === Current Status === Database RUP is running on node raclinux1 [oracle@raclinux2 ~]$ [oracle@raclinux2 ~]$ Omotion RAC One Node databases on this cluster: # Database Server Fix Required === ======== ============================== ============ [1] RUP raclinux1 N Enter number of the database to migrate [1]: 1 Specify maximum time in minutes for migration to complete (max 30) [30]: 30 Available Target Server(s) : # Server Available === ================== ========= [1] raclinux2 Y Enter number of the target node [1]: 1 Omotion Started... Starting target instance on raclinux2... Migrating sessions... Stopping source instance on raclinux1... Omotion Completed... === Current Status === Database RUP is running on node raclinux2 [oracle@raclinux2 ~]$
Summary
We configured a TAF tnsnames.ora entry and used Omotion to move the RUP database to another node of the cluster. Using the defined RUPTAF connect string we confirmed that the session fails over to the second instance started by Omotion using the defined failover attributes.
Annex
The RUPTAF tnsnames.ora entry and the test script follows.
# in tnsnames.ora RUPTAF= (DESCRIPTION = (ENABLE=BROKEN) (LOAD_BALANCE = OFF) (FAILOVER = ON) (ADDRESS = (PROTOCOL = TCP)(HOST = rac-scan)(PORT = 1521)) (ADDRESS = (PROTOCOL = TCP)(HOST = raclinux1-vip)(PORT = 1521)) (ADDRESS = (PROTOCOL = TCP)(HOST = raclinux2-vip)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = RUP) (FAILOVER_MODE = (TYPE=SELECT) (METHOD=BASIC) (BACKUP=RUP) ) ) ) [oracle@raclinux2 admin]$ cat /u03/testtaf.sql col sid format 999 col serial# format 99999999 col failover_type format a13 col failover_method format a15 col failed_over format a11 select sid, serial#, failover_type, failover_method, failed_over from v$session where username ='SYSTEM'; select instance_name from v$instance; select count(*) from ( select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source ); col sid format 999 col serial# format 99999999 col failover_type format a13 col failover_method format a15 col failed_over format a11 select sid, serial#, failover_type, failover_method, failed_over from v$session where username ='SYSTEM'; select instance_name from v$instance; select count(*) from ( select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source union select * from dba_source ); [oracle@raclinux2 admin]$
Multiple OCRs and vote disks on ASM in Oracle 11gR2
Multiple OCRs and vote disks on ASM in Oracle 11gR2
Edited: 3-August-2012
Oracle Clusterware used to provide installation option for multiple locations for the OCR and vote disks prior to 11gR2. In Oracle 11gR2 ASM disk groups are used to store the OCR and vote disks. While we can use the mirroring capabilities of ASM disk groups we still can manually add additional OCR disks in Oracle 11gR2 on ASM disk groups to provide redundancy and High Availability. Here in this post we will look at adding additional OCR disks to new ASM disks groups and moving the vote disk to a disk group with high redundancy.
Initially we have Oracle GI installed on a disk group DATA with external redundancy. We created a disk group dgdup1 high redundancy with 5 disks and a disk group dgdup2 with external redundancy.
Make sure that Oracle GI is running
[root@raclinux2 ~]# cd /u01/app/11.2.0.2/grid [root@raclinux2 grid]# cd bin [root@raclinux2 bin]# ./crsctl check cluster -all ************************************************************** raclinux1: CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online ************************************************************** raclinux2: CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online ************************************************************** [root@raclinux2 bin]#
Vote disks
Let’s move the vote disk to a high redundancy group.
[root@raclinux2 bin]# [root@raclinux2 bin]# ./crsctl query css votedisk ## STATE File Universal Id File Name Disk group -- ----- ----------------- --------- --------- 1. ONLINE 7b7f9d9ae9484f2cbf7e62c164aa221b (/dev/oracleasm/disks/DISK1) [DATA] Located 1 voting disk(s). [root@raclinux2 bin]# ./crsctl replace votedisk +dgdup1 Successful addition of voting disk 786515d3e2cc4fa7bfc5fd0b6e87cfeb. Successful addition of voting disk afe5c0e3da484f62bfcc36b1c0eb4aa4. Successful addition of voting disk 1ec83edcf92b4f55bfdc9711c49e0ddd. Successful addition of voting disk 97278bb3f36f4f84bf5965682efe87d2. Successful addition of voting disk 46021fd53e394fb8bf0f476f1fa210dc. Successful deletion of voting disk 7b7f9d9ae9484f2cbf7e62c164aa221b. Successfully replaced voting disk group with +dgdup1. CRS-4266: Voting file(s) successfully replaced [root@raclinux2 bin]# ./crsctl query css votedisk ## STATE File Universal Id File Name Disk group -- ----- ----------------- --------- --------- 1. ONLINE 786515d3e2cc4fa7bfc5fd0b6e87cfeb (/dev/oracleasm/disks/DISK11) [DGDUP1] 2. ONLINE afe5c0e3da484f62bfcc36b1c0eb4aa4 (/dev/oracleasm/disks/DISK12) [DGDUP1] 3. ONLINE 1ec83edcf92b4f55bfdc9711c49e0ddd (/dev/oracleasm/disks/DISK13) [DGDUP1] 4. ONLINE 97278bb3f36f4f84bf5965682efe87d2 (/dev/oracleasm/disks/DISK14) [DGDUP1] 5. ONLINE 46021fd53e394fb8bf0f476f1fa210dc (/dev/oracleasm/disks/DISK15) [DGDUP1] Located 5 voting disk(s). [root@raclinux2 bin]# [root@raclinux2 bin]# ./crsctl add css votedisk +data CRS-4671: This command is not supported for ASM diskgroups. CRS-4000: Command Add failed, or completed with errors. [root@raclinux2 bin]# ./crsctl add css votedisk /u03/vote_acfs.dsk CRS-4258: Addition and deletion of voting files are not allowed because the voting files are on ASM [root@raclinux2 bin]#
As seen from above we can move a vote disk to a high redundancy disk group dgdup1 and benefit from the High Redundancy of the group. We cannot use the ‘crsctl add css votedisk’ to add a vote disk on ASM disk group or ACFS file system.
OCR disks
While we can benefit from the redundancy that the disk group storing the OCR provides we still can manually add OCR to different disk groups.
[root@raclinux2 bin]# cat /etc/oracle/ocr.loc #Device/file getting replaced by device +dgdup2 ocrconfig_loc=+DATA local_only=false [root@raclinux2 bin]# ./ocrcheck Status of Oracle Cluster Registry is as follows : Version : 3 Total space (kbytes) : 262120 Used space (kbytes) : 3380 Available space (kbytes) : 258740 ID : 1332773503 Device/File Name : +DATA Device/File integrity check succeeded Device/File not configured Device/File not configured Device/File not configured Device/File not configured Cluster registry integrity check succeeded Logical corruption check succeeded [root@raclinux2 bin]# [root@raclinux2 bin]# ./ocrconfig -h Name: ocrconfig - Configuration tool for Oracle Cluster/Local Registry. Synopsis: ocrconfig [option] option: [-local] -export - Export OCR/OLR contents to a file [-local] -import - Import OCR/OLR contents from a file [-local] -upgrade [ []] - Upgrade OCR from previous version -downgrade [-version ] - Downgrade OCR to the specified version [-local] -backuploc - Configure OCR/OLR backup location [-local] -showbackup [auto|manual] - Show OCR/OLR backup information [-local] -manualbackup - Perform OCR/OLR backup [-local] -restore - Restore OCR/OLR from physical backup -replace -replacement - Replace a OCR device/file <filename1> with <filename2> -add - Add a new OCR device/file -delete - Remove a OCR device/file -overwrite - Overwrite OCR configuration on disk -repair -add | -delete | -replace -replacement - Repair OCR configuration on the local node -help - Print out this help information Note: * A log file will be created in $ORACLE_HOME/log//client/ocrconfig_.log. Please ensure you have file creation privileges in the above directory before running this tool. * Only -local -showbackup [manual] is supported. * Use option '-local' to indicate that the operation is to be performed on the Oracle Local Registry. [root@raclinux2 bin]# [root@raclinux2 bin]# [root@raclinux2 bin]# ./ocrconfig -add +dgdup2 [root@raclinux2 bin]# ./ocrconfig -add +dgdup1 [root@raclinux2 bin]# ./ocrcheck Status of Oracle Cluster Registry is as follows : Version : 3 Total space (kbytes) : 262120 Used space (kbytes) : 3380 Available space (kbytes) : 258740 ID : 1332773503 Device/File Name : +DATA Device/File integrity check succeeded Device/File Name : +dgdup2 Device/File integrity check succeeded Device/File Name : +dgdup1 Device/File integrity check succeeded Device/File not configured Device/File not configured Cluster registry integrity check succeeded Logical corruption check succeeded [root@raclinux2 bin]# [root@raclinux2 bin]# cat /etc/oracle/ocr.loc #Device/file getting replaced by device +dgdup1 ocrconfig_loc=+DATA ocrmirrorconfig_loc=+dgdup2 ocrconfig_loc3=+dgdup1 local_only=false[root@raclinux2 bin]#
Summary:
While we benefit from the redundancy a disk group provides we can add multiple OCR on a different disk groups. We can move the VOTE disk to a disk group with high redundancy.
To be precise vote disk in ASM has the following specifics.
No file in ASM spans a disk group including a vote file corresponding to the vote disks. Difference is that vote file is not mirrored and stripped like any other files (data, redo, control, OCR etc…) and has a fixed assignment to a failure group in a disk. Vote disk can be accessed even if ASM is not up whereas another file cannot be accessed if ASM is not up and running.
You cannot mix vote files on ASM and vote files on non-ASM. Instead you can dedicate either one ASM DG for all you cluster vote disk files or you can use non-ASM but never both except for initial migration to 11.2 when previous versions were on non-ASM storage.
For vote files you need a special number of failure groups in a disk groups as indicated below
- For External redundancy DG you need 1 failure group for one Vote disk file
- For Normal redundancy DG you need 3 failure groups for 3 vote disk file
- For High redundancy DG you need 5 failure groups for 5 vote disk files
-
OraNA
-
Archives
- December 2012 (2)
- November 2012 (8)
- October 2012 (6)
- July 2012 (8)
- June 2012 (4)
- February 2012 (2)
- January 2012 (2)
- November 2011 (6)
- October 2011 (3)
- September 2011 (1)
- August 2011 (2)
- July 2011 (9)
-
Categories
-
RSS
Entries RSS
Comments RSS
