Guenadi N Jilevski's Oracle BLOG

OUI 11gR2 while installing grid hangs at 65% on Linux OEL 5.X

OUI 11gR2 while installing grid hangs at 65% on Linux OEL 5.X

While installing Oracle 11gR2 grid OUI hangs at 65% completed and shows that remote copy is in progress. The output of

./runcluvfy.sh stage -pre crsinst -n raclinux1,raclinux2 -verbose

prior to running the 11gR2 grid install is as displayed in Appendix A. As we can see Oracle cluvfy utility did not find out anything suggesting that OUI will not proceed as expected. Examining the logs revealed the following error.

After disabling the firewall and making sure that SELINUX is disabled as show below I rebooted raclinux1 and raclinux2 nodes of the cluster.

I started a fresh Oracle grid installation. This time the installation completed successfully.

Appendix A

Performing pre-checks for cluster services setup

Checking node reachability…

Check: Node reachability from node “raclinux1”

Destination Node Reachable?

———————————— ————————

raclinux1 yes

raclinux2 yes

Result: Node reachability check passed from node “raclinux1”

Checking user equivalence…

Check: User equivalence for user “oracle”

Node Name Comment

———————————— ————————

raclinux2 passed

raclinux1 passed

Result: User equivalence check passed for user “oracle”

Checking node connectivity…

Checking hosts config file…

Node Name Status Comment

———— ———————— ————————

raclinux2 passed

raclinux1 passed

Verification of the hosts config file successful

Interface information for node “raclinux2”

Name IP Address Subnet Gateway Def. Gateway HW Address MTU

—— ————— ————— ————— ————— —————– ——

virbr0 192.168.122.1 192.168.122.0 0.0.0.0 UNKNOWN 00:00:00:00:00:00 1500

bond0 192.168.2.132 192.168.2.0 0.0.0.0 UNKNOWN 00:0C:29:85:B0:7E 1500

bond1 10.10.4.132 10.10.4.0 0.0.0.0 UNKNOWN 00:0C:29:85:B0:88 1500

Interface information for node “raclinux1”

Name IP Address Subnet Gateway Def. Gateway HW Address MTU

—— ————— ————— ————— ————— —————– ——

virbr0 192.168.122.1 192.168.122.0 0.0.0.0 UNKNOWN 00:00:00:00:00:00 1500

bond0 192.168.2.131 192.168.2.0 0.0.0.0 UNKNOWN 00:0C:29:38:C8:4B 1500

bond1 10.10.4.131 10.10.4.0 0.0.0.0 UNKNOWN 00:0C:29:38:C8:55 1500

Check: Node connectivity of subnet “192.168.122.0”

Source Destination Connected?

—————————— —————————— —————-

raclinux2:virbr0 raclinux1:virbr0 yes

Result: Node connectivity passed for subnet “192.168.122.0” with node(s) raclinux2,raclinux1

Check: TCP connectivity of subnet “192.168.122.0”

Result: TCP connectivity check failed for subnet “192.168.122.0”

Check: Node connectivity of subnet “192.168.2.0”

Source Destination Connected?

—————————— —————————— —————-

raclinux2:bond0 raclinux1:bond0 yes

Result: Node connectivity passed for subnet “192.168.2.0” with node(s) raclinux2,raclinux1

Check: TCP connectivity of subnet “192.168.2.0”

Source Destination Connected?

—————————— —————————— —————-

raclinux1:192.168.2.131 raclinux2:192.168.2.132 failed

Result: TCP connectivity check failed for subnet “192.168.2.0”

Check: Node connectivity of subnet “10.10.4.0”

Source Destination Connected?

—————————— —————————— —————-

raclinux2:bond1 raclinux1:bond1 yes

Result: Node connectivity passed for subnet “10.10.4.0” with node(s) raclinux2,raclinux1

Check: TCP connectivity of subnet “10.10.4.0”

Source Destination Connected?

—————————— —————————— —————-

raclinux1:10.10.4.131 raclinux2:10.10.4.132 failed

Result: TCP connectivity check failed for subnet “10.10.4.0”

Interfaces found on subnet “192.168.122.0” that are likely candidates for a private interconnect are:

raclinux2 virbr0:192.168.122.1

raclinux1 virbr0:192.168.122.1

Interfaces found on subnet “192.168.2.0” that are likely candidates for a private interconnect are:

raclinux2 bond0:192.168.2.132

raclinux1 bond0:192.168.2.131

Interfaces found on subnet “10.10.4.0” that are likely candidates for a private interconnect are:

raclinux2 bond1:10.10.4.132

raclinux1 bond1:10.10.4.131

WARNING:

Could not find a suitable set of interfaces for VIPs

Result: Node connectivity check passed

Check: Total memory

Node Name Available Required Comment

———— ———————— ———————— ———-

raclinux2 1.98GB (2075488.0KB) 1.5GB (1572864.0KB) passed

raclinux1 1.98GB (2075488.0KB) 1.5GB (1572864.0KB) passed

Result: Total memory check passed

Check: Available memory

Node Name Available Required Comment

———— ———————— ———————— ———-

raclinux2 1.79GB (1880648.0KB) 50MB (51200.0KB) passed

raclinux1 1.77GB (1856332.0KB) 50MB (51200.0KB) passed

Result: Available memory check passed

Check: Swap space

Node Name Available Required Comment

———— ———————— ———————— ———-

raclinux2 3.94GB (4128760.0KB) 2.97GB (3113232.0KB) passed

raclinux1 3.94GB (4128760.0KB) 2.97GB (3113232.0KB) passed

Result: Swap space check passed

Check: Free disk space for “raclinux2:/tmp”

Path Node Name Mount point Available Required Comment

—————- ———— ———— ———— ———— ————

/tmp raclinux2 / 26.18GB 1GB passed

Result: Free disk space check passed for “raclinux2:/tmp”

Check: Free disk space for “raclinux1:/tmp”

Path Node Name Mount point Available Required Comment

—————- ———— ———— ———— ———— ————

/tmp raclinux1 / 11.33GB 1GB passed

Result: Free disk space check passed for “raclinux1:/tmp”

Check: User existence for “oracle”

Node Name Status Comment

———— ———————— ————————

raclinux2 exists passed

raclinux1 exists passed

Result: User existence check passed for “oracle”

Check: Group existence for “oinstall”

Node Name Status Comment

———— ———————— ————————

raclinux2 exists passed

raclinux1 exists passed

Result: Group existence check passed for “oinstall”

Check: Group existence for “dba”

Node Name Status Comment

———— ———————— ————————

raclinux2 exists passed

raclinux1 exists passed

Result: Group existence check passed for “dba”

Check: Membership of user “oracle” in group “oinstall” [as Primary]

Node Name User Exists Group Exists User in Group Primary Comment

—————- ———— ———— ———— ———— ————

raclinux2 yes yes yes yes passed

raclinux1 yes yes yes yes passed

Result: Membership check for user “oracle” in group “oinstall” [as Primary] passed

Check: Membership of user “oracle” in group “dba”

Node Name User Exists Group Exists User in Group Comment

—————- ———— ———— ———— —————-

raclinux2 yes yes yes passed

raclinux1 yes yes yes passed

Result: Membership check for user “oracle” in group “dba” passed

Check: Run level

Node Name run level Required Comment

———— ———————— ———————— ———-

raclinux2 5 3,5 passed

raclinux1 5 3,5 passed

Result: Run level check passed

Check: Hard limits for “maximum open file descriptors”

Node Name Type Available Required Comment

—————- ———— ———— ———— —————-

raclinux2 hard 131072 65536 passed

raclinux1 hard 131072 65536 passed

Result: Hard limits check passed for “maximum open file descriptors”

Check: Soft limits for “maximum open file descriptors”

Node Name Type Available Required Comment

—————- ———— ———— ———— —————-

raclinux2 soft 131072 1024 passed

raclinux1 soft 131072 1024 passed

Result: Soft limits check passed for “maximum open file descriptors”

Check: Hard limits for “maximum user processes”

Node Name Type Available Required Comment

—————- ———— ———— ———— —————-

raclinux2 hard 131072 16384 passed

raclinux1 hard 131072 16384 passed

Result: Hard limits check passed for “maximum user processes”

Check: Soft limits for “maximum user processes”

Node Name Type Available Required Comment

—————- ———— ———— ———— —————-

raclinux2 soft 131072 2047 passed

raclinux1 soft 131072 2047 passed

Result: Soft limits check passed for “maximum user processes”

Check: System architecture

Node Name Available Required Comment

———— ———————— ———————— ———-

raclinux2 i686 x86 passed

raclinux1 i686 x86 passed

Result: System architecture check passed

Check: Kernel version

Node Name Available Required Comment

———— ———————— ———————— ———-

raclinux2 2.6.18-194.el5PAE 2.6.18 passed

raclinux1 2.6.18-194.el5PAE 2.6.18 passed

Result: Kernel version check passed

Check: Kernel parameter for “semmsl”

Node Name Configured Required Comment

———— ———————— ———————— ———-

raclinux2 250 250 passed

raclinux1 250 250 passed

Result: Kernel parameter check passed for “semmsl”

Check: Kernel parameter for “semmns”

Node Name Configured Required Comment

———— ———————— ———————— ———-

raclinux2 32000 32000 passed

raclinux1 32000 32000 passed

Result: Kernel parameter check passed for “semmns”

Check: Kernel parameter for “semopm”

Node Name Configured Required Comment

———— ———————— ———————— ———-

raclinux2 100 100 passed

raclinux1 100 100 passed

Result: Kernel parameter check passed for “semopm”

Check: Kernel parameter for “semmni”

Node Name Configured Required Comment

———— ———————— ———————— ———-

raclinux2 128 128 passed

raclinux1 128 128 passed

Result: Kernel parameter check passed for “semmni”

Check: Kernel parameter for “shmmax”

Node Name Configured Required Comment

———— ———————— ———————— ———-

raclinux2 536870912 536870912 passed

raclinux1 536870912 536870912 passed

Result: Kernel parameter check passed for “shmmax”

Check: Kernel parameter for “shmmni”

Node Name Configured Required Comment

———— ———————— ———————— ———-

raclinux2 4096 4096 passed

raclinux1 4096 4096 passed

Result: Kernel parameter check passed for “shmmni”

Check: Kernel parameter for “shmall”

Node Name Configured Required Comment

———— ———————— ———————— ———-

raclinux2 2097152 2097152 passed

raclinux1 2097152 2097152 passed

Result: Kernel parameter check passed for “shmall”

Check: Kernel parameter for “file-max”

Node Name Configured Required Comment

———— ———————— ———————— ———-

raclinux2 6815744 6815744 passed

raclinux1 6815744 6815744 passed

Result: Kernel parameter check passed for “file-max”

Check: Kernel parameter for “ip_local_port_range”

Node Name Configured Required Comment

———— ———————— ———————— ———-

raclinux2 between 9000 & 65500 between 9000 & 65500 passed

raclinux1 between 9000 & 65500 between 9000 & 65500 passed

Result: Kernel parameter check passed for “ip_local_port_range”

Check: Kernel parameter for “rmem_default”

Node Name Configured Required Comment

———— ———————— ———————— ———-

raclinux2 262144 262144 passed

raclinux1 262144 262144 passed

Result: Kernel parameter check passed for “rmem_default”

Check: Kernel parameter for “rmem_max”

Node Name Configured Required Comment

———— ———————— ———————— ———-

raclinux2 4194304 4194304 passed

raclinux1 4194304 4194304 passed

Result: Kernel parameter check passed for “rmem_max”

Check: Kernel parameter for “wmem_default”

Node Name Configured Required Comment

———— ———————— ———————— ———-

raclinux2 262144 262144 passed

raclinux1 262144 262144 passed

Result: Kernel parameter check passed for “wmem_default”

Check: Kernel parameter for “wmem_max”

Node Name Configured Required Comment

———— ———————— ———————— ———-

raclinux2 1048586 1048576 passed

raclinux1 1048586 1048576 passed

Result: Kernel parameter check passed for “wmem_max”

Check: Kernel parameter for “aio-max-nr”

Node Name Configured Required Comment

———— ———————— ———————— ———-

raclinux2 1048576 1048576 passed

raclinux1 1048576 1048576 passed

Result: Kernel parameter check passed for “aio-max-nr”

Check: Package existence for “make-3.81”

Node Name Available Required Comment

———— ———————— ———————— ———-

raclinux2 make-3.81-3.el5 make-3.81 passed

raclinux1 make-3.81-3.el5 make-3.81 passed

Result: Package existence check passed for “make-3.81”

Check: Package existence for “binutils-2.17.50.0.6”

Node Name Available Required Comment

———— ———————— ———————— ———-

raclinux2 binutils-2.17.50.0.6-14.el5 binutils-2.17.50.0.6 passed

raclinux1 binutils-2.17.50.0.6-14.el5 binutils-2.17.50.0.6 passed

Result: Package existence check passed for “binutils-2.17.50.0.6”

Check: Package existence for “gcc-4.1.2”

Node Name Available Required Comment

———— ———————— ———————— ———-

raclinux2 gcc-4.1.2-48.el5 gcc-4.1.2 passed

raclinux1 gcc-4.1.2-48.el5 gcc-4.1.2 passed

Result: Package existence check passed for “gcc-4.1.2”

Check: Package existence for “gcc-c++-4.1.2”

Node Name Available Required Comment

———— ———————— ———————— ———-

raclinux2 gcc-c++-4.1.2-48.el5 gcc-c++-4.1.2 passed

raclinux1 gcc-c++-4.1.2-48.el5 gcc-c++-4.1.2 passed

Result: Package existence check passed for “gcc-c++-4.1.2”

Check: Package existence for “libgomp-4.1.2”

Node Name Available Required Comment

———— ———————— ———————— ———-

raclinux2 libgomp-4.4.0-6.el5 libgomp-4.1.2 passed

raclinux1 libgomp-4.4.0-6.el5 libgomp-4.1.2 passed

Result: Package existence check passed for “libgomp-4.1.2”

Check: Package existence for “libaio-0.3.106”

Node Name Available Required Comment

———— ———————— ———————— ———-

raclinux2 libaio-0.3.106-5 libaio-0.3.106 passed

raclinux1 libaio-0.3.106-5 libaio-0.3.106 passed

Result: Package existence check passed for “libaio-0.3.106”

Check: Package existence for “glibc-2.5-24”

Node Name Available Required Comment

———— ———————— ———————— ———-

raclinux2 glibc-2.5-49 glibc-2.5-24 passed

raclinux1 glibc-2.5-49 glibc-2.5-24 passed

Result: Package existence check passed for “glibc-2.5-24”

Check: Package existence for “compat-libstdc++-33-3.2.3”

Node Name Available Required Comment

———— ———————— ———————— ———-

raclinux2 compat-libstdc++-33-3.2.3-61 compat-libstdc++-33-3.2.3 passed

raclinux1 compat-libstdc++-33-3.2.3-61 compat-libstdc++-33-3.2.3 passed

Result: Package existence check passed for “compat-libstdc++-33-3.2.3”

Check: Package existence for “elfutils-libelf-0.125”

Node Name Available Required Comment

———— ———————— ———————— ———-

raclinux2 elfutils-libelf-0.137-3.el5 elfutils-libelf-0.125 passed

raclinux1 elfutils-libelf-0.137-3.el5 elfutils-libelf-0.125 passed

Result: Package existence check passed for “elfutils-libelf-0.125”

Check: Package existence for “elfutils-libelf-devel-0.125”

Node Name Available Required Comment

———— ———————— ———————— ———-

raclinux2 elfutils-libelf-devel-0.137-3.el5 elfutils-libelf-devel-0.125 passed

raclinux1 elfutils-libelf-devel-0.137-3.el5 elfutils-libelf-devel-0.125 passed

Result: Package existence check passed for “elfutils-libelf-devel-0.125”

Check: Package existence for “glibc-common-2.5”

Node Name Available Required Comment

———— ———————— ———————— ———-

raclinux2 glibc-common-2.5-49 glibc-common-2.5 passed

raclinux1 glibc-common-2.5-49 glibc-common-2.5 passed

Result: Package existence check passed for “glibc-common-2.5”

Check: Package existence for “glibc-devel-2.5”

Node Name Available Required Comment

———— ———————— ———————— ———-

raclinux2 glibc-devel-2.5-49 glibc-devel-2.5 passed

raclinux1 glibc-devel-2.5-49 glibc-devel-2.5 passed

Result: Package existence check passed for “glibc-devel-2.5”

Check: Package existence for “glibc-headers-2.5”

Node Name Available Required Comment

———— ———————— ———————— ———-

raclinux2 glibc-headers-2.5-49 glibc-headers-2.5 passed

raclinux1 glibc-headers-2.5-49 glibc-headers-2.5 passed

Result: Package existence check passed for “glibc-headers-2.5”

Check: Package existence for “libaio-devel-0.3.106”

Node Name Available Required Comment

———— ———————— ———————— ———-

raclinux2 libaio-devel-0.3.106-5 libaio-devel-0.3.106 passed

raclinux1 libaio-devel-0.3.106-5 libaio-devel-0.3.106 passed

Result: Package existence check passed for “libaio-devel-0.3.106”

Check: Package existence for “libgcc-4.1.2”

Node Name Available Required Comment

———— ———————— ———————— ———-

raclinux2 libgcc-4.1.2-48.el5 libgcc-4.1.2 passed

raclinux1 libgcc-4.1.2-48.el5 libgcc-4.1.2 passed

Result: Package existence check passed for “libgcc-4.1.2”

Check: Package existence for “libstdc++-4.1.2”

Node Name Available Required Comment

———— ———————— ———————— ———-

raclinux2 libstdc++-4.1.2-48.el5 libstdc++-4.1.2 passed

raclinux1 libstdc++-4.1.2-48.el5 libstdc++-4.1.2 passed

Result: Package existence check passed for “libstdc++-4.1.2”

Check: Package existence for “libstdc++-devel-4.1.2”

Node Name Available Required Comment

———— ———————— ———————— ———-

raclinux2 libstdc++-devel-4.1.2-48.el5 libstdc++-devel-4.1.2 passed

raclinux1 libstdc++-devel-4.1.2-48.el5 libstdc++-devel-4.1.2 passed

Result: Package existence check passed for “libstdc++-devel-4.1.2”

Check: Package existence for “sysstat-7.0.2”

Node Name Available Required Comment

———— ———————— ———————— ———-

raclinux2 sysstat-7.0.2-3.el5 sysstat-7.0.2 passed

raclinux1 sysstat-7.0.2-3.el5 sysstat-7.0.2 passed

Result: Package existence check passed for “sysstat-7.0.2”

Check: Package existence for “unixODBC-2.2.11”

Node Name Available Required Comment

———— ———————— ———————— ———-

raclinux2 unixODBC-2.2.11-7.1 unixODBC-2.2.11 passed

raclinux1 unixODBC-2.2.11-7.1 unixODBC-2.2.11 passed

Result: Package existence check passed for “unixODBC-2.2.11”

Check: Package existence for “unixODBC-devel-2.2.11”

Node Name Available Required Comment

———— ———————— ———————— ———-

raclinux2 unixODBC-devel-2.2.11-7.1 unixODBC-devel-2.2.11 passed

raclinux1 unixODBC-devel-2.2.11-7.1 unixODBC-devel-2.2.11 passed

Result: Package existence check passed for “unixODBC-devel-2.2.11”

Check: Package existence for “ksh-20060214”

Node Name Available Required Comment

———— ———————— ———————— ———-

raclinux2 ksh-20100202-1.el5 ksh-20060214 passed

raclinux1 ksh-20100202-1.el5 ksh-20060214 passed

Result: Package existence check passed for “ksh-20060214”

Checking for multiple users with UID value 0

Result: Check for multiple users with UID value 0 passed

Check: Current group ID

Result: Current group ID check passed

Checking Core file name pattern consistency…

Core file name pattern consistency check passed.

Checking to make sure user “oracle” is not in “root” group

Node Name Status Comment

———— ———————— ————————

raclinux2 does not exist passed

raclinux1 does not exist passed

Result: User “oracle” is not part of “root” group. Check passed

Check default user file creation mask

Node Name Available Required Comment

———— ———————— ———————— ———-

raclinux2 0022 0022 passed

raclinux1 0022 0022 passed

Result: Default user file creation mask check passed

Starting Clock synchronization checks using Network Time Protocol(NTP)…

NTP Configuration file check started…

The NTP configuration file “/etc/ntp.conf” is available on all nodes

NTP Configuration file check passed

Checking daemon liveness…

Check: Liveness for “ntpd”

Node Name Running?

———————————— ————————

raclinux2 yes

raclinux1 yes

Result: Liveness check passed for “ntpd”

Checking NTP daemon command line for slewing option “-x”

Check: NTP daemon command line

Node Name Slewing Option Set?

———————————— ————————

raclinux2 yes

raclinux1 yes

Result:

NTP daemon slewing option check passed

Checking NTP daemon’s boot time configuration, in file “/etc/sysconfig/ntpd”, for slewing option “-x”

Check: NTP daemon’s boot time configuration

Node Name Slewing Option Set?

———————————— ————————

raclinux2 yes

raclinux1 yes

Result:

NTP daemon’s boot time configuration check for slewing option passed

NTP common Time Server Check started…

NTP Time Server “.LOCL.” is common to all nodes on which the NTP daemon is running

Check of common NTP Time Server passed

Clock time offset check from NTP Time Server started…

Checking on nodes “[raclinux2, raclinux1]”…

Check: Clock time offset from NTP Time Server

Time Server: .LOCL.

Time Offset Limit: 1000.0 msecs

Node Name Time Offset Status

———— ———————— ————————

raclinux2 0.0 passed

raclinux1 0.0 passed

Time Server “.LOCL.” has time offsets that are within permissible limits for nodes “[raclinux2, raclinux1]”.

Clock time offset check passed

Result: Clock synchronization check using Network Time Protocol(NTP) passed

Pre-check for cluster services setup was successful.

June 8, 2010 - Posted by gjilevski | oracle