Guenadi N Jilevski's Oracle BLOG

Oracle RAC, DG, EBS, DR and HA DBA BLOG

Build HA for third party application with Oracle GI 11.2.0.3

Build HA for third party application with Oracle GI 11.2.0.3

In the article you will have a look at how to protect an Apache from node failure by registering it with Oracle Clusterware (OC) for monitoring, restart, failover and high availability.

Overview of using Oracle Clusterware for protecting third party application

Starting with Oracle release 10.2 Oracle Clusterware provided high availability for third party applications. An application should be associated with a profile that contain a set of attributes for the Oracle Clusterware(OC) to manage (stop, start, monitor, restart, failover) it. OCR stores the application profile. Prior to Oracle 11.2 crs_* utilities ( crs_profile, crs_register, crs_setprm, crs_start, crs_relocate, crs_stop, crs_unregister) were used for management of third party applications brought under control of OC. In Oracle 11.2 management of third party applications is standardized and has a common management interface using crsctl. Oracle 11.2 also introduces appvipcfg utility for creating VIPs. Upon node failure application VIP fails over to a surviving node along with the protected application. It is the Application VIP that is used for accessing the application, thus in case of failure the application will be highly.

In this article you will look at how to register Apache for management with OC 11.2.0.3. The process comprises several ordered steps as listed below.

  1. Create an Application VIP MyTestVIP on 192.168.20.111.
  2. Modify Apache configuration to start on the newly create Application VIP.
  3. Create an action perl script for monitoring, start, stop and clean Apache.
  4. Create a MyTest resource
  5. Test the implementation.

  1. Create an Application VIP MyTestVIP on 192.168.20.111

    Allocate a non-assigned and non-pingable IP for the application VIP. After creating the Application VIP the address will be available and used for accessing Apache. Use the following command to create an Application VIP named MyTestVIP as root.

    appvipcfg create -network=1 -ip=192.168.20.111 -vipname=MyTestVIP -user=grid

    [root@raclinux1 bin]# ./appvipcfg create -network=1 -ip=192.168.20.111 -vipname=MyTestVIP -user=root

    Production Copyright 2007, 2008, Oracle.All rights reserved

    2011-11-12 20:09:34: Creating Resource Type

    2011-11-12 20:09:34: Executing /u01/app/11.2.0/grid/bin/crsctl add type app.appvip_net1.type -basetype ora.cluster_vip_net1.type -file /u01/app/11.2.0/grid/crs/template/appvip.type

    2011-11-12 20:09:34: Executing cmd: /u01/app/11.2.0/grid/bin/crsctl add type app.appvip_net1.type -basetype ora.cluster_vip_net1.type -file /u01/app/11.2.0/grid/crs/template/appvip.type

    2011-11-12 20:09:34: Create the Resource

    2011-11-12 20:09:34: Executing /u01/app/11.2.0/grid/bin/crsctl add resource MyTestVIP -type app.appvip_net1.type -attr “USR_ORA_VIP=192.168.20.111,START_DEPENDENCIES=hard(ora.net1.network) pullup(ora.net1.network),STOP_DEPENDENCIES=hard(ora.net1.network),ACL=’owner:root:rwx,pgrp:root:r-x,other::r–,user:root:r-x’,HOSTING_MEMBERS=raclinux1.gj.com,APPSVIP_FAILBACK=”

    2011-11-12 20:09:34: Executing cmd: /u01/app/11.2.0/grid/bin/crsctl add resource MyTestVIP -type app.appvip_net1.type -attr “USR_ORA_VIP=192.168.20.111,START_DEPENDENCIES=hard(ora.net1.network) pullup(ora.net1.network),STOP_DEPENDENCIES=hard(ora.net1.network),ACL=’owner:root:rwx,pgrp:root:r-x,other::r–,user:root:r-x’,HOSTING_MEMBERS=raclinux1.gj.com,APPSVIP_FAILBACK=”

    [root@raclinux1 bin]#

    Note that appvipcfg is located in $GI_HOME/bin and is a wrapper for calling crsctl.

    [grid@raclinux1 ~]$ appvipcfg -h

    Production Copyright 2007, 2008, Oracle.All rights reserved

    Unknown option: h

     Usage: appvipcfg create -network= -ip= -vipname=

    -user=<user_name>[-group=<group_name>] [-failback=0 | 1]

     delete -vipname=

    [grid@raclinux1 ~]$

    Verify that the resource was created successfully.

    [root@raclinux1 bin]# ./crsctl stat res MyTestVIP

    NAME=MyTestVIP

    TYPE=app.appvip_net1.type

    TARGET=OFFLINE

    STATE=OFFLINE on raclinux1

    [root@raclinux1 bin]# ./crsctl stat res MyTestVIP -t

    ——————————————————————————–

    NAME TARGET STATE SERVER STATE_DETAILS

    ——————————————————————————–

    Cluster Resources

    ——————————————————————————–

    MyTestVIP

     1 OFFLINE OFFLINE raclinux1

    [root@raclinux1 bin]#

    Start the resource

    [root@raclinux1 bin]# ./crsctl start res MyTestVIP

    CRS-2672: Attempting to start ‘MyTestVIP’ on ‘raclinux1’

    CRS-2676: Start of ‘MyTestVIP1’ on ‘raclinux1’ succeeded

    [root@raclinux1 bin]#

    Verify the resource status

    [root@raclinux1 bin]# ./crsctl stat res MyTestVIP

    NAME=MyTestVIP

    TYPE=app.appvip_net1.type

    TARGET=ONLINE

    STATE=ONLINE on raclinux1

    [root@raclinux1 bin]# ./crsctl stat res MyTestVIP -t

    ——————————————————————————–

    NAME TARGET STATE SERVER STATE_DETAILS

    ——————————————————————————–

    Cluster Resources

    ——————————————————————————–

    MyTestVIP

     1 ONLINE ONLINE raclinux1

    [root@raclinux1 bin]#

    [root@raclinux1 bin]# ping -c 3 192.168.20.111

    PING 192.168.20.111 (192.168.20.111) 56(84) bytes of data.

    64 bytes from 192.168.20.111: icmp_seq=1 ttl=64 time=0.019 ms

    64 bytes from 192.168.20.111: icmp_seq=2 ttl=64 time=0.022 ms

    64 bytes from 192.168.20.111: icmp_seq=3 ttl=64 time=0.013 ms

    — 192.168.20.111 ping statistics —

    3 packets transmitted, 3 received, 0% packet loss, time 2005ms

    rtt min/avg/max/mdev = 0.013/0.018/0.022/0.003 ms

    [root@raclinux1 bin]#

  2. Modify Apache configuration to start on the newly created Application VIP.

    Modify /etc/httpd/conf/httpd.conf to adjust the IP and port by setting the Listen IP:Port. In my example Apache listens on 192.168.20.111:80.

    [root@raclinux1 conf]# grep 111 /etc/httpd/conf/httpd.conf

    Listen 192.168.20.111:80

    [root@raclinux1 conf]#

  3. Create an action script for monitoring, starting, stopping and cleaning Apache.

    An Action script must be created for OC to control the Apache. The important actions are stop, start, monitor and clean. The commands shown below will be embedded into the script for start, stop, clean and monitoring Apache.

    [root@raclinux1 bin]# apachectl stop

    [root@raclinux1 bin]# /usr/bin/wget -q –delete-after http://192.168.20.111:80/icons/apache_pb.gif

    [root@raclinux1 bin]# echo $?

    1

    [root@raclinux1 bin]#

    [root@raclinux1 bin]# apachectl start

    [root@raclinux1 bin]# /usr/bin/wget -q –delete-after http://192.168.20.111:80/icons/apache_pb.gif

    [root@raclinux1 bin]# echo $?

    0

    [root@raclinux1 bin]# apachectl stop

    [root@raclinux1 bin]# /usr/bin/wget -q –delete-after http://192.168.20.111:80/icons/apache_pb.gif

    [root@raclinux1 bin]# echo $?

    1

    [root@raclinux1 bin]#

    A perl script is created as follows in /u01/app/11.2.0/grid/crs/public directory.

    [root@raclinux1 conf]# cat /u01/app/11.2.0.3/grid/crs/public/myTest_actionScript.pl

    #!/usr/bin/perl

    if ($#ARGV != 0) {

     # This is an unreacheable condition

     # Oracle Clusterware would always invoke with a valid argument

    exit;

    }

    $command = $ARGV[0];

    # Start command

    if ($command eq “start”) {

    system(“sh /usr/sbin/apachectl -k start”);

    exit $?

    # steps to start the resource

     # exit 0 if the start is successfull

    # exit 1 if the start has failed

    }

    # Stop command

    if ($command eq “stop”) {

    system(“/usr/sbin/apachectl -k stop”);

    exit $?

    # steps to stop the resource

     # exit 0 if the stop is successfull

    # exit 1 if the stop has failed

    }

    # Check command

    if ($command eq “check”) {

    system(“/usr/bin/wget -q –delete-after http://192.168.20.111:80/icons/apache_pb.gif&#8221;);

    #if ($? == -1) { print “failed to execute: $!\n” } else

    #{ printf “check exited with value %d\n”, $? >> 8; };

    exit $?>>8

    # steps to check the status of the resource

    # exit 0 if the resource is online

    # exit 1 otherwise

    }

    # Clean command

    if ($command eq “clean”) {

    system(“/usr/sbin/apachectl -k stop”);

    exit $?

    # steps to force stop the resource

     # exit 0 if the force stop is successfull

    # exit 1 if the force stop has failed

    }

    You have new mail in /var/spool/mail/root

    [root@raclinux1 conf]#

    Test the script

    [root@raclinux1 bin]# /u01/app/11.2.0.3/grid/crs/public/myTest_actionScript.pl start

    [root@raclinux1 bin]# /u01/app/11.2.0.3/grid/crs/public/myTest_actionScript.pl check

    [root@raclinux1 bin]# echo $?

    0

    [root@raclinux1 bin]# /u01/app/11.2.0.3/grid/crs/public/myTest_actionScript.pl stop

    [root@raclinux1 bin]# /u01/app/11.2.0.3/grid/crs/public/myTest_actionScript.pl check

    [root@raclinux1 bin]# echo $?

    1

    [root@raclinux1 bin]# /u01/app/11.2.0.3/grid/crs/public/myTest_actionScript.pl stop

    httpd (no pid file) not running

    [root@raclinux1 bin]# /u01/app/11.2.0.3/grid/crs/public/myTest_actionScript.pl check

    [root@raclinux1 bin]# echo $?

    1

    [root@raclinux1 bin]# /u01/app/11.2.0.3/grid/crs/public/myTest_actionScript.pl start

    [root@raclinux1 bin]# /u01/app/11.2.0.3/grid/crs/public/myTest_actionScript.pl start

    httpd (pid 10924) already running

    [root@raclinux1 bin]# /u01/app/11.2.0.3/grid/crs/public/myTest_actionScript.pl check

    [root@raclinux1 bin]# echo $?

    1

    [root@raclinux1 bin]# /u01/app/11.2.0.3/grid/crs/public/myTest_actionScript.pl check

    [root@raclinux1 bin]# echo $?

    1

    [root@raclinux1 bin]#

    The script performs as expected to star/stop/monitor/clean a resource.

  4. Create a MyTest resource

    Here I will create a resource MyTest using the following below crsctl command. The objective is to configure the resource MyTest to be monitored for 1 hour before considered it stable and to run on MyTestVIP. Resource will be restarted 2 times and will be able to run on any server on the cluster upon node failure. At any time the MyTest resource will run on one cluster node only and only one instance of the resource will be running at any given time.

    [root@raclinux1 bin]# ./crsctl add resource MyTest -type cluster_resource -attr ” AGENT_FILENAME=”%”CRS_HOME”%”/bin/scriptagent, ACTION_SCRIPT=/u01/app/11.2.0.3/grid/crs/public/myTest_actionScript.pl, DESCRIPTION=, DEGREE=1, ENABLED=1, AUTO_START=always, START_TIMEOUT=0, UPTIME_THRESHOLD=1h, CHECK_INTERVAL=10, STOP_TIMEOUT=0, SCRIPT_TIMEOUT=60, RESTART_ATTEMPTS=2, OFFLINE_CHECK_INTERVAL=0, START_DEPENDENCIES=hard(MyTestVIP) pullup(MyTestVIP), STOP_DEPENDENCIES=hard(intermediate:MyTestVIP),CARDINALITY=1, FAILURE_INTERVAL=0, FAILURE_THRESHOLD=0, SERVER_POOLS=*, PLACEMENT=balanced, LOAD=1, ACTIVE_PLACEMENT=1″

    [root@raclinux1 bin]#

    [root@raclinux1 bin]#

    Where the attributes and attribute values are summarized in the table below ( See Oracle® Clusterware Administration and Deployment Guide 11g Release 2 (11.2) E16794-09 ):

Attribute Attribute meaning Attribute Value
AGENT_FILENAME A fully qualified file name of an agent program that a resource type uses to manage its resources. There are two script agents included with Oracle Clusterware 11g release 2 (11.2): application and scriptagent. Oracle Clusterware uses the application script agent for resources of the deprecated application resource type. The default value for this attribute is scriptagent.
=”%”CRS_HOME”%”/bin/scriptagent
ACTION_SCRIPT An absolute file name that includes the path and file name of an action script. The agent specified in the AGENT_FILENAME attribute calls the script specified in the ACTION_SCRIPT attribute. /u01/app/11.2.0.3/grid/crs/public/myTest_actionScript.pl
DESCRIPTION Enter a description of the resource you are adding.
DEGREE The number of instances of a cluster resource that can run on a single server 1
ENABLED Oracle Clusterware uses this attribute to manage the state of the resource. Oracle Clusterware does not attempt to manage a disabled (ENABLED=0) resource either
directly or because of a dependency to another resource. A disabled resource cannot be started but it can be stopped. Oracle Clusterware does not actively monitor disabled resources, meaning that Oracle Clusterware does not check their state. However, when Oracle Clusterware starts, it does query the state of disabled resources.
1
AUTO_START Indicates whether Oracle Clusterware automatically starts a resource after a cluster server restart. always
START_TIMEOUT The maximum time (in seconds) in which a start action can run. Oracle Clusterware returns an error message if the action does not complete within the time specified. If you do not specify a value for this attribute or you specify 0 seconds, then Oracle
Clusterware uses the value of the SCRIPT_TIMEOUT attribute.
0
UPTIME_THRESHOLD The value for UPTIME_THRESHOLD represents the length of time that a resource must be up before Oracle Clusterware considers the resource to be stable. By setting a value for the UPTIME_THRESHOLD attribute, you can indicate the stability of a resource.
After the time period you specify for UPTIME_THRESHOLD elapses, Oracle Clusterware resets the value for RESTART_COUNT to 0.
1h
CHECK_INTERVAL The time interval, in seconds, between repeated executions of the check action. 10
STOP_TIMEOUT The maximum time (in seconds) in which a stop or clean action can run. 0
SCRIPT_TIMEOUT The maximum time (in seconds) for an action to run. 60
RESTART_ATTEMPTS The number of times that Oracle Clusterware attempts to restart a resource on the resource’s current server before attempting to relocate it. 2
OFFLINE_CHECK_INTERVAL Controls offline monitoring of a resource. The value represents the interval (in
seconds) that Oracle Clusterware monitors a resource when its state is OFFLINE. Monitoring is disabled if the value is 0.
0
START_DEPENDENCIES Specifies a set of relationships that Oracle Clusterware considers when starting a resource. hard(MyTestVIP) pullup(MyTestVIP)
STOP_DEPENDENCIES Specifies a set of relationships that Oracle Clusterware considers when stopping a resource.
hard(intermediate:MyTestVIP)
CARDINALITY The number of servers on which a resource can run, simultaneously. This is the upper
limit for resource cardinality.
1
FAILURE_INTERVAL The interval, in seconds, during which Oracle Clusterware applies the FAILURE_THRESHOLD attribute. If the value is zero (0), then tracking of failures is disabled.
0
FAILURE_THRESHOLD The number of failures detected within a specified FAILURE_INTERVAL for a resource before Oracle Clusterware marks the resource as unavailable and no longer monitors it. If a resource fails the specified number of times, then Oracle Clusterware stops the resource. If the value is zero (0), then tracking of failures is disabled. The maximum value is 20. 0
SERVER_POOLS This attribute creates an affinity between a resource and one or more server pools
regarding placement, and is dependent on the value of the PLACEMENT attribute.
If a resource can run on any server in a cluster, then use the default value, *, unless the resource is a cluster_resource type, in which case, the default value for the

SERVER_POOLS attribute is empty. Only cluster administrators can specify * as the value for this attribute. Otherwise, you must specify in the SERVER_POOLS attribute a space-separated list of the server pools to which a particular resource can belong.

*
PLACEMENT Specifies how Oracle Clusterware selects a cluster server on which to start a resource.
Valid values are balanced, favored, or restricted.
balanced
LOAD Oracle Clusterware interprets the value of this attribute along with that of the PLACEMENT attribute. When the value of PLACEMENT is balanced, the value of LOAD determines where best to place a resource. A nonnegative, numeric value that quantitatively represents how much server capacity an instance of a resource consumes relative to other resources. Oracle Clusterware attempts to place resources on servers with the least total load of running resources. 1
ACTIVE_PLACEMENT When set to 1, Oracle Clusterware uses this attribute to reevaluate the placement of a resource during addition or restart of a cluster server. 1

In a nutshell, I created a MyTest resource as cluster_resource type. Resources create as a cluster_resource type can run on any cluster node as opposed to a local resource which runs only on a single node. I specified start and stop dependencies on MyTestVIP resource. MyTest resource has CARDINALITY of 1 and DEGREE 1 so that to be able to run on only one server and as single instance. The attribute UPTIME_THRESHOLD implements stability requirement of 1 hour. PLACEMENT and SERVER_POOLS attribute together allow for MyTest resource placement on any cluster node. The hard start dependency indicates that MyTest requires MyTestVIP started for its operation. Pullup start dependency indicates that whenever MyTestVIP starts it starts MyTest as well. The stop dependency indicates that stop of MyTestVIP should stop MyTest.

Start the resource.

[root@raclinux1 bin]#./crsctl start res MyTest

CRS-2672: Attempting to start ‘MyTest’ on ‘raclinux1’

CRS-2676: Start of ‘MyTest’ on ‘raclinux1’ succeeded

[root@raclinux1 bin]#

Verify the resource status.

[root@raclinux1 bin]# ./crsctl stat res MyTest -t

——————————————————————————–

NAME TARGET STATE SERVER STATE_DETAILS

——————————————————————————–

Cluster Resources

——————————————————————————–

MyTest

 1 ONLINE ONLINE raclinux1

[root@raclinux1 bin]# ./crsctl stat res MyTestVIP -t

——————————————————————————–

NAME TARGET STATE SERVER STATE_DETAILS

——————————————————————————–

Cluster Resources

——————————————————————————–

MyTestVIP

 1 ONLINE ONLINE raclinux1

[root@raclinux1 bin]#

Access the URL: 192.168.20.111:80 or 192.168.20.111 as it defaults to 80.


  1. Test the implementation.

There two test that will be conducted

  1. Failure of Apache daemon
  2. Failure on the node running Apache
    1. Failure of Apache Daemon

Here I will kill an Apache daemons and will verify the restart. From a terminal session make sure that MyTest resource is running and issue consecutive ps and kill commands to observe the restart. As can be see from the example below OC restarts the httpd successfully.

[root@raclinux2 bin]# ps -ef | grep http

root 7196 1 0 07:07 ? 00:00:00 /usr/sbin/nss_pcache off /etc/httpd/alias

root 7199 1 0 07:07 ? 00:00:00 /usr/sbin/httpd -k start

apache 7205 7199 0 07:07 ? 00:00:00 /usr/sbin/httpd -k start

apache 7206 7199 0 07:07 ? 00:00:00 /usr/sbin/httpd -k start

apache 7207 7199 0 07:07 ? 00:00:00 /usr/sbin/httpd -k start

apache 7208 7199 0 07:07 ? 00:00:00 /usr/sbin/httpd -k start

apache 7209 7199 0 07:07 ? 00:00:00 /usr/sbin/httpd -k start

apache 7210 7199 0 07:07 ? 00:00:00 /usr/sbin/httpd -k start

apache 7211 7199 0 07:07 ? 00:00:00 /usr/sbin/httpd -k start

apache 7212 7199 0 07:07 ? 00:00:00 /usr/sbin/httpd -k start

root 7315 6538 0 07:08 pts/1 00:00:00 grep http

[root@raclinux2 bin]# kill -9 7212 7211 7210 7209 7208 7207 7206 7205 7205 7196

[root@raclinux2 bin]# ps -ef | grep http

root 7199 1 0 07:07 ? 00:00:00 /usr/sbin/httpd -k start

apache 7354 7199 0 07:09 ? 00:00:00 /usr/sbin/httpd -k start

apache 7355 7199 0 07:09 ? 00:00:00 /usr/sbin/httpd -k start

apache 7356 7199 0 07:09 ? 00:00:00 /usr/sbin/httpd -k start

apache 7357 7199 0 07:09 ? 00:00:00 /usr/sbin/httpd -k start

apache 7358 7199 0 07:09 ? 00:00:00 /usr/sbin/httpd -k start

apache 7359 7199 0 07:09 ? 00:00:00 /usr/sbin/httpd -k start

apache 7360 7199 0 07:09 ? 00:00:00 /usr/sbin/httpd -k start

root 7379 6538 0 07:09 pts/1 00:00:00 grep http

[root@raclinux2 bin]#

  1. Failure of the node running Apache

I will reboot each of the cluster node one after another and will monitor failover of the MyTestVIP and MyTest resources. As can be seen from the examples below MyTest resource and MyTestVIP successfully failover to the surviving nodes and continues to provide the service through the MyTestVIP.

Before rebooting node raclinux1

[root@raclinux2 bin]# ./crsctl stat res -t

——————————————————————————–

NAME TARGET STATE SERVER STATE_DETAILS

——————————————————————————–

Local Resources

——————————————————————————–

ora.DATA.dg

 ONLINE ONLINE raclinux1

 ONLINE ONLINE raclinux2

ora.DATADG.dg

 ONLINE ONLINE raclinux1

 ONLINE ONLINE raclinux2

ora.LISTENER.lsnr

 ONLINE ONLINE raclinux1

 ONLINE ONLINE raclinux2

ora.asm

 ONLINE ONLINE raclinux1 Started

 ONLINE ONLINE raclinux2 Started

ora.gsd

 OFFLINE OFFLINE raclinux1

 OFFLINE OFFLINE raclinux2

ora.net1.network

 ONLINE ONLINE raclinux1

 ONLINE ONLINE raclinux2

ora.ons

 ONLINE ONLINE raclinux1

 ONLINE ONLINE raclinux2

ora.registry.acfs

 ONLINE ONLINE raclinux1

 ONLINE ONLINE raclinux2

——————————————————————————–

Cluster Resources

——————————————————————————–

MyTest

 1 ONLINE ONLINE raclinux1

MyTestVIP

 1 ONLINE ONLINE raclinux1

ora.LISTENER_SCAN1.lsnr

 1 ONLINE ONLINE raclinux1

ora.RAC10G.RAC10G1.inst

 1 ONLINE ONLINE raclinux1

ora.RAC10G.RAC10G2.inst

 1 ONLINE ONLINE raclinux2

ora.RAC10G.db

 1 ONLINE ONLINE raclinux1

ora.cvu

 1 ONLINE ONLINE raclinux1

ora.oc4j

 1 ONLINE ONLINE raclinux1

ora.racdb.db

 1 ONLINE ONLINE raclinux1 Open

 2 ONLINE ONLINE raclinux2 Open

ora.raclinux1.vip

 1 ONLINE ONLINE raclinux1

ora.raclinux2.vip

 1 ONLINE ONLINE raclinux2

ora.scan1.vip

 1 ONLINE ONLINE raclinux1

[root@raclinux2 bin]#

[root@raclinux2 bin]#

After rebooting raclinux1 on raclinux2 we have:

[root@raclinux2 bin]# ./crsctl stat res -t

——————————————————————————–

NAME TARGET STATE SERVER STATE_DETAILS

——————————————————————————–

Local Resources

——————————————————————————–

ora.DATA.dg

 ONLINE ONLINE raclinux1 STOPPING

 ONLINE ONLINE raclinux2

ora.DATADG.dg

 ONLINE ONLINE raclinux1 STOPPING

 ONLINE ONLINE raclinux2

ora.LISTENER.lsnr

 ONLINE OFFLINE raclinux1

 ONLINE ONLINE raclinux2

ora.asm

 ONLINE ONLINE raclinux1 Started

 ONLINE ONLINE raclinux2 Started

ora.gsd

 OFFLINE OFFLINE raclinux1

 OFFLINE OFFLINE raclinux2

ora.net1.network

 ONLINE ONLINE raclinux1

 ONLINE ONLINE raclinux2

ora.ons

 ONLINE ONLINE raclinux1

 ONLINE ONLINE raclinux2

ora.registry.acfs

 ONLINE OFFLINE raclinux1

 ONLINE ONLINE raclinux2

——————————————————————————–

Cluster Resources

——————————————————————————–

MyTest

 1 ONLINE ONLINE raclinux2

MyTestVIP

 1 ONLINE ONLINE raclinux2

ora.LISTENER_SCAN1.lsnr

 1 ONLINE ONLINE raclinux2

ora.RAC10G.RAC10G1.inst

1 ONLINE OFFLINE

ora.RAC10G.RAC10G2.inst

 1 ONLINE ONLINE raclinux2

ora.RAC10G.db

 1 ONLINE ONLINE raclinux2

ora.cvu

 1 ONLINE ONLINE raclinux2

ora.oc4j

1 ONLINE OFFLINE STARTING

ora.racdb.db

1 ONLINE OFFLINE Instance Shutdown

 2 ONLINE ONLINE raclinux2 Open

ora.raclinux1.vip

 1 ONLINE INTERMEDIATE raclinux2 FAILED OVER

ora.raclinux2.vip

 1 ONLINE ONLINE raclinux2

ora.scan1.vip

 1 ONLINE ONLINE raclinux2

[root@raclinux2 bin]#

After rebooting raclinux2 on raclinux1 we have

[root@raclinux1 bin]# ./crsctl stat res -t

——————————————————————————–

NAME TARGET STATE SERVER STATE_DETAILS

——————————————————————————–

Local Resources

——————————————————————————–

ora.DATA.dg

 ONLINE ONLINE raclinux1

 ONLINE ONLINE raclinux2 STOPPING

ora.DATADG.dg

 ONLINE ONLINE raclinux1

 ONLINE ONLINE raclinux2 STOPPING

ora.LISTENER.lsnr

 ONLINE ONLINE raclinux1

 ONLINE OFFLINE raclinux2

ora.asm

 ONLINE ONLINE raclinux1 Started

 ONLINE ONLINE raclinux2 Started

ora.gsd

 OFFLINE OFFLINE raclinux1

 OFFLINE OFFLINE raclinux2

ora.net1.network

 ONLINE ONLINE raclinux1

 ONLINE ONLINE raclinux2

ora.ons

 ONLINE ONLINE raclinux1

 ONLINE ONLINE raclinux2

ora.registry.acfs

 ONLINE ONLINE raclinux1

 ONLINE OFFLINE raclinux2

——————————————————————————–

Cluster Resources

——————————————————————————–

MyTest

 1 ONLINE ONLINE raclinux1

MyTestVIP

 1 ONLINE ONLINE raclinux1

ora.LISTENER_SCAN1.lsnr

 1 ONLINE ONLINE raclinux1

ora.RAC10G.RAC10G1.inst

 1 ONLINE ONLINE raclinux1

ora.RAC10G.RAC10G2.inst

1 ONLINE OFFLINE

ora.RAC10G.db

 1 ONLINE ONLINE raclinux2

ora.cvu

 1 ONLINE ONLINE raclinux1

ora.oc4j

 1 ONLINE ONLINE raclinux1

ora.racdb.db

 1 ONLINE ONLINE raclinux1 Open

2 ONLINE OFFLINE Instance Shutdown

ora.raclinux1.vip

 1 ONLINE ONLINE raclinux1

ora.raclinux2.vip

 1 ONLINE INTERMEDIATE raclinux1 FAILED OVER

ora.scan1.vip

 1 ONLINE ONLINE raclinux1

[root@raclinux1 bin]#

Summary

In the article you had a look at the steps to protect a third party application by registering it as OC resource. You created an application VIP MyTestVIP. You had a glimpse at an action script for the OC to manage Apache. You create MyTest OC resource. You conducted tests aiming at proving that OC successfully restart Apache after httpd failure and successfully relocate MyTest to surviving cluster node upon node failure.

November 13, 2011 - Posted by | oracle

7 Comments »

  1. […] related to building High Availability for third party applications with Oracle GI 11.2.0.3 click here. In the article you will find out how to use the GI 11.2.0.3 to set up an Active/Passive […]

    Pingback by Build Active-Passive HA configuration for single instance database with Oracle GI 11.2.0.3 « Guenadi N Jilevski's Oracle BLOG | January 9, 2012 | Reply

  2. I found this very useful in my endeavor to configure CMAN to be a GI managed resource.

    Comment by discojohnson | August 1, 2012 | Reply

    • Hi,

      I am glad that it helps.

      Regards,

      Comment by gjilevski | August 1, 2012 | Reply

  3. Hi,

    Thanks for this write up. It helped to setup and test app HA using GI (11203).

    Everything went through successfully until the last test case i.e. Network Failure. That is when I hit the bug and finally the note on MOS 1457380.1.

    So appvipcfg has a bug which is likely to be fixed onIy in 11.2.0.4 and until then adding custom vip is suggested via crsctl add resource.

    But with that, the app vip just does not come up (atleast with appvipcfg it came up and worked fine for almost all scenarios). Will update further as I find out.

    Thanks.

    Comment by PC | June 25, 2013 | Reply

  4. Hi Guenadi, Thanks for this article. It was helpful.

    To add, for this to work, it requires 112035 (or probably 11204 whenever it is out). Reason being, there is a bug with appvipcfg – if the network fails on the node on which the app vip and the app are active, then neither the app vip (and hence the app) nor the node vip would failover to other node. Pls refer MOS note 1457380.1.

    Comment by PC | June 27, 2013 | Reply

  5. sorry, for some reason, I could not see the previous comment and hence re-commented.

    Comment by PC | June 27, 2013 | Reply

  6. Hi Guenadi Jilevski;
    First of all thank you for sharing this wonderful details. I just found the same with the help of one my team member- and cant stop my self to thanks for this.. I have some question on this. I would like to configure rpas fusion client application; which get data from RPAS server. its take data in file based.(The Oracle Retail Predictive Application Server (commonly referred to as RPAS) is a configurable software platform for developing forecasting and planning applications, following a Client/Server OLAP model.) In this configuration we get RPAS daemon Domain process when my RPAS server runs; for the same we wanted to configure HA solution. In same we have application deployed on webLogic which work only when we have daemon domain process running. we wanted a solution where we can configure Oracle grid infrastructure (cluster ware) and then can attached LUN (storage) with cluster ware from out side but in not form of ASM-OCFS-ACFS- As they don’t support flat file data and available on both nodes as well. Is this above solution will work on my situation.

    Please suggest on this.

    Thanks.

    Comment by Raj Gupta | March 14, 2016 | Reply


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: