Guenadi N Jilevski's Oracle BLOG

Oracle RAC, DG, EBS, DR and HA DBA BLOG

Backup and Restore of OCR in Grid Infrastructure 11G R2 11.2

Backup and Restore of OCR in Grid Infrastructure 11G R2 11.2

In this article are going to have a look at the components of the Oracle Clusterware existing in Oracle GI 11.2 that need attention to ensure high availability of the RAC system. The article will focus on backup and restore of the OCR and will point out some practices for high availability on the GI Clusterware components. Oracle Clusterware in Oracle 11gR2 Grid Infrastructure consists of the OCR, vote disk and OLR click here for more info. Here in the article Oracle ASM is installed and used for Oracle GI and RDBMS storage and we will look at how to implement a robust backup and restore strategy for the clusterware components. To ensure that RAC is deployed and operating successfully we need to ensure that the following policies listed in the bullets are in place.

  • High availability for OCR and vote disk.
  • Robust backup and recovery policy for the Oracle GI clusterware components.

 

How to configure multiple OCR and vote disk high availability using ASM disk groups is elaborated here. Note that vote disk can be on only one ASM disk group and the disk group itself provides redundancy.

Staring with Oracle 11gR2 vote disk is automatically backed up in the OCR and there is need to perform manual backups as in earlier pre 11.2 Oracle versions. Having configured multiple OCR and vote disk on a high redundancy ASM disk group we will continue with the ways to ensure that OCR backup and recovery strategy is in place.

Backing up and Recovering OCR

 

Oracle Clusterware automatically creates OCR backups every 4 hours. At any one time, Oracle Clusterware always retains the latest 3 backup copies of the OCR that are 4 hours old, 1 day old and 1 week old. There is no way to customize the backup frequencies or the number of files that Oracle Grid Infrastructure retains while automatically backing OCR. In addition a custom OCR backup can be made at any time. However any backup software can be used to copy the automatically or manually generated backup files at least once daily to a different destination. The default location for generating backups on OEL systems is GI_home/cdata/cluster_name where the cluster_name is the name of your cluster and GI_home is the home directory of your Oracle GI installation. RAC-SCAN is the scan cluster specified during the installation.

To perform a manual backup use the following command:     

ocrconfig -manualbackup


To view the all backups generated automatically or manually by Oracle Clusterware use the ocrconfig
utility. To find the most recent backup of the OCR, run the following command on any node in the cluster:

ocrconfig -showbackup


This command can be used to show both automatically or manually created backups.


In addition to using the automatically or manually created OCR backup files, you can also export the OCR contents to a file before and after making significant configuration changes such as adding or deleting nodes from your environment, modifying Oracle Clusterware resources, or creating a database.

Exporting the OCR contents to a file lets you restore the OCR if your configuration changes cause errors. For example, if you have un-resolvable configuration problems, or if you are unable to restart your cluster database after such changes, then you can restore your configuration by importing the saved OCR content from the valid configuration. To export the contents of the OCR to a file log as the root user, use the following command:

ocrconfig -export backup_file_name

The
backup_file_name
is the name of the OCR backup file you want to create.


Recovering the OCR using manually or auto generated backups

There are two methods when recovering the OCR. The first method uses automatic or user generated OCR file backup copies and the second method uses manually created OCR export files. In this section, we will look at recovering the OCR using the manual or auto generated backups. In the event of a failure, before you attempt to restore the OCR, ensure that the OCR is unavailable. To check the status of the OCR, run the following command:

ocrcheck


If this command does not display the message Device/File integrity check succeeded for at least one copy of the OCR, then both the primary OCR and the OCR mirrors have failed. You must restore the OCR from a backup. If there is at least one copy of the OCR available, you can use that copy to restore the other copies of the OCR.

When restoring the OCR from automatically generated backups, you first have to determine which backup file you will use for the recovery. Identify the available OCR backups using the ocrconfig command while logged in as root user:

ocrconfig -showbackup

[root@raclinux2 bin]# ./ocrconfig -showbackup

raclinux1 2010/12/04 04:07:18 /u01/app/11.2.0.2/grid/cdata/rac-scan/backup00.ocr

raclinux1 2010/12/04 00:07:08 /u01/app/11.2.0.2/grid/cdata/rac-scan/backup01.ocr

raclinux1 2010/12/03 20:07:02 /u01/app/11.2.0.2/grid/cdata/rac-scan/backup02.ocr

raclinux1 2010/12/03 16:06:53 /u01/app/11.2.0.2/grid/cdata/rac-scan/day.ocr

raclinux1 2010/12/03 16:06:53 /u01/app/11.2.0.2/grid/cdata/rac-scan/week.ocr

raclinux2 2010/12/19 20:37:14 /u01/app/11.2.0.2/grid/cdata/rac-scan/backup_20101219_203714.ocr

raclinux2 2010/12/19 20:35:51 /u01/app/11.2.0.2/grid/cdata/rac-scan/backup_20101219_203551.ocr

raclinux2 2010/12/19 20:35:36 /u01/app/11.2.0.2/grid/cdata/rac-scan/backup_20101219_203536.ocr

raclinux2 2010/12/19 20:35:24 /u01/app/11.2.0.2/grid/cdata/rac-scan/backup_20101219_203524.ocr

raclinux2 2010/12/19 20:33:56 /u01/app/11.2.0.2/grid/cdata/rac-scan/backup_20101219_203356.ocr

[root@raclinux2 bin]# ./ocrconfig -showbackup manual

raclinux2 2010/12/19 20:37:14 /u01/app/11.2.0.2/grid/cdata/rac-scan/backup_20101219_203714.ocr

raclinux2 2010/12/19 20:35:51 /u01/app/11.2.0.2/grid/cdata/rac-scan/backup_20101219_203551.ocr

raclinux2 2010/12/19 20:35:36 /u01/app/11.2.0.2/grid/cdata/rac-scan/backup_20101219_203536.ocr

raclinux2 2010/12/19 20:35:24 /u01/app/11.2.0.2/grid/cdata/rac-scan/backup_20101219_203524.ocr

raclinux2 2010/12/19 20:33:56 /u01/app/11.2.0.2/grid/cdata/rac-scan/backup_20101219_203356.ocr

[root@raclinux2 bin]#

Review the contents of the backup using the following ocrdump
command, where file_name is the name of the OCR backup file. The command generates an ASCII file OCRDUMPFILE:

ocrdump -backupfile OCR_backup_file_name


 

[root@raclinux2 bin]# ./ocrdump -backupfile /u01/app/11.2.0.2/grid/cdata/rac-scan/backup_20101219_203714.ocr

[root@raclinux2 bin]# ls OCRDUMPFILE

OCRDUMPFILE

[root@raclinux2 bin]#

As the root user, stop Oracle Clusterware on all the nodes in your Oracle RAC cluster by executing the following command:

crsctl stop crs

The outputs from the command on both nodes are in the Annex.

As the OCR is on ASM we need to start the ASM on the node where we will recover the OCR. The output of the command is in the Annex.

crsctl start crs -excl

After the startup verify that the diskgroups we need are mounted.

SQL> select name, state from v$asm_diskgroup;

NAME STATE

—————————— ———–

DATA MOUNTED

DGDUP1 MOUNTED

DGDUP2 MOUNTED

DGDUP MOUNTED

PRIM MOUNTED

SEC MOUNTED

6 rows selected.

SQL>

Stop crsd if running

crsctl stop resource ora.crsd -init

Failure to do so will result in error

[root@raclinux2 bin]# ./ocrconfig -restore /u01/app/11.2.0.2/grid/cdata/rac-scan/backup_20101219_203714.ocr

PROT-19: Cannot proceed while the Cluster Ready Service is running

[root@raclinux2 bin]# ./crsctl stop resource ora.crsd -init

CRS-2673: Attempting to stop ‘ora.crsd’ on ‘raclinux2’

CRS-2677: Stop of ‘ora.crsd’ on ‘raclinux2’ succeeded

[root@raclinux2 bin]#

As the root user, restore the OCR by applying an OCR backup file using the following command, where file_name is the name of the OCR that you want to restore:

ocrconfig -restore file_name


 

[root@raclinux2 bin]# ./ocrconfig -restore /u01/app/11.2.0.2/grid/cdata/rac-scan/backup_20101219_203714.ocr

[root@raclinux2 bin]#

As root verify the OCR

[root@raclinux2 bin]# ./ocrcheck

Status of Oracle Cluster Registry is as follows :

Version : 3

Total space (kbytes) : 262120

Used space (kbytes) : 4128

Available space (kbytes) : 257992

ID : 1332773503

Device/File Name : +DATA

Device/File integrity check succeeded

Device/File Name : +dgdup2

Device/File integrity check succeeded

Device/File Name : +dgdup1

Device/File integrity check succeeded

Device/File not configured

Device/File not configured

Cluster registry integrity check succeeded

Logical corruption check succeeded

[root@raclinux2 bin]#

As root stop the Oracle clusterware as it is running in exclusive mode.

[root@raclinux2 bin]# ./crsctl stop crs -f

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on ‘raclinux2’

CRS-2673: Attempting to stop ‘ora.mdnsd’ on ‘raclinux2’

CRS-2673: Attempting to stop ‘ora.ctssd’ on ‘raclinux2’

CRS-2673: Attempting to stop ‘ora.asm’ on ‘raclinux2’

CRS-2673: Attempting to stop ‘ora.drivers.acfs’ on ‘raclinux2’

CRS-2677: Stop of ‘ora.asm’ on ‘raclinux2’ succeeded

CRS-2673: Attempting to stop ‘ora.cluster_interconnect.haip’ on ‘raclinux2’

CRS-2677: Stop of ‘ora.cluster_interconnect.haip’ on ‘raclinux2’ succeeded

CRS-2677: Stop of ‘ora.mdnsd’ on ‘raclinux2’ succeeded

CRS-2677: Stop of ‘ora.ctssd’ on ‘raclinux2’ succeeded

CRS-2673: Attempting to stop ‘ora.cssd’ on ‘raclinux2’

CRS-2677: Stop of ‘ora.cssd’ on ‘raclinux2’ succeeded

CRS-2673: Attempting to stop ‘ora.gipcd’ on ‘raclinux2’

CRS-2673: Attempting to stop ‘ora.diskmon’ on ‘raclinux2’

CRS-2677: Stop of ‘ora.drivers.acfs’ on ‘raclinux2’ succeeded

CRS-2677: Stop of ‘ora.gipcd’ on ‘raclinux2’ succeeded

CRS-2673: Attempting to stop ‘ora.gpnpd’ on ‘raclinux2’

CRS-2677: Stop of ‘ora.diskmon’ on ‘raclinux2’ succeeded

CRS-2677: Stop of ‘ora.gpnpd’ on ‘raclinux2’ succeeded

CRS-2793: Shutdown of Oracle High Availability Services-managed resources on ‘raclinux2’ has completed

CRS-4133: Oracle High Availability Services has been stopped.

[root@raclinux2 bin]#

As the root user, restart Oracle Clusterware on all the nodes in your cluster by executing the following command:

crsctl start crs

[root@raclinux1 bin]# ./crsctl start crs

CRS-4123: Oracle High Availability Services has been started.

[root@raclinux2 bin]# ./crsctl start crs

CRS-4123: Oracle High Availability Services has been started.

Use the Cluster Verification Utility (CVU) to verify the OCR integrity. Run the following command, where the -n all argument retrieves a list of all the cluster nodes that are configured as part of your cluster:

cluvfy comp ocr -n all [-verbose]

The output as show below confirms that OCR was successfully restored on raclinux2.


The output as show below confirms that OCR was successfully restored on raclinux1.


Recovering the OCR using manually generated exports

 

There are two methods when recovering the OCR. The first method uses automatic or user generated OCR file backup copies as we already covered in the section above and the second method uses manually created OCR export files. In this section we will look at recovering the OCR using the manually generated exports.

The ocrconfig -export command creates an export backup of the OCR, enabling you to restore the OCR using the ocrconfig -import option if your configuration changes cause errors. To restore the previous configuration stored in the OCR from an OCR export file as the root user, stop Oracle Clusterware on all the nodes in your Oracle RAC cluster by executing the following command:

crsctl stop crs

The output from the commands on both nodes are in the Annex.

As root start the Oracle clusterware on only one node in exclusive mode so that to be able to perform the import.

crsctl start crs -excl -nocrs

The output is in the Annex. Make sure that all ASM disk groups that are to be used for the OCR are mounted by ASM prior to the import as how below.

SQL> select name, state from v$asm_diskgroup;

NAME STATE

—————————— ———–

DATA MOUNTED

DGDUP1 MOUNTED

DGDUP2 MOUNTED

DGDUP MOUNTED

PRIM MOUNTED

SEC MOUNTED

6 rows selected.

SQL>

Restore the OCR data by importing the contents of the OCR export file using the following command, where file_name is the name of the OCR export file created with the ocrconfig –export backup_file_name:

ocrconfig -import file_name

[root@raclinux2 bin]# ls /tmp

CVU_11.2.0.2.0_oracle keyring-DogWA0 keyring-Rz8s0k keyring-YLYULe OCR_export12202001.exp orbit-root

gconfd-root keyring-DsfLiM keyring-XneWAG lost+found OCR_export12202010.exp ssh-qEcgof5487

hsperfdata_oracle keyring-QiN0FU keyring-Y3hxJA mapping-root OCR_export12212010.exp virtual-root.uFl45n

[root@raclinux2 bin]# ./ocrconfig -import /tmp/OCR_export12202010.exp

[root@raclinux2 bin]#

As root verify OCR.

[root@raclinux2 bin]# ./ocrcheck

Status of Oracle Cluster Registry is as follows :

Version : 3

Total space (kbytes) : 262120

Used space (kbytes) : 4128

Available space (kbytes) : 257992

ID : 437081935

Device/File Name : +DATA

Device/File integrity check succeeded

Device/File Name : +dgdup2

Device/File integrity check succeeded

Device/File Name : +dgdup1

Device/File integrity check succeeded

Device/File not configured

Device/File not configured

Cluster registry integrity check succeeded

Logical corruption check succeeded

[root@raclinux2 bin]#

As root stop the CRS on raclinux2 temporarily started for the import.

[root@raclinux2 bin]# ./crsctl stop crs -f

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on ‘raclinux2’

CRS-2673: Attempting to stop ‘ora.drivers.acfs’ on ‘raclinux2’

CRS-2673: Attempting to stop ‘ora.mdnsd’ on ‘raclinux2’

CRS-2673: Attempting to stop ‘ora.ctssd’ on ‘raclinux2’

CRS-2673: Attempting to stop ‘ora.asm’ on ‘raclinux2’

CRS-2677: Stop of ‘ora.drivers.acfs’ on ‘raclinux2’ succeeded

CRS-2677: Stop of ‘ora.asm’ on ‘raclinux2’ succeeded

CRS-2673: Attempting to stop ‘ora.cluster_interconnect.haip’ on ‘raclinux2’

CRS-2677: Stop of ‘ora.mdnsd’ on ‘raclinux2’ succeeded

CRS-2677: Stop of ‘ora.cluster_interconnect.haip’ on ‘raclinux2’ succeeded

CRS-2677: Stop of ‘ora.ctssd’ on ‘raclinux2’ succeeded

CRS-2673: Attempting to stop ‘ora.cssd’ on ‘raclinux2’

CRS-2677: Stop of ‘ora.cssd’ on ‘raclinux2’ succeeded

CRS-2673: Attempting to stop ‘ora.gipcd’ on ‘raclinux2’

CRS-2673: Attempting to stop ‘ora.diskmon’ on ‘raclinux2’

CRS-2677: Stop of ‘ora.diskmon’ on ‘raclinux2’ succeeded

CRS-2677: Stop of ‘ora.gipcd’ on ‘raclinux2’ succeeded

CRS-2673: Attempting to stop ‘ora.gpnpd’ on ‘raclinux2’

CRS-2677: Stop of ‘ora.gpnpd’ on ‘raclinux2’ succeeded

CRS-2793: Shutdown of Oracle High Availability Services-managed resources on ‘raclinux2’ has completed

CRS-4133: Oracle High Availability Services has been stopped.

[root@raclinux2 bin]#

Logged in as the root user, restart Oracle Clusterware on all the nodes in your cluster by restarting each node, or by running the following command:

crsctl start crs

[root@raclinux1 bin]# ./crsctl start crs

CRS-4123: Oracle High Availability Services has been started.

[root@raclinux2 bin]# ./crsctl start crs

CRS-4123: Oracle High Availability Services has been started.

Use the
CVU
to verify the OCR integrity. Run the following command, where the
-n all argument retrieves a list of all the cluster nodes that are configured as part of your cluster:

cluvfy comp ocr -n all [-verbose]

On node raclinux1 the output shown below confirms the OCR integrity.


On node raclinux2 the output shown below confirms the OCR integrity.


Please note that you cannot use the ocrconfig command to import an OCR backup file, only an OCR export file.

Summary

 

In the article we looked at a step by step implementation of a backup and recovery procedure for the OCR. As Oracle GI provides both backup and export features related to OCR we detailed two alternative ways for OCR backup. First is using automatic or manual backups of OCR and the latter is using OCR exports. Pointers were provided to additional information related to OCR redundancy and vote disk high availability implementations. As the article is based on the assumption that ASM diskgroups are used for both OCR and vote disks in great details step by step the procedure for restore or import was covered to illustrate the importance of starting the CRS in exclusive mode to make the ASM disk groups available during either the restore or the import. In both cases the OCR integrity was verified successfully confirming that both procedures can be used with setups where OCR is based on ASM disk groups.

Annex

 

Output from raclinux2

[root@raclinux2 bin]# ./crsctl stop crs

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on ‘raclinux2’

CRS-2673: Attempting to stop ‘ora.crsd’ on ‘raclinux2’

CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on ‘raclinux2’

CRS-2673: Attempting to stop ‘ora.repl.main.sec.sec.acfs’ on ‘raclinux2’

CRS-2673: Attempting to stop ‘ora.repl.main.prim.prim.acfs’ on ‘raclinux2’

CRS-2673: Attempting to stop ‘ora.LISTENER_SCAN1.lsnr’ on ‘raclinux2’

CRS-2673: Attempting to stop ‘ora.cvu’ on ‘raclinux2’

CRS-2673: Attempting to stop ‘ora.repl.init.prim.prim.acfs’ on ‘raclinux2’

CRS-2673: Attempting to stop ‘ora.LISTENER.lsnr’ on ‘raclinux2’

CRS-2673: Attempting to stop ‘ora.DGDUP1.dg’ on ‘raclinux2’

CRS-2673: Attempting to stop ‘ora.DGDUP2.dg’ on ‘raclinux2’

CRS-2673: Attempting to stop ‘ora.PRIM.dg’ on ‘raclinux2’

CRS-2673: Attempting to stop ‘ora.SEC.dg’ on ‘raclinux2’

CRS-2673: Attempting to stop ‘ora.registry.acfs’ on ‘raclinux2’

CRS-2673: Attempting to stop ‘ora.racdb.db’ on ‘raclinux2’

CRS-2673: Attempting to stop ‘ora.repl.apply.sec.sec.acfs’ on ‘raclinux2’

CRS-2673: Attempting to stop ‘ora.oc4j’ on ‘raclinux2’

CRS-2677: Stop of ‘ora.LISTENER_SCAN1.lsnr’ on ‘raclinux2’ succeeded

CRS-2673: Attempting to stop ‘ora.scan1.vip’ on ‘raclinux2’

CRS-2677: Stop of ‘ora.scan1.vip’ on ‘raclinux2’ succeeded

CRS-2672: Attempting to start ‘ora.scan1.vip’ on ‘raclinux1’

CRS-2677: Stop of ‘ora.LISTENER.lsnr’ on ‘raclinux2’ succeeded

CRS-2673: Attempting to stop ‘ora.raclinux2.vip’ on ‘raclinux2’

CRS-2677: Stop of ‘ora.raclinux2.vip’ on ‘raclinux2’ succeeded

CRS-2672: Attempting to start ‘ora.raclinux2.vip’ on ‘raclinux1’

CRS-2677: Stop of ‘ora.cvu’ on ‘raclinux2’ succeeded

CRS-2672: Attempting to start ‘ora.cvu’ on ‘raclinux1’

CRS-2676: Start of ‘ora.scan1.vip’ on ‘raclinux1’ succeeded

CRS-2676: Start of ‘ora.raclinux2.vip’ on ‘raclinux1’ succeeded

CRS-2675: Stop of ‘ora.repl.init.prim.prim.acfs’ on ‘raclinux2’ failed

CRS-2679: Attempting to clean ‘ora.repl.init.prim.prim.acfs’ on ‘raclinux2’

CRS-2676: Start of ‘ora.cvu’ on ‘raclinux1’ succeeded

CRS-2675: Stop of ‘ora.repl.apply.sec.sec.acfs’ on ‘raclinux2’ failed

CRS-2679: Attempting to clean ‘ora.repl.apply.sec.sec.acfs’ on ‘raclinux2’

CRS-2681: Clean of ‘ora.repl.init.prim.prim.acfs’ on ‘raclinux2’ succeeded

CRS-2672: Attempting to start ‘ora.repl.init.prim.prim.acfs’ on ‘raclinux1’

CRS-2681: Clean of ‘ora.repl.apply.sec.sec.acfs’ on ‘raclinux2’ succeeded

CRS-2672: Attempting to start ‘ora.repl.apply.sec.sec.acfs’ on ‘raclinux1’

CRS-2676: Start of ‘ora.repl.init.prim.prim.acfs’ on ‘raclinux1’ succeeded

CRS-2677: Stop of ‘ora.racdb.db’ on ‘raclinux2’ succeeded

CRS-2673: Attempting to stop ‘ora.DGDUP.dg’ on ‘raclinux2’

CRS-2673: Attempting to stop ‘ora.DATA.dg’ on ‘raclinux2’

CRS-2676: Start of ‘ora.repl.apply.sec.sec.acfs’ on ‘raclinux1’ succeeded

CRS-2675: Stop of ‘ora.repl.main.prim.prim.acfs’ on ‘raclinux2’ failed

CRS-2679: Attempting to clean ‘ora.repl.main.prim.prim.acfs’ on ‘raclinux2’

CRS-2675: Stop of ‘ora.repl.main.sec.sec.acfs’ on ‘raclinux2’ failed

CRS-2679: Attempting to clean ‘ora.repl.main.sec.sec.acfs’ on ‘raclinux2’

CRS-2681: Clean of ‘ora.repl.main.prim.prim.acfs’ on ‘raclinux2’ succeeded

CRS-2672: Attempting to start ‘ora.repl.main.prim.prim.acfs’ on ‘raclinux1’

CRS-2681: Clean of ‘ora.repl.main.sec.sec.acfs’ on ‘raclinux2’ succeeded

CRS-2672: Attempting to start ‘ora.repl.main.sec.sec.acfs’ on ‘raclinux1’

CRS-2672: Attempting to start ‘ora.LISTENER_SCAN1.lsnr’ on ‘raclinux1’

CRS-2676: Start of ‘ora.repl.main.prim.prim.acfs’ on ‘raclinux1’ succeeded

CRS-2676: Start of ‘ora.repl.main.sec.sec.acfs’ on ‘raclinux1’ succeeded

CRS-2676: Start of ‘ora.LISTENER_SCAN1.lsnr’ on ‘raclinux1’ succeeded

CRS-2677: Stop of ‘ora.SEC.dg’ on ‘raclinux2’ succeeded

CRS-2677: Stop of ‘ora.registry.acfs’ on ‘raclinux2’ succeeded

CRS-2677: Stop of ‘ora.oc4j’ on ‘raclinux2’ succeeded

CRS-2677: Stop of ‘ora.DGDUP.dg’ on ‘raclinux2’ succeeded

CRS-2677: Stop of ‘ora.PRIM.dg’ on ‘raclinux2’ succeeded

CRS-2677: Stop of ‘ora.DGDUP2.dg’ on ‘raclinux2’ succeeded

CRS-2677: Stop of ‘ora.DGDUP1.dg’ on ‘raclinux2’ succeeded

CRS-2677: Stop of ‘ora.DATA.dg’ on ‘raclinux2’ succeeded

CRS-2673: Attempting to stop ‘ora.asm’ on ‘raclinux2’

CRS-2677: Stop of ‘ora.asm’ on ‘raclinux2’ succeeded

CRS-2673: Attempting to stop ‘ora.ons’ on ‘raclinux2’

CRS-2677: Stop of ‘ora.ons’ on ‘raclinux2’ succeeded

CRS-2673: Attempting to stop ‘ora.net1.network’ on ‘raclinux2’

CRS-2677: Stop of ‘ora.net1.network’ on ‘raclinux2’ succeeded

CRS-2792: Shutdown of Cluster Ready Services-managed resources on ‘raclinux2’ has completed

CRS-2677: Stop of ‘ora.crsd’ on ‘raclinux2’ succeeded

CRS-2673: Attempting to stop ‘ora.ctssd’ on ‘raclinux2’

CRS-2673: Attempting to stop ‘ora.evmd’ on ‘raclinux2’

CRS-2673: Attempting to stop ‘ora.asm’ on ‘raclinux2’

CRS-2673: Attempting to stop ‘ora.mdnsd’ on ‘raclinux2’

CRS-2673: Attempting to stop ‘ora.drivers.acfs’ on ‘raclinux2’

CRS-2677: Stop of ‘ora.drivers.acfs’ on ‘raclinux2’ succeeded

CRS-2677: Stop of ‘ora.asm’ on ‘raclinux2’ succeeded

CRS-2673: Attempting to stop ‘ora.cluster_interconnect.haip’ on ‘raclinux2’

CRS-2677: Stop of ‘ora.evmd’ on ‘raclinux2’ succeeded

CRS-2677: Stop of ‘ora.mdnsd’ on ‘raclinux2’ succeeded

CRS-2677: Stop of ‘ora.ctssd’ on ‘raclinux2’ succeeded

CRS-2677: Stop of ‘ora.cluster_interconnect.haip’ on ‘raclinux2’ succeeded

CRS-2673: Attempting to stop ‘ora.cssd’ on ‘raclinux2’

CRS-2677: Stop of ‘ora.cssd’ on ‘raclinux2’ succeeded

CRS-2673: Attempting to stop ‘ora.diskmon’ on ‘raclinux2’

CRS-2673: Attempting to stop ‘ora.crf’ on ‘raclinux2’

CRS-2677: Stop of ‘ora.diskmon’ on ‘raclinux2’ succeeded

CRS-2677: Stop of ‘ora.crf’ on ‘raclinux2’ succeeded

CRS-2673: Attempting to stop ‘ora.gipcd’ on ‘raclinux2’

CRS-2677: Stop of ‘ora.gipcd’ on ‘raclinux2’ succeeded

CRS-2673: Attempting to stop ‘ora.gpnpd’ on ‘raclinux2’

CRS-2677: Stop of ‘ora.gpnpd’ on ‘raclinux2’ succeeded

CRS-2793: Shutdown of Oracle High Availability Services-managed resources on ‘raclinux2’ has completed

CRS-4133: Oracle High Availability Services has been stopped.

[root@raclinux2 bin]#

Output from raclinux1

[root@raclinux1 bin]# ./crsctl stop crs

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on ‘raclinux1’

CRS-2673: Attempting to stop ‘ora.crsd’ on ‘raclinux1’

CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on ‘raclinux1’

CRS-2673: Attempting to stop ‘ora.DGDUP1.dg’ on ‘raclinux1’

CRS-2673: Attempting to stop ‘ora.DGDUP2.dg’ on ‘raclinux1’

CRS-2673: Attempting to stop ‘ora.PRIM.dg’ on ‘raclinux1’

CRS-2673: Attempting to stop ‘ora.SEC.dg’ on ‘raclinux1’

CRS-2673: Attempting to stop ‘ora.registry.acfs’ on ‘raclinux1’

CRS-2673: Attempting to stop ‘ora.racdb.db’ on ‘raclinux1’

CRS-2673: Attempting to stop ‘ora.rone.racone.svc’ on ‘raclinux1’

CRS-2673: Attempting to stop ‘ora.repl.transport.sec.sec.acfs’ on ‘raclinux1’

CRS-2673: Attempting to stop ‘ora.repl.transport.prim.prim.acfs’ on ‘raclinux1’

CRS-2673: Attempting to stop ‘ora.LISTENER.lsnr’ on ‘raclinux1’

CRS-2677: Stop of ‘ora.LISTENER.lsnr’ on ‘raclinux1’ succeeded

CRS-2677: Stop of ‘ora.rone.racone.svc’ on ‘raclinux1’ succeeded

CRS-2673: Attempting to stop ‘ora.rone.db’ on ‘raclinux1’

CRS-2675: Stop of ‘ora.repl.transport.sec.sec.acfs’ on ‘raclinux1’ failed

CRS-2679: Attempting to clean ‘ora.repl.transport.sec.sec.acfs’ on ‘raclinux1’

CRS-2681: Clean of ‘ora.repl.transport.sec.sec.acfs’ on ‘raclinux1’ succeeded

CRS-2675: Stop of ‘ora.repl.transport.prim.prim.acfs’ on ‘raclinux1’ failed

CRS-2679: Attempting to clean ‘ora.repl.transport.prim.prim.acfs’ on ‘raclinux1’

CRS-2681: Clean of ‘ora.repl.transport.prim.prim.acfs’ on ‘raclinux1’ succeeded

CRS-2673: Attempting to stop ‘ora.raclinux2.vip’ on ‘raclinux1’

CRS-2677: Stop of ‘ora.raclinux2.vip’ on ‘raclinux1’ succeeded

CRS-2673: Attempting to stop ‘ora.cvu’ on ‘raclinux1’

CRS-2673: Attempting to stop ‘ora.raclinux1.vip’ on ‘raclinux1’

CRS-2677: Stop of ‘ora.raclinux1.vip’ on ‘raclinux1’ succeeded

CRS-2677: Stop of ‘ora.cvu’ on ‘raclinux1’ succeeded

CRS-2673: Attempting to stop ‘ora.repl.init.prim.prim.acfs’ on ‘raclinux1’

CRS-2675: Stop of ‘ora.repl.init.prim.prim.acfs’ on ‘raclinux1’ failed

CRS-2679: Attempting to clean ‘ora.repl.init.prim.prim.acfs’ on ‘raclinux1’

CRS-2681: Clean of ‘ora.repl.init.prim.prim.acfs’ on ‘raclinux1’ succeeded

CRS-2673: Attempting to stop ‘ora.repl.apply.sec.sec.acfs’ on ‘raclinux1’

CRS-2675: Stop of ‘ora.repl.apply.sec.sec.acfs’ on ‘raclinux1’ failed

CRS-2679: Attempting to clean ‘ora.repl.apply.sec.sec.acfs’ on ‘raclinux1’

CRS-2681: Clean of ‘ora.repl.apply.sec.sec.acfs’ on ‘raclinux1’ succeeded

CRS-2673: Attempting to stop ‘ora.repl.main.prim.prim.acfs’ on ‘raclinux1’

CRS-2677: Stop of ‘ora.rone.db’ on ‘raclinux1’ succeeded

CRS-2673: Attempting to stop ‘ora.repl.main.sec.sec.acfs’ on ‘raclinux1’

CRS-2677: Stop of ‘ora.racdb.db’ on ‘raclinux1’ succeeded

CRS-2673: Attempting to stop ‘ora.DGDUP.dg’ on ‘raclinux1’

CRS-2673: Attempting to stop ‘ora.DATA.dg’ on ‘raclinux1’

CRS-2675: Stop of ‘ora.repl.main.prim.prim.acfs’ on ‘raclinux1’ failed

CRS-2679: Attempting to clean ‘ora.repl.main.prim.prim.acfs’ on ‘raclinux1’

CRS-2681: Clean of ‘ora.repl.main.prim.prim.acfs’ on ‘raclinux1’ succeeded

CRS-2673: Attempting to stop ‘ora.LISTENER_SCAN1.lsnr’ on ‘raclinux1’

CRS-2677: Stop of ‘ora.LISTENER_SCAN1.lsnr’ on ‘raclinux1’ succeeded

CRS-2673: Attempting to stop ‘ora.scan1.vip’ on ‘raclinux1’

CRS-2677: Stop of ‘ora.scan1.vip’ on ‘raclinux1’ succeeded

CRS-2675: Stop of ‘ora.repl.main.sec.sec.acfs’ on ‘raclinux1’ failed

CRS-2679: Attempting to clean ‘ora.repl.main.sec.sec.acfs’ on ‘raclinux1’

CRS-2681: Clean of ‘ora.repl.main.sec.sec.acfs’ on ‘raclinux1’ succeeded

CRS-2677: Stop of ‘ora.registry.acfs’ on ‘raclinux1’ succeeded

CRS-2677: Stop of ‘ora.PRIM.dg’ on ‘raclinux1’ succeeded

CRS-2677: Stop of ‘ora.DGDUP.dg’ on ‘raclinux1’ succeeded

CRS-2677: Stop of ‘ora.SEC.dg’ on ‘raclinux1’ succeeded

CRS-2677: Stop of ‘ora.DGDUP1.dg’ on ‘raclinux1’ succeeded

CRS-2677: Stop of ‘ora.DGDUP2.dg’ on ‘raclinux1’ succeeded

CRS-2677: Stop of ‘ora.DATA.dg’ on ‘raclinux1’ succeeded

CRS-2673: Attempting to stop ‘ora.asm’ on ‘raclinux1’

CRS-2677: Stop of ‘ora.asm’ on ‘raclinux1’ succeeded

CRS-2673: Attempting to stop ‘ora.ons’ on ‘raclinux1’

CRS-2677: Stop of ‘ora.ons’ on ‘raclinux1’ succeeded

CRS-2673: Attempting to stop ‘ora.net1.network’ on ‘raclinux1’

CRS-2677: Stop of ‘ora.net1.network’ on ‘raclinux1’ succeeded

CRS-2792: Shutdown of Cluster Ready Services-managed resources on ‘raclinux1’ has completed

CRS-2677: Stop of ‘ora.crsd’ on ‘raclinux1’ succeeded

CRS-2673: Attempting to stop ‘ora.drivers.acfs’ on ‘raclinux1’

CRS-2673: Attempting to stop ‘ora.ctssd’ on ‘raclinux1’

CRS-2673: Attempting to stop ‘ora.evmd’ on ‘raclinux1’

CRS-2673: Attempting to stop ‘ora.asm’ on ‘raclinux1’

CRS-2673: Attempting to stop ‘ora.mdnsd’ on ‘raclinux1’

CRS-2677: Stop of ‘ora.drivers.acfs’ on ‘raclinux1’ succeeded

CRS-2677: Stop of ‘ora.asm’ on ‘raclinux1’ succeeded

CRS-2673: Attempting to stop ‘ora.cluster_interconnect.haip’ on ‘raclinux1’

CRS-2677: Stop of ‘ora.evmd’ on ‘raclinux1’ succeeded

CRS-2677: Stop of ‘ora.mdnsd’ on ‘raclinux1’ succeeded

CRS-2677: Stop of ‘ora.ctssd’ on ‘raclinux1’ succeeded

CRS-2677: Stop of ‘ora.cluster_interconnect.haip’ on ‘raclinux1’ succeeded

CRS-2673: Attempting to stop ‘ora.cssd’ on ‘raclinux1’

CRS-2677: Stop of ‘ora.cssd’ on ‘raclinux1’ succeeded

CRS-2673: Attempting to stop ‘ora.diskmon’ on ‘raclinux1’

CRS-2673: Attempting to stop ‘ora.crf’ on ‘raclinux1’

CRS-2677: Stop of ‘ora.crf’ on ‘raclinux1’ succeeded

CRS-2673: Attempting to stop ‘ora.gipcd’ on ‘raclinux1’

CRS-2677: Stop of ‘ora.diskmon’ on ‘raclinux1’ succeeded

CRS-2677: Stop of ‘ora.gipcd’ on ‘raclinux1’ succeeded

CRS-2673: Attempting to stop ‘ora.gpnpd’ on ‘raclinux1’

CRS-2677: Stop of ‘ora.gpnpd’ on ‘raclinux1’ succeeded

CRS-2793: Shutdown of Oracle High Availability Services-managed resources on ‘raclinux1’ has completed

CRS-4133: Oracle High Availability Services has been stopped.

[root@raclinux1 bin]#

Output from raclinux2

[root@raclinux2 bin]# ./crsctl start crs -excl

CRS-4123: Oracle High Availability Services has been started.

CRS-2672: Attempting to start ‘ora.mdnsd’ on ‘raclinux2’

CRS-2676: Start of ‘ora.mdnsd’ on ‘raclinux2’ succeeded

CRS-2672: Attempting to start ‘ora.gpnpd’ on ‘raclinux2’

CRS-2676: Start of ‘ora.gpnpd’ on ‘raclinux2’ succeeded

CRS-2672: Attempting to start ‘ora.cssdmonitor’ on ‘raclinux2’

CRS-2672: Attempting to start ‘ora.gipcd’ on ‘raclinux2’

CRS-2676: Start of ‘ora.cssdmonitor’ on ‘raclinux2’ succeeded

CRS-2676: Start of ‘ora.gipcd’ on ‘raclinux2’ succeeded

CRS-2672: Attempting to start ‘ora.cssd’ on ‘raclinux2’

CRS-2672: Attempting to start ‘ora.diskmon’ on ‘raclinux2’

CRS-2676: Start of ‘ora.diskmon’ on ‘raclinux2’ succeeded

CRS-2676: Start of ‘ora.cssd’ on ‘raclinux2’ succeeded

CRS-2672: Attempting to start ‘ora.ctssd’ on ‘raclinux2’

CRS-2672: Attempting to start ‘ora.drivers.acfs’ on ‘raclinux2’

CRS-2672: Attempting to start ‘ora.cluster_interconnect.haip’ on ‘raclinux2’

CRS-2676: Start of ‘ora.drivers.acfs’ on ‘raclinux2’ succeeded

CRS-2676: Start of ‘ora.ctssd’ on ‘raclinux2’ succeeded

CRS-2676: Start of ‘ora.cluster_interconnect.haip’ on ‘raclinux2’ succeeded

CRS-2672: Attempting to start ‘ora.asm’ on ‘raclinux2’

CRS-2676: Start of ‘ora.asm’ on ‘raclinux2’ succeeded

CRS-2672: Attempting to start ‘ora.crsd’ on ‘raclinux2’

CRS-2676: Start of ‘ora.crsd’ on ‘raclinux2’ succeeded

[root@raclinux2 bin]#

Output from raclinux2

[root@raclinux2 bin]# ./crsctl start crs -excl -nocrs

CRS-4123: Oracle High Availability Services has been started.

CRS-2672: Attempting to start ‘ora.mdnsd’ on ‘raclinux2’

CRS-2676: Start of ‘ora.mdnsd’ on ‘raclinux2’ succeeded

CRS-2672: Attempting to start ‘ora.gpnpd’ on ‘raclinux2’

CRS-2676: Start of ‘ora.gpnpd’ on ‘raclinux2’ succeeded

CRS-2672: Attempting to start ‘ora.cssdmonitor’ on ‘raclinux2’

CRS-2672: Attempting to start ‘ora.gipcd’ on ‘raclinux2’

CRS-2676: Start of ‘ora.cssdmonitor’ on ‘raclinux2’ succeeded

CRS-2676: Start of ‘ora.gipcd’ on ‘raclinux2’ succeeded

CRS-2672: Attempting to start ‘ora.cssd’ on ‘raclinux2’

CRS-2672: Attempting to start ‘ora.diskmon’ on ‘raclinux2’

CRS-2676: Start of ‘ora.diskmon’ on ‘raclinux2’ succeeded

CRS-2676: Start of ‘ora.cssd’ on ‘raclinux2’ succeeded

CRS-2672: Attempting to start ‘ora.drivers.acfs’ on ‘raclinux2’

CRS-2672: Attempting to start ‘ora.cluster_interconnect.haip’ on ‘raclinux2’

CRS-2672: Attempting to start ‘ora.ctssd’ on ‘raclinux2’

CRS-2676: Start of ‘ora.drivers.acfs’ on ‘raclinux2’ succeeded

CRS-2676: Start of ‘ora.ctssd’ on ‘raclinux2’ succeeded

CRS-2676: Start of ‘ora.cluster_interconnect.haip’ on ‘raclinux2’ succeeded

CRS-2672: Attempting to start ‘ora.asm’ on ‘raclinux2’

CRS-2676: Start of ‘ora.asm’ on ‘raclinux2’ succeeded

[root@raclinux2 bin]#

December 20, 2010 - Posted by | oracle

7 Comments »

  1. This post just finally fixed a problem I’ve had for 3 days in 11.2.0.1. NOWHERE does it say one must run “crsctl stop resource ora.crsd -init”… THANK YOU!!
    You’ve totally made my day!!!

    Comment by Tyfanie | January 7, 2011 | Reply

  2. Very good!

    Comment by Carlos Francisco | May 9, 2012 | Reply

  3. Thanks a lot! Very helpful

    Comment by kamachi | July 20, 2012 | Reply

  4. Excellent article.

    Comment by Shanaka | July 1, 2015 | Reply

  5. Many thanks, it was very clear, detailed useful. Again, many thanks.

    Comment by Giuseppe Lottini | August 15, 2015 | Reply

  6. useful. and there is another tesseract ocr tool for others like.

    Comment by Jony Green | September 5, 2015 | Reply

  7. Thank you so much. The document is very clear and useful. Keep sharing…….
    Venkatesh RK

    Comment by Venkatesh RK | November 5, 2016 | Reply


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: