Guenadi N Jilevski's Oracle BLOG

Oracle RAC, DG, EBS, DR and HA DBA BLOG

Oracle Virtualization – Installing Oracle VM Server 3.1.1, Oracle VM Manager 3.1.1 and Deploying Oracle RAC 11gR2 (11.2.0.3) Oracle VM templates Linux x86 64 bit for test configuration

Oracle Virtualization – Installing Oracle VM Server 3.1.1, Oracle VM Manager 3.1.1 and Deploying Oracle RAC 11gR2 (11.2.0.3) Oracle VM templates Linux x86 64 bit for test configuration

In the second post, first is here, for Oracle VM 3.1.1 technology you will look at how to upgrade the existing Oracle VM server and Oracle VM manager to version 3.1.1 and deploy Oracle RAC templates to build two node Oracle 11gR2 (11.2.0.3) cluster.

All software can be downloaded from edelivery.oracle.com (Oracle Software Delivery Cloud) and the templates (version 11.2.0.3) can be downloaded as a patch from MOS (patch 12694014. The approach described in the article is applicable on any supported platform although I am using Oracle VirtualBox for testing.

Software versions used are:

  • Oracle VirtualBox 4.1.16
  • Oracle VM Server 3.1.1
  • Oracle VM Manager 3.1.1
  • OEL 6.2
  • Oracle RAC templates (patch 12694014)

The article will cover the following:

  • VirtualBox configuration for VM Server machine and VMManager machine
  • Installation of Oracle VM Server
  • Installation of Oracle VM Manager
  • Deployment of Oracle RAC 11gR2 Oracle VM templates Linux x86 64bit for test configuration

In real life Oracle VM server runs on a dedicated set of servers and is installed as a bare metal product without the need of an Operating system. Oracle VM Manager is installed on a Linux box. Oracle VirtualBox can be successfully used for testing and self-study of the new features.

The main components in the architecture are two virtual machines.

Virtual Machine IP Address OS Purpose
OracleVMServer303 192.168.2.21 OEL XEN with Oracle VM Server 3.0.3 Oracle VM Server 3.0.3
OracleVMManager303 192.168.2.31 OEL 6.2 Oracle VM Manager 3.0.3

Product documentation for Oracle VM server 3.1.1 and Oracle VM Manager 3.1.1 is available here.

  1. Upgrade Oracle VM server 3.0.3 to Oracle VM server 3.1.1

    Make the VMserver311 ISO available to the virtual machine OracleVMserver303 and start the VM. Once the boot screen appears press Enter.


    Press Skip.


    Select keyboard.


    Accept the license agreement.


    Select an UPGRADE option and the Oracle VM server 3.0.3 to upgrade.


    Upgrade the boot loader.


    Take a note of the log file location and press OK.


    Wait for the upgrade to continue.


    Reboot.


    Press the highlighted selection.


    Wait for the boot to complete.


    Press Alt-F2 to login


    Login


  2. Upgrade Oracle VM Manager 3.0.3 to Oracle VM Manager 3.1.1

    Transfer to OracleVMManager303 virtual machine OracleVMManager311.iso image a loop mount it.

    [root@vmmgr u01]# ls

    app OracleVMManager311.iso r.txt stage t.txt VMManager303.iso

    [root@vmmgr u01]# mount -o loop OracleVMManager311.iso /mnt/stage

    [root@vmmgr u01]#

    From the stage directory invoke the upgrade script.

    [root@vmmgr stage]# ./runUpgrader.sh

    Stating OVM Manager upgrade on Sun May 20 16:28:43 EEST 2012

    Oracle VM Manager 3.1.1.305 upgrade utility

    Upgrade logfile : /tmp/upgrade-2012-05-20-28.log

    It is highly recommended to do a full database repository backup prior to upgrading Oracle VM Manager …

    Press any key to continue …

    Oracle VM Manager is running …

    Verifying installation status …

    Read Oracle VM Manager config file …

    Found Oracle VM Manager install files …

    Found Oracle VM Manager upgrader …

    Found Oracle WebLogic Server …

    Found Java …

    Using the following information :

    Database Host : localhost

    Database SID : XE

    Database LSNR : 1521

    Oracle VM Schema : ovs

    Oracle VM Manager UUID : 0004fb0000010000f8a01f5a1718ffd9

    Current Build ID : 3.0.3.126

    Upgrade from version : 3.0.3

    Upgrade to version : 3.1.1

    Using /tmp/workdir.0zgO6hQ9um for backup and export location.

    Using /tmp/patchdir.Glh0EQQcD for patching.

    Enter password for user ovs :

    Undeploying previous version of Oracle VM Manager application …

    Undeploying Oracle VM Manager help …

    Undeploying Oracle VM Manager console …

    Undeploying Oracle VM Manager core …

    Waiting for Oracle VM Manager core to fully undeploy…

    Waiting…

    Finished undeploying previous version …

    Exporting Oracle VM Manager repository …

    Please wait as this can take a long time …

    Oracle VM Manager repository export completed …

    Creating backup file …

    Oracle VM Manager repository backup in /tmp/ovm-manager-3-backup-2012-05-20.zip

    Upgrading Oracle VM Manager …

    Backing up old files to /tmp/ovm-manager-3-backup-2012-05-20-163052…

    Removing old files …

    Unpacking Oracle VM Manager 3.1.1.305

    `transform_003001001000_010.xsl’ -> `/tmp/patchdir.Glh0EQQcD/transform_003001001000_010.xsl’

    `transform_003001001000_020.xsl’ -> `/tmp/patchdir.Glh0EQQcD/transform_003001001000_020.xsl’

    `deletedClasses.xml’ -> `/tmp/patchdir.Glh0EQQcD/deletedClasses.xml’

    Filtering full repository export to the selective export subset at /tmp/workdir_sel.XRjFLORxBo …

    cp: omitting directory `/tmp/workdir.0zgO6hQ9um/jrnl’

    cp: omitting directory `/tmp/workdir.0zgO6hQ9um/objs’

    adding: objs/63/639.cl.xml (deflated 81%)

    adding: objs/9.cl.xml (deflated 93%)

    adding: objs/16/161.cl.xml (deflated 67%)

    adding: objs/19/193.cl.xml (deflated 70%)

    Selective export is at /tmp/workdir_sel.XRjFLORxBo

    4 objects selected (out of 508) to be upgraded

    Transform XSL files used:

    -rw-r–r–. 1 root root 56678 May 20 16:30 /tmp/patchdir.Glh0EQQcD/transform_003001001000_010.xsl

    -rw-r–r–. 1 root root 10079 May 20 16:30 /tmp/patchdir.Glh0EQQcD/transform_003001001000_020.xsl

    Changed classes encountered in selective export set:

    com.oracle.ovm.mgr.api.manager.BusinessManagerDbImpl

    com.oracle.ovm.mgr.api.manager.ModelManagerDbImpl

    com.oracle.ovm.mgr.api.manager.RasManagerDbImpl

    com.oracle.ovm.mgr.api.physical.ServerDbImpl

    Upgrading Oracle VM Manager repository …

    Please wait as this can take a long time …

    Oracle VM Manager repository upgrade completed …

    Validating Oracle VM Manager repository …

    Oracle VM Manager repository validation completed …

    Refresh system-jazn-data.xml file …

    Redeploying Oracle VM Manager core container …

    Redeploying Oracle VM Manager console …

    Redeploying Oracle VM Manager help …

    Install ADF Patch …

    Completed upgrade to 3.1.1.305 …

    Writing updated config in /u01/app/oracle/ovm-manager-3/.config

    Restart WebLogic …

    Stopping Oracle VM Manager [ OK ]

    Starting Oracle VM Managernohup: ignoring input and redirecting stderr to stdout

    [ OK ]

    OVM Manager upgrade finished on Sun May 20 16:35:47 EEST 2012

    [root@vmmgr stage]#

    Let’s test Oracle VM Manager 3.1.1


    After login you have.


    Starting and Stopping the OVM from OEL.

    [root@vmmgr ~]# service ovmm stop

    Stopping Oracle VM Manager

    [ OK ]

    [root@vmmgr ~]#

    [root@vmmgr ~]# service ovmm start

    Starting Oracle VM Managernohup: ignoring input and redirecting stderr to stdout

    [ OK ]

    [root@vmmgr ~]#

  3. Deploy two node Oracle 11gR2 (11.2.0.3) RAC
    1. Add a file system /nfs that is to NFS mounted.

      Add 50GB disk to the OracleVMManager303 virtual machine. This disk is visible as /dev/sdb.

      Create a volume, file system and mount the file system on this disk.

      [root@vmmgr ~]# fdisk -l /dev/sdb

      Disk /dev/sdb: 53.7 GB, 53687091200 bytes

      255 heads, 63 sectors/track, 6527 cylinders

      Units = cylinders of 16065 * 512 = 8225280 bytes

      Sector size (logical/physical): 512 bytes / 512 bytes

      I/O size (minimum/optimal): 512 bytes / 512 bytes

      Disk identifier: 0x00000000

      Disk /dev/sdb doesn’t contain a valid partition table

      [root@vmmgr ~]#

      [root@vmmgr ~]# pvcreate /dev/sdb

      Writing physical volume data to disk “/dev/sdb”

      Physical volume “/dev/sdb” successfully created

      [root@vmmgr ~]#

      [root@vmmgr ~]# fdisk /dev/sdb

      Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel

      Building a new DOS disklabel with disk identifier 0xb618dbd7.

      Changes will remain in memory only, until you decide to write them.

      After that, of course, the previous content won’t be recoverable.

      Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

      WARNING: DOS-compatible mode is deprecated. It’s strongly recommended to

      switch off the mode (command ‘c’) and change display units to

      sectors (command ‘u’).

      Command (m for help): c

      DOS Compatibility flag is not set

      Command (m for help): u

      Changing display/entry units to sectors

      Command (m for help): n

      Command action

      e extended

      p primary partition (1-4)

      p

      Partition number (1-4): 1

      First sector (2048-104857599, default 2048):

      Using default value 2048

      Last sector, +sectors or +size{K,M,G} (2048-104857599, default 104857599): +50G

      Value out of range.

      Last sector, +sectors or +size{K,M,G} (2048-104857599, default 104857599): +49G

      Command (m for help): t

      Selected partition 1

      Hex code (type L to list codes): 8e

      Changed system type of partition 1 to 8e (Linux LVM)

      Command (m for help): w

      The partition table has been altered!

      Calling ioctl() to re-read partition table.

      Syncing disks.

      [root@vmmgr ~]#

      [root@vmmgr ~]# fdisk -l /dev/sdb

      Disk /dev/sdb: 53.7 GB, 53687091200 bytes

      171 heads, 46 sectors/track, 13330 cylinders

      Units = cylinders of 7866 * 512 = 4027392 bytes

      Sector size (logical/physical): 512 bytes / 512 bytes

      I/O size (minimum/optimal): 512 bytes / 512 bytes

      Disk identifier: 0xb618dbd7

      Device Boot Start End Blocks Id System

      /dev/sdb1 1 13065 51380224 8e Linux LVM

      [root@vmmgr ~]#

      [root@vmmgr ~]# pvcreate /dev/sdb1

      Writing physical volume data to disk “/dev/sdb1”

      Physical volume “/dev/sdb1” successfully created

      [root@vmmgr ~]#

      vgcreate NFSVolGroup /dev/sdb1

      [root@vmmgr ~]# vgcreate NFSVolGroup /dev/sdb1

      Volume group “NFSVolGroup” successfully created

      [root@vmmgr ~]#

      [root@vmmgr ~]# vgdisplay NFSVolGroup

      — Volume group —

      VG Name NFSVolGroup

      System ID

      Format lvm2

      Metadata Areas 1

      Metadata Sequence No 1

      VG Access read/write

      VG Status resizable

      MAX LV 0

      Cur LV 0

      Open LV 0

      Max PV 0

      Cur PV 1

      Act PV 1

      VG Size 49.00 GiB

      PE Size 4.00 MiB

      Total PE 12543

      Alloc PE / Size 0 / 0

      Free PE / Size 12543 / 49.00 GiB

      VG UUID RnSh8A-fvtJ-97hA-KyX3-Lizm-Dpat-sDcWM9

      [root@vmmgr ~]#

      lvcreate –size 48G NFSVolGroup

      [root@vmmgr ~]# lvcreate –size 48G NFSVolGroup

      Logical volume “lvol0” created

      [root@vmmgr ~]#

      [root@vmmgr ~]# lvdisplay

      — Logical volume —

      LV Name /dev/NFSVolGroup/lvol0

      VG Name NFSVolGroup

      LV UUID 0M6eKi-2uiG-Chjl-rk6E-8BUc-WngI-SvvZpl

      LV Write Access read/write

      LV Status available

      # open 0

      LV Size 48.00 GiB

      Current LE 12288

      Segments 1

      Allocation inherit

      Read ahead sectors auto

      – currently set to 256

      Block device 253:3

      — Logical volume —

      LV Name /dev/vg_vmmgr/lv_root

      VG Name vg_vmmgr

      LV UUID kpXFYY-eAPW-zCXW-wHnN-QuKw-kFxh-vjeaRX

      LV Write Access read/write

      LV Status available

      # open 1

      LV Size 28.37 GiB

      Current LE 7263

      Segments 1

      Allocation inherit

      Read ahead sectors auto

      – currently set to 256

      Block device 253:0

      — Logical volume —

      LV Name /dev/vg_vmmgr/lv_home

      VG Name vg_vmmgr

      LV UUID gaY4wY-aN48-SemS-iUps-L8CX-w27t-OQqPdF

      LV Write Access read/write

      LV Status available

      # open 1

      LV Size 25.26 GiB

      Current LE 6467

      Segments 1

      Allocation inherit

      Read ahead sectors auto

      – currently set to 256

      Block device 253:2

      — Logical volume —

      LV Name /dev/vg_vmmgr/lv_swap

      VG Name vg_vmmgr

      LV UUID f3L33X-DIly-6iO3-hGgu-DrWM-0g1O-mACi0e

      LV Write Access read/write

      LV Status available

      # open 1

      LV Size 5.88 GiB

      Current LE 1504

      Segments 1

      Allocation inherit

      Read ahead sectors auto

      – currently set to 256

      Block device 253:1

      [root@vmmgr ~]#

      /dev/NFSVolGroup/lvol0

      [root@vmmgr ~]# mkfs.ext3 /dev/NFSVolGroup/lvol0

      mke2fs 1.41.12 (17-May-2010)

      Filesystem label=

      OS type: Linux

      Block size=4096 (log=2)

      Fragment size=4096 (log=2)

      Stride=0 blocks, Stripe width=0 blocks

      3145728 inodes, 12582912 blocks

      629145 blocks (5.00%) reserved for the super user

      First data block=0

      Maximum filesystem blocks=4294967296

      384 block groups

      32768 blocks per group, 32768 fragments per group

      8192 inodes per group

      Superblock backups stored on blocks:

          32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,

          4096000, 7962624, 11239424

      Writing inode tables: done

      Creating journal (32768 blocks): done

      Writing superblocks and filesystem accounting information:

      done

      This filesystem will be automatically checked every 21 mounts or

      180 days, whichever comes first. Use tune2fs -c or -i to override.

      [root@vmmgr ~]#

      [root@vmmgr ~]#

      # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info

      #

      /dev/mapper/vg_vmmgr-lv_root / ext4 defaults 1 1

      UUID=19fcf4b8-c245-4f71-bd73-5bf56c6dada5 /boot ext4 defaults 1 2

      /dev/mapper/vg_vmmgr-lv_home /home ext4 defaults 1 2

      /dev/mapper/NFSVolGroup-lvol0 /nfs ext4 defaults 1 2

      /dev/mapper/vg_vmmgr-lv_swap swap swap defaults 0 0

      tmpfs /dev/shm tmpfs defaults 0 0

      devpts /dev/pts devpts gid=5,mode=620 0 0

      sysfs /sys sysfs defaults 0 0

      proc /proc proc defaults 0 0

      [root@vmmgr mapper]#

    2. Create NFS

      Modify /etc/exports as follows.

      [root@vmmgr ~]# cat /etc/export

      cat: /etc/export: No such file or directory

      [root@vmmgr ~]# cat /etc/exports

      /nfs/stage *(rw,sync,no_wdelay,insecure_locks,no_root_squash)

      /nfs/stage1 *(rw,sync,no_wdelay,insecure_locks,no_root_squash)

      [root@vmmgr ~]#

      Create /nfs/stage and /nfs/stage1

      Enable and start nfs service

      [root@vmmgr stage]# chkconfig nfs on

      [root@vmmgr stage]# service nfs restart

      Shutting down NFS mountd: [FAILED]

      Shutting down NFS daemon: [FAILED]

      Shutting down NFS quotas: [FAILED]

      Starting NFS services: [ OK ]

      Starting NFS quotas: [ OK ]

      Starting NFS daemon: [ OK ]

      Starting NFS mountd: [ OK ]

      Starting and Stopping the OVM

      [root@vmmgr ~]# service ovmm stop

      Stopping Oracle VM Manager

      [ OK ]

      [root@vmmgr ~]#

      [root@vmmgr ~]# service ovmm start

      Starting Oracle VM Managernohup: ignoring input and redirecting stderr to stdout

      [ OK ]

      [root@vmmgr ~]#

    3. Discover a VM server from the Oracle VM Manager 3.1.1

      Click discover servers in the tool bar.


      Wait until the server get discovered.


    4. Create a network

      Click the Networking tab, then the Networks subtab. Click Create New Network + to start.


      Select ‘Create a network with bonds/ports only. Press Next.


      Specify Name.


      Select servers.


      Select ports. Make sure ONLY eth0 is selected. Picture is for illustration and should have only eth0.


      Configure IP addresses. Select None and press Finish.


      After the network is created it will be displayed.


    5. Create a file server

      On the storage tab select discover file servers and press Next once filled in the data.


      Select server.


      Select the nfs from /etc/exports. Here should show one of it and press Finish. (Instead of nfs:/u01/VM/nfs should be nfs:/nfs/stage or nfs:/nfs/stage1) /nfs/stage is for the file system and /nfs/stage1 is for the repository.


      You will get something like


  1. Create a Server pool

    From Servers and VMs click Create Server Pool. Press Next.


    Select the server press Finish.


    You will see the nfs created earlier. Press OK.


    Examine the Pool successfully created.


  2. Create Storage Repository.

    Make sure that storage is refreshed. If not refresh from file servers.


    Click Create Repository.


    Enter Repository name. Press the magnifier to select a nfs location.


    Press OK and press Next.


    Select server.


    Verify the result.


  3. Import the templates.

    Templates are not anymore stored directly onto the Oracle VM server 3.1.1. Instead Oracle has provided a way to access them using either ftp or http.

    I will use ftp. So I will configure the ftp service.

    [root@vmmgr stage]# service vsftpd start

    Starting vsftpd for vsftpd: [ OK ]

    [root@vmmgr stage]# chkconfig vsftpd on

    [root@vmmgr stage]#

    I am using OS user gjilevski on the OEL6.2 linux VM running the Oracle VM Manager 3.1.1 with home directory /home/gjilevski.

    Copy the templates into /home/gjilevski. When specifying the templates make sure that you list second before the first template.

    ftp://gjilevski:password@192.168.2.31/OVM_OL5U7_X86_64_11203RAC_PVM-2of2.tgz

    ftp://gjilevski:password@192.168.2.31/OVM_OL5U7_X86_64_11203RAC_PVM-1of2.tgz

    In the URL gjilevski is the OS user identified by the password. File names follows without a path as they are in the default user directory.

    From VM templates import the templates.


    In the screen that appear enter

    ftp://gjilevski:password@192.168.2.31/OVM_OL5U7_X86_64_11203RAC_PVM-2of2.tgz

    ftp://gjilevski:password@192.168.2.31/OVM_OL5U7_X86_64_11203RAC_PVM-1of2.tgz

    instead of what is displayed.


    Wait for the import to completes.


    It will looks like this.


    If you edit it properties will be as follows.





  4. Create shared virtual disk

    Installing RAC requires 5 shared disks each 2GB. From the screen click create virtual disk.


    Create ASM1


    Create ASM2


    Create ASM3


    Repeat same steps for ASM4 and ASM5. Verify that all are created.


  5. Create VNIC

    From Networking tab click virtual NICs.


    Click Auto Fill and Create.


    Initially I did not created VNIC and got the error.

    begin()

    Appended operation ‘Adding a queued job operation’ to object ‘0004fb00001900008ea92f8eb1b6abc2 (Create Statistics Log)’.

    com.oracle.odof.exception.ObjectException: Caught during invoke method: com.oracle.ovm.mgr.api.exception.IllegalOperationException: OVMAPI_8004E There are no more VNICs available. Create More MAC Addresses.

    Mon May 21 17:33:13 EEST 2012

    at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:151)

    at com.oracle.odof.command.InvokeMethodCommand.process(InvokeMethodCommand.java:100)

    at com.oracle.odof.core.BasicWork.processCommand(BasicWork.java:81)

    at com.oracle.odof.core.TransactionManager.processCommand(TransactionManager.java:773)

    at com.oracle.odof.core.WorkflowManager.processCommand(WorkflowManager.java:401)

    at com.oracle.odof.core.WorkflowManager.processWork(WorkflowManager.java:459)

    at com.oracle.odof.io.AbstractClient.run(AbstractClient.java:42)

    at java.lang.Thread.run(Thread.java:662)

    Caused by: com.oracle.ovm.mgr.api.exception.IllegalOperationException: OVMAPI_8004E There are no more VNICs available. Create More MAC Addresses.

    Mon May 21 17:33:13 EEST 2012

    at com.oracle.ovm.mgr.api.manager.VnicManagerDbImpl.getAvailableVnic(VnicManagerDbImpl.java:353)

    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330)

    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)

    at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:822)

    at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:245)

    at com.oracle.ovm.mgr.api.manager.VnicManagerProxy.getAvailableVnic(Unknown Source)

    at com.oracle.ovm.mgr.api.virtual.VirtualMachineConfigDbImpl.cloneNetworks(VirtualMachineConfigDbImpl.java:1065)

    at com.oracle.ovm.mgr.api.virtual.VirtualMachineConfigDbImpl.clone(VirtualMachineConfigDbImpl.java:916)

    at com.oracle.ovm.mgr.api.virtual.VirtualMachineConfigDbImpl.clone(VirtualMachineConfigDbImpl.java:860)

    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330)

    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)

    at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:136)

    … 7 more

    rollback()

  6. Create Virtual Machines



    Once both are done you will have.


  7. Assign NICs to Virtual Machines and make sure that xenbr0 and xenbr1 networks exists.

    In the network you need at the end something like this.


    If not create/modify xenbr0 and xenbr1.

    For xenbr0 press Next.


    Select server.


    Select the ports.


    Press Next.


    Press Next.


    For xenbr1

    Press Next.


    Select server name and press Next.


    Select ports and press Next.


    Press Finish.


    If you do not do so will get similar to

    Job Construction Phase

    ———————-

    begin()

    Appended operation ‘Adding a queued job operation’ to object ‘0004fb000019000038ea52da9f8f21ee (Create Statistics Log)’.

    Appended operation ‘Virtual Disk/Virtual Cdrom Clone’ to object ‘0004fb00001200001de8348bd5ec83ab.img (System.img (2))’.

    Appended operation ‘Repository Element files refresh’ to object ‘0004fb00001200001de8348bd5ec83ab.img (System.img (2))’.

    Appended operation ‘Server Copy File’ to object ‘0004fb00001200001de8348bd5ec83ab.img (System.img (2))’.

    Appended operation ‘Virtual Disk/Virtual Cdrom Clone’ to object ‘0004fb000012000064a859339197b7ee.img (Oracle11203RAC_x86_64-xvdb.img (2))’.

    Appended operation ‘Repository Element files refresh’ to object ‘0004fb000012000064a859339197b7ee.img (Oracle11203RAC_x86_64-xvdb.img (2))’.

    Appended operation ‘Server Copy File’ to object ‘0004fb000012000064a859339197b7ee.img (Oracle11203RAC_x86_64-xvdb.img (2))’.

    Appended operation ‘Virtual Machine Cfg File Construct’ to object ‘0004fb0000220000aa9f213a1f521585’.

    com.oracle.odof.exception.ObjectException: Caught during invoke method: com.oracle.ovm.mgr.api.exception.IllegalOperationException: OVMAPI_9039E Cannot place clone VM: OVM_OL5U7_X86_64_11203RAC_PVM-2of2.tgz, in Server Pool: MyRACPool. That server pool has no servers that can run the VM

    Mon May 21 18:42:36 EEST 2012

    at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:151)

    at com.oracle.odof.command.InvokeMethodCommand.process(InvokeMethodCommand.java:100)

    at com.oracle.odof.core.BasicWork.processCommand(BasicWork.java:81)

    at com.oracle.odof.core.TransactionManager.processCommand(TransactionManager.java:773)

    at com.oracle.odof.core.WorkflowManager.processCommand(WorkflowManager.java:401)

    at com.oracle.odof.core.WorkflowManager.processWork(WorkflowManager.java:459)

    at com.oracle.odof.io.AbstractClient.run(AbstractClient.java:42)

    at java.lang.Thread.run(Thread.java:662)

    Caused by: com.oracle.ovm.mgr.api.exception.IllegalOperationException: OVMAPI_9039E Cannot place clone VM: OVM_OL5U7_X86_64_11203RAC_PVM-2of2.tgz, in Server Pool: MyRACPool. That server pool has no servers that can run the VM

    Mon May 21 18:42:36 EEST 2012

    at com.oracle.ovm.mgr.api.virtual.VirtualMachineConfigDbImpl.clone(VirtualMachineConfigDbImpl.java:939)

    at com.oracle.ovm.mgr.api.virtual.VirtualMachineConfigDbImpl.clone(VirtualMachineConfigDbImpl.java:860)

    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330)

    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)

    at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:136)

    … 7 more

  8. Assign ASM shared disks to VM

    Make sure that you have the following assignment for the VMs.

    For racnode1





    For racnode2





    The file are as follows.

    [root@vmmgr u01]# cat vm.cfg

    vif = [‘mac=00:21:f6:00:00:2f,bridge=xenbr0’, ‘mac=00:21:f6:00:00:30,bridge=xenbr1’]

    OVM_simple_name = ‘racnode2’

    disk = [‘file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb00001200008699ec66aee7821a.img,xvda,w’, ‘file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb00001200006e90573639dac725.img,xvdb,w’, ‘file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb0000120000cdf8b3cd91c68ebd.img,xvdc,w!’, ‘file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb0000120000fb986a9247406641.img,xvdd,w!’, ‘file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb00001200008d3f9019fe32a304.img,xvde,w!’, ‘file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb000012000046a344c5efa19a02.img,xvdf,w!’, ‘file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb0000120000c65d6d0a62ec75a0.img,xvdg,w!’]

    uuid = ‘0004fb00-0006-0000-a2a3-03e4d1281181’

    on_reboot = ‘restart’

    cpu_weight = 27500

    memory = 2048

    cpu_cap = 0

    maxvcpus = 1

    OVM_high_availability = False

    maxmem = 2048

    OVM_description = ‘racnode2’

    on_poweroff = ‘destroy’

    on_crash = ‘restart’

    bootloader = ‘/usr/bin/pygrub’

    name = ‘0004fb0000060000a2a303e4d1281181’

    guest_os_type = ‘default’

    vif_other_config = []

    vfb = [‘type=vnc,vncunused=1,vnclisten=127.0.0.1,keymap=en-us’]

    vcpus = 1

    OVM_os_type = ‘None’

    OVM_cpu_compat_group = ”

    OVM_domain_type = ‘xen_pvm’

    [root@vmmgr u01]#

    [root@vmmgr u01]# cat vm.cfg

    vif = [‘mac=00:21:f6:00:00:1a,bridge=xenbr0’, ‘mac=00:21:f6:00:00:05,bridge=xenbr1’]

    OVM_simple_name = ‘racnode1’

    disk = [‘file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb0000120000a3949c3f3d394b21.img,xvda,w’, ‘file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb000012000065511085c4ad6864.img,xvdb,w’, ‘file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb0000120000cdf8b3cd91c68ebd.img,xvdc,w!’, ‘file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb0000120000fb986a9247406641.img,xvdd,w!’, ‘file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb00001200008d3f9019fe32a304.img,xvde,w!’, ‘file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb000012000046a344c5efa19a02.img,xvdf,w!’, ‘file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb0000120000c65d6d0a62ec75a0.img,xvdg,w!’]

    uuid = ‘0004fb00-0006-0000-cfab-2af11ac1f125’

    on_reboot = ‘restart’

    cpu_weight = 27500

    memory = 2048

    cpu_cap = 0

    maxvcpus = 1

    OVM_high_availability = False

    maxmem = 2048

    OVM_description = ‘racnode1’

    on_poweroff = ‘destroy’

    on_crash = ‘restart’

    bootloader = ‘/usr/bin/pygrub’

    name = ‘0004fb0000060000cfab2af11ac1f125’

    guest_os_type = ‘default’

    vif_other_config = []

    vfb = [‘type=vnc,vncunused=1,vnclisten=127.0.0.1,keymap=en-us’]

    vcpus = 1

    OVM_os_type = ‘None’

    OVM_cpu_compat_group = ”

    OVM_domain_type = ‘xen_pvm’

    [root@vmmgr u01]#

  9. Start the VMS


  10. Establish VNC to both nodes

    After both VM are started establish a VNC sessions to both nodes for configuration of GI and RAC.

    Select and dedicate which node will be the first and second node.


    Enter for each node hostname, public IP and private IP address, VIP, VIP name, domain, DNS, public and private interface etc as prompted.


    Review and acknowledge by pressing YES.


    Press YES


    Wait for the installation to complete.

    Progress is…


    At the end


  11. Verify Oracle 11gR2 11.2.03 GI and RAC installation.

    You can verify the status using sqlplus.


    You can verify the status using srvctl.


    Some notes.

    Acknowledging events may clear some errors after a crash of the Oracle VM Server 3.1.1 VirtualBox VM.


    Verifying the VM properties resolves the error

    Unexpected error during login (com.oracle.ovm.mgr.api.exception.FailedOperationException: OVMAPI_6000E Internal Error: Connection refused Connection refused Mon May 21 23:18:19 EEST 2012), please consult logs for details.

    Job Construction Phase

    ———————-

    begin()

    Appended operation ‘Virtual Machine Start’ to object ‘0004fb0000060000cfab2af11ac1f125 (racnode1)’.

    Appended operation ‘Bridge Configure Operation’ to object ‘0004fb0000200000129d832619e996f5 (network.EthernetPort (4) in vmsrv33.gj.com)’.

    Appended operation ‘Bridge Configure Operation’ to object ‘0004fb0000200000604c446c90acae98 (network.EthernetPort (3) in vmsrv33.gj.com)’.

    commit()

    Completed Step: COMMIT

    Objects and Operations

    ———————-

    Object (IN_USE): [EthernetPort] 0004fb0000200000604c446c90acae98 (network.EthernetPort (3) in vmsrv33.gj.com)

    Operation: Bridge Configure Operation

    Object (IN_USE): [Vnic] 0004fb000007000047de4cdfa34b0a41 (00:21:f6:00:00:1a)

    Object (IN_USE): [Vnic] 0004fb000007000020f4233448d8d770 (00:21:f6:00:00:05)

    Object (IN_USE): [VmDiskMapping] 0004fb00001300006e3c1bc79509f857

    Object (IN_USE): [VmDiskMapping] 0004fb0000130000ce67b1e5241157d7

    Object (IN_USE): [VirtualDisk] 0004fb00001200008d3f9019fe32a304.img (ASM3)

    Object (IN_USE): [VmDiskMapping] 0004fb0000130000494d049341598540

    Object (IN_USE): [VirtualDisk] 0004fb0000120000c65d6d0a62ec75a0.img (ASM5)

    Object (IN_USE): [VmDiskMapping] 0004fb0000130000267c771177271b00

    Object (IN_USE): [VirtualMachine] 0004fb0000060000cfab2af11ac1f125 (racnode1)

    Operation: Virtual Machine Start

    Object (IN_USE): [VirtualDisk] 0004fb0000120000cdf8b3cd91c68ebd.img (ASM1)

    Object (IN_USE): [VmDiskMapping] 0004fb00001300002036b8246f86e2c4

    Object (IN_USE): [VirtualDisk] 0004fb000012000046a344c5efa19a02.img (ASM4)

    Object (IN_USE): [VirtualDisk] 0004fb000012000065511085c4ad6864.img (Oracle11203RAC_x86_64-xvdb.img (2))

    Object (IN_USE): [EthernetPort] 0004fb0000200000129d832619e996f5 (network.EthernetPort (4) in vmsrv33.gj.com)

    Operation: Bridge Configure Operation

    Object (IN_USE): [VirtualDisk] 0004fb0000120000fb986a9247406641.img (ASM2)

    Object (IN_USE): [VmDiskMapping] 0004fb00001300007b4b45d8dab8f783

    Object (IN_USE): [VirtualDisk] 0004fb0000120000a3949c3f3d394b21.img (System.img (2))

    Object (IN_USE): [VmDiskMapping] 0004fb000013000040f7df759023d559

    Job Running Phase at 23:27 on Mon, May 21, 2012

    ———————————————-

    Job Participants: [b6:10:7c:ba:39:89:4e:e8:95:b7:ca:1a:61:e9:81:69 (vmsrv33.gj.com)]

    Actioner

    ——–

    Starting operation ‘Bridge Configure Operation’ on object ‘0004fb0000200000604c446c90acae98 (network.EthernetPort (3) in vmsrv33.gj.com)’

    Bridge [xenbr1] already exists (and should exist) on interface [eth2] on server [vmsrv33.gj.com]; skipping bridge creation

    Completed operation ‘Bridge Configure Operation’ completed with direction ==> DONE

    Starting operation ‘Virtual Machine Start’ on object ‘0004fb0000060000cfab2af11ac1f125 (racnode1)’

    Completed operation ‘Virtual Machine Start’ completed with direction ==> LATER

    Starting operation ‘Bridge Configure Operation’ on object ‘0004fb0000200000129d832619e996f5 (network.EthernetPort (4) in vmsrv33.gj.com)’

    Bridge [xenbr0] already exists (and should exist) on interface [eth3] on server [vmsrv33.gj.com]; skipping bridge creation

    Completed operation ‘Bridge Configure Operation’ completed with direction ==> DONE

    Starting operation ‘Virtual Machine Start’ on object ‘0004fb0000060000cfab2af11ac1f125 (racnode1)’

    Job Internal Error (Operation)com.oracle.ovm.mgr.api.exception.FailedOperationException: OVMAPI_4010E Attempt to send command: dispatch to server: vmsrv33.gj.com failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@192.168.2.21:8899/api/2 start_vm 0004fb000003000009747581081557e5 0004fb0000060000cfab2af11ac1f125, Status: org.apache.xmlrpc.XmlRpcException: exceptions.RuntimeError:Command: [‘xm’, ‘create’, ‘/OVS/Repositories/0004fb000003000009747581081557e5/VirtualMachines/0004fb0000060000cfab2af11ac1f125/vm.cfg’] failed (1): stderr: Error: Device 51744 (vbd) could not be connected. /etc/xen/scripts/block failed; error detected.

    stdout: Using config file “/OVS/Repositories/0004fb000003000009747581081557e5/VirtualMachines/0004fb0000060000cfab2af11ac1f125/vm.cfg”.

    Mon May 21 23:28:12 EEST 2012

    Mon May 21 23:28:12 EEST 2012

    at com.oracle.ovm.mgr.action.ActionEngine.sendCommandToServer(ActionEngine.java:507)

    at com.oracle.ovm.mgr.action.ActionEngine.sendDispatchedServerCommand(ActionEngine.java:444)

    at com.oracle.ovm.mgr.action.ActionEngine.sendServerCommand(ActionEngine.java:378)

    at com.oracle.ovm.mgr.action.VirtualMachineAction.sendStartVmCommand(VirtualMachineAction.java:121)

    at com.oracle.ovm.mgr.op.virtual.VirtualMachineStart.asyncAction(VirtualMachineStart.java:65)

    at com.oracle.ovm.mgr.api.collectable.ManagedObjectDbImpl.executeCurrentJobOperationAsyncAction(ManagedObjectDbImpl.java:999)

    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330)

    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)

    at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:822)

    at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:245)

    at com.oracle.ovm.mgr.api.virtual.VirtualMachineProxy.executeCurrentJobOperationAsyncAction(Unknown Source)

    at com.oracle.ovm.mgr.api.job.JobEngine.objectAsyncStart(JobEngine.java:596)

    at com.oracle.ovm.mgr.op.virtual.VirtualMachineStart.action(VirtualMachineStart.java:52)

    at com.oracle.ovm.mgr.api.collectable.ManagedObjectDbImpl.executeCurrentJobOperationAction(ManagedObjectDbImpl.java:1009)

    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330)

    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)

    at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:822)

    at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:245)

    at com.oracle.ovm.mgr.api.virtual.VirtualMachineProxy.executeCurrentJobOperationAction(Unknown Source)

    at com.oracle.ovm.mgr.api.job.JobEngine.operationActioner(JobEngine.java:218)

    at com.oracle.ovm.mgr.api.job.JobEngine.objectActioner(JobEngine.java:309)

    at com.oracle.ovm.mgr.api.job.InternalJobDbImpl.objectCommitter(InternalJobDbImpl.java:1140)

    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330)

    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)

    at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:136)

    at com.oracle.odof.command.InvokeMethodCommand.process(InvokeMethodCommand.java:100)

    at com.oracle.odof.core.BasicWork.processCommand(BasicWork.java:81)

    at com.oracle.odof.core.TransactionManager.processCommand(TransactionManager.java:773)

    at com.oracle.odof.core.WorkflowManager.processCommand(WorkflowManager.java:401)

    at com.oracle.odof.core.WorkflowManager.processWork(WorkflowManager.java:459)

    at com.oracle.odof.io.AbstractClient.run(AbstractClient.java:42)

    at java.lang.Thread.run(Thread.java:662)

    Caused by: com.oracle.ovm.mgr.api.exception.IllegalOperationException: OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@192.168.2.21:8899/api/2 start_vm 0004fb000003000009747581081557e5 0004fb0000060000cfab2af11ac1f125, Status: org.apache.xmlrpc.XmlRpcException: exceptions.RuntimeError:Command: [‘xm’, ‘create’, ‘/OVS/Repositories/0004fb000003000009747581081557e5/VirtualMachines/0004fb0000060000cfab2af11ac1f125/vm.cfg’] failed (1): stderr: Error: Device 51744 (vbd) could not be connected. /etc/xen/scripts/block failed; error detected.

    stdout: Using config file “/OVS/Repositories/0004fb000003000009747581081557e5/VirtualMachines/0004fb0000060000cfab2af11ac1f125/vm.cfg”.

    Mon May 21 23:28:12 EEST 2012

    at com.oracle.ovm.mgr.action.ActionEngine.sendAction(ActionEngine.java:798)

    at com.oracle.ovm.mgr.action.ActionEngine.sendCommandToServer(ActionEngine.java:503)

    … 42 more

    FailedOperationCleanup

    ———-

    Starting failed operation ‘Virtual Machine Start’ cleanup on object ‘racnode1’

    Complete rollback operation ‘Virtual Machine Start’ completed with direction=racnode1

    Rollbacker

    ———-

    Executing rollback operation ‘Bridge Configure Operation’ on object ‘0004fb0000200000604c446c90acae98 (network.EthernetPort (3) in vmsrv33.gj.com)’

    Complete rollback operation ‘Bridge Configure Operation’ completed with direction=DONE

    Executing rollback operation ‘Virtual Machine Start’ on object ‘0004fb0000060000cfab2af11ac1f125 (racnode1)’

    Complete rollback operation ‘Virtual Machine Start’ completed with direction=DONE

    Executing rollback operation ‘Bridge Configure Operation’ on object ‘0004fb0000200000129d832619e996f5 (network.EthernetPort (4) in vmsrv33.gj.com)’

    Complete rollback operation ‘Bridge Configure Operation’ completed with direction=DONE

    Objects To Be Rolled Back

    ————————-

    Object (IN_USE): [EthernetPort] 0004fb0000200000604c446c90acae98 (network.EthernetPort (3) in vmsrv33.gj.com)

    Object (IN_USE): [Vnic] 0004fb000007000047de4cdfa34b0a41 (00:21:f6:00:00:1a)

    Object (IN_USE): [Vnic] 0004fb000007000020f4233448d8d770 (00:21:f6:00:00:05)

    Object (IN_USE): [VmDiskMapping] 0004fb00001300006e3c1bc79509f857

    Object (IN_USE): [VmDiskMapping] 0004fb0000130000ce67b1e5241157d7

    Object (IN_USE): [VirtualDisk] 0004fb00001200008d3f9019fe32a304.img (ASM3)

    Object (IN_USE): [VmDiskMapping] 0004fb0000130000494d049341598540

    Object (IN_USE): [VirtualDisk] 0004fb0000120000c65d6d0a62ec75a0.img (ASM5)

    Object (IN_USE): [VmDiskMapping] 0004fb0000130000267c771177271b00

    Object (IN_USE): [VirtualMachine] 0004fb0000060000cfab2af11ac1f125 (racnode1)

    Object (IN_USE): [VirtualDisk] 0004fb0000120000cdf8b3cd91c68ebd.img (ASM1)

    Object (IN_USE): [VmDiskMapping] 0004fb00001300002036b8246f86e2c4

    Object (IN_USE): [VirtualDisk] 0004fb000012000046a344c5efa19a02.img (ASM4)

    Object (IN_USE): [VirtualDisk] 0004fb000012000065511085c4ad6864.img (Oracle11203RAC_x86_64-xvdb.img (2))

    Object (IN_USE): [EthernetPort] 0004fb0000200000129d832619e996f5 (network.EthernetPort (4) in vmsrv33.gj.com)

    Object (IN_USE): [VirtualDisk] 0004fb0000120000fb986a9247406641.img (ASM2)

    Object (IN_USE): [VmDiskMapping] 0004fb00001300007b4b45d8dab8f783

    Object (IN_USE): [VirtualDisk] 0004fb0000120000a3949c3f3d394b21.img (System.img (2))

    Object (IN_USE): [VmDiskMapping] 0004fb000013000040f7df759023d559

    Write Methods Invoked

    ——————-

    Class=InternalJobDbImpl vessel_id=10367 method=addTransactionIdentifier accessLevel=6

    Class=VirtualMachineDbImpl vessel_id=9317 method=start accessLevel=6

    Class=VnicDbImpl vessel_id=8010 method=lock accessLevel=6

    Class=VnicDbImpl vessel_id=7888 method=lock accessLevel=6

    Class=VirtualDiskDbImpl vessel_id=9329 method=lock accessLevel=6

    Class=VirtualDiskDbImpl vessel_id=9339 method=lock accessLevel=6

    Class=VirtualDiskDbImpl vessel_id=7354 method=lock accessLevel=6

    Class=VirtualDiskDbImpl vessel_id=7404 method=lock accessLevel=6

    Class=VirtualDiskDbImpl vessel_id=7529 method=lock accessLevel=6

    Class=VirtualDiskDbImpl vessel_id=7571 method=lock accessLevel=6

    Class=VirtualDiskDbImpl vessel_id=7590 method=lock accessLevel=6

    Class=VnicDbImpl vessel_id=8010 method=lock accessLevel=6

    Class=VnicDbImpl vessel_id=7888 method=lock accessLevel=6

    Class=VmDiskMappingDbImpl vessel_id=9334 method=lock accessLevel=6

    Class=VmDiskMappingDbImpl vessel_id=9344 method=lock accessLevel=6

    Class=VmDiskMappingDbImpl vessel_id=9726 method=lock accessLevel=6

    Class=VmDiskMappingDbImpl vessel_id=9731 method=lock accessLevel=6

    Class=VmDiskMappingDbImpl vessel_id=9736 method=lock accessLevel=6

    Class=VmDiskMappingDbImpl vessel_id=9741 method=lock accessLevel=6

    Class=VmDiskMappingDbImpl vessel_id=9746 method=lock accessLevel=6

    Class=VirtualDiskDbImpl vessel_id=9329 method=lock accessLevel=6

    Class=VirtualDiskDbImpl vessel_id=9339 method=lock accessLevel=6

    Class=VirtualDiskDbImpl vessel_id=7354 method=lock accessLevel=6

    Class=VirtualDiskDbImpl vessel_id=7404 method=lock accessLevel=6

    Class=VirtualDiskDbImpl vessel_id=7529 method=lock accessLevel=6

    Class=VirtualDiskDbImpl vessel_id=7571 method=lock accessLevel=6

    Class=VirtualDiskDbImpl vessel_id=7590 method=lock accessLevel=6

    Class=VirtualMachineStartingEventDbImpl vessel_id=10376 method=notify accessLevel=6

    Class=VirtualMachineStartingEventDbImpl vessel_id=10376 method=setDescription accessLevel=6

    Class=EthernetPortDbImpl vessel_id=1888 method=addJobOperation accessLevel=6

    Class=EthernetPortDbImpl vessel_id=1882 method=addJobOperation accessLevel=6

    Class=InternalJobDbImpl vessel_id=10367 method=setCompletedStep accessLevel=6

    Class=InternalJobDbImpl vessel_id=10367 method=setAssociatedHandles accessLevel=6

    Class=EthernetPortDbImpl vessel_id=1882 method=setCurrentJobOperationComplete accessLevel=6

    Class=EthernetPortDbImpl vessel_id=1882 method=nextJobOperation accessLevel=6

    Class=InternalJobDbImpl vessel_id=10367 method=setTuringMachineFlag accessLevel=6

    Class=VirtualMachineDbImpl vessel_id=9317 method=setCurrentOperationToLater accessLevel=6

    Class=InternalJobDbImpl vessel_id=10367 method=setTuringMachineFlag accessLevel=6

    Class=EthernetPortDbImpl vessel_id=1888 method=setCurrentJobOperationComplete accessLevel=6

    Class=EthernetPortDbImpl vessel_id=1888 method=nextJobOperation accessLevel=6

    Class=VirtualMachineDbImpl vessel_id=9317 method=nextJobOperation accessLevel=6

    Class=InternalJobDbImpl vessel_id=10367 method=setFailedOperation accessLevel=6

    Class=EthernetPortDbImpl vessel_id=1882 method=nextJobOperation accessLevel=6

    Class=VnicDbImpl vessel_id=8010 method=nextJobOperation accessLevel=6

    Class=VnicDbImpl vessel_id=7888 method=nextJobOperation accessLevel=6

    Class=VmDiskMappingDbImpl vessel_id=9741 method=nextJobOperation accessLevel=6

    Class=VmDiskMappingDbImpl vessel_id=9344 method=nextJobOperation accessLevel=6

    Class=VirtualDiskDbImpl vessel_id=7529 method=nextJobOperation accessLevel=6

    Class=VmDiskMappingDbImpl vessel_id=9736 method=nextJobOperation accessLevel=6

    Class=VirtualDiskDbImpl vessel_id=7590 method=nextJobOperation accessLevel=6

    Class=VmDiskMappingDbImpl vessel_id=9726 method=nextJobOperation accessLevel=6

    Class=VirtualMachineDbImpl vessel_id=9317 method=nextJobOperation accessLevel=6

    Class=VirtualDiskDbImpl vessel_id=7354 method=nextJobOperation accessLevel=6

    Class=VmDiskMappingDbImpl vessel_id=9731 method=nextJobOperation accessLevel=6

    Class=VirtualDiskDbImpl vessel_id=7571 method=nextJobOperation accessLevel=6

    Class=VirtualDiskDbImpl vessel_id=9339 method=nextJobOperation accessLevel=6

    Class=EthernetPortDbImpl vessel_id=1888 method=nextJobOperation accessLevel=6

    Class=VirtualDiskDbImpl vessel_id=7404 method=nextJobOperation accessLevel=6

    Class=VmDiskMappingDbImpl vessel_id=9334 method=nextJobOperation accessLevel=6

    Class=VirtualDiskDbImpl vessel_id=9329 method=nextJobOperation accessLevel=6

    Class=VmDiskMappingDbImpl vessel_id=9746 method=nextJobOperation accessLevel=6

    Class=EthernetPortDbImpl vessel_id=1882 method=nextJobOperation accessLevel=6

    Class=VirtualMachineDbImpl vessel_id=9317 method=nextJobOperation accessLevel=6

    Class=EthernetPortDbImpl vessel_id=1888 method=nextJobOperation accessLevel=6

    Completed Step: ROLLBACK

    Job failed commit (internal) due to OVMAPI_4010E Attempt to send command: dispatch to server: vmsrv33.gj.com failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@192.168.2.21:8899/api/2 start_vm 0004fb000003000009747581081557e5 0004fb0000060000cfab2af11ac1f125, Status: org.apache.xmlrpc.XmlRpcException: exceptions.RuntimeError:Command: [‘xm’, ‘create’, ‘/OVS/Repositories/0004fb000003000009747581081557e5/VirtualMachines/0004fb0000060000cfab2af11ac1f125/vm.cfg’] failed (1): stderr: Error: Device 51744 (vbd) could not be connected. /etc/xen/scripts/block failed; error detected.

    stdout: Using config file “/OVS/Repositories/0004fb000003000009747581081557e5/VirtualMachines/0004fb0000060000cfab2af11ac1f125/vm.cfg”.

    Mon May 21 23:28:12 EEST 2012

    Mon May 21 23:28:12 EEST 2012

    com.oracle.ovm.mgr.api.exception.FailedOperationException: OVMAPI_4010E Attempt to send command: dispatch to server: vmsrv33.gj.com failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@192.168.2.21:8899/api/2 start_vm 0004fb000003000009747581081557e5 0004fb0000060000cfab2af11ac1f125, Status: org.apache.xmlrpc.XmlRpcException: exceptions.RuntimeError:Command: [‘xm’, ‘create’, ‘/OVS/Repositories/0004fb000003000009747581081557e5/VirtualMachines/0004fb0000060000cfab2af11ac1f125/vm.cfg’] failed (1): stderr: Error: Device 51744 (vbd) could not be connected. /etc/xen/scripts/block failed; error detected.

    stdout: Using config file “/OVS/Repositories/0004fb000003000009747581081557e5/VirtualMachines/0004fb0000060000cfab2af11ac1f125/vm.cfg”.

    Mon May 21 23:28:12 EEST 2012

    Mon May 21 23:28:12 EEST 2012

    at com.oracle.ovm.mgr.action.ActionEngine.sendCommandToServer(ActionEngine.java:507)

    at com.oracle.ovm.mgr.action.ActionEngine.sendDispatchedServerCommand(ActionEngine.java:444)

    at com.oracle.ovm.mgr.action.ActionEngine.sendServerCommand(ActionEngine.java:378)

    at com.oracle.ovm.mgr.action.VirtualMachineAction.sendStartVmCommand(VirtualMachineAction.java:121)

    at com.oracle.ovm.mgr.op.virtual.VirtualMachineStart.asyncAction(VirtualMachineStart.java:65)

    at com.oracle.ovm.mgr.api.collectable.ManagedObjectDbImpl.executeCurrentJobOperationAsyncAction(ManagedObjectDbImpl.java:999)

    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

    at java.lang.reflect.Method.invoke(Method.java:597)

    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330)

    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)

    at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:822)

    at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:245)

    at com.oracle.ovm.mgr.api.virtual.VirtualMachineProxy.executeCurrentJobOperationAsyncAction(Unknown Source)

    at com.oracle.ovm.mgr.api.job.JobEngine.objectAsyncStart(JobEngine.java:596)

    at com.oracle.ovm.mgr.op.virtual.VirtualMachineStart.action(VirtualMachineStart.java:52)

    at com.oracle.ovm.mgr.api.collectable.ManagedObjectDbImpl.executeCurrentJobOperationAction(ManagedObjectDbImpl.java:1009)

    at sun.reflect.GeneratedMethodAccessor886.invoke(Unknown Source)

    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

    at java.lang.reflect.Method.invoke(Method.java:597)

    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330)

    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)

    at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:822)

    at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:245)

    at com.oracle.ovm.mgr.api.virtual.VirtualMachineProxy.executeCurrentJobOperationAction(Unknown Source)

    at com.oracle.ovm.mgr.api.job.JobEngine.operationActioner(JobEngine.java:218)

    at com.oracle.ovm.mgr.api.job.JobEngine.objectActioner(JobEngine.java:309)

    at com.oracle.ovm.mgr.api.job.InternalJobDbImpl.objectCommitter(InternalJobDbImpl.java:1140)

    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

    at java.lang.reflect.Method.invoke(Method.java:597)

    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330)

    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)

    at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:136)

    at com.oracle.odof.command.InvokeMethodCommand.process(InvokeMethodCommand.java:100)

    at com.oracle.odof.core.BasicWork.processCommand(BasicWork.java:81)

    at com.oracle.odof.core.TransactionManager.processCommand(TransactionManager.java:773)

    at com.oracle.odof.core.WorkflowManager.processCommand(WorkflowManager.java:401)

    at com.oracle.odof.core.WorkflowManager.processWork(WorkflowManager.java:459)

    at com.oracle.odof.io.AbstractClient.run(AbstractClient.java:42)

    at java.lang.Thread.run(Thread.java:662)

    Caused by: com.oracle.ovm.mgr.api.exception.IllegalOperationException: OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@192.168.2.21:8899/api/2 start_vm 0004fb000003000009747581081557e5 0004fb0000060000cfab2af11ac1f125, Status: org.apache.xmlrpc.XmlRpcException: exceptions.RuntimeError:Command: [‘xm’, ‘create’, ‘/OVS/Repositories/0004fb000003000009747581081557e5/VirtualMachines/0004fb0000060000cfab2af11ac1f125/vm.cfg’] failed (1): stderr: Error: Device 51744 (vbd) could not be connected. /etc/xen/scripts/block failed; error detected.

    stdout: Using config file “/OVS/Repositories/0004fb000003000009747581081557e5/VirtualMachines/0004fb0000060000cfab2af11ac1f125/vm.cfg”.

    Mon May 21 23:28:12 EEST 2012

    at com.oracle.ovm.mgr.action.ActionEngine.sendAction(ActionEngine.java:798)

    at com.oracle.ovm.mgr.action.ActionEngine.sendCommandToServer(ActionEngine.java:503)

    … 42 more

    ———-

    End of Job

    ———-

  1. Conclusion

You had a look at the steps to upgrade Oracle VM server and Oracle VM Manager to version 3.1.1. You looked at the step by step guide to install two node Oracle 11gR2 11.2.0.3 RAC using templates. The errors that I encountered were also included as well as A way to work it around.

 

Download Oracle Virtualization – Installing Oracle VM Server 3.1.1, Oracle VM Manager 3.1.2 and Deploying Oracle RAC 11gR2 (11.2.0.3) Oracle VM templates Linux x86 64 bit for test configuration

June 20, 2012 - Posted by | oracle

20 Comments »

  1. Thanks for your efforts. If, I have a hard disk which is directly attached to the OVM server’s OS, how to deal with that? The OVM Manager is not able to pick up the hard disk partition Eg. /dev/sda4

    Comment by Praveen | August 24, 2012 | Reply

  2. Thanks for the article. But I have a FC SAN as the storage. I found it impossible to create the server pool in VM Manager since it always says that job is time-out. I searched all the docs on the net but found nothing helpful. Any suggestions? Thanks.

    Comment by Michael | September 4, 2012 | Reply

    • Somebody suggests me to wipe out all the data on the storage. I’ve used dd command to write the first 100M of the SAN to zero. But it’s in vain. The same error showed up again. To my surprise, Oracle official documentation on this is unclear and vague. I can’t believe that Oracle VM, as Oracle’s the main virtualization product for x86, is so fragile.

      Comment by Michael | September 4, 2012 | Reply

    • Hi,

      Look at the error and work it out. I cannot understand what is the problem that you are facing other than that the Server pool creation times out.

      Could you bounce the VM manager and give it a try? Make sure that you follow the ordered steps, do not miss a step, and each step completes succesfully before moving to the nest one. Did you synchronise the NFS and repository configuration? Try to get something that provides an indication of a problem…

      Regards,

      Comment by gjilevski | September 4, 2012 | Reply

  3. @Praveen,
    What you should do is:
    login to your OVM server as root and use parted command to look at the unused disk.
    So, for example, to carve out ~50GB space and make it available to the VM Manager:
    [root@ovm1 dev]# parted
    GNU Parted 1.8.1
    Using /dev/sda
    Welcome to GNU Parted! Type ‘help’ to view a list of commands.
    (parted) print free

    Model: HITACHI H101414SCSUN146G (scsi)
    Disk /dev/sda: 147GB
    Sector size (logical/physical): 512B/512B
    Partition Table: msdos

    Number Start End Size Type File system Flags
    1 32.3kB 107MB 107MB primary ext3 boot
    2 107MB 1184MB 1078MB primary linux-swap
    1184MB 147GB 146GB Free Space

    (parted) mkpart primary 1184MB 50GB
    (parted) print free

    Model: HITACHI H101414SCSUN146G (scsi)
    Disk /dev/sda: 147GB
    Sector size (logical/physical): 512B/512B
    Partition Table: msdos

    Number Start End Size Type File system Flags
    1 32.3kB 107MB 107MB primary ext3 boot
    2 107MB 1184MB 1078MB primary linux-swap
    3 1184MB 50.0GB 48.8GB primary
    50.0GB 147GB 96.8GB Free Space

    Now, you will see a /dev/sda3 device.
    Just create a fs on it:

    [root@ovm1 ~]# ll /dev/sd*
    brw-r—– 1 root disk 8, 0 Sep 6 16:41 /dev/sda
    brw-r—– 1 root disk 8, 1 Sep 6 16:27 /dev/sda1
    brw-r—– 1 root disk 8, 2 Sep 6 16:27 /dev/sda2
    brw-r—– 1 root disk 8, 3 Sep 6 16:41 /dev/sda3
    brw-r—– 1 root disk 8, 16 Sep 6 16:27 /dev/sdb
    brw-r—– 1 root disk 8, 17 Sep 6 16:27 /dev/sdb1
    brw-r—– 1 root disk 8, 18 Sep 6 16:27 /dev/sdb2
    [root@ovm1 ~]#
    [root@ovm1 ~]#
    [root@ovm1 ~]# mkfs -t ext3 /dev/sda3
    mke2fs 1.39 (29-May-2006)
    Filesystem label=
    OS type: Linux
    Block size=4096 (log=2)
    Fragment size=4096 (log=2)
    5963776 inodes, 11918221 blocks
    595911 blocks (5.00%) reserved for the super user
    First data block=0
    Maximum filesystem blocks=4294967296
    364 block groups
    32768 blocks per group, 32768 fragments per group
    16384 inodes per group
    Superblock backups stored on blocks:
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
    4096000, 7962624, 11239424

    Writing inode tables: done
    Creating journal (32768 blocks): done
    Writing superblocks and filesystem accounting information: done

    This filesystem will be automatically checked every 39 mounts or
    180 days, whichever comes first. Use tune2fs -c or -i to override.
    [root@ovm1 ~]# mkdir /1st_50G
    [root@ovm1 ~]# mount /dev/sda3 /1st_50G
    [root@ovm1 ~]# df -h
    Filesystem Size Used Avail Use% Mounted on
    /dev/sdb1 3.0G 635M 2.2G 23% /
    /dev/sda1 99M 28M 67M 30% /boot
    tmpfs 360M 0 360M 0% /dev/shm
    none 360M 40K 360M 1% /var/lib/xenstored
    /dev/sda3 45G 181M 43G 1% /1st_50G
    [root@ovm1 ~]#

    Add to /etc/fstab:
    /dev/sda3 /1st_50G ext3 defaults 0 0

    Now, you can export it as NFS share as Guenadi has shown above:

    [root@ovm1 ~]# cat /etc/exports
    /ist_50G *(rw,sync,no_wdelay,insecure_locks,no_root_squash)

    [root@ovm1 ~]# service nfs status
    rpc.mountd (pid 3699) is running…
    nfsd (pid 3696 3695 3694 3693 3692 3691 3690 3689) is running…
    [root@ovm1 ~]# service nfs restart
    Shutting down NFS mountd: [ OK ]
    Shutting down NFS daemon: [ OK ]
    Shutting down NFS services: [ OK ]
    Starting NFS services: [ OK ]
    Starting NFS daemon: [ OK ]
    Starting NFS mountd: [ OK ]
    [root@ovm1 ~]#

    Now, you can use this 50G space from your OVM Manager console.
    Hope that helps.

    Comment by jcnars | September 6, 2012 | Reply

    • Thanks Guenadi for taking the time to list the steps. I got it working(finally) with the local hard disk by keeping it as “raw” – i.e. initialization from BIOS but NO initialization from the OS and file system of these hard disks. This works – not THE solution to be used in production env. though. I will keep your NFS suggestion if need be for future additional disks.
      Thanks again.

      Regards,
      Praveen

      Comment by Praveen | September 7, 2012 | Reply

      • @Praveen,
        Thanks. Could you be more specific as to the exact details? I didn’t quite follow how you made it to work.

        Maybe a blog entry? I am sure there’s a lot of interest in the solution of using up the local disks work (for TEST/DEV OVM setup) and yours could be a useful one.

        Regards
        jcnars

        Comment by jcnars | September 7, 2012

      • Hi,

        Hi, Could you please elaborate further on the steps that you did as I believe readers can be interested in

        Regards,

        Comment by gjilevski | September 7, 2012

      • Happy to do this, when I can do something from scratch so that I could get the right screen-shots. Not sure how soon but. Thanks for your interest.

        -Praveen

        On Sat, Sep 8, 2012 at 5:38 AM, Guenadi N Jilevski’s Oracle BLOG wrote:

        > ** > gjilevski commented: “Hi, Could you please elaborate further on he > steps that you did as I belive reades can be interested in. Regards,” >

        Comment by Praveen Rajendra Babu | September 13, 2012

      • Happy to do this, when I can do something from scratch so that I could get the right screen-shots. Not sure how soon but.
        Thanks for your interest.
        If you have clean slate hard-disks, I could spare an hour for a remote session for Aus.(Melbourne) time. My config. is 4 separate hard disks. Kept 1 disk separate and merged the other 3 as RAID. The 1 separate hard disk for the OVM server installation and the other for the OVM repository.

        -Praveen

        Comment by Praveen | September 13, 2012

      • Hi,

        Do not worry. If you can point it out is fine else no biggie.

        Regards,

        Comment by gjilevski | September 13, 2012

  4. could you please point out the Oracle VM Manager Installation Instructions and filesystems layout
    thanks in advance

    Comment by samjaz | September 16, 2012 | Reply

  5. Reblogged this on Apps DBA notes.

    Comment by Mohammed | September 26, 2012 | Reply

  6. Hi Guenadi – thanks for the info on OVM3.1 – quite different to OVM 2 – nice note.

    Regards,
    Gavin

    Comment by Gavin | October 17, 2012 | Reply

  7. Hi gjilevski,

    Great post! Very useful information.

    Few question tos ask:

    1. do you need to build the Virtual Machine racnode1 and racnode2 on VirtualBox like you did for svr33 & vmmgr?
    2. how do you logged into racnode1 and racnode2?

    3. Have you tried the newest deploycluster tool with OVM 3.1.1? (http://www.oracle.com/technetwork/products/clusterware/overview/racovm-deploycluster-tool-1635519.pdf)
    According to the Doc, I can skip your step 9 “Start the VMS”. But not sure!

    Possible to update this post with the newest deploycluster tool?.

    Best Regards,

    Comment by jlong | November 8, 2012 | Reply

    • Hi,

      1. No. I build using VirtualBox only vmmgr and vmsrv33. The virtual servers raclinux1 and raclinux2 are built on vmsrv33 as a result of creating virtual machines using the RAC templates from VM Manager.

      2. Look in the post for establishing a vnc connection. I did it from the VM Manager. You could try using vncviewer as well. It is the same but invoke it on the command line instead of from the VMManager.

      3. The tool was not available when I wrote the article. I can give it a try. It should simplify the setup. I did not use it. I can update the article when get a chance to play with the tool.

      Regards,

      Comment by gjilevski | November 8, 2012 | Reply

  8. Hi, In step4 you created MYRAC network. So you have two networks – the management network and MYRAC, which you did not apply an IP address to.
    In step 7 you suddenly have xenbr0 and xenbr1 – where did they come from ? – Do you need to have extra NICs configured in the Virtualbox VM before these networks can be created?

    Comment by dodgydba | May 12, 2013 | Reply

  9. I have two questions regarding the disk space and network.

    1. How much disk space is needed for the share disks?

    2. In Step 6 – Create Virtual Disks, it took me 15-30mins to create a single node. I checked what was the resources used, apparently the network was the main heavy activity. System Monitor showed the network packet sent was hovering around 20MB+. I am not sure if we can improve the speed?

    Thank you.

    Comment by Des | July 17, 2013 | Reply

    • Hi,

      1. Look at the article for the disk sizes I used. Your disk sizes could be different. The ASM disk should be >=2GB to get it started.

      The repository (stage) area is 50GB and each ASMx disk is 2GB in the article.

      2. Look at your configuration for the network. Are you tesing on a bare metal or using any virtualization software?

      Best Regards,

      Comment by gjilevski | July 21, 2013 | Reply


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: