Oracle Virtualization – Installing Oracle VM Server 3.1.1, Oracle VM Manager 3.1.1 and Deploying Oracle RAC 11gR2 (11.2.0.3) Oracle VM templates Linux x86 64 bit for test configuration
Oracle Virtualization – Installing Oracle VM Server 3.1.1, Oracle VM Manager 3.1.1 and Deploying Oracle RAC 11gR2 (11.2.0.3) Oracle VM templates Linux x86 64 bit for test configuration
In the second post, first is here, for Oracle VM 3.1.1 technology you will look at how to upgrade the existing Oracle VM server and Oracle VM manager to version 3.1.1 and deploy Oracle RAC templates to build two node Oracle 11gR2 (11.2.0.3) cluster.
All software can be downloaded from edelivery.oracle.com (Oracle Software Delivery Cloud) and the templates (version 11.2.0.3) can be downloaded as a patch from MOS (patch 12694014. The approach described in the article is applicable on any supported platform although I am using Oracle VirtualBox for testing.
Software versions used are:
- Oracle VirtualBox 4.1.16
- Oracle VM Server 3.1.1
- Oracle VM Manager 3.1.1
- OEL 6.2
- Oracle RAC templates (patch 12694014)
The article will cover the following:
- VirtualBox configuration for VM Server machine and VMManager machine
- Installation of Oracle VM Server
- Installation of Oracle VM Manager
- Deployment of Oracle RAC 11gR2 Oracle VM templates Linux x86 64bit for test configuration
In real life Oracle VM server runs on a dedicated set of servers and is installed as a bare metal product without the need of an Operating system. Oracle VM Manager is installed on a Linux box. Oracle VirtualBox can be successfully used for testing and self-study of the new features.
The main components in the architecture are two virtual machines.
Virtual Machine | IP Address | OS | Purpose |
OracleVMServer303 | 192.168.2.21 | OEL XEN with Oracle VM Server 3.0.3 | Oracle VM Server 3.0.3 |
OracleVMManager303 | 192.168.2.31 | OEL 6.2 | Oracle VM Manager 3.0.3 |
Product documentation for Oracle VM server 3.1.1 and Oracle VM Manager 3.1.1 is available here.
-
Upgrade Oracle VM server 3.0.3 to Oracle VM server 3.1.1
Make the VMserver311 ISO available to the virtual machine OracleVMserver303 and start the VM. Once the boot screen appears press Enter.
Press Skip.
Select keyboard.
Accept the license agreement.
Select an UPGRADE option and the Oracle VM server 3.0.3 to upgrade.
Upgrade the boot loader.
Take a note of the log file location and press OK.
Wait for the upgrade to continue.
Reboot.
Press the highlighted selection.
Wait for the boot to complete.
Press Alt-F2 to login
Login
-
Upgrade Oracle VM Manager 3.0.3 to Oracle VM Manager 3.1.1
Transfer to OracleVMManager303 virtual machine OracleVMManager311.iso image a loop mount it.
[root@vmmgr u01]# ls
app OracleVMManager311.iso r.txt stage t.txt VMManager303.iso
[root@vmmgr u01]# mount -o loop OracleVMManager311.iso /mnt/stage
[root@vmmgr u01]#
From the stage directory invoke the upgrade script.
[root@vmmgr stage]# ./runUpgrader.sh
Stating OVM Manager upgrade on Sun May 20 16:28:43 EEST 2012
Oracle VM Manager 3.1.1.305 upgrade utility
Upgrade logfile : /tmp/upgrade-2012-05-20-28.log
It is highly recommended to do a full database repository backup prior to upgrading Oracle VM Manager …
Press any key to continue …
Oracle VM Manager is running …
Verifying installation status …
Read Oracle VM Manager config file …
Found Oracle VM Manager install files …
Found Oracle VM Manager upgrader …
Found Oracle WebLogic Server …
Found Java …
Using the following information :
Database Host : localhost
Database SID : XE
Database LSNR : 1521
Oracle VM Schema : ovs
Oracle VM Manager UUID : 0004fb0000010000f8a01f5a1718ffd9
Current Build ID : 3.0.3.126
Upgrade from version : 3.0.3
Upgrade to version : 3.1.1
Using /tmp/workdir.0zgO6hQ9um for backup and export location.
Using /tmp/patchdir.Glh0EQQcD for patching.
Enter password for user ovs :
Undeploying previous version of Oracle VM Manager application …
Undeploying Oracle VM Manager help …
Undeploying Oracle VM Manager console …
Undeploying Oracle VM Manager core …
Waiting for Oracle VM Manager core to fully undeploy…
Waiting…
Finished undeploying previous version …
Exporting Oracle VM Manager repository …
Please wait as this can take a long time …
Oracle VM Manager repository export completed …
Creating backup file …
Oracle VM Manager repository backup in /tmp/ovm-manager-3-backup-2012-05-20.zip
Upgrading Oracle VM Manager …
Backing up old files to /tmp/ovm-manager-3-backup-2012-05-20-163052…
Removing old files …
Unpacking Oracle VM Manager 3.1.1.305
`transform_003001001000_010.xsl’ -> `/tmp/patchdir.Glh0EQQcD/transform_003001001000_010.xsl’
`transform_003001001000_020.xsl’ -> `/tmp/patchdir.Glh0EQQcD/transform_003001001000_020.xsl’
`deletedClasses.xml’ -> `/tmp/patchdir.Glh0EQQcD/deletedClasses.xml’
Filtering full repository export to the selective export subset at /tmp/workdir_sel.XRjFLORxBo …
cp: omitting directory `/tmp/workdir.0zgO6hQ9um/jrnl’
cp: omitting directory `/tmp/workdir.0zgO6hQ9um/objs’
adding: objs/63/639.cl.xml (deflated 81%)
adding: objs/9.cl.xml (deflated 93%)
adding: objs/16/161.cl.xml (deflated 67%)
adding: objs/19/193.cl.xml (deflated 70%)
Selective export is at /tmp/workdir_sel.XRjFLORxBo
4 objects selected (out of 508) to be upgraded
Transform XSL files used:
-rw-r–r–. 1 root root 56678 May 20 16:30 /tmp/patchdir.Glh0EQQcD/transform_003001001000_010.xsl
-rw-r–r–. 1 root root 10079 May 20 16:30 /tmp/patchdir.Glh0EQQcD/transform_003001001000_020.xsl
Changed classes encountered in selective export set:
com.oracle.ovm.mgr.api.manager.BusinessManagerDbImpl
com.oracle.ovm.mgr.api.manager.ModelManagerDbImpl
com.oracle.ovm.mgr.api.manager.RasManagerDbImpl
com.oracle.ovm.mgr.api.physical.ServerDbImpl
Upgrading Oracle VM Manager repository …
Please wait as this can take a long time …
Oracle VM Manager repository upgrade completed …
Validating Oracle VM Manager repository …
Oracle VM Manager repository validation completed …
Refresh system-jazn-data.xml file …
Redeploying Oracle VM Manager core container …
Redeploying Oracle VM Manager console …
Redeploying Oracle VM Manager help …
Install ADF Patch …
Completed upgrade to 3.1.1.305 …
Writing updated config in /u01/app/oracle/ovm-manager-3/.config
Restart WebLogic …
Stopping Oracle VM Manager [ OK ]
Starting Oracle VM Managernohup: ignoring input and redirecting stderr to stdout
[ OK ]
OVM Manager upgrade finished on Sun May 20 16:35:47 EEST 2012
[root@vmmgr stage]#
Let’s test Oracle VM Manager 3.1.1
After login you have.
Starting and Stopping the OVM from OEL.
[root@vmmgr ~]# service ovmm stop
Stopping Oracle VM Manager
[ OK ]
[root@vmmgr ~]#
[root@vmmgr ~]# service ovmm start
Starting Oracle VM Managernohup: ignoring input and redirecting stderr to stdout
[ OK ]
[root@vmmgr ~]#
-
Deploy two node Oracle 11gR2 (11.2.0.3) RAC
-
Add a file system /nfs that is to NFS mounted.
Add 50GB disk to the OracleVMManager303 virtual machine. This disk is visible as /dev/sdb.
Create a volume, file system and mount the file system on this disk.
[root@vmmgr ~]# fdisk -l /dev/sdb
Disk /dev/sdb: 53.7 GB, 53687091200 bytes
255 heads, 63 sectors/track, 6527 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/sdb doesn’t contain a valid partition table
[root@vmmgr ~]#
[root@vmmgr ~]# pvcreate /dev/sdb
Writing physical volume data to disk “/dev/sdb”
Physical volume “/dev/sdb” successfully created
[root@vmmgr ~]#
[root@vmmgr ~]# fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0xb618dbd7.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won’t be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
WARNING: DOS-compatible mode is deprecated. It’s strongly recommended to
switch off the mode (command ‘c’) and change display units to
sectors (command ‘u’).
Command (m for help): c
DOS Compatibility flag is not set
Command (m for help): u
Changing display/entry units to sectors
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First sector (2048-104857599, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-104857599, default 104857599): +50G
Value out of range.
Last sector, +sectors or +size{K,M,G} (2048-104857599, default 104857599): +49G
Command (m for help): t
Selected partition 1
Hex code (type L to list codes): 8e
Changed system type of partition 1 to 8e (Linux LVM)
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
[root@vmmgr ~]#
[root@vmmgr ~]# fdisk -l /dev/sdb
Disk /dev/sdb: 53.7 GB, 53687091200 bytes
171 heads, 46 sectors/track, 13330 cylinders
Units = cylinders of 7866 * 512 = 4027392 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xb618dbd7
Device Boot Start End Blocks Id System
/dev/sdb1 1 13065 51380224 8e Linux LVM
[root@vmmgr ~]#
[root@vmmgr ~]# pvcreate /dev/sdb1
Writing physical volume data to disk “/dev/sdb1”
Physical volume “/dev/sdb1” successfully created
[root@vmmgr ~]#
vgcreate NFSVolGroup /dev/sdb1
[root@vmmgr ~]# vgcreate NFSVolGroup /dev/sdb1
Volume group “NFSVolGroup” successfully created
[root@vmmgr ~]#
[root@vmmgr ~]# vgdisplay NFSVolGroup
— Volume group —
VG Name NFSVolGroup
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 1
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 0
Open LV 0
Max PV 0
Cur PV 1
Act PV 1
VG Size 49.00 GiB
PE Size 4.00 MiB
Total PE 12543
Alloc PE / Size 0 / 0
Free PE / Size 12543 / 49.00 GiB
VG UUID RnSh8A-fvtJ-97hA-KyX3-Lizm-Dpat-sDcWM9
[root@vmmgr ~]#
lvcreate –size 48G NFSVolGroup
[root@vmmgr ~]# lvcreate –size 48G NFSVolGroup
Logical volume “lvol0” created
[root@vmmgr ~]#
[root@vmmgr ~]# lvdisplay
— Logical volume —
LV Name /dev/NFSVolGroup/lvol0
VG Name NFSVolGroup
LV UUID 0M6eKi-2uiG-Chjl-rk6E-8BUc-WngI-SvvZpl
LV Write Access read/write
LV Status available
# open 0
LV Size 48.00 GiB
Current LE 12288
Segments 1
Allocation inherit
Read ahead sectors auto
– currently set to 256
Block device 253:3
— Logical volume —
LV Name /dev/vg_vmmgr/lv_root
VG Name vg_vmmgr
LV UUID kpXFYY-eAPW-zCXW-wHnN-QuKw-kFxh-vjeaRX
LV Write Access read/write
LV Status available
# open 1
LV Size 28.37 GiB
Current LE 7263
Segments 1
Allocation inherit
Read ahead sectors auto
– currently set to 256
Block device 253:0
— Logical volume —
LV Name /dev/vg_vmmgr/lv_home
VG Name vg_vmmgr
LV UUID gaY4wY-aN48-SemS-iUps-L8CX-w27t-OQqPdF
LV Write Access read/write
LV Status available
# open 1
LV Size 25.26 GiB
Current LE 6467
Segments 1
Allocation inherit
Read ahead sectors auto
– currently set to 256
Block device 253:2
— Logical volume —
LV Name /dev/vg_vmmgr/lv_swap
VG Name vg_vmmgr
LV UUID f3L33X-DIly-6iO3-hGgu-DrWM-0g1O-mACi0e
LV Write Access read/write
LV Status available
# open 1
LV Size 5.88 GiB
Current LE 1504
Segments 1
Allocation inherit
Read ahead sectors auto
– currently set to 256
Block device 253:1
[root@vmmgr ~]#
/dev/NFSVolGroup/lvol0
[root@vmmgr ~]# mkfs.ext3 /dev/NFSVolGroup/lvol0
mke2fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
3145728 inodes, 12582912 blocks
629145 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
384 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information:
done
This filesystem will be automatically checked every 21 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
[root@vmmgr ~]#
[root@vmmgr ~]#
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/vg_vmmgr-lv_root / ext4 defaults 1 1
UUID=19fcf4b8-c245-4f71-bd73-5bf56c6dada5 /boot ext4 defaults 1 2
/dev/mapper/vg_vmmgr-lv_home /home ext4 defaults 1 2
/dev/mapper/NFSVolGroup-lvol0 /nfs ext4 defaults 1 2
/dev/mapper/vg_vmmgr-lv_swap swap swap defaults 0 0
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
[root@vmmgr mapper]#
-
Create NFS
Modify /etc/exports as follows.
[root@vmmgr ~]# cat /etc/export
cat: /etc/export: No such file or directory
[root@vmmgr ~]# cat /etc/exports
/nfs/stage *(rw,sync,no_wdelay,insecure_locks,no_root_squash)
/nfs/stage1 *(rw,sync,no_wdelay,insecure_locks,no_root_squash)
[root@vmmgr ~]#
Create /nfs/stage and /nfs/stage1
Enable and start nfs service
[root@vmmgr stage]# chkconfig nfs on
[root@vmmgr stage]# service nfs restart
Shutting down NFS mountd: [FAILED]
Shutting down NFS daemon: [FAILED]
Shutting down NFS quotas: [FAILED]
Starting NFS services: [ OK ]
Starting NFS quotas: [ OK ]
Starting NFS daemon: [ OK ]
Starting NFS mountd: [ OK ]
Starting and Stopping the OVM
[root@vmmgr ~]# service ovmm stop
Stopping Oracle VM Manager
[ OK ]
[root@vmmgr ~]#
[root@vmmgr ~]# service ovmm start
Starting Oracle VM Managernohup: ignoring input and redirecting stderr to stdout
[ OK ]
[root@vmmgr ~]#
-
Discover a VM server from the Oracle VM Manager 3.1.1
Click discover servers in the tool bar.
Wait until the server get discovered.
-
Create a network
Click the Networking tab, then the Networks subtab. Click Create New Network + to start.
Select ‘Create a network with bonds/ports only. Press Next.
Specify Name.
Select servers.
Select ports. Make sure ONLY eth0 is selected. Picture is for illustration and should have only eth0.
Configure IP addresses. Select None and press Finish.
After the network is created it will be displayed.
-
Create a file server
On the storage tab select discover file servers and press Next once filled in the data.
Select server.
Select the nfs from /etc/exports. Here should show one of it and press Finish. (Instead of nfs:/u01/VM/nfs should be nfs:/nfs/stage or nfs:/nfs/stage1) /nfs/stage is for the file system and /nfs/stage1 is for the repository.
You will get something like
-
-
Create a Server pool
From Servers and VMs click Create Server Pool. Press Next.
Select the server press Finish.
You will see the nfs created earlier. Press OK.
Examine the Pool successfully created.
-
Create Storage Repository.
Make sure that storage is refreshed. If not refresh from file servers.
Click Create Repository.
Enter Repository name. Press the magnifier to select a nfs location.
Press OK and press Next.
Select server.
Verify the result.
-
Import the templates.
Templates are not anymore stored directly onto the Oracle VM server 3.1.1. Instead Oracle has provided a way to access them using either ftp or http.
I will use ftp. So I will configure the ftp service.
[root@vmmgr stage]# service vsftpd start
Starting vsftpd for vsftpd: [ OK ]
[root@vmmgr stage]# chkconfig vsftpd on
[root@vmmgr stage]#
I am using OS user gjilevski on the OEL6.2 linux VM running the Oracle VM Manager 3.1.1 with home directory /home/gjilevski.
Copy the templates into /home/gjilevski. When specifying the templates make sure that you list second before the first template.
ftp://gjilevski:password@192.168.2.31/OVM_OL5U7_X86_64_11203RAC_PVM-2of2.tgz
ftp://gjilevski:password@192.168.2.31/OVM_OL5U7_X86_64_11203RAC_PVM-1of2.tgz
In the URL gjilevski is the OS user identified by the password. File names follows without a path as they are in the default user directory.
From VM templates import the templates.
In the screen that appear enter
ftp://gjilevski:password@192.168.2.31/OVM_OL5U7_X86_64_11203RAC_PVM-2of2.tgz
ftp://gjilevski:password@192.168.2.31/OVM_OL5U7_X86_64_11203RAC_PVM-1of2.tgz
instead of what is displayed.
Wait for the import to completes.
It will looks like this.
If you edit it properties will be as follows.
-
Create shared virtual disk
Installing RAC requires 5 shared disks each 2GB. From the screen click create virtual disk.
Create ASM1
Create ASM2
Create ASM3
Repeat same steps for ASM4 and ASM5. Verify that all are created.
-
Create VNIC
From Networking tab click virtual NICs.
Click Auto Fill and Create.
Initially I did not created VNIC and got the error.
begin()
Appended operation ‘Adding a queued job operation’ to object ‘0004fb00001900008ea92f8eb1b6abc2 (Create Statistics Log)’.
com.oracle.odof.exception.ObjectException: Caught during invoke method: com.oracle.ovm.mgr.api.exception.IllegalOperationException: OVMAPI_8004E There are no more VNICs available. Create More MAC Addresses.
Mon May 21 17:33:13 EEST 2012
at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:151)
at com.oracle.odof.command.InvokeMethodCommand.process(InvokeMethodCommand.java:100)
at com.oracle.odof.core.BasicWork.processCommand(BasicWork.java:81)
at com.oracle.odof.core.TransactionManager.processCommand(TransactionManager.java:773)
at com.oracle.odof.core.WorkflowManager.processCommand(WorkflowManager.java:401)
at com.oracle.odof.core.WorkflowManager.processWork(WorkflowManager.java:459)
at com.oracle.odof.io.AbstractClient.run(AbstractClient.java:42)
at java.lang.Thread.run(Thread.java:662)
Caused by: com.oracle.ovm.mgr.api.exception.IllegalOperationException: OVMAPI_8004E There are no more VNICs available. Create More MAC Addresses.
Mon May 21 17:33:13 EEST 2012
at com.oracle.ovm.mgr.api.manager.VnicManagerDbImpl.getAvailableVnic(VnicManagerDbImpl.java:353)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)
at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:822)
at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:245)
at com.oracle.ovm.mgr.api.manager.VnicManagerProxy.getAvailableVnic(Unknown Source)
at com.oracle.ovm.mgr.api.virtual.VirtualMachineConfigDbImpl.cloneNetworks(VirtualMachineConfigDbImpl.java:1065)
at com.oracle.ovm.mgr.api.virtual.VirtualMachineConfigDbImpl.clone(VirtualMachineConfigDbImpl.java:916)
at com.oracle.ovm.mgr.api.virtual.VirtualMachineConfigDbImpl.clone(VirtualMachineConfigDbImpl.java:860)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)
at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:136)
… 7 more
rollback()
-
Create Virtual Machines
Once both are done you will have.
-
Assign NICs to Virtual Machines and make sure that xenbr0 and xenbr1 networks exists.
In the network you need at the end something like this.
If not create/modify xenbr0 and xenbr1.
For xenbr0 press Next.
Select server.
Select the ports.
Press Next.
Press Next.
For xenbr1
Press Next.
Select server name and press Next.
Select ports and press Next.
Press Finish.
If you do not do so will get similar to
Job Construction Phase
———————-
begin()
Appended operation ‘Adding a queued job operation’ to object ‘0004fb000019000038ea52da9f8f21ee (Create Statistics Log)’.
Appended operation ‘Virtual Disk/Virtual Cdrom Clone’ to object ‘0004fb00001200001de8348bd5ec83ab.img (System.img (2))’.
Appended operation ‘Repository Element files refresh’ to object ‘0004fb00001200001de8348bd5ec83ab.img (System.img (2))’.
Appended operation ‘Server Copy File’ to object ‘0004fb00001200001de8348bd5ec83ab.img (System.img (2))’.
Appended operation ‘Virtual Disk/Virtual Cdrom Clone’ to object ‘0004fb000012000064a859339197b7ee.img (Oracle11203RAC_x86_64-xvdb.img (2))’.
Appended operation ‘Repository Element files refresh’ to object ‘0004fb000012000064a859339197b7ee.img (Oracle11203RAC_x86_64-xvdb.img (2))’.
Appended operation ‘Server Copy File’ to object ‘0004fb000012000064a859339197b7ee.img (Oracle11203RAC_x86_64-xvdb.img (2))’.
Appended operation ‘Virtual Machine Cfg File Construct’ to object ‘0004fb0000220000aa9f213a1f521585’.
com.oracle.odof.exception.ObjectException: Caught during invoke method: com.oracle.ovm.mgr.api.exception.IllegalOperationException: OVMAPI_9039E Cannot place clone VM: OVM_OL5U7_X86_64_11203RAC_PVM-2of2.tgz, in Server Pool: MyRACPool. That server pool has no servers that can run the VM
Mon May 21 18:42:36 EEST 2012
at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:151)
at com.oracle.odof.command.InvokeMethodCommand.process(InvokeMethodCommand.java:100)
at com.oracle.odof.core.BasicWork.processCommand(BasicWork.java:81)
at com.oracle.odof.core.TransactionManager.processCommand(TransactionManager.java:773)
at com.oracle.odof.core.WorkflowManager.processCommand(WorkflowManager.java:401)
at com.oracle.odof.core.WorkflowManager.processWork(WorkflowManager.java:459)
at com.oracle.odof.io.AbstractClient.run(AbstractClient.java:42)
at java.lang.Thread.run(Thread.java:662)
Caused by: com.oracle.ovm.mgr.api.exception.IllegalOperationException: OVMAPI_9039E Cannot place clone VM: OVM_OL5U7_X86_64_11203RAC_PVM-2of2.tgz, in Server Pool: MyRACPool. That server pool has no servers that can run the VM
Mon May 21 18:42:36 EEST 2012
at com.oracle.ovm.mgr.api.virtual.VirtualMachineConfigDbImpl.clone(VirtualMachineConfigDbImpl.java:939)
at com.oracle.ovm.mgr.api.virtual.VirtualMachineConfigDbImpl.clone(VirtualMachineConfigDbImpl.java:860)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)
at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:136)
… 7 more
-
Assign ASM shared disks to VM
Make sure that you have the following assignment for the VMs.
For racnode1
For racnode2
The file are as follows.
[root@vmmgr u01]# cat vm.cfg
vif = [‘mac=00:21:f6:00:00:2f,bridge=xenbr0’, ‘mac=00:21:f6:00:00:30,bridge=xenbr1’]
OVM_simple_name = ‘racnode2’
disk = [‘file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb00001200008699ec66aee7821a.img,xvda,w’, ‘file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb00001200006e90573639dac725.img,xvdb,w’, ‘file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb0000120000cdf8b3cd91c68ebd.img,xvdc,w!’, ‘file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb0000120000fb986a9247406641.img,xvdd,w!’, ‘file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb00001200008d3f9019fe32a304.img,xvde,w!’, ‘file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb000012000046a344c5efa19a02.img,xvdf,w!’, ‘file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb0000120000c65d6d0a62ec75a0.img,xvdg,w!’]
uuid = ‘0004fb00-0006-0000-a2a3-03e4d1281181’
on_reboot = ‘restart’
cpu_weight = 27500
memory = 2048
cpu_cap = 0
maxvcpus = 1
OVM_high_availability = False
maxmem = 2048
OVM_description = ‘racnode2’
on_poweroff = ‘destroy’
on_crash = ‘restart’
bootloader = ‘/usr/bin/pygrub’
name = ‘0004fb0000060000a2a303e4d1281181’
guest_os_type = ‘default’
vif_other_config = []
vfb = [‘type=vnc,vncunused=1,vnclisten=127.0.0.1,keymap=en-us’]
vcpus = 1
OVM_os_type = ‘None’
OVM_cpu_compat_group = ”
OVM_domain_type = ‘xen_pvm’
[root@vmmgr u01]#
[root@vmmgr u01]# cat vm.cfg
vif = [‘mac=00:21:f6:00:00:1a,bridge=xenbr0’, ‘mac=00:21:f6:00:00:05,bridge=xenbr1’]
OVM_simple_name = ‘racnode1’
disk = [‘file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb0000120000a3949c3f3d394b21.img,xvda,w’, ‘file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb000012000065511085c4ad6864.img,xvdb,w’, ‘file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb0000120000cdf8b3cd91c68ebd.img,xvdc,w!’, ‘file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb0000120000fb986a9247406641.img,xvdd,w!’, ‘file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb00001200008d3f9019fe32a304.img,xvde,w!’, ‘file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb000012000046a344c5efa19a02.img,xvdf,w!’, ‘file:/OVS/Repositories/0004fb000003000009747581081557e5/VirtualDisks/0004fb0000120000c65d6d0a62ec75a0.img,xvdg,w!’]
uuid = ‘0004fb00-0006-0000-cfab-2af11ac1f125’
on_reboot = ‘restart’
cpu_weight = 27500
memory = 2048
cpu_cap = 0
maxvcpus = 1
OVM_high_availability = False
maxmem = 2048
OVM_description = ‘racnode1’
on_poweroff = ‘destroy’
on_crash = ‘restart’
bootloader = ‘/usr/bin/pygrub’
name = ‘0004fb0000060000cfab2af11ac1f125’
guest_os_type = ‘default’
vif_other_config = []
vfb = [‘type=vnc,vncunused=1,vnclisten=127.0.0.1,keymap=en-us’]
vcpus = 1
OVM_os_type = ‘None’
OVM_cpu_compat_group = ”
OVM_domain_type = ‘xen_pvm’
[root@vmmgr u01]#
-
Start the VMS
-
Establish VNC to both nodes
After both VM are started establish a VNC sessions to both nodes for configuration of GI and RAC.
Select and dedicate which node will be the first and second node.
Enter for each node hostname, public IP and private IP address, VIP, VIP name, domain, DNS, public and private interface etc as prompted.
Review and acknowledge by pressing YES.
Press YES
Wait for the installation to complete.
Progress is…
At the end
-
Verify Oracle 11gR2 11.2.03 GI and RAC installation.
You can verify the status using sqlplus.
You can verify the status using srvctl.
Some notes.
Acknowledging events may clear some errors after a crash of the Oracle VM Server 3.1.1 VirtualBox VM.
Verifying the VM properties resolves the error
Unexpected error during login (com.oracle.ovm.mgr.api.exception.FailedOperationException: OVMAPI_6000E Internal Error: Connection refused Connection refused Mon May 21 23:18:19 EEST 2012), please consult logs for details.
Job Construction Phase
———————-
begin()
Appended operation ‘Virtual Machine Start’ to object ‘0004fb0000060000cfab2af11ac1f125 (racnode1)’.
Appended operation ‘Bridge Configure Operation’ to object ‘0004fb0000200000129d832619e996f5 (network.EthernetPort (4) in vmsrv33.gj.com)’.
Appended operation ‘Bridge Configure Operation’ to object ‘0004fb0000200000604c446c90acae98 (network.EthernetPort (3) in vmsrv33.gj.com)’.
commit()
Completed Step: COMMIT
Objects and Operations
———————-
Object (IN_USE): [EthernetPort] 0004fb0000200000604c446c90acae98 (network.EthernetPort (3) in vmsrv33.gj.com)
Operation: Bridge Configure Operation
Object (IN_USE): [Vnic] 0004fb000007000047de4cdfa34b0a41 (00:21:f6:00:00:1a)
Object (IN_USE): [Vnic] 0004fb000007000020f4233448d8d770 (00:21:f6:00:00:05)
Object (IN_USE): [VmDiskMapping] 0004fb00001300006e3c1bc79509f857
Object (IN_USE): [VmDiskMapping] 0004fb0000130000ce67b1e5241157d7
Object (IN_USE): [VirtualDisk] 0004fb00001200008d3f9019fe32a304.img (ASM3)
Object (IN_USE): [VmDiskMapping] 0004fb0000130000494d049341598540
Object (IN_USE): [VirtualDisk] 0004fb0000120000c65d6d0a62ec75a0.img (ASM5)
Object (IN_USE): [VmDiskMapping] 0004fb0000130000267c771177271b00
Object (IN_USE): [VirtualMachine] 0004fb0000060000cfab2af11ac1f125 (racnode1)
Operation: Virtual Machine Start
Object (IN_USE): [VirtualDisk] 0004fb0000120000cdf8b3cd91c68ebd.img (ASM1)
Object (IN_USE): [VmDiskMapping] 0004fb00001300002036b8246f86e2c4
Object (IN_USE): [VirtualDisk] 0004fb000012000046a344c5efa19a02.img (ASM4)
Object (IN_USE): [VirtualDisk] 0004fb000012000065511085c4ad6864.img (Oracle11203RAC_x86_64-xvdb.img (2))
Object (IN_USE): [EthernetPort] 0004fb0000200000129d832619e996f5 (network.EthernetPort (4) in vmsrv33.gj.com)
Operation: Bridge Configure Operation
Object (IN_USE): [VirtualDisk] 0004fb0000120000fb986a9247406641.img (ASM2)
Object (IN_USE): [VmDiskMapping] 0004fb00001300007b4b45d8dab8f783
Object (IN_USE): [VirtualDisk] 0004fb0000120000a3949c3f3d394b21.img (System.img (2))
Object (IN_USE): [VmDiskMapping] 0004fb000013000040f7df759023d559
Job Running Phase at 23:27 on Mon, May 21, 2012
———————————————-
Job Participants: [b6:10:7c:ba:39:89:4e:e8:95:b7:ca:1a:61:e9:81:69 (vmsrv33.gj.com)]
Actioner
——–
Starting operation ‘Bridge Configure Operation’ on object ‘0004fb0000200000604c446c90acae98 (network.EthernetPort (3) in vmsrv33.gj.com)’
Bridge [xenbr1] already exists (and should exist) on interface [eth2] on server [vmsrv33.gj.com]; skipping bridge creation
Completed operation ‘Bridge Configure Operation’ completed with direction ==> DONE
Starting operation ‘Virtual Machine Start’ on object ‘0004fb0000060000cfab2af11ac1f125 (racnode1)’
Completed operation ‘Virtual Machine Start’ completed with direction ==> LATER
Starting operation ‘Bridge Configure Operation’ on object ‘0004fb0000200000129d832619e996f5 (network.EthernetPort (4) in vmsrv33.gj.com)’
Bridge [xenbr0] already exists (and should exist) on interface [eth3] on server [vmsrv33.gj.com]; skipping bridge creation
Completed operation ‘Bridge Configure Operation’ completed with direction ==> DONE
Starting operation ‘Virtual Machine Start’ on object ‘0004fb0000060000cfab2af11ac1f125 (racnode1)’
Job Internal Error (Operation)com.oracle.ovm.mgr.api.exception.FailedOperationException: OVMAPI_4010E Attempt to send command: dispatch to server: vmsrv33.gj.com failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@192.168.2.21:8899/api/2 start_vm 0004fb000003000009747581081557e5 0004fb0000060000cfab2af11ac1f125, Status: org.apache.xmlrpc.XmlRpcException: exceptions.RuntimeError:Command: [‘xm’, ‘create’, ‘/OVS/Repositories/0004fb000003000009747581081557e5/VirtualMachines/0004fb0000060000cfab2af11ac1f125/vm.cfg’] failed (1): stderr: Error: Device 51744 (vbd) could not be connected. /etc/xen/scripts/block failed; error detected.
stdout: Using config file “/OVS/Repositories/0004fb000003000009747581081557e5/VirtualMachines/0004fb0000060000cfab2af11ac1f125/vm.cfg”.
Mon May 21 23:28:12 EEST 2012
Mon May 21 23:28:12 EEST 2012
at com.oracle.ovm.mgr.action.ActionEngine.sendCommandToServer(ActionEngine.java:507)
at com.oracle.ovm.mgr.action.ActionEngine.sendDispatchedServerCommand(ActionEngine.java:444)
at com.oracle.ovm.mgr.action.ActionEngine.sendServerCommand(ActionEngine.java:378)
at com.oracle.ovm.mgr.action.VirtualMachineAction.sendStartVmCommand(VirtualMachineAction.java:121)
at com.oracle.ovm.mgr.op.virtual.VirtualMachineStart.asyncAction(VirtualMachineStart.java:65)
at com.oracle.ovm.mgr.api.collectable.ManagedObjectDbImpl.executeCurrentJobOperationAsyncAction(ManagedObjectDbImpl.java:999)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)
at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:822)
at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:245)
at com.oracle.ovm.mgr.api.virtual.VirtualMachineProxy.executeCurrentJobOperationAsyncAction(Unknown Source)
at com.oracle.ovm.mgr.api.job.JobEngine.objectAsyncStart(JobEngine.java:596)
at com.oracle.ovm.mgr.op.virtual.VirtualMachineStart.action(VirtualMachineStart.java:52)
at com.oracle.ovm.mgr.api.collectable.ManagedObjectDbImpl.executeCurrentJobOperationAction(ManagedObjectDbImpl.java:1009)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)
at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:822)
at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:245)
at com.oracle.ovm.mgr.api.virtual.VirtualMachineProxy.executeCurrentJobOperationAction(Unknown Source)
at com.oracle.ovm.mgr.api.job.JobEngine.operationActioner(JobEngine.java:218)
at com.oracle.ovm.mgr.api.job.JobEngine.objectActioner(JobEngine.java:309)
at com.oracle.ovm.mgr.api.job.InternalJobDbImpl.objectCommitter(InternalJobDbImpl.java:1140)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)
at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:136)
at com.oracle.odof.command.InvokeMethodCommand.process(InvokeMethodCommand.java:100)
at com.oracle.odof.core.BasicWork.processCommand(BasicWork.java:81)
at com.oracle.odof.core.TransactionManager.processCommand(TransactionManager.java:773)
at com.oracle.odof.core.WorkflowManager.processCommand(WorkflowManager.java:401)
at com.oracle.odof.core.WorkflowManager.processWork(WorkflowManager.java:459)
at com.oracle.odof.io.AbstractClient.run(AbstractClient.java:42)
at java.lang.Thread.run(Thread.java:662)
Caused by: com.oracle.ovm.mgr.api.exception.IllegalOperationException: OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@192.168.2.21:8899/api/2 start_vm 0004fb000003000009747581081557e5 0004fb0000060000cfab2af11ac1f125, Status: org.apache.xmlrpc.XmlRpcException: exceptions.RuntimeError:Command: [‘xm’, ‘create’, ‘/OVS/Repositories/0004fb000003000009747581081557e5/VirtualMachines/0004fb0000060000cfab2af11ac1f125/vm.cfg’] failed (1): stderr: Error: Device 51744 (vbd) could not be connected. /etc/xen/scripts/block failed; error detected.
stdout: Using config file “/OVS/Repositories/0004fb000003000009747581081557e5/VirtualMachines/0004fb0000060000cfab2af11ac1f125/vm.cfg”.
Mon May 21 23:28:12 EEST 2012
at com.oracle.ovm.mgr.action.ActionEngine.sendAction(ActionEngine.java:798)
at com.oracle.ovm.mgr.action.ActionEngine.sendCommandToServer(ActionEngine.java:503)
… 42 more
FailedOperationCleanup
———-
Starting failed operation ‘Virtual Machine Start’ cleanup on object ‘racnode1’
Complete rollback operation ‘Virtual Machine Start’ completed with direction=racnode1
Rollbacker
———-
Executing rollback operation ‘Bridge Configure Operation’ on object ‘0004fb0000200000604c446c90acae98 (network.EthernetPort (3) in vmsrv33.gj.com)’
Complete rollback operation ‘Bridge Configure Operation’ completed with direction=DONE
Executing rollback operation ‘Virtual Machine Start’ on object ‘0004fb0000060000cfab2af11ac1f125 (racnode1)’
Complete rollback operation ‘Virtual Machine Start’ completed with direction=DONE
Executing rollback operation ‘Bridge Configure Operation’ on object ‘0004fb0000200000129d832619e996f5 (network.EthernetPort (4) in vmsrv33.gj.com)’
Complete rollback operation ‘Bridge Configure Operation’ completed with direction=DONE
Objects To Be Rolled Back
————————-
Object (IN_USE): [EthernetPort] 0004fb0000200000604c446c90acae98 (network.EthernetPort (3) in vmsrv33.gj.com)
Object (IN_USE): [Vnic] 0004fb000007000047de4cdfa34b0a41 (00:21:f6:00:00:1a)
Object (IN_USE): [Vnic] 0004fb000007000020f4233448d8d770 (00:21:f6:00:00:05)
Object (IN_USE): [VmDiskMapping] 0004fb00001300006e3c1bc79509f857
Object (IN_USE): [VmDiskMapping] 0004fb0000130000ce67b1e5241157d7
Object (IN_USE): [VirtualDisk] 0004fb00001200008d3f9019fe32a304.img (ASM3)
Object (IN_USE): [VmDiskMapping] 0004fb0000130000494d049341598540
Object (IN_USE): [VirtualDisk] 0004fb0000120000c65d6d0a62ec75a0.img (ASM5)
Object (IN_USE): [VmDiskMapping] 0004fb0000130000267c771177271b00
Object (IN_USE): [VirtualMachine] 0004fb0000060000cfab2af11ac1f125 (racnode1)
Object (IN_USE): [VirtualDisk] 0004fb0000120000cdf8b3cd91c68ebd.img (ASM1)
Object (IN_USE): [VmDiskMapping] 0004fb00001300002036b8246f86e2c4
Object (IN_USE): [VirtualDisk] 0004fb000012000046a344c5efa19a02.img (ASM4)
Object (IN_USE): [VirtualDisk] 0004fb000012000065511085c4ad6864.img (Oracle11203RAC_x86_64-xvdb.img (2))
Object (IN_USE): [EthernetPort] 0004fb0000200000129d832619e996f5 (network.EthernetPort (4) in vmsrv33.gj.com)
Object (IN_USE): [VirtualDisk] 0004fb0000120000fb986a9247406641.img (ASM2)
Object (IN_USE): [VmDiskMapping] 0004fb00001300007b4b45d8dab8f783
Object (IN_USE): [VirtualDisk] 0004fb0000120000a3949c3f3d394b21.img (System.img (2))
Object (IN_USE): [VmDiskMapping] 0004fb000013000040f7df759023d559
Write Methods Invoked
——————-
Class=InternalJobDbImpl vessel_id=10367 method=addTransactionIdentifier accessLevel=6
Class=VirtualMachineDbImpl vessel_id=9317 method=start accessLevel=6
Class=VnicDbImpl vessel_id=8010 method=lock accessLevel=6
Class=VnicDbImpl vessel_id=7888 method=lock accessLevel=6
Class=VirtualDiskDbImpl vessel_id=9329 method=lock accessLevel=6
Class=VirtualDiskDbImpl vessel_id=9339 method=lock accessLevel=6
Class=VirtualDiskDbImpl vessel_id=7354 method=lock accessLevel=6
Class=VirtualDiskDbImpl vessel_id=7404 method=lock accessLevel=6
Class=VirtualDiskDbImpl vessel_id=7529 method=lock accessLevel=6
Class=VirtualDiskDbImpl vessel_id=7571 method=lock accessLevel=6
Class=VirtualDiskDbImpl vessel_id=7590 method=lock accessLevel=6
Class=VnicDbImpl vessel_id=8010 method=lock accessLevel=6
Class=VnicDbImpl vessel_id=7888 method=lock accessLevel=6
Class=VmDiskMappingDbImpl vessel_id=9334 method=lock accessLevel=6
Class=VmDiskMappingDbImpl vessel_id=9344 method=lock accessLevel=6
Class=VmDiskMappingDbImpl vessel_id=9726 method=lock accessLevel=6
Class=VmDiskMappingDbImpl vessel_id=9731 method=lock accessLevel=6
Class=VmDiskMappingDbImpl vessel_id=9736 method=lock accessLevel=6
Class=VmDiskMappingDbImpl vessel_id=9741 method=lock accessLevel=6
Class=VmDiskMappingDbImpl vessel_id=9746 method=lock accessLevel=6
Class=VirtualDiskDbImpl vessel_id=9329 method=lock accessLevel=6
Class=VirtualDiskDbImpl vessel_id=9339 method=lock accessLevel=6
Class=VirtualDiskDbImpl vessel_id=7354 method=lock accessLevel=6
Class=VirtualDiskDbImpl vessel_id=7404 method=lock accessLevel=6
Class=VirtualDiskDbImpl vessel_id=7529 method=lock accessLevel=6
Class=VirtualDiskDbImpl vessel_id=7571 method=lock accessLevel=6
Class=VirtualDiskDbImpl vessel_id=7590 method=lock accessLevel=6
Class=VirtualMachineStartingEventDbImpl vessel_id=10376 method=notify accessLevel=6
Class=VirtualMachineStartingEventDbImpl vessel_id=10376 method=setDescription accessLevel=6
Class=EthernetPortDbImpl vessel_id=1888 method=addJobOperation accessLevel=6
Class=EthernetPortDbImpl vessel_id=1882 method=addJobOperation accessLevel=6
Class=InternalJobDbImpl vessel_id=10367 method=setCompletedStep accessLevel=6
Class=InternalJobDbImpl vessel_id=10367 method=setAssociatedHandles accessLevel=6
Class=EthernetPortDbImpl vessel_id=1882 method=setCurrentJobOperationComplete accessLevel=6
Class=EthernetPortDbImpl vessel_id=1882 method=nextJobOperation accessLevel=6
Class=InternalJobDbImpl vessel_id=10367 method=setTuringMachineFlag accessLevel=6
Class=VirtualMachineDbImpl vessel_id=9317 method=setCurrentOperationToLater accessLevel=6
Class=InternalJobDbImpl vessel_id=10367 method=setTuringMachineFlag accessLevel=6
Class=EthernetPortDbImpl vessel_id=1888 method=setCurrentJobOperationComplete accessLevel=6
Class=EthernetPortDbImpl vessel_id=1888 method=nextJobOperation accessLevel=6
Class=VirtualMachineDbImpl vessel_id=9317 method=nextJobOperation accessLevel=6
Class=InternalJobDbImpl vessel_id=10367 method=setFailedOperation accessLevel=6
Class=EthernetPortDbImpl vessel_id=1882 method=nextJobOperation accessLevel=6
Class=VnicDbImpl vessel_id=8010 method=nextJobOperation accessLevel=6
Class=VnicDbImpl vessel_id=7888 method=nextJobOperation accessLevel=6
Class=VmDiskMappingDbImpl vessel_id=9741 method=nextJobOperation accessLevel=6
Class=VmDiskMappingDbImpl vessel_id=9344 method=nextJobOperation accessLevel=6
Class=VirtualDiskDbImpl vessel_id=7529 method=nextJobOperation accessLevel=6
Class=VmDiskMappingDbImpl vessel_id=9736 method=nextJobOperation accessLevel=6
Class=VirtualDiskDbImpl vessel_id=7590 method=nextJobOperation accessLevel=6
Class=VmDiskMappingDbImpl vessel_id=9726 method=nextJobOperation accessLevel=6
Class=VirtualMachineDbImpl vessel_id=9317 method=nextJobOperation accessLevel=6
Class=VirtualDiskDbImpl vessel_id=7354 method=nextJobOperation accessLevel=6
Class=VmDiskMappingDbImpl vessel_id=9731 method=nextJobOperation accessLevel=6
Class=VirtualDiskDbImpl vessel_id=7571 method=nextJobOperation accessLevel=6
Class=VirtualDiskDbImpl vessel_id=9339 method=nextJobOperation accessLevel=6
Class=EthernetPortDbImpl vessel_id=1888 method=nextJobOperation accessLevel=6
Class=VirtualDiskDbImpl vessel_id=7404 method=nextJobOperation accessLevel=6
Class=VmDiskMappingDbImpl vessel_id=9334 method=nextJobOperation accessLevel=6
Class=VirtualDiskDbImpl vessel_id=9329 method=nextJobOperation accessLevel=6
Class=VmDiskMappingDbImpl vessel_id=9746 method=nextJobOperation accessLevel=6
Class=EthernetPortDbImpl vessel_id=1882 method=nextJobOperation accessLevel=6
Class=VirtualMachineDbImpl vessel_id=9317 method=nextJobOperation accessLevel=6
Class=EthernetPortDbImpl vessel_id=1888 method=nextJobOperation accessLevel=6
Completed Step: ROLLBACK
Job failed commit (internal) due to OVMAPI_4010E Attempt to send command: dispatch to server: vmsrv33.gj.com failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@192.168.2.21:8899/api/2 start_vm 0004fb000003000009747581081557e5 0004fb0000060000cfab2af11ac1f125, Status: org.apache.xmlrpc.XmlRpcException: exceptions.RuntimeError:Command: [‘xm’, ‘create’, ‘/OVS/Repositories/0004fb000003000009747581081557e5/VirtualMachines/0004fb0000060000cfab2af11ac1f125/vm.cfg’] failed (1): stderr: Error: Device 51744 (vbd) could not be connected. /etc/xen/scripts/block failed; error detected.
stdout: Using config file “/OVS/Repositories/0004fb000003000009747581081557e5/VirtualMachines/0004fb0000060000cfab2af11ac1f125/vm.cfg”.
Mon May 21 23:28:12 EEST 2012
Mon May 21 23:28:12 EEST 2012
com.oracle.ovm.mgr.api.exception.FailedOperationException: OVMAPI_4010E Attempt to send command: dispatch to server: vmsrv33.gj.com failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@192.168.2.21:8899/api/2 start_vm 0004fb000003000009747581081557e5 0004fb0000060000cfab2af11ac1f125, Status: org.apache.xmlrpc.XmlRpcException: exceptions.RuntimeError:Command: [‘xm’, ‘create’, ‘/OVS/Repositories/0004fb000003000009747581081557e5/VirtualMachines/0004fb0000060000cfab2af11ac1f125/vm.cfg’] failed (1): stderr: Error: Device 51744 (vbd) could not be connected. /etc/xen/scripts/block failed; error detected.
stdout: Using config file “/OVS/Repositories/0004fb000003000009747581081557e5/VirtualMachines/0004fb0000060000cfab2af11ac1f125/vm.cfg”.
Mon May 21 23:28:12 EEST 2012
Mon May 21 23:28:12 EEST 2012
at com.oracle.ovm.mgr.action.ActionEngine.sendCommandToServer(ActionEngine.java:507)
at com.oracle.ovm.mgr.action.ActionEngine.sendDispatchedServerCommand(ActionEngine.java:444)
at com.oracle.ovm.mgr.action.ActionEngine.sendServerCommand(ActionEngine.java:378)
at com.oracle.ovm.mgr.action.VirtualMachineAction.sendStartVmCommand(VirtualMachineAction.java:121)
at com.oracle.ovm.mgr.op.virtual.VirtualMachineStart.asyncAction(VirtualMachineStart.java:65)
at com.oracle.ovm.mgr.api.collectable.ManagedObjectDbImpl.executeCurrentJobOperationAsyncAction(ManagedObjectDbImpl.java:999)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)
at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:822)
at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:245)
at com.oracle.ovm.mgr.api.virtual.VirtualMachineProxy.executeCurrentJobOperationAsyncAction(Unknown Source)
at com.oracle.ovm.mgr.api.job.JobEngine.objectAsyncStart(JobEngine.java:596)
at com.oracle.ovm.mgr.op.virtual.VirtualMachineStart.action(VirtualMachineStart.java:52)
at com.oracle.ovm.mgr.api.collectable.ManagedObjectDbImpl.executeCurrentJobOperationAction(ManagedObjectDbImpl.java:1009)
at sun.reflect.GeneratedMethodAccessor886.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)
at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:822)
at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:245)
at com.oracle.ovm.mgr.api.virtual.VirtualMachineProxy.executeCurrentJobOperationAction(Unknown Source)
at com.oracle.ovm.mgr.api.job.JobEngine.operationActioner(JobEngine.java:218)
at com.oracle.ovm.mgr.api.job.JobEngine.objectActioner(JobEngine.java:309)
at com.oracle.ovm.mgr.api.job.InternalJobDbImpl.objectCommitter(InternalJobDbImpl.java:1140)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)
at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:136)
at com.oracle.odof.command.InvokeMethodCommand.process(InvokeMethodCommand.java:100)
at com.oracle.odof.core.BasicWork.processCommand(BasicWork.java:81)
at com.oracle.odof.core.TransactionManager.processCommand(TransactionManager.java:773)
at com.oracle.odof.core.WorkflowManager.processCommand(WorkflowManager.java:401)
at com.oracle.odof.core.WorkflowManager.processWork(WorkflowManager.java:459)
at com.oracle.odof.io.AbstractClient.run(AbstractClient.java:42)
at java.lang.Thread.run(Thread.java:662)
Caused by: com.oracle.ovm.mgr.api.exception.IllegalOperationException: OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@192.168.2.21:8899/api/2 start_vm 0004fb000003000009747581081557e5 0004fb0000060000cfab2af11ac1f125, Status: org.apache.xmlrpc.XmlRpcException: exceptions.RuntimeError:Command: [‘xm’, ‘create’, ‘/OVS/Repositories/0004fb000003000009747581081557e5/VirtualMachines/0004fb0000060000cfab2af11ac1f125/vm.cfg’] failed (1): stderr: Error: Device 51744 (vbd) could not be connected. /etc/xen/scripts/block failed; error detected.
stdout: Using config file “/OVS/Repositories/0004fb000003000009747581081557e5/VirtualMachines/0004fb0000060000cfab2af11ac1f125/vm.cfg”.
Mon May 21 23:28:12 EEST 2012
at com.oracle.ovm.mgr.action.ActionEngine.sendAction(ActionEngine.java:798)
at com.oracle.ovm.mgr.action.ActionEngine.sendCommandToServer(ActionEngine.java:503)
… 42 more
———-
End of Job
———-
- Conclusion
You had a look at the steps to upgrade Oracle VM server and Oracle VM Manager to version 3.1.1. You looked at the step by step guide to install two node Oracle 11gR2 11.2.0.3 RAC using templates. The errors that I encountered were also included as well as A way to work it around.
20 Comments »
Leave a Reply
-
Archives
- February 2017 (1)
- November 2016 (1)
- October 2016 (1)
- May 2016 (2)
- March 2016 (3)
- December 2014 (2)
- July 2014 (1)
- June 2014 (6)
- May 2014 (5)
- February 2014 (1)
- December 2012 (2)
- November 2012 (8)
-
Categories
-
RSS
Entries RSS
Comments RSS
Thanks for your efforts. If, I have a hard disk which is directly attached to the OVM server’s OS, how to deal with that? The OVM Manager is not able to pick up the hard disk partition Eg. /dev/sda4
Thanks for the article. But I have a FC SAN as the storage. I found it impossible to create the server pool in VM Manager since it always says that job is time-out. I searched all the docs on the net but found nothing helpful. Any suggestions? Thanks.
Somebody suggests me to wipe out all the data on the storage. I’ve used dd command to write the first 100M of the SAN to zero. But it’s in vain. The same error showed up again. To my surprise, Oracle official documentation on this is unclear and vague. I can’t believe that Oracle VM, as Oracle’s the main virtualization product for x86, is so fragile.
Hi,
Look at the error and work it out. I cannot understand what is the problem that you are facing other than that the Server pool creation times out.
Could you bounce the VM manager and give it a try? Make sure that you follow the ordered steps, do not miss a step, and each step completes succesfully before moving to the nest one. Did you synchronise the NFS and repository configuration? Try to get something that provides an indication of a problem…
Regards,
@Praveen,
What you should do is:
login to your OVM server as root and use parted command to look at the unused disk.
So, for example, to carve out ~50GB space and make it available to the VM Manager:
[root@ovm1 dev]# parted
GNU Parted 1.8.1
Using /dev/sda
Welcome to GNU Parted! Type ‘help’ to view a list of commands.
(parted) print free
Model: HITACHI H101414SCSUN146G (scsi)
Disk /dev/sda: 147GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
1 32.3kB 107MB 107MB primary ext3 boot
2 107MB 1184MB 1078MB primary linux-swap
1184MB 147GB 146GB Free Space
(parted) mkpart primary 1184MB 50GB
(parted) print free
Model: HITACHI H101414SCSUN146G (scsi)
Disk /dev/sda: 147GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
1 32.3kB 107MB 107MB primary ext3 boot
2 107MB 1184MB 1078MB primary linux-swap
3 1184MB 50.0GB 48.8GB primary
50.0GB 147GB 96.8GB Free Space
Now, you will see a /dev/sda3 device.
Just create a fs on it:
[root@ovm1 ~]# ll /dev/sd*
brw-r—– 1 root disk 8, 0 Sep 6 16:41 /dev/sda
brw-r—– 1 root disk 8, 1 Sep 6 16:27 /dev/sda1
brw-r—– 1 root disk 8, 2 Sep 6 16:27 /dev/sda2
brw-r—– 1 root disk 8, 3 Sep 6 16:41 /dev/sda3
brw-r—– 1 root disk 8, 16 Sep 6 16:27 /dev/sdb
brw-r—– 1 root disk 8, 17 Sep 6 16:27 /dev/sdb1
brw-r—– 1 root disk 8, 18 Sep 6 16:27 /dev/sdb2
[root@ovm1 ~]#
[root@ovm1 ~]#
[root@ovm1 ~]# mkfs -t ext3 /dev/sda3
mke2fs 1.39 (29-May-2006)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
5963776 inodes, 11918221 blocks
595911 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
364 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 39 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
[root@ovm1 ~]# mkdir /1st_50G
[root@ovm1 ~]# mount /dev/sda3 /1st_50G
[root@ovm1 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sdb1 3.0G 635M 2.2G 23% /
/dev/sda1 99M 28M 67M 30% /boot
tmpfs 360M 0 360M 0% /dev/shm
none 360M 40K 360M 1% /var/lib/xenstored
/dev/sda3 45G 181M 43G 1% /1st_50G
[root@ovm1 ~]#
Add to /etc/fstab:
/dev/sda3 /1st_50G ext3 defaults 0 0
Now, you can export it as NFS share as Guenadi has shown above:
[root@ovm1 ~]# cat /etc/exports
/ist_50G *(rw,sync,no_wdelay,insecure_locks,no_root_squash)
[root@ovm1 ~]# service nfs status
rpc.mountd (pid 3699) is running…
nfsd (pid 3696 3695 3694 3693 3692 3691 3690 3689) is running…
[root@ovm1 ~]# service nfs restart
Shutting down NFS mountd: [ OK ]
Shutting down NFS daemon: [ OK ]
Shutting down NFS services: [ OK ]
Starting NFS services: [ OK ]
Starting NFS daemon: [ OK ]
Starting NFS mountd: [ OK ]
[root@ovm1 ~]#
Now, you can use this 50G space from your OVM Manager console.
Hope that helps.
Thanks Guenadi for taking the time to list the steps. I got it working(finally) with the local hard disk by keeping it as “raw” – i.e. initialization from BIOS but NO initialization from the OS and file system of these hard disks. This works – not THE solution to be used in production env. though. I will keep your NFS suggestion if need be for future additional disks.
Thanks again.
Regards,
Praveen
@Praveen,
Thanks. Could you be more specific as to the exact details? I didn’t quite follow how you made it to work.
Maybe a blog entry? I am sure there’s a lot of interest in the solution of using up the local disks work (for TEST/DEV OVM setup) and yours could be a useful one.
Regards
jcnars
Hi,
Hi, Could you please elaborate further on the steps that you did as I believe readers can be interested in
Regards,
Happy to do this, when I can do something from scratch so that I could get the right screen-shots. Not sure how soon but. Thanks for your interest.
-Praveen
On Sat, Sep 8, 2012 at 5:38 AM, Guenadi N Jilevski’s Oracle BLOG wrote:
> ** > gjilevski commented: “Hi, Could you please elaborate further on he > steps that you did as I belive reades can be interested in. Regards,” >
Happy to do this, when I can do something from scratch so that I could get the right screen-shots. Not sure how soon but.
Thanks for your interest.
If you have clean slate hard-disks, I could spare an hour for a remote session for Aus.(Melbourne) time. My config. is 4 separate hard disks. Kept 1 disk separate and merged the other 3 as RAID. The 1 separate hard disk for the OVM server installation and the other for the OVM repository.
-Praveen
Hi,
Do not worry. If you can point it out is fine else no biggie.
Regards,
could you please point out the Oracle VM Manager Installation Instructions and filesystems layout
thanks in advance
Hi,
Look at
Upgrade Oracle VM Manager 3.0.3 to Oracle VM Manager 3.1.1 https://gjilevski.wordpress.com/2012/06/20/oracle-virtualization-installing-oracle-vm-server-3-1-1-oracle-vm-manager-3-1-1-and-deploying-oracle-rac-11gr2-11-2-0-3-oracle-vm-templates-linux-x86-64-bit-for-test-configuration/?preview=true&preview_id=4644&preview_nonce=605f7b57dd
Install and configure Oracle VM Manager 3.0.3 https://gjilevski.wordpress.com/2012/06/19/oracle-virtualization-installing-oracle-vm-server-3-0-3-oracle-vm-manager-3-0-3-and-deploying-oracle-rac-11gr2-11-2-0-3-oracle-vm-templates-linux-x86-64-bit-for-test-configuration/
Regards,
Reblogged this on Apps DBA notes.
Hi Guenadi – thanks for the info on OVM3.1 – quite different to OVM 2 – nice note.
Regards,
Gavin
Hi gjilevski,
Great post! Very useful information.
Few question tos ask:
1. do you need to build the Virtual Machine racnode1 and racnode2 on VirtualBox like you did for svr33 & vmmgr?
2. how do you logged into racnode1 and racnode2?
3. Have you tried the newest deploycluster tool with OVM 3.1.1? (http://www.oracle.com/technetwork/products/clusterware/overview/racovm-deploycluster-tool-1635519.pdf)
According to the Doc, I can skip your step 9 “Start the VMS”. But not sure!
Possible to update this post with the newest deploycluster tool?.
Best Regards,
Hi,
1. No. I build using VirtualBox only vmmgr and vmsrv33. The virtual servers raclinux1 and raclinux2 are built on vmsrv33 as a result of creating virtual machines using the RAC templates from VM Manager.
2. Look in the post for establishing a vnc connection. I did it from the VM Manager. You could try using vncviewer as well. It is the same but invoke it on the command line instead of from the VMManager.
3. The tool was not available when I wrote the article. I can give it a try. It should simplify the setup. I did not use it. I can update the article when get a chance to play with the tool.
Regards,
Hi, In step4 you created MYRAC network. So you have two networks – the management network and MYRAC, which you did not apply an IP address to.
In step 7 you suddenly have xenbr0 and xenbr1 – where did they come from ? – Do you need to have extra NICs configured in the Virtualbox VM before these networks can be created?
I have two questions regarding the disk space and network.
1. How much disk space is needed for the share disks?
2. In Step 6 – Create Virtual Disks, it took me 15-30mins to create a single node. I checked what was the resources used, apparently the network was the main heavy activity. System Monitor showed the network packet sent was hovering around 20MB+. I am not sure if we can improve the speed?
Thank you.
Hi,
1. Look at the article for the disk sizes I used. Your disk sizes could be different. The ASM disk should be >=2GB to get it started.
The repository (stage) area is 50GB and each ASMx disk is 2GB in the article.
2. Look at your configuration for the network. Are you tesing on a bare metal or using any virtualization software?
Best Regards,