This discussion is archived
8 Replies Latest reply: Jan 17, 2013 9:23 PM by ovmnewbee RSS

OVMAPI_4004E Server Failed Command:- NFS repo VM starting issue

ovmnewbee Newbie
Currently Being Moderated
Hi

I have setup a POC environment on OVM 3.1.1 with two OVS servers mapped to a cluster and one OVM server managing these servers . I have mapped a 600GB iscsi LUN as well as a 300GB NFS mount on both the OVS servers.

I created 2 VM's with the iscsi LUN mapped as the storage repo and VM's seem to be working fine

The issue that I am facing is with one of VM's on the NFS mount. I created a VM ( win2k3nfs01) on one of the servers (englabbld10) and mapped the NFS mount as the storage repository. The issue that I am facing is that I am not able to start the VM on the server englabbld10 I get the error which I have mentioned below, but if I migrate the VM to the other server englabbld09 I am able to start the VM. I am enclosing the error details of englabbld10.

The VM’s with iscsi repository work perfectly fine in englabbld10, I can migrate start stop the same.

I have cross validated the nfs permission, DNS records etc. Everything seems to be just fine.

Please help in resolve the same

Job Construction Phase
----------------------
begin()
Appended operation 'Virtual Machine Start' to object '0004fb0000060000ff2b96ea9cc79078 (win2k3nfs01)'.
Appended operation 'Bridge Configure Operation' to object '0004fb00002000005e3d744bbd8de7da (network.BondPort (1) in englabbld10)'.
Appended operation 'VM Set HA' to object '0004fb0000060000ff2b96ea9cc79078 (win2k3nfs01)'.
commit()
Completed Step: COMMIT

Objects and Operations
----------------------
Object (IN_USE): [VirtualMachine] 0004fb0000060000ff2b96ea9cc79078 (win2k3nfs01)
Operation: Virtual Machine Start
Operation: VM Set HA
Object (IN_USE): [VmDiskMapping] 0004fb0000130000f8a0f77cdb2d1976
Object (IN_USE): [VirtualCdrom] 0004fb000015000066df9e28a1f4976e.iso (win2k3_64bit.iso)
Object (IN_USE): [Vnic] 0004fb00000700003454c955f4b4660b (00:21:f6:00:00:00)
Object (IN_USE): [BondPort] 0004fb00002000005e3d744bbd8de7da (network.BondPort (1) in englabbld10)
Operation: Bridge Configure Operation
Object (IN_USE): [VmDiskMapping] 0004fb000013000008cf2ea56564db67
Object (IN_USE): [VirtualDisk] 0004fb00001200006e9c05a32bf7ad47.img (win2k3vm02)

Job Running Phase at 09:50 on Wed, Jan 16, 2013
----------------------------------------------
Job Participants: [ff:20:00:08:ff:ff:ff:ff:ff:ff:48:0f:bc:28:21:00 (englabbld10)]


Actioner
--------
Starting operation 'Virtual Machine Start' on object '0004fb0000060000ff2b96ea9cc79078 (win2k3nfs01)'
Completed operation 'Virtual Machine Start' completed with direction ==> LATER
Starting operation 'Bridge Configure Operation' on object '0004fb00002000005e3d744bbd8de7da (network.BondPort (1) in englabbld10)'
Bridge [206.236.134.0] already exists (and should exist) on interface [bond0] on server [englabbld10]; skipping bridge creation
Completed operation 'Bridge Configure Operation' completed with direction ==> DONE
Starting operation 'VM Set HA' on object '0004fb0000060000ff2b96ea9cc79078 (win2k3nfs01)'
Completed operation 'VM Set HA' completed with direction ==> LATER
Starting operation 'Virtual Machine Start' on object '0004fb0000060000ff2b96ea9cc79078 (win2k3nfs01)'
Job Internal Error (Operation)com.oracle.ovm.mgr.api.exception.FailedOperationException: OVMAPI_4010E Attempt to send command: dispatch to server: englabbld10 failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@206.236.134.19:8899/api/2 start_vm 0004fb0000030000e40574ce6f97e906 0004fb0000060000ff2b96ea9cc79078, Status: org.apache.xmlrpc.XmlRpcException: exceptions.RuntimeError:Command: ['xm', 'create', '/OVS/Repositories/0004fb0000030000e40574ce6f97e906/VirtualMachines/0004fb0000060000ff2b96ea9cc79078/vm.cfg'] failed (1): stderr: Error: Unable to open config file: /OVS/Repositories/0004fb0000030000e40574ce6f97e906/VirtualMachines/0004fb0000060000ff2b96ea9cc79078/vm.cfg
stdout: Usage: xm create [options] [vars]

Create a domain based on .

Options:

-h, --help Print this help.
--help_config Print the available configuration variables (vars)
for the configuration script.
-q, --quiet Quiet.
--path=PATH Search path for configuration scripts. The value of
PATH is a colon-separated directory list.
-f=FILE, --defconfig=FILE
Use the given Python configuration script.The
configuration script is loaded after arguments have
been processed. Each command-line option sets a
configuration variable named after its long option
name, and these variables are placed in the
environment of the script before it is loaded.
Variables for options that may be repeated have list
values. Other variables can be set using VAR=VAL on
the command line. After the script is loaded, option
values that were not set on the command line are
replaced by the values set in the script.
-F=FILE, --config=FILE
Domain configuration to use (SXP).
SXP is the underlying configuration format used by
Xen.
SXP configurations can be hand-written or generated
from Python configuration scripts, using the -n
(dryrun) option to print the configuration.
-n, --dryrun Dry run - prints the resulting configuration in SXP
but does not create the domain.
-x, --xmldryrun XML dry run - prints the resulting configuration in
XML but does not create the domain.
-s, --skipdtd Skip DTD checking - skips checks on XML before
creating. Experimental. Can decrease create time.
-p, --paused Leave the domain paused after it is created.
-c, --console_autoconnect
Connect to the console after the domain is created.
--vncviewer Connect to the VNC display after the domain is
created.
--vncviewer-autopass Pass VNC password to viewer via stdin and -autopass.

[options] [vars]

Create a domain.

Domain creation parameters can be set by command-line switches, from
a python configuration script or an SXP config file. See documentation
for defconfig, config. Configuration variables can be set using
VAR=VAL on the command line. For example vmid=3 sets vmid to 3.


...
Wed Jan 16 09:50:04 IST 2013...
Wed Jan 16 09:50:04 IST 2013
at com.oracle.ovm.mgr.action.ActionEngine.sendCommandToServer(ActionEngine.java:507)
at com.oracle.ovm.mgr.action.ActionEngine.sendDispatchedServerCommand(ActionEngine.java:444)
at com.oracle.ovm.mgr.action.ActionEngine.sendServerCommand(ActionEngine.java:378)
at com.oracle.ovm.mgr.action.VirtualMachineAction.sendStartVmCommand(VirtualMachineAction.java:121)
at com.oracle.ovm.mgr.op.virtual.VirtualMachineStart.asyncAction(VirtualMachineStart.java:65)
at com.oracle.ovm.mgr.api.collectable.ManagedObjectDbImpl.executeCurrentJobOperationAsyncAction(ManagedObjectDbImpl.java:1002)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:329)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:289)
at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:826)
at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:245)
at com.oracle.ovm.mgr.api.virtual.VirtualMachineProxy.executeCurrentJobOperationAsyncAction(Unknown Source)
at com.oracle.ovm.mgr.api.job.JobEngine.objectAsyncStart(JobEngine.java:596)
at com.oracle.ovm.mgr.op.virtual.VirtualMachineStart.action(VirtualMachineStart.java:52)
at com.oracle.ovm.mgr.api.collectable.ManagedObjectDbImpl.executeCurrentJobOperationAction(ManagedObjectDbImpl.java:1012)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:329)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:289)
at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:826)
at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:245)
at com.oracle.ovm.mgr.api.virtual.VirtualMachineProxy.executeCurrentJobOperationAction(Unknown Source)
at com.oracle.ovm.mgr.api.job.JobEngine.operationActioner(JobEngine.java:218)
at com.oracle.ovm.mgr.api.job.JobEngine.objectActioner(JobEngine.java:309)
at com.oracle.ovm.mgr.api.job.InternalJobDbImpl.objectCommitter(InternalJobDbImpl.java:1140)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:329)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:289)
at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:136)
at com.oracle.odof.command.InvokeMethodCommand.process(InvokeMethodCommand.java:100)
at com.oracle.odof.core.BasicWork.processCommand(BasicWork.java:81)
at com.oracle.odof.core.TransactionManager.processCommand(TransactionManager.java:773)
at com.oracle.odof.core.WorkflowManager.processCommand(WorkflowManager.java:401)
at com.oracle.odof.core.WorkflowManager.processWork(WorkflowManager.java:459)
at com.oracle.odof.io.AbstractClient.run(AbstractClient.java:42)
at java.lang.Thread.run(Thread.java:662)
Caused by: com.oracle.ovm.mgr.api.exception.IllegalOperationException: OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@206.236.134.19:8899/api/2 start_vm 0004fb0000030000e40574ce6f97e906 0004fb0000060000ff2b96ea9cc79078, Status: org.apache.xmlrpc.XmlRpcException: exceptions.RuntimeError:Command: ['xm', 'create', '/OVS/Repositories/0004fb0000030000e40574ce6f97e906/VirtualMachines/0004fb0000060000ff2b96ea9cc79078/vm.cfg'] failed (1): stderr: Error: Unable to open config file: /OVS/Repositories/0004fb0000030000e40574ce6f97e906/VirtualMachines/0004fb0000060000ff2b96ea9cc79078/vm.cfg
stdout: Usage: xm create [options] [vars]

Create a domain based on .

Options:

-h, --help Print this help.
--help_config Print the available configuration variables (vars)
for the configuration script.
-q, --quiet Quiet.
--path=PATH Search path for configuration scripts. The value of
PATH is a colon-separated directory list.
-f=FILE, --defconfig=FILE
Use the given Python configuration script.The
configuration script is loaded after arguments have
been processed. Each command-line option sets a
configuration variable named after its long option
name, and these variables are placed in the
environment of the script before it is loaded.
Variables for options that may be repeated have list
values. Other variables can be set using VAR=VAL on
the command line. After the script is loaded, option
values that were not set on the command line are
replaced by the values set in the script.
-F=FILE, --config=FILE
Domain configuration to use (SXP).
SXP is the underlying configuration format used by
Xen.
SXP configurations can be hand-written or generated
from Python configuration scripts, using the -n
(dryrun) option to print the configuration.
-n, --dryrun Dry run - prints the resulting configuration in SXP
but does not create the domain.
-x, --xmldryrun XML dry run - prints the resulting configuration in
XML but does not create the domain.
-s, --skipdtd Skip DTD checking - skips checks on XML before
creating. Experimental. Can decrease create time.
-p, --paused Leave the domain paused after it is created.
-c, --console_autoconnect
Connect to the console after the domain is created.
--vncviewer Connect to the VNC display after the domain is
created.
--vncviewer-autopass Pass VNC password to viewer via stdin and -autopass.

[options] [vars]

Create a domain.

Domain creation parameters can be set by command-line switches, from
a python configuration script or an SXP config file. See documentation
for defconfig, config. Configuration variables can be set using
VAR=VAL on the command line. For example vmid=3 sets vmid to 3.


...
Wed Jan 16 09:50:04 IST 2013
at com.oracle.ovm.mgr.action.ActionEngine.sendAction(ActionEngine.java:798)
at com.oracle.ovm.mgr.action.ActionEngine.sendCommandToServer(ActionEngine.java:503)
... 40 more


FailedOperationCleanup
----------
Starting failed operation 'Virtual Machine Start' cleanup on object 'win2k3nfs01'
Complete rollback operation 'Virtual Machine Start' completed with direction=win2k3nfs01

Rollbacker
----------
Executing rollback operation 'Virtual Machine Start' on object '0004fb0000060000ff2b96ea9cc79078 (win2k3nfs01)'
Complete rollback operation 'Virtual Machine Start' completed with direction=DONE
Executing rollback operation 'Bridge Configure Operation' on object '0004fb00002000005e3d744bbd8de7da (network.BondPort (1) in englabbld10)'
Complete rollback operation 'Bridge Configure Operation' completed with direction=DONE

Objects To Be Rolled Back
-------------------------
Object (IN_USE): [VirtualMachine] 0004fb0000060000ff2b96ea9cc79078 (win2k3nfs01)
Object (IN_USE): [VmDiskMapping] 0004fb0000130000f8a0f77cdb2d1976
Object (IN_USE): [VirtualCdrom] 0004fb000015000066df9e28a1f4976e.iso (win2k3_64bit.iso)
Object (IN_USE): [Vnic] 0004fb00000700003454c955f4b4660b (00:21:f6:00:00:00)
Object (IN_USE): [BondPort] 0004fb00002000005e3d744bbd8de7da (network.BondPort (1) in englabbld10)
Object (IN_USE): [VmDiskMapping] 0004fb000013000008cf2ea56564db67
Object (IN_USE): [VirtualDisk] 0004fb00001200006e9c05a32bf7ad47.img (win2k3vm02)


Write Methods Invoked
-------------------
Class=InternalJobDbImpl vessel_id=12116 method=addTransactionIdentifier accessLevel=6
Class=VirtualMachineDbImpl vessel_id=10564 method=start accessLevel=6
Class=VnicDbImpl vessel_id=2129 method=lock accessLevel=6
Class=VirtualCdromDbImpl vessel_id=9293 method=lock accessLevel=6
Class=VirtualDiskDbImpl vessel_id=10585 method=lock accessLevel=6
Class=VnicDbImpl vessel_id=2129 method=lock accessLevel=6
Class=VmDiskMappingDbImpl vessel_id=10592 method=lock accessLevel=6
Class=VmDiskMappingDbImpl vessel_id=10597 method=lock accessLevel=6
Class=VirtualCdromDbImpl vessel_id=9293 method=lock accessLevel=6
Class=VirtualDiskDbImpl vessel_id=10585 method=lock accessLevel=6
Class=VirtualMachineStartingEventDbImpl vessel_id=12125 method=notify accessLevel=6
Class=VirtualMachineStartingEventDbImpl vessel_id=12125 method=setDescription accessLevel=6
Class=BondPortDbImpl vessel_id=1473 method=addJobOperation accessLevel=6
Class=InternalJobDbImpl vessel_id=12116 method=setCompletedStep accessLevel=6
Class=InternalJobDbImpl vessel_id=12116 method=setAssociatedHandles accessLevel=6
Class=InternalJobDbImpl vessel_id=12116 method=setTuringMachineFlag accessLevel=6
Class=VirtualMachineDbImpl vessel_id=10564 method=setCurrentOperationToLater accessLevel=6
Class=InternalJobDbImpl vessel_id=12116 method=setTuringMachineFlag accessLevel=6
Class=BondPortDbImpl vessel_id=1473 method=setCurrentJobOperationComplete accessLevel=6
Class=BondPortDbImpl vessel_id=1473 method=nextJobOperation accessLevel=6
Class=InternalJobDbImpl vessel_id=12116 method=setTuringMachineFlag accessLevel=6
Class=VirtualMachineDbImpl vessel_id=10564 method=setCurrentOperationToLater accessLevel=6
Class=InternalJobDbImpl vessel_id=12116 method=setTuringMachineFlag accessLevel=6
Class=VirtualMachineDbImpl vessel_id=10564 method=nextJobOperation accessLevel=6
Class=InternalJobDbImpl vessel_id=12116 method=setFailedOperation accessLevel=6
Class=VirtualMachineDbImpl vessel_id=10564 method=nextJobOperation accessLevel=6
Class=VmDiskMappingDbImpl vessel_id=10597 method=nextJobOperation accessLevel=6
Class=VirtualCdromDbImpl vessel_id=9293 method=nextJobOperation accessLevel=6
Class=VnicDbImpl vessel_id=2129 method=nextJobOperation accessLevel=6
Class=BondPortDbImpl vessel_id=1473 method=nextJobOperation accessLevel=6
Class=VmDiskMappingDbImpl vessel_id=10592 method=nextJobOperation accessLevel=6
Class=VirtualDiskDbImpl vessel_id=10585 method=nextJobOperation accessLevel=6
Class=VirtualMachineDbImpl vessel_id=10564 method=nextJobOperation accessLevel=6
Class=BondPortDbImpl vessel_id=1473 method=nextJobOperation accessLevel=6
Completed Step: ROLLBACK
Job failed commit (internal) due to OVMAPI_4010E Attempt to send command: dispatch to server: englabbld10 failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@206.236.134.19:8899/api/2 start_vm 0004fb0000030000e40574ce6f97e906 0004fb0000060000ff2b96ea9cc79078, Status: org.apache.xmlrpc.XmlRpcException: exceptions.RuntimeError:Command: ['xm', 'create', '/OVS/Repositories/0004fb0000030000e40574ce6f97e906/VirtualMachines/0004fb0000060000ff2b96ea9cc79078/vm.cfg'] failed (1): stderr: Error: Unable to open config file: /OVS/Repositories/0004fb0000030000e40574ce6f97e906/VirtualMachines/0004fb0000060000ff2b96ea9cc79078/vm.cfg
stdout: Usage: xm create [options] [vars]

Create a domain based on .

Options:

-h, --help Print this help.
--help_config Print the available configuration variables (vars)
for the configuration script.
-q, --quiet Quiet.
--path=PATH Search path for configuration scripts. The value of
PATH is a colon-separated directory list.
-f=FILE, --defconfig=FILE
Use the given Python configuration script.The
configuration script is loaded after arguments have
been processed. Each command-line option sets a
configuration variable named after its long option
name, and these variables are placed in the
environment of the script before it is loaded.
Variables for options that may be repeated have list
values. Other variables can be set using VAR=VAL on
the command line. After the script is loaded, option
values that were not set on the command line are
replaced by the values set in the script.
-F=FILE, --config=FILE
Domain configuration to use (SXP).
SXP is the underlying configuration format used by
Xen.
SXP configurations can be hand-written or generated
from Python configuration scripts, using the -n
(dryrun) option to print the configuration.
-n, --dryrun Dry run - prints the resulting configuration in SXP
but does not create the domain.
-x, --xmldryrun XML dry run - prints the resulting configuration in
XML but does not create the domain.
-s, --skipdtd Skip DTD checking - skips checks on XML before
creating. Experimental. Can decrease create time.
-p, --paused Leave the domain paused after it is created.
-c, --console_autoconnect
Connect to the console after the domain is created.
--vncviewer Connect to the VNC display after the domain is
created.
--vncviewer-autopass Pass VNC password to viewer via stdin and -autopass.

[options] [vars]

Create a domain.

Domain creation parameters can be set by command-line switches, from
a python configuration script or an SXP config file. See documentation
for defconfig, config. Configuration variables can be set using
VAR=VAL on the command line. For example vmid=3 sets vmid to 3.


...
Wed Jan 16 09:50:04 IST 2013...
Wed Jan 16 09:50:04 IST 2013
com.oracle.ovm.mgr.api.exception.FailedOperationException: OVMAPI_4010E Attempt to send command: dispatch to server: englabbld10 failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@206.236.134.19:8899/api/2 start_vm 0004fb0000030000e40574ce6f97e906 0004fb0000060000ff2b96ea9cc79078, Status: org.apache.xmlrpc.XmlRpcException: exceptions.RuntimeError:Command: ['xm', 'create', '/OVS/Repositories/0004fb0000030000e40574ce6f97e906/VirtualMachines/0004fb0000060000ff2b96ea9cc79078/vm.cfg'] failed (1): stderr: Error: Unable to open config file: /OVS/Repositories/0004fb0000030000e40574ce6f97e906/VirtualMachines/0004fb0000060000ff2b96ea9cc79078/vm.cfg
stdout: Usage: xm create [options] [vars]

Create a domain based on .

Options:

-h, --help Print this help.
--help_config Print the available configuration variables (vars)
for the configuration script.
-q, --quiet Quiet.
--path=PATH Search path for configuration scripts. The value of
PATH is a colon-separated directory list.
-f=FILE, --defconfig=FILE
Use the given Python configuration script.The
configuration script is loaded after arguments have
been processed. Each command-line option sets a
configuration variable named after its long option
name, and these variables are placed in the
environment of the script before it is loaded.
Variables for options that may be repeated have list
values. Other variables can be set using VAR=VAL on
the command line. After the script is loaded, option
values that were not set on the command line are
replaced by the values set in the script.
-F=FILE, --config=FILE
Domain configuration to use (SXP).
SXP is the underlying configuration format used by
Xen.
SXP configurations can be hand-written or generated
from Python configuration scripts, using the -n
(dryrun) option to print the configuration.
-n, --dryrun Dry run - prints the resulting configuration in SXP
but does not create the domain.
-x, --xmldryrun XML dry run - prints the resulting configuration in
XML but does not create the domain.
-s, --skipdtd Skip DTD checking - skips checks on XML before
creating. Experimental. Can decrease create time.
-p, --paused Leave the domain paused after it is created.
-c, --console_autoconnect
Connect to the console after the domain is created.
--vncviewer Connect to the VNC display after the domain is
created.
--vncviewer-autopass Pass VNC password to viewer via stdin and -autopass.

[options] [vars]

Create a domain.

Domain creation parameters can be set by command-line switches, from
a python configuration script or an SXP config file. See documentation
for defconfig, config. Configuration variables can be set using
VAR=VAL on the command line. For example vmid=3 sets vmid to 3.


...
Wed Jan 16 09:50:04 IST 2013...
Wed Jan 16 09:50:04 IST 2013
at com.oracle.ovm.mgr.action.ActionEngine.sendCommandToServer(ActionEngine.java:507)
at com.oracle.ovm.mgr.action.ActionEngine.sendDispatchedServerCommand(ActionEngine.java:444)
at com.oracle.ovm.mgr.action.ActionEngine.sendServerCommand(ActionEngine.java:378)
at com.oracle.ovm.mgr.action.VirtualMachineAction.sendStartVmCommand(VirtualMachineAction.java:121)
at com.oracle.ovm.mgr.op.virtual.VirtualMachineStart.asyncAction(VirtualMachineStart.java:65)
at com.oracle.ovm.mgr.api.collectable.ManagedObjectDbImpl.executeCurrentJobOperationAsyncAction(ManagedObjectDbImpl.java:1002)
at sun.reflect.GeneratedMethodAccessor3034.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:329)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:289)
at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:826)
at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:245)
at com.oracle.ovm.mgr.api.virtual.VirtualMachineProxy.executeCurrentJobOperationAsyncAction(Unknown Source)
at com.oracle.ovm.mgr.api.job.JobEngine.objectAsyncStart(JobEngine.java:596)
at com.oracle.ovm.mgr.op.virtual.VirtualMachineStart.action(VirtualMachineStart.java:52)
at com.oracle.ovm.mgr.api.collectable.ManagedObjectDbImpl.executeCurrentJobOperationAction(ManagedObjectDbImpl.java:1012)
at sun.reflect.GeneratedMethodAccessor1231.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:329)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:289)
at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:826)
at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:245)
at com.oracle.ovm.mgr.api.virtual.VirtualMachineProxy.executeCurrentJobOperationAction(Unknown Source)
at com.oracle.ovm.mgr.api.job.JobEngine.operationActioner(JobEngine.java:218)
at com.oracle.ovm.mgr.api.job.JobEngine.objectActioner(JobEngine.java:309)
at com.oracle.ovm.mgr.api.job.InternalJobDbImpl.objectCommitter(InternalJobDbImpl.java:1140)
at sun.reflect.GeneratedMethodAccessor1239.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:329)
at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:289)
at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:136)
at com.oracle.odof.command.InvokeMethodCommand.process(InvokeMethodCommand.java:100)
at com.oracle.odof.core.BasicWork.processCommand(BasicWork.java:81)
at com.oracle.odof.core.TransactionManager.processCommand(TransactionManager.java:773)
at com.oracle.odof.core.WorkflowManager.processCommand(WorkflowManager.java:401)
at com.oracle.odof.core.WorkflowManager.processWork(WorkflowManager.java:459)
at com.oracle.odof.io.AbstractClient.run(AbstractClient.java:42)
at java.lang.Thread.run(Thread.java:662)
Caused by: com.oracle.ovm.mgr.api.exception.IllegalOperationException: OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@206.236.134.19:8899/api/2 start_vm 0004fb0000030000e40574ce6f97e906 0004fb0000060000ff2b96ea9cc79078, Status: org.apache.xmlrpc.XmlRpcException: exceptions.RuntimeError:Command: ['xm', 'create', '/OVS/Repositories/0004fb0000030000e40574ce6f97e906/VirtualMachines/0004fb0000060000ff2b96ea9cc79078/vm.cfg'] failed (1): stderr: Error: Unable to open config file: /OVS/Repositories/0004fb0000030000e40574ce6f97e906/VirtualMachines/0004fb0000060000ff2b96ea9cc79078/vm.cfg
stdout: Usage: xm create [options] [vars]

Create a domain based on .

Options:

-h, --help Print this help.
--help_config Print the available configuration variables (vars)
for the configuration script.
-q, --quiet Quiet.
--path=PATH Search path for configuration scripts. The value of
PATH is a colon-separated directory list.
-f=FILE, --defconfig=FILE
Use the given Python configuration script.The
configuration script is loaded after arguments have
been processed. Each command-line option sets a
configuration variable named after its long option
name, and these variables are placed in the
environment of the script before it is loaded.
Variables for options that may be repeated have list
values. Other variables can be set using VAR=VAL on
the command line. After the script is loaded, option
values that were not set on the command line are
replaced by the values set in the script.
-F=FILE, --config=FILE
Domain configuration to use (SXP).
SXP is the underlying configuration format used by
Xen.
SXP configurations can be hand-written or generated
from Python configuration scripts, using the -n
(dryrun) option to print the configuration.
-n, --dryrun Dry run - prints the resulting configuration in SXP
but does not create the domain.
-x, --xmldryrun XML dry run - prints the resulting configuration in
XML but does not create the domain.
-s, --skipdtd Skip DTD checking - skips checks on XML before
creating. Experimental. Can decrease create time.
-p, --paused Leave the domain paused after it is created.
-c, --console_autoconnect
Connect to the console after the domain is created.
--vncviewer Connect to the VNC display after the domain is
created.
--vncviewer-autopass Pass VNC password to viewer via stdin and -autopass.

[options] [vars]

Create a domain.

Domain creation parameters can be set by command-line switches, from
a python configuration script or an SXP config file. See documentation
for defconfig, config. Configuration variables can be set using
VAR=VAL on the command line. For example vmid=3 sets vmid to 3.


...
Wed Jan 16 09:50:04 IST 2013
at com.oracle.ovm.mgr.action.ActionEngine.sendAction(ActionEngine.java:798)
at com.oracle.ovm.mgr.action.ActionEngine.sendCommandToServer(ActionEngine.java:503)
... 40 more


----------
End of Job
----------
  • 1. Re: OVMAPI_4004E Server Failed Command:- NFS repo VM starting issue
    984927 Newbie
    Currently Being Moderated
    Hi,

    This may be a naive question, but I take it you have verfied that this NFS filesystem is mounted on both your OVM servers and at the same location (/OVS/Repositories/0004fb0000030000e40574ce6f97e906/VirtualMachines)? It seems to me the error is suggesting that one of our servers isn't seeing this storage mounted..., or is not seeing the vm.cfg.

    Regards

    Julian
  • 2. Re: OVMAPI_4004E Server Failed Command:- NFS repo VM starting issue
    ovmnewbee Newbie
    Currently Being Moderated
    Hi Julian

    thanks for your response. I have cross validated and confirm that NFS filesystem is mounted on both the OVM servers

    Server englabbld10

    [root@englabbld10 /]# df -kh
    Filesystem Size Used Avail Use% Mounted on
    /dev/sda2 3.0G 683M 2.1G 25% /
    /dev/sda1 99M 28M 67M 29% /boot
    tmpfs 503M 0 503M 0% /dev/shm
    none 503M 80K 503M 1% /var/lib/xenstored
    /dev/mapper/360a98000433465644e346f2d4836514d
    10G 398M 9.7G 4% /poolfsmnt/0004fb000005000046c0cb2a4467e2d6
    206.236.134.65:/vol/ovmnfs
    300G 5.2G 295G 2% /OVS/Repositories/0004fb0000030000e40574ce6f97e906
    /dev/mapper/360a98000433465644e346f2d48377670
    476G 23G 453G 5% /OVS/Repositories/0004fb000003000036658ed4a80c8716
    [root@englabbld10 /]#


    Server englabbld09
    [root@englabbld09 ~]# df -kh
    Filesystem Size Used Avail Use% Mounted on
    /dev/sda2 3.0G 683M 2.1G 25% /
    /dev/sda1 99M 28M 67M 29% /boot
    tmpfs 503M 0 503M 0% /dev/shm
    none 503M 152K 503M 1% /var/lib/xenstored
    /dev/mapper/360a98000433465644e346f2d4836514d
    10G 398M 9.7G 4% /poolfsmnt/0004fb000005000046c0cb2a4467e2d6
    206.236.134.65:/vol/ovmnfs
    300G 5.2G 295G 2% /OVS/Repositories/0004fb0000030000e40574ce6f97e906
    /dev/mapper/360a98000433465644e346f2d48377670
    476G 23G 453G 5% /OVS/Repositories/0004fb000003000036658ed4a80c8716
    [root@englabbld09 ~]#

    Edited by: ovmnewbee on Jan 16, 2013 5:04 PM
  • 3. Re: OVMAPI_4004E Server Failed Command:- NFS repo VM starting issue
    984927 Newbie
    Currently Being Moderated
    Hi,

    Thanks - I can see it's mounted on both - have you verified you can actually see the vm.cfg on both servers, and tested that your NFS mount is actually r/w by attempting to create a file?

    Cheers

    Julian
  • 4. Re: OVMAPI_4004E Server Failed Command:- NFS repo VM starting issue
    ovmnewbee Newbie
    Currently Being Moderated
    Hi Julian

    I was able to create/delete a folder called "test" in the NFS filesystem on both the servers,

    however can you please let me know in which directory the vm.cfg file is located?
  • 5. Re: OVMAPI_4004E Server Failed Command:- NFS repo VM starting issue
    984927 Newbie
    Currently Being Moderated
    Hi,

    According to the output you uploaded, it is trying to find:
    /OVS/Repositories/0004fb0000030000e40574ce6f97e906/VirtualMachines/0004fb0000060000ff2b96ea9cc79078/vm.cfg
  • 6. Re: OVMAPI_4004E Server Failed Command:- NFS repo VM starting issue
    ovmnewbee Newbie
    Currently Being Moderated
    Yes Julian...you are right...I am not able to access it from englabbld10 getting a permission denied error. But I am able to access it from englabbld09. How do I resolve this issue ? ...please advise. I tried changing the permission on englabbld10 but was not able to do it getting an "Operation not permitted" error.


    root@englabbld10 VirtualMachines]# pwd
    /OVS/Repositories/0004fb0000030000e40574ce6f97e906/VirtualMachines
    [root@englabbld10 VirtualMachines]# cd 0004fb0000060000ff2b96ea9cc79078
    -bash: cd: 0004fb0000060000ff2b96ea9cc79078: Permission denied

    [root@englabbld09 VirtualMachines]# cd 0004fb0000060000ff2b96ea9cc79078
    [root@englabbld09 0004fb0000060000ff2b96ea9cc79078]# ls
    vm.cfg
  • 7. Re: OVMAPI_4004E Server Failed Command:- NFS repo VM starting issue
    984927 Newbie
    Currently Being Moderated
    I think I would probably start by investigating the exports on the NFS server, ensuring you are exporting RW= and root= to both hosts. Could be as simple as a typo or omission...
  • 8. Re: OVMAPI_4004E Server Failed Command:- NFS repo VM starting issue
    ovmnewbee Newbie
    Currently Being Moderated
    yes..Julian you were right it was a typo error in regard to the export permission on the NFS server. I re-created the permissions and it seems to have worked...many thanks for your inputs.

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points