9 Replies Latest reply on Apr 22, 2012 9:33 AM by 463096

    Templates fails to upload

    Jrl-Oracle
      I am getting an error while uploading a template.
      In the message console I am getting this error
      Job Construction Phase
      ----------------------
      begin()
      Appended operation 'Import Virtual Machine Template' to object 'cfgFile_0004fb00001400005c33cf92650cd43f'.
      Appended operation 'Virtual Machine Cfg File Configure' to object 'cfgFile_0004fb00001400005c33cf92650cd43f'.
      commit()
      Completed Step: COMMIT

      Objects and Operations
      ----------------------
      Object (CREATED): [VirtualMachineTemplate] 0004fb00001400005c33cf92650cd43f (E1_DB.tgz)
      Object (CREATED): [VirtualMachineCfgFile] cfgFile_0004fb00001400005c33cf92650cd43f
      Operation: Import Virtual Machine Template
      Operation: Virtual Machine Cfg File Configure

      Job Running Phase at 07:20 on Wed, Feb 8, 2012
      ----------------------------------------------
      Job Participants: [34:30:33:34:31:33:55:53:45:36:32:33:4e:36:4e:36 (dnptovm200)]


      Actioner
      --------
      Starting operation 'Import Virtual Machine Template' on object 'cfgFile_0004fb00001400005c33cf92650cd43f'
      Job Internal Error (Operation)com.oracle.ovm.mgr.api.exception.JobFailureException: OVMAPI_5001E Job "1328710821448" Failed due to failure "(02/08/2012 08:01:45:873 AM)
      Async operation failed on server: dnptovm200.
      Object: cfgFile_0004fb00001400005c33cf92650cd43f,
      PID: 25479,
      Server error message: Template import error: No VM configure file found.
      " on server "34:30:33:34:31:33:55:53:45:36:32:33:4e:36:4e:36" associated with object "cfgFile_0004fb00001400005c33cf92650cd43f"
      Wed Feb 08 08:01:46 MST 2012
      at com.oracle.ovm.mgr.api.job.JobEngine.checkJobStatus(JobEngine.java:602)
      at com.oracle.ovm.mgr.api.job.JobEngine.objectAsyncStart(JobEngine.java:580)
      at com.oracle.ovm.mgr.api.job.JobEngine.objectAsyncStart(JobEngine.java:564)
      at com.oracle.ovm.mgr.op.virtual.VirtualMachineTemplateImport.action(VirtualMachineTemplateImport.java:68)
      at com.oracle.ovm.mgr.api.job.JobEngine.operationActioner(JobEngine.java:193)
      at com.oracle.ovm.mgr.api.job.JobEngine.objectActioner(JobEngine.java:264)
      at com.oracle.ovm.mgr.api.job.InternalJobDbImpl.objectCommitter(InternalJobDbImpl.java:1090)
      at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:247)
      at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:207)
      at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:136)
      at com.oracle.odof.command.InvokeMethodCommand.process(InvokeMethodCommand.java:100)
      at com.oracle.odof.core.BasicWork.processCommand(BasicWork.java:81)
      at com.oracle.odof.core.TransactionManager.processCommand(TransactionManager.java:751)
      at com.oracle.odof.core.WorkflowManager.processCommand(WorkflowManager.java:401)
      at com.oracle.odof.core.WorkflowManager.processWork(WorkflowManager.java:459)
      at com.oracle.odof.io.AbstractClient.run(AbstractClient.java:42)
      at java.lang.Thread.run(Thread.java:662)


      FailedOperationCleanup
      ----------
      Starting failed operation 'Import Virtual Machine Template' cleanup on object 'cfgFile_0004fb00001400005c33cf92650cd43f'
      Complete rollback operation 'Import Virtual Machine Template' completed with direction=cfgFile_0004fb00001400005c33cf92650cd43f

      Rollbacker
      ----------

      Objects To Be Rolled Back
      -------------------------
      Object (CREATED): [VirtualMachineTemplate] 0004fb00001400005c33cf92650cd43f (E1_DB.tgz)
      Object (CREATED): [VirtualMachineCfgFile] cfgFile_0004fb00001400005c33cf92650cd43f


      Write Methods Invoked
      -------------------
      Class=InternalJobDbImpl vessel_id=5391 method=addTransactionIdentifier accessLevel=6
      Class=VirtualMachineTemplateDbImpl vessel_id=5409 method=setName accessLevel=6
      Class=VirtualMachineTemplateDbImpl vessel_id=5409 method=setFoundryContext accessLevel=6
      Class=VirtualMachineTemplateDbImpl vessel_id=5409 method=onPersistableCreate accessLevel=6
      Class=VirtualMachineTemplateDbImpl vessel_id=5409 method=setLifecycleState accessLevel=6
      Class=VirtualMachineTemplateDbImpl vessel_id=5409 method=setRollbackLifecycleState accessLevel=6
      Class=VirtualMachineCfgFileDbImpl vessel_id=5414 method=setName accessLevel=6
      Class=VirtualMachineCfgFileDbImpl vessel_id=5414 method=setFoundryContext accessLevel=6
      Class=VirtualMachineCfgFileDbImpl vessel_id=5414 method=onPersistableCreate accessLevel=6
      Class=VirtualMachineCfgFileDbImpl vessel_id=5414 method=setLifecycleState accessLevel=6
      Class=VirtualMachineCfgFileDbImpl vessel_id=5414 method=setRollbackLifecycleState accessLevel=6
      Class=VirtualMachineCfgFileDbImpl vessel_id=5414 method=setJobAsset accessLevel=6
      Class=VirtualMachineTemplateDbImpl vessel_id=5409 method=setVirtualMachineCfgFile accessLevel=6
      Class=VirtualMachineCfgFileDbImpl vessel_id=5414 method=setVirtualMachine accessLevel=6
      Class=VirtualMachineCfgFileDbImpl vessel_id=5414 method=importVirtualMachineTemplate accessLevel=6
      Class=VirtualMachineTemplateDbImpl vessel_id=5409 method=setSimpleName accessLevel=6
      Class=VirtualMachineCfgFileDbImpl vessel_id=5414 method=setAsset accessLevel=6
      Class=VirtualMachineCfgFileDbImpl vessel_id=5414 method=configure accessLevel=6
      Class=VirtualMachineTemplateDbImpl vessel_id=5409 method=setOrigin accessLevel=6
      Class=VirtualMachineTemplateDbImpl vessel_id=5409 method=setDescription accessLevel=6
      Class=VirtualMachineCfgFileDbImpl vessel_id=5414 method=setAsset accessLevel=6
      Class=VirtualMachineCfgFileDbImpl vessel_id=5414 method=configure accessLevel=6
      Class=InternalJobDbImpl vessel_id=5391 method=setCompletedStep accessLevel=6
      Class=InternalJobDbImpl vessel_id=5391 method=setAssociatedHandles accessLevel=6
      Class=InternalJobDbImpl vessel_id=5391 method=setFailedOperation accessLevel=6
      Class=VirtualMachineTemplateDbImpl vessel_id=5409 method=nextJobOperation accessLevel=6
      Class=VirtualMachineCfgFileDbImpl vessel_id=5414 method=nextJobOperation accessLevel=6
      Completed Step: ROLLBACK


      I then went and check the ovs-agent.log and found this:
      [2012-02-08 07:51:20 6926] DEBUG (OVSNotificationServer:138) Sending notification: {ASYNC_PROC} progress PID 25479
      [2012-02-08 07:51:41 25479] DEBUG (OVSAsyncProc:192) async_proc_prog: notification {ASYNC_PROC} progress PID 25479 status {'OBJECT': 'template', 'TOTAL_OBJECT_COUNT': 1, 'CURRENT_OBJECT_COUNT': 1, 'TOTAL_BYTES': 11713453161L, 'CURRENT_BYTES': 11587552675L, 'OPERATION': 'download'}
      [2012-02-08 07:51:41 25479] DEBUG (OVSNotificationProxy:45) Notification sent: {ASYNC_PROC} progress PID 25479
      [2012-02-08 07:51:41 6926] DEBUG (OVSNotificationServer:138) Sending notification: {ASYNC_PROC} progress PID 25479
      [2012-02-08 07:52:03 25479] DEBUG (OVSAsyncProc:192) async_proc_prog: notification {ASYNC_PROC} progress PID 25479 status {'OBJECT': 'template', 'TOTAL_OBJECT_COUNT': 1, 'CURRENT_OBJECT_COUNT': 1, 'TOTAL_BYTES': 11713453161L, 'CURRENT_BYTES': 11685037467L, 'OPERATION': 'download'}
      [2012-02-08 07:52:03 25479] DEBUG (OVSNotificationProxy:45) Notification sent: {ASYNC_PROC} progress PID 25479
      [2012-02-08 07:52:03 6926] DEBUG (OVSNotificationServer:138) Sending notification: {ASYNC_PROC} progress PID 25479
      [2012-02-08 07:52:16 25479] DEBUG (OVSAsyncProc:192) async_proc_prog: notification {ASYNC_PROC} progress PID 25479 status {'OBJECT': 'template', 'TOTAL_OBJECT_COUNT': 1, 'CURRENT_OBJECT_COUNT': 1, 'TOTAL_BYTES': 11713453161L, 'CURRENT_BYTES': 11713453161L, 'OPERATION': 'download'}
      [2012-02-08 07:52:16 25479] DEBUG (OVSNotificationProxy:45) Notification sent: {ASYNC_PROC} progress PID 25479
      [2012-02-08 07:52:16 6926] DEBUG (OVSNotificationServer:138) Sending notification: {ASYNC_PROC} progress PID 25479
      *[2012-02-08 07:52:20 25479] DEBUG (OVSAsyncProc:104) ovs_async_proc: pid 25479 status 1 output Template import error: No VM configure file found.
      [2012-02-08 07:52:20 25479] DEBUG (OVSNotificationProxy:45) Notification sent: {ASYNC_PROC} exit PID 25479
      [2012-02-08 07:52:20 6926] DEBUG (OVSNotificationServer:138) Sending notification: {ASYNC_PROC} exit PID 25479

      The template comes from a OVM 2 version where it contains
      [root@dendell08 E1_DB_898.4.1_11GR2_X86_64]# ls -lrt
      total 79221520
      -rwxrwxrwx 1 root root 13316728320 Nov 23 2010 system.img
      -rwxrwxrwx 1 root root 350 Jan 11 2011 vm.cfg
      -rwxrwxrwx 1 root root 78643200000 Jan 11 2011 E1_DB.img
      [root@dendell08 E1_DB_898.4.1_11GR2_X86_64]#

      The vm.cfg file contains the following:
      # Automatically generated xen config file
      name = "orclvmdb"
      memory = "4096"
      disk = [ 'file:/OVS/running_pool/orclvmdb/system.img,hda,w',
      'file:/OVS/running_pool/orclvmdb/E1_DB.img,hdb,w', ]
      vif = [ '', ]
      bootloader="/usr/bin/pygrub"
      vcpus=4
      vfb = [ 'type=vnc,vncunused=1,vnclisten=0.0.0.0' ]
      on_reboot = 'restart'
      on_crash = 'restart'


      Is there something wrong with the vm.cfg?
        • 1. Re: Templates fails to upload
          Avi Miller-Oracle
          John Lee wrote:
          Is there something wrong with the vm.cfg?
          The vm.cfg looks OK to me. You could also try unpacking the tarball on your webserver and importing each file individually. Oracle VM 3.0 supports a multi-file import for templates, so just add the URL for all three files (the vm.cfg and the two disk images).
          1 person found this helpful
          • 2. Re: Templates fails to upload
            Jrl-Oracle
            Seperating the file didn't solve the problem but may have changed it. In the ovs-agent.log I see the following:
            [2012-02-09 07:43:21 20641] DEBUG (OVSNotificationProxy:45) Notification sent: {ASYNC_PROC} progress PID 20641
            [2012-02-09 07:43:21 6926] DEBUG (OVSNotificationServer:138) Sending notification: {ASYNC_PROC} progress PID 20641
            [2012-02-09 07:43:44 20641] DEBUG (OVSAsyncProc:192) async_proc_prog: notification {ASYNC_PROC} progress PID 20641 status {'OBJECT': 'template', 'TOTAL_OBJECT_COUNT': 3, 'CURRENT_OBJECT_COUNT': 3, 'TOTAL_BYTES': 78643200000L, 'CURRENT_BYTES': 15838708632L, 'OPERATION': 'download'}
            [2012-02-09 07:43:44 20641] DEBUG (OVSNotificationProxy:45) Notification sent: {ASYNC_PROC} progress PID 20641
            [2012-02-09 07:43:44 6926] DEBUG (OVSNotificationServer:138) Sending notification: {ASYNC_PROC} progress PID 20641
            [2012-02-09 07:43:53 20641] DEBUG (OVSAsyncProc:104) ovs_async_proc: pid 20641 status 1 output Template import error: [Errno 14] curl#23 - "Failed writing body"

            What could casue this error?
            • 3. Re: Templates fails to upload
              911168
              Wild guess here: Out of space?
              • 4. Re: Templates fails to upload
                Jrl-Oracle
                Yeah, that seems possible.

                But to claim back some space I have to master deleting a virtual machine I don't want anymore.

                I have tried to remove the machine with the console and the ovs_update.sh -fixup but the VM won't go away.

                I really don't want to reinstall and recreate the VM that are good.

                Anyone have idea on this error or removing VMs from the server pool?
                • 5. Re: Templates fails to upload
                  Jrl-Oracle
                  The images I am trying to upload are the following size:

                  system.img 12.6 Gigabytes
                  DBase.img 73.2 Gigabytes

                  The repository has 98 Gigabytes of available space.

                  When I tried loading as one tgz file I got errors of
                  *[2012-02-08 07:52:20 25479] DEBUG (OVSAsyncProc:104) ovs_async_proc: pid 25479 status 1 output Template import error: No VM configure file found.

                  When I tried uploading as seperate files in the upload menu I am pretty sure I ran out of space and got the error:
                  [2012-02-09 07:43:53 20641] DEBUG (OVSAsyncProc:104) ovs_async_proc: pid 20641 status 1 output Template import error: [Errno 14] curl#23 - "Failed writing body"

                  Pretty sure this is due to running out of space.

                  Worked through the issues with deleting other images files to free up more space in the respository. Tried again and am nowing getting the following error:

                  (02/16/2012 03:00:21:237 PM) Async operation failed on server: dnptovm200. Object: cfgFile_0004fb0000140000dda73e3bb2a9dfec, PID: 21608, Server error message: Template import error: Cannot find disk file: system.img

                  If I change the order of the upload I get the same error.

                  Could this still be a space issue? Can't add anymore space to repository.

                  This isn't an issue when trying to clone the images it happens when uploading. However, I am concerned that when I do get thsi uploaded I won't have enought space to create a clone of this image to run.

                  Does anyone have any ideas or comments?

                  Edited by: JRL on Feb 17, 2012 9:41 AM
                  • 6. Re: Templates fails to upload
                    911168
                    A template is just a compressed, tar file. So, to make a clone of a tar/gz file, you'll have multiple copies of the object at any time. If you don't have at least triple the size of the final disk with which to work, You are bound to fail.

                    You can un-zip, un-tar the disk elsewhere, and then copy over it images. That's done on the console not in the manager so it doesn't belong on this forum, but if you have the space somewhere else to extract the disk images, you just might get it to fit. You'll have to customize the vm.cfg by hand.

                    I don't know how much concurrency is going on under the covers, but at the worst case the "deploy" does:
                    source un-gzip temp
                    temp un-tar dest

                    So, there are 3 copies (temp is deleted AFTER success). It may not be that bad, but it's not going to fit on your disk.
                    • 7. Re: Templates fails to upload
                      Jrl-Oracle
                      I am uploading untar and zipped files. They are the straight *IMG files and I have already modified the vm.cfg.  So the system is trying to untar and unzip the system.img from the uploaded img files.

                      Repository is 3x the size of uploaded templates is this documented anywhere?


                      This is in OVM 3.0.3. I don't think that you can or should copy images over to the OVS in OVM 3.0.3.

                      Edited by: JRL on Feb 17, 2012 11:55 AM
                      • 8. Re: Templates fails to upload
                        911168
                        I see where Avi Miller suggested this earlier, without using the console. Good idea. untar/unzip it to your webserver and then import the untarred files. You do not have room to clone disks that nearly fill your repository.

                        The system won't "untar and unzip the system.img from the uploaded img files", they are already expanded. It may try to retar and rezip them if you import them as a template. That won't fit by the same reasoning - the uncompressed version must coexist with the compressed version at some point. The system wont' have to compress/uncompress them if you import them as Virtual Disks instead of as a template.

                        "This is in OVM 3.0.3. I don't think that you can or should copy images over to the OVS in OVM 3.0.3."

                        I'd said, "That's done on the console not in the manager so it doesn't belong on this forum" but Avi showed us a way to do it using the Manager. Is that not working?

                        Build (with the manager) a VM with a tiny disk, but the vm.cfg you want. Remove the disk and add in the ones you imported.

                        No, I don't believe you'll find anywhere in the documentation of Oracle VM that you need enought space to copy a file for the copy to work. It's probably assumed. It might need to be explained somewhere, however. And, untarring/unzipping a .tgz file does copy it, and cloning it copies it also.

                        In Summary (you already know this), you need more space.
                        1 person found this helpful
                        • 9. Re: Templates fails to upload
                          463096
                          I have found that templates files >2GB with HTTP or HTTPS protocols using OVMM 3.0.3 running on Linux 6.2 X86_64 fail consistently. Using the FTP protocol from an FTP server setup for Anonymous, Read-Only access resolved the issue for me. Your mileage may vary.

                          By Anonymous, I mean that the FTP server is setup to NOT require a login with a userid/password.

                          A URL of the form ftp://server-ip-address/volume/file-name provided to the OVMM 3.0.3 GUI results in a successful import of files larger than 2GB and larger than 4GB. My FTP Server is a D-LINK DNS-320.

                          There may be other ways of resolving the issues I encountered with files > 2GB using HTTP or HTTPS protocols. Using the FTP protocol was an expedient work-around for me as the NAS device was immediately available.

                          Good luck.