2 Replies Latest reply: Dec 3, 2012 10:10 AM by budachst RSS

    Repository Creation fails with Time Out Error

    itsakc
      Hi
      I am trying a test setup of OVM3.1.1-305. I am using OpenFiler for iSCSI disks and have created 20GB iSCSI disk for Pool FileSystem and another 260GB iSCSI disk for Data Repository. The server pool gets created fine with the Pool FS and HA and Cluster options enabled. But while exposing the iSCSI disk for data store the job times out. Details of the error message is as below, please help!



      The Job has timed out while executing, and will be aborted.

      #####################################

      Job Construction Phase
      ----------------------
      begin()
      Appended operation 'File System Construct' to object '0004fb000009000001b30fa7bc24bef1 (Local FS ibcovs1)'.
      Appended operation 'Cluster File System Present' to object 'fb5840a9a3a78f51'.
      Appended operation 'Repository Construct' to object '0004fb00000300009bfb51738df3f282 (OPNFILER (2))'.
      commit()
      Completed Step: COMMIT

      Objects and Operations
      ----------------------
      Object (IN_USE): [LocalFileServer] 0004fb000009000001b30fa7bc24bef1 (Local FS ibcovs1)
      Operation: File System Construct
      Object (CREATED): [Repository] 0004fb00000300009bfb51738df3f282 (OVSRepos)
      Operation: Repository Construct
      Object (IN_USE): [LocalFileServer] 0004fb0000090000a8d525219658c28a (Local FS ibcovs2)
      Object (IN_USE): [Cluster] fb5840a9a3a78f51
      Operation: Cluster File System Present
      Object (CREATED): [LocalFileSystem] 0004fb00000500000cd1d94d87e829e8 (fs_OVSRepos)
      Object (IN_USE): [StorageElement] 0004fb0000180000bf6e8d3ff957b21b (OPNFILER (2))

      Job Running Phase at 04:01 on Mon, Dec 3, 2012
      ----------------------------------------------
      Job Participants: [44:45:4c:4c:48:00:10:44:80:52:b4:c0:4f:33:42:53 (ibcovs1), 44:45:4c:4c:48:00:10:46:80:32:c6:c0:4f:34:42:53 (ibcovs2)]


      Actioner
      --------
      Starting operation 'File System Construct' on object '0004fb00000500000cd1d94d87e829e8 (fs_OVSRepos)'
      Job: 1354487506506, aborted post-commit by user: admin

      Write Methods Invoked
      -------------------
      Class=InternalJobDbImpl vessel_id=2477 method=addTransactionIdentifier accessLevel=6
      Class=LocalFileServerDbImpl vessel_id=788 method=createFileSystem accessLevel=6
      Class=LocalFileSystemDbImpl vessel_id=2486 method=setName accessLevel=6
      Class=LocalFileSystemDbImpl vessel_id=2486 method=setFoundryContext accessLevel=6
      Class=LocalFileSystemDbImpl vessel_id=2486 method=onPersistableCreate accessLevel=6
      Class=LocalFileSystemDbImpl vessel_id=2486 method=setLifecycleState accessLevel=6
      Class=LocalFileSystemDbImpl vessel_id=2486 method=setRollbackLifecycleState accessLevel=6
      Class=LocalFileSystemDbImpl vessel_id=2486 method=setRefreshed accessLevel=6
      Class=LocalFileSystemDbImpl vessel_id=2486 method=setBackingDevices accessLevel=6
      Class=LocalFileSystemDbImpl vessel_id=2486 method=setUuid accessLevel=6
      Class=LocalFileSystemDbImpl vessel_id=2486 method=setPath accessLevel=6
      Class=LocalFileSystemDbImpl vessel_id=2486 method=setSimpleName accessLevel=6
      Class=LocalFileSystemDbImpl vessel_id=2486 method=addFileServer accessLevel=6
      Class=LocalFileSystemDbImpl vessel_id=2486 method=setStorageDevice accessLevel=6
      Class=StorageElementDbImpl vessel_id=1121 method=addLayeredFileSystem accessLevel=6
      Class=LocalFileSystemDbImpl vessel_id=2486 method=setSimpleName accessLevel=6
      Class=LocalFileServerDbImpl vessel_id=922 method=addFileSystem accessLevel=6
      Class=LocalFileSystemDbImpl vessel_id=2486 method=addFileServer accessLevel=6
      Class=ClusterDbImpl vessel_id=1340 method=addLocalFileSystem accessLevel=6
      Class=LocalFileSystemDbImpl vessel_id=2486 method=setCluster accessLevel=6
      Class=LocalFileSystemDbImpl vessel_id=2486 method=setAsset accessLevel=6
      Class=LocalFileSystemDbImpl vessel_id=2486 method=createRepository accessLevel=6
      Class=RepositoryDbImpl vessel_id=2492 method=setName accessLevel=6
      Class=RepositoryDbImpl vessel_id=2492 method=setFoundryContext accessLevel=6
      Class=RepositoryDbImpl vessel_id=2492 method=onPersistableCreate accessLevel=6
      Class=RepositoryDbImpl vessel_id=2492 method=setLifecycleState accessLevel=6
      Class=RepositoryDbImpl vessel_id=2492 method=setRollbackLifecycleState accessLevel=6
      Class=RepositoryDbImpl vessel_id=2492 method=setRefreshed accessLevel=6
      Class=RepositoryDbImpl vessel_id=2492 method=setDom0Uuid accessLevel=6
      Class=RepositoryDbImpl vessel_id=2492 method=setSharePath accessLevel=6
      Class=RepositoryDbImpl vessel_id=2492 method=setSimpleName accessLevel=6
      Class=RepositoryDbImpl vessel_id=2492 method=setFileSystem accessLevel=6
      Class=LocalFileSystemDbImpl vessel_id=2486 method=addRepository accessLevel=6
      Class=RepositoryDbImpl vessel_id=2492 method=setManagerUuid accessLevel=6
      Class=RepositoryDbImpl vessel_id=2492 method=setVersion accessLevel=6
      Class=RepositoryDbImpl vessel_id=2492 method=addJobOperation accessLevel=6
      Class=RepositoryDbImpl vessel_id=2492 method=setSimpleName accessLevel=6
      Class=RepositoryDbImpl vessel_id=2492 method=setDescription accessLevel=6
      Class=InternalJobDbImpl vessel_id=2477 method=setCompletedStep accessLevel=6
      Class=InternalJobDbImpl vessel_id=2477 method=setAssociatedHandles accessLevel=6
      Job Internal Error (Operation)com.oracle.ovm.mgr.api.exception.FailedOperationException: OVMAPI_B000E Storage plugin command [storage_plugin_createFileSystem] failed for storage server [0004fb000009000001b30fa7bc24bef1] failed with [com.oracle.ovm.mgr.api.exception.FailedOperationException: OVMAPI_4010E Attempt to send command: dispatch to server: ibcovs1 failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@10.183.104.24:8899/api/2 storage_plugin_createFileSystem oracle.ocfs2.OCFS2.OCFS2Plugin 0004fb00000500000cd1d94d87e829e8 /dev/mapper/14f504e46494c4552637a6f5154372d7643426d2d49333763 0, Status: java.lang.InterruptedException
      Mon Dec 03 04:03:48 IST 2012
      Mon Dec 03 04:03:48 IST 2012] OVMAPI_4010E Attempt to send command: dispatch to server: ibcovs1 failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@10.183.104.24:8899/api/2 storage_plugin_createFileSystem oracle.ocfs2.OCFS2.OCFS2Plugin 0004fb00000500000cd1d94d87e829e8 /dev/mapper/14f504e46494c4552637a6f5154372d7643426d2d49333763 0, Status: java.lang.InterruptedException
      Mon Dec 03 04:03:48 IST 2012
      Mon Dec 03 04:03:48 IST 2012
      Mon Dec 03 04:03:48 IST 2012
      at com.oracle.ovm.mgr.action.StoragePluginAction.processException(StoragePluginAction.java:1371)
      at com.oracle.ovm.mgr.action.StoragePluginAction.createFileSystem(StoragePluginAction.java:894)
      at com.oracle.ovm.mgr.op.physical.storage.FileSystemConstruct.createFileSystem(FileSystemConstruct.java:57)
      at com.oracle.ovm.mgr.op.physical.storage.FileSystemConstruct.action(FileSystemConstruct.java:49)
      at com.oracle.ovm.mgr.api.collectable.ManagedObjectDbImpl.executeCurrentJobOperationAction(ManagedObjectDbImpl.java:1009)
      at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330)
      at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)
      at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:822)
      at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:245)
      at com.oracle.ovm.mgr.api.physical.storage.LocalFileServerProxy.executeCurrentJobOperationAction(Unknown Source)
      at com.oracle.ovm.mgr.api.job.JobEngine.operationActioner(JobEngine.java:218)
      at com.oracle.ovm.mgr.api.job.JobEngine.objectActioner(JobEngine.java:309)
      at com.oracle.ovm.mgr.api.job.InternalJobDbImpl.objectCommitter(InternalJobDbImpl.java:1140)
      at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330)
      at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)
      at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:136)
      at com.oracle.odof.command.InvokeMethodCommand.process(InvokeMethodCommand.java:100)
      at com.oracle.odof.core.BasicWork.processCommand(BasicWork.java:81)
      at com.oracle.odof.core.TransactionManager.processCommand(TransactionManager.java:773)
      at com.oracle.odof.core.WorkflowManager.processCommand(WorkflowManager.java:401)
      at com.oracle.odof.core.WorkflowManager.processWork(WorkflowManager.java:459)
      at com.oracle.odof.io.AbstractClient.run(AbstractClient.java:42)
      at java.lang.Thread.run(Thread.java:662)
      Caused by: com.oracle.ovm.mgr.api.exception.FailedOperationException: OVMAPI_4010E Attempt to send command: dispatch to server: ibcovs1 failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@10.183.104.24:8899/api/2 storage_plugin_createFileSystem oracle.ocfs2.OCFS2.OCFS2Plugin 0004fb00000500000cd1d94d87e829e8 /dev/mapper/14f504e46494c4552637a6f5154372d7643426d2d49333763 0, Status: java.lang.InterruptedException
      Mon Dec 03 04:03:48 IST 2012
      Mon Dec 03 04:03:48 IST 2012
      at com.oracle.ovm.mgr.action.ActionEngine.sendCommandToServer(ActionEngine.java:507)
      at com.oracle.ovm.mgr.action.ActionEngine.sendDispatchedServerCommand(ActionEngine.java:444)
      at com.oracle.ovm.mgr.action.ActionEngine.sendServerCommand(ActionEngine.java:378)
      at com.oracle.ovm.mgr.action.StoragePluginAction.createFileSystem(StoragePluginAction.java:890)
      ... 27 more
      Caused by: com.oracle.ovm.mgr.api.exception.IllegalOperationException: OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@10.183.104.24:8899/api/2 storage_plugin_createFileSystem oracle.ocfs2.OCFS2.OCFS2Plugin 0004fb00000500000cd1d94d87e829e8 /dev/mapper/14f504e46494c4552637a6f5154372d7643426d2d49333763 0, Status: java.lang.InterruptedException
      Mon Dec 03 04:03:48 IST 2012
      at com.oracle.ovm.mgr.action.ActionEngine.sendAction(ActionEngine.java:798)
      at com.oracle.ovm.mgr.action.ActionEngine.sendCommandToServer(ActionEngine.java:503)
      ... 30 more


      FailedOperationCleanup
      ----------
      Starting failed operation 'File System Construct' cleanup on object 'fs_OVSRepos'
      Complete rollback operation 'File System Construct' completed with direction=fs_OVSRepos

      Rollbacker
      ----------
      Executing rollback operation 'File System Construct' on object '0004fb00000500000cd1d94d87e829e8 (fs_OVSRepos)'
      Complete rollback operation 'File System Construct' completed with direction=DONE

      Objects To Be Rolled Back
      -------------------------
      Object (IN_USE): [LocalFileServer] 0004fb000009000001b30fa7bc24bef1 (Local FS ibcovs1)
      Object (CREATED): [Repository] 0004fb00000300009bfb51738df3f282 (OVSRepos)
      Object (IN_USE): [LocalFileServer] 0004fb0000090000a8d525219658c28a (Local FS ibcovs2)
      Object (IN_USE): [Cluster] fb5840a9a3a78f51
      Object (CREATED): [LocalFileSystem] 0004fb00000500000cd1d94d87e829e8 (fs_OVSRepos)
      Object (IN_USE): [StorageElement] 0004fb0000180000bf6e8d3ff957b21b (OPNFILER (2))


      ###############################

      Thanks & Regards