1 Reply Latest reply: Feb 11, 2013 3:05 PM by Looney128 RSS

    Problem refreshing ISCSI SAN on 3.2.1

    Looney128
      Any one else have this problem?

      If I want to add a new LUN from my ISCSI SAN....I give the servers access....refresh the SAN Server within OVM....nothing happens.
      I re-scan the physical disks on the servers.....I get these errors below.
      If I reboot the server...then it can see the new LUNs.
      The LUNs where there when I first discovered the SAN server, no reboot required?????? Just when I add new LUNs.

      Job Construction Phase
      ----------------------
      Job ID: 1360612584011

      begin()
      Appended operation 'Storage Array Discover Storage Elements' to object '31:33:37:35:32:32:53:55:45:31:35:33:52:30:4c:32 (ovmsco1)'.
      Appended operation 'ISCSI Storage Array Discover Targets' to object '0004fb0000090000894cc51c39b1b988 (Juneau)'.
      Appended operation 'Server Refresh' to object '31:33:37:35:32:32:53:55:45:31:35:33:52:30:4c:32 (ovmsco1)'.
      Appended operation 'Storage Array Refresh' to object '31:33:37:35:32:32:53:55:45:31:35:33:52:30:4c:32 (ovmsco1)'.
      Appended operation 'Storage Array Rescan' to object '31:33:37:35:32:32:53:55:45:31:35:33:52:30:4c:32 (ovmsco1)'.
      commit()
      Completed Step: COMMIT

      Objects and Operations
      ----------------------
      Object (IN_USE): [Server] 31:33:37:35:32:32:53:55:45:31:35:33:52:30:4c:32 (ovmsco1)
      Operation: Storage Array Discover Storage Elements
      Operation: Server Refresh
      Operation: Storage Array Refresh
      Operation: Storage Array Rescan
      Object (IN_USE): [IscsiStorageArray] 0004fb0000090000894cc51c39b1b988 (Juneau)
      Operation: ISCSI Storage Array Discover Targets

      Job Running Phase at 2013-02-11 13:56:24,011
      ----------------------------------------------
      Job Participants: []


      Actioner
      --------
      13:56:24,192: Starting operation 'Storage Array Discover Storage Elements' on object '0004fb0000090000894cc51c39b1b988 (Juneau)'
      13:56:24,193: Completed operation 'Storage Array Discover Storage Elements' with direction ==> LATER
      13:56:24,197: Starting operation 'ISCSI Storage Array Discover Targets' on object '0004fb0000090000894cc51c39b1b988 (Juneau)'
      Setting Context to model only in job with id=1360612584011
      Setting Context to default in job with id=1360612584011
      13:56:24,348: Completed operation 'ISCSI Storage Array Discover Targets' with direction ==> DONE
      13:56:24,351: Starting operation 'Server Refresh' on object '31:33:37:35:32:32:53:55:45:31:35:33:52:30:4c:32 (ovmsco1)'
      13:56:24,353: Completed operation 'Server Refresh' with direction ==> LATER
      13:56:24,357: Starting operation 'Storage Array Refresh' on object '31:33:37:35:32:32:53:55:45:31:35:33:52:30:4c:32 (ovmsco1)'
      Job Internal Error (Operation)com.oracle.ovm.mgr.api.exception.FailedOperationException: OVMAPI_B000E Storage plugin command [storage_plugin_refresh] failed for storage server [0004fb0000090000894cc51c39b1b988] failed with [com.oracle.ovm.mgr.api.exception.FailedOperationException: OVMAPI_4010E Attempt to send command: dispatch to server: ovmsco1 failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@172.20.1.11:8899/api/3 storage_plugin_refresh oracle.generic.SCSIPlugin.GenericPlugin, Status: OSCPlugin.RefreshFailedEx:'Target refresh failed: iscsiadm: No portal found.\n' [Mon Feb 11 13:56:27 CST 2013] [Mon Feb 11 13:56:27 CST 2013]] OVMAPI_4010E Attempt to send command: dispatch to server: ovmsco1 failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@172.20.1.11:8899/api/3 storage_plugin_refresh oracle.generic.SCSIPlugin.GenericPlugin, Status: org.apache.xmlrpc.XmlRpcException: OSCPlugin.RefreshFailedEx:'Target refresh failed: iscsiadm: No portal found.\n' [Mon Feb 11 13:56:27 CST 2013] [Mon Feb 11 13:56:27 CST 2013] [Mon Feb 11 13:56:27 CST 2013]
      at com.oracle.ovm.mgr.action.StoragePluginAction.processException(StoragePluginAction.java:1358)
      at com.oracle.ovm.mgr.action.StoragePluginAction.refresh(StoragePluginAction.java:126)
      at com.oracle.ovm.mgr.op.physical.storage.StorageRefresh.refresh(StorageRefresh.java:59)
      at com.oracle.ovm.mgr.op.physical.storage.StorageRefresh.action(StorageRefresh.java:50)
      at com.oracle.ovm.mgr.api.collectable.ManagedObjectDbImpl.executeCurrentJobOperationAction(ManagedObjectDbImpl.java:1156)
      at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:356)
      at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:333)
      at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:865)
      at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:244)
      at com.oracle.ovm.mgr.api.physical.ServerProxy.executeCurrentJobOperationAction(Unknown Source)
      at com.oracle.ovm.mgr.api.job.JobEngine.operationActioner(JobEngine.java:230)
      at com.oracle.ovm.mgr.api.job.JobEngine.objectActioner(JobEngine.java:322)
      at com.oracle.ovm.mgr.api.job.InternalJobDbImpl.objectCommitter(InternalJobDbImpl.java:1340)
      at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:356)
      at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:333)
      at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:106)
      at com.oracle.odof.command.InvokeMethodCommand.process(InvokeMethodCommand.java:92)
      at com.oracle.odof.core.BasicWork.processCommand(BasicWork.java:81)
      at com.oracle.odof.core.TransactionManager.processCommand(TransactionManager.java:752)
      at com.oracle.odof.core.WorkflowManager.processCommand(WorkflowManager.java:456)
      at com.oracle.odof.core.WorkflowManager.processWork(WorkflowManager.java:514)
      at com.oracle.odof.io.AbstractClient.run(AbstractClient.java:42)
      at java.lang.Thread.run(Thread.java:662)
      Caused by: com.oracle.ovm.mgr.api.exception.FailedOperationException: OVMAPI_4010E Attempt to send command: dispatch to server: ovmsco1 failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@172.20.1.11:8899/api/3 storage_plugin_refresh oracle.generic.SCSIPlugin.GenericPlugin, Status: org.apache.xmlrpc.XmlRpcException: OSCPlugin.RefreshFailedEx:'Target refresh failed: iscsiadm: No portal found.\n' [Mon Feb 11 13:56:27 CST 2013] [Mon Feb 11 13:56:27 CST 2013]
      at com.oracle.ovm.mgr.action.ActionEngine.sendCommandToServer(ActionEngine.java:512)
      at com.oracle.ovm.mgr.action.ActionEngine.sendDispatchedServerCommand(ActionEngine.java:449)
      at com.oracle.ovm.mgr.action.ActionEngine.sendServerCommand(ActionEngine.java:383)
      at com.oracle.ovm.mgr.action.StoragePluginAction.refresh(StoragePluginAction.java:122)
      ... 27 more
      Caused by: com.oracle.ovm.mgr.api.exception.IllegalOperationException: OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@172.20.1.11:8899/api/3 storage_plugin_refresh oracle.generic.SCSIPlugin.GenericPlugin, Status: org.apache.xmlrpc.XmlRpcException: OSCPlugin.RefreshFailedEx:'Target refresh failed: iscsiadm: No portal found.\n' [Mon Feb 11 13:56:27 CST 2013]
      at com.oracle.ovm.mgr.action.ActionEngine.sendAction(ActionEngine.java:803)
      at com.oracle.ovm.mgr.action.ActionEngine.sendCommandToServer(ActionEngine.java:508)
      ... 30 more


      FailedOperationCleanup
      ----------
      Starting failed operation 'Storage Array Refresh' cleanup on object 'ovmsco1'
      Complete rollback operation 'Storage Array Refresh' cleanup on object 'ovmsco1'

      Rollbacker
      ----------
      13:56:27,224: Starting rollbacker...
      Executing rollback operation 'Storage Array Refresh' on object '31:33:37:35:32:32:53:55:45:31:35:33:52:30:4c:32 (ovmsco1)'
      Complete rollback operation 'Storage Array Refresh' completed with direction=DONE
      Executing rollback operation 'ISCSI Storage Array Discover Targets' on object '0004fb0000090000894cc51c39b1b988 (Juneau)'
      Complete rollback operation 'ISCSI Storage Array Discover Targets' completed with direction=DONE
      13:56:27,237: Rollbacker completed...

      Objects To Be Rolled Back
      -------------------------
      Object (IN_USE): [Server] 31:33:37:35:32:32:53:55:45:31:35:33:52:30:4c:32 (ovmsco1)
      Object (IN_USE): [IscsiStorageArray] 0004fb0000090000894cc51c39b1b988 (Juneau)


      Write Methods Invoked
      -------------------
      13:56:24,107 Class=InternalJobDbImpl vessel_id=4678 method=addTransactionIdentifier accessLevel=6 owningTx=1360612584108
      13:56:24,127 Class=ServerDbImpl vessel_id=512 method=refreshStorageLayer accessLevel=6 owningTx=1360612584108
      13:56:24,131 Class=IscsiStorageArrayDbImpl vessel_id=2676 method=addJobOperation accessLevel=6 owningTx=1360612584108
      13:56:24,141 Class=InternalJobDbImpl vessel_id=4678 method=setCompletedStep accessLevel=6 owningTx=1360612584108
      13:56:24,142 Class=InternalJobDbImpl vessel_id=4678 method=setAssociatedHandles accessLevel=6 owningTx=1360612584108
      13:56:24,197 Class=InternalJobDbImpl vessel_id=4678 method=setTuringMachineFlag accessLevel=6 owningTx=1360612584108
      13:56:24,197 Class=ServerDbImpl vessel_id=512 method=setCurrentOperationToLater accessLevel=6 owningTx=1360612584108
      13:56:24,197 Class=InternalJobDbImpl vessel_id=4678 method=setTuringMachineFlag accessLevel=6 owningTx=1360612584108
      13:56:24,199 Class=InternalJobDbImpl vessel_id=4678 method=setContext accessLevel=6 owningTx=1360612584108
      13:56:24,346 Class=InternalJobDbImpl vessel_id=4678 method=setContext accessLevel=6 owningTx=1360612584108
      13:56:24,351 Class=IscsiStorageArrayDbImpl vessel_id=2676 method=setCurrentJobOperationComplete accessLevel=6 owningTx=1360612584108
      13:56:24,351 Class=IscsiStorageArrayDbImpl vessel_id=2676 method=nextJobOperation accessLevel=6 owningTx=1360612584108
      13:56:24,356 Class=InternalJobDbImpl vessel_id=4678 method=setTuringMachineFlag accessLevel=6 owningTx=1360612584108
      13:56:24,356 Class=ServerDbImpl vessel_id=512 method=setCurrentOperationToLater accessLevel=6 owningTx=1360612584108
      13:56:24,357 Class=InternalJobDbImpl vessel_id=4678 method=setTuringMachineFlag accessLevel=6 owningTx=1360612584108
      13:56:27,026 Class=ServerDbImpl vessel_id=512 method=nextJobOperation accessLevel=6 owningTx=1360612584108
      13:56:27,026 Class=InternalJobDbImpl vessel_id=4678 method=setFailedOperation accessLevel=6 owningTx=1360612584108
      13:56:27,226 Class=ServerDbImpl vessel_id=512 method=nextJobOperation accessLevel=6 owningTx=1360612584108
      13:56:27,226 Class=IscsiStorageArrayDbImpl vessel_id=2676 method=nextJobOperation accessLevel=6 owningTx=1360612584108
      13:56:27,232 Class=ServerDbImpl vessel_id=512 method=nextJobOperation accessLevel=6 owningTx=1360612584108
      13:56:27,237 Class=IscsiStorageArrayDbImpl vessel_id=2676 method=nextJobOperation accessLevel=6 owningTx=1360612584108
      Completed Step: ROLLBACK
      Job failed commit (internal) due to OVMAPI_B000E Storage plugin command [storage_plugin_refresh] failed for storage server [0004fb0000090000894cc51c39b1b988] failed with [com.oracle.ovm.mgr.api.exception.FailedOperationException: OVMAPI_4010E Attempt to send command: dispatch to server: ovmsco1 failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@172.20.1.11:8899/api/3 storage_plugin_refresh oracle.generic.SCSIPlugin.GenericPlugin, Status: OSCPlugin.RefreshFailedEx:'Target refresh failed: iscsiadm: No portal found.\n' [Mon Feb 11 13:56:27 CST 2013] [Mon Feb 11 13:56:27 CST 2013]] OVMAPI_4010E Attempt to send command: dispatch to server: ovmsco1 failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@172.20.1.11:8899/api/3 storage_plugin_refresh oracle.generic.SCSIPlugin.GenericPlugin, Status: org.apache.xmlrpc.XmlRpcException: OSCPlugin.RefreshFailedEx:'Target refresh failed: iscsiadm: No portal found.\n' [Mon Feb 11 13:56:27 CST 2013] [Mon Feb 11 13:56:27 CST 2013] [Mon Feb 11 13:56:27 CST 2013]
      com.oracle.ovm.mgr.api.exception.FailedOperationException: OVMAPI_B000E Storage plugin command [storage_plugin_refresh] failed for storage server [0004fb0000090000894cc51c39b1b988] failed with [com.oracle.ovm.mgr.api.exception.FailedOperationException: OVMAPI_4010E Attempt to send command: dispatch to server: ovmsco1 failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@172.20.1.11:8899/api/3 storage_plugin_refresh oracle.generic.SCSIPlugin.GenericPlugin, Status: OSCPlugin.RefreshFailedEx:'Target refresh failed: iscsiadm: No portal found.\n' [Mon Feb 11 13:56:27 CST 2013] [Mon Feb 11 13:56:27 CST 2013]] OVMAPI_4010E Attempt to send command: dispatch to server: ovmsco1 failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@172.20.1.11:8899/api/3 storage_plugin_refresh oracle.generic.SCSIPlugin.GenericPlugin, Status: org.apache.xmlrpc.XmlRpcException: OSCPlugin.RefreshFailedEx:'Target refresh failed: iscsiadm: No portal found.\n' [Mon Feb 11 13:56:27 CST 2013] [Mon Feb 11 13:56:27 CST 2013] [Mon Feb 11 13:56:27 CST 2013]
      at com.oracle.ovm.mgr.action.StoragePluginAction.processException(StoragePluginAction.java:1358)
      at com.oracle.ovm.mgr.action.StoragePluginAction.refresh(StoragePluginAction.java:126)
      at com.oracle.ovm.mgr.op.physical.storage.StorageRefresh.refresh(StorageRefresh.java:59)
      at com.oracle.ovm.mgr.op.physical.storage.StorageRefresh.action(StorageRefresh.java:50)
      at com.oracle.ovm.mgr.api.collectable.ManagedObjectDbImpl.executeCurrentJobOperationAction(ManagedObjectDbImpl.java:1156)
      at sun.reflect.GeneratedMethodAccessor690.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:356)
      at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:333)
      at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:865)
      at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:244)
      at com.oracle.ovm.mgr.api.physical.ServerProxy.executeCurrentJobOperationAction(Unknown Source)
      at com.oracle.ovm.mgr.api.job.JobEngine.operationActioner(JobEngine.java:230)
      at com.oracle.ovm.mgr.api.job.JobEngine.objectActioner(JobEngine.java:322)
      at com.oracle.ovm.mgr.api.job.InternalJobDbImpl.objectCommitter(InternalJobDbImpl.java:1340)
      at sun.reflect.GeneratedMethodAccessor858.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:356)
      at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:333)
      at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:106)
      at com.oracle.odof.command.InvokeMethodCommand.process(InvokeMethodCommand.java:92)
      at com.oracle.odof.core.BasicWork.processCommand(BasicWork.java:81)
      at com.oracle.odof.core.TransactionManager.processCommand(TransactionManager.java:752)
      at com.oracle.odof.core.WorkflowManager.processCommand(WorkflowManager.java:456)
      at com.oracle.odof.core.WorkflowManager.processWork(WorkflowManager.java:514)
      at com.oracle.odof.io.AbstractClient.run(AbstractClient.java:42)
      at java.lang.Thread.run(Thread.java:662)
      Caused by: com.oracle.ovm.mgr.api.exception.FailedOperationException: OVMAPI_4010E Attempt to send command: dispatch to server: ovmsco1 failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@172.20.1.11:8899/api/3 storage_plugin_refresh oracle.generic.SCSIPlugin.GenericPlugin, Status: org.apache.xmlrpc.XmlRpcException: OSCPlugin.RefreshFailedEx:'Target refresh failed: iscsiadm: No portal found.\n' [Mon Feb 11 13:56:27 CST 2013] [Mon Feb 11 13:56:27 CST 2013]
      at com.oracle.ovm.mgr.action.ActionEngine.sendCommandToServer(ActionEngine.java:512)
      at com.oracle.ovm.mgr.action.ActionEngine.sendDispatchedServerCommand(ActionEngine.java:449)
      at com.oracle.ovm.mgr.action.ActionEngine.sendServerCommand(ActionEngine.java:383)
      at com.oracle.ovm.mgr.action.StoragePluginAction.refresh(StoragePluginAction.java:122)
      ... 27 more
      Caused by: com.oracle.ovm.mgr.api.exception.IllegalOperationException: OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@172.20.1.11:8899/api/3 storage_plugin_refresh oracle.generic.SCSIPlugin.GenericPlugin, Status: org.apache.xmlrpc.XmlRpcException: OSCPlugin.RefreshFailedEx:'Target refresh failed: iscsiadm: No portal found.\n' [Mon Feb 11 13:56:27 CST 2013]
      at com.oracle.ovm.mgr.action.ActionEngine.sendAction(ActionEngine.java:803)
      at com.oracle.ovm.mgr.action.ActionEngine.sendCommandToServer(ActionEngine.java:508)
      ... 30 more


      ----------
      End of Job
      ----------
        • 1. Re: Problem refreshing ISCSI SAN on 3.2.1
          Looney128
          Here is the other error

          Job Construction Phase
          ----------------------
          Job ID: 1360612831751

          begin()
          Appended operation 'Storage Array Discover Storage Elements' to object '31:33:37:35:32:32:53:55:45:31:35:33:52:30:4c:32 (ovmsco1)'.
          Appended operation 'ISCSI Storage Array Discover Targets' to object '0004fb0000090000894cc51c39b1b988 (Juneau)'.
          Appended operation 'Server Refresh' to object '31:33:37:35:32:32:53:55:45:31:35:33:52:30:4c:32 (ovmsco1)'.
          Appended operation 'Storage Array Refresh' to object '31:33:37:35:32:32:53:55:45:31:35:33:52:30:4c:32 (ovmsco1)'.
          Appended operation 'Storage Array Rescan' to object '31:33:37:35:32:32:53:55:45:31:35:33:52:30:4c:32 (ovmsco1)'.
          Operation 'Storage Array Discover Storage Elements' was not added, it already exists on object '31:33:37:35:32:32:53:55:45:31:35:33:52:30:4c:32 (ovmsco1)'.
          Appended operation 'ISCSI Storage Array Discover Targets' to object '0004fb0000090000f8f2530e60d3227c (Group2)'.
          Operation 'Server Refresh' was not added, it already exists on object '31:33:37:35:32:32:53:55:45:31:35:33:52:30:4c:32 (ovmsco1)'.
          Operation 'Storage Array Refresh' was not added, it already exists on object '31:33:37:35:32:32:53:55:45:31:35:33:52:30:4c:32 (ovmsco1)'.
          Operation 'Storage Array Rescan' was not added, it already exists on object '31:33:37:35:32:32:53:55:45:31:35:33:52:30:4c:32 (ovmsco1)'.
          Operation 'Storage Array Discover Storage Elements' was not added, it already exists on object '31:33:37:35:32:32:53:55:45:31:35:33:52:30:4c:32 (ovmsco1)'.
          Appended operation 'ISCSI Storage Array Discover Targets' to object '0004fb0000090000231ddadb0304bbfb (Group1)'.
          Operation 'Server Refresh' was not added, it already exists on object '31:33:37:35:32:32:53:55:45:31:35:33:52:30:4c:32 (ovmsco1)'.
          Operation 'Storage Array Refresh' was not added, it already exists on object '31:33:37:35:32:32:53:55:45:31:35:33:52:30:4c:32 (ovmsco1)'.
          Operation 'Storage Array Rescan' was not added, it already exists on object '31:33:37:35:32:32:53:55:45:31:35:33:52:30:4c:32 (ovmsco1)'.
          commit()
          Completed Step: COMMIT

          Objects and Operations
          ----------------------
          Object (IN_USE): [Server] 31:33:37:35:32:32:53:55:45:31:35:33:52:30:4c:32 (ovmsco1)
          Operation: Storage Array Discover Storage Elements
          Operation: Server Refresh
          Operation: Storage Array Refresh
          Operation: Storage Array Rescan
          Object (IN_USE): [IscsiStorageArray] 0004fb0000090000231ddadb0304bbfb (Group1)
          Operation: ISCSI Storage Array Discover Targets
          Object (IN_USE): [IscsiStorageArray] 0004fb0000090000894cc51c39b1b988 (Juneau)
          Operation: ISCSI Storage Array Discover Targets
          Object (IN_USE): [IscsiStorageArray] 0004fb0000090000f8f2530e60d3227c (Group2)
          Operation: ISCSI Storage Array Discover Targets

          Job Running Phase at 2013-02-11 14:00:31,751
          ----------------------------------------------
          Job Participants: []


          Actioner
          --------
          14:00:32,186: Starting operation 'Storage Array Discover Storage Elements' on object '0004fb0000090000894cc51c39b1b988 (Juneau)'
          14:00:32,188: Completed operation 'Storage Array Discover Storage Elements' with direction ==> LATER
          14:00:32,191: Starting operation 'ISCSI Storage Array Discover Targets' on object '0004fb0000090000231ddadb0304bbfb (Group1)'
          Setting Context to model only in job with id=1360612831751
          Setting Context to default in job with id=1360612831751
          14:00:32,340: Completed operation 'ISCSI Storage Array Discover Targets' with direction ==> DONE
          14:00:32,344: Starting operation 'ISCSI Storage Array Discover Targets' on object '0004fb0000090000894cc51c39b1b988 (Juneau)'
          Setting Context to model only in job with id=1360612831751
          Setting Context to default in job with id=1360612831751
          14:00:32,488: Completed operation 'ISCSI Storage Array Discover Targets' with direction ==> DONE
          14:00:32,559: Starting operation 'ISCSI Storage Array Discover Targets' on object '0004fb0000090000f8f2530e60d3227c (Group2)'
          Setting Context to model only in job with id=1360612831751
          Setting Context to default in job with id=1360612831751
          14:00:32,720: Completed operation 'ISCSI Storage Array Discover Targets' with direction ==> DONE
          14:00:32,736: Starting operation 'Server Refresh' on object '31:33:37:35:32:32:53:55:45:31:35:33:52:30:4c:32 (ovmsco1)'
          14:00:32,738: Completed operation 'Server Refresh' with direction ==> LATER
          14:00:32,756: Starting operation 'Storage Array Refresh' on object '31:33:37:35:32:32:53:55:45:31:35:33:52:30:4c:32 (ovmsco1)'
          Job Internal Error (Operation)com.oracle.ovm.mgr.api.exception.FailedOperationException: OVMAPI_B000E Storage plugin command [storage_plugin_refresh] failed for storage server [0004fb0000090000894cc51c39b1b988] failed with [com.oracle.ovm.mgr.api.exception.FailedOperationException: OVMAPI_4010E Attempt to send command: dispatch to server: ovmsco1 failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@172.20.1.11:8899/api/3 storage_plugin_refresh oracle.generic.SCSIPlugin.GenericPlugin, Status: OSCPlugin.RefreshFailedEx:'Target refresh failed: iscsiadm: No portal found.\n' [Mon Feb 11 14:00:35 CST 2013] [Mon Feb 11 14:00:35 CST 2013]] OVMAPI_4010E Attempt to send command: dispatch to server: ovmsco1 failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@172.20.1.11:8899/api/3 storage_plugin_refresh oracle.generic.SCSIPlugin.GenericPlugin, Status: org.apache.xmlrpc.XmlRpcException: OSCPlugin.RefreshFailedEx:'Target refresh failed: iscsiadm: No portal found.\n' [Mon Feb 11 14:00:35 CST 2013] [Mon Feb 11 14:00:35 CST 2013] [Mon Feb 11 14:00:35 CST 2013]
          at com.oracle.ovm.mgr.action.StoragePluginAction.processException(StoragePluginAction.java:1358)
          at com.oracle.ovm.mgr.action.StoragePluginAction.refresh(StoragePluginAction.java:126)
          at com.oracle.ovm.mgr.op.physical.storage.StorageRefresh.refresh(StorageRefresh.java:59)
          at com.oracle.ovm.mgr.op.physical.storage.StorageRefresh.action(StorageRefresh.java:50)
          at com.oracle.ovm.mgr.api.collectable.ManagedObjectDbImpl.executeCurrentJobOperationAction(ManagedObjectDbImpl.java:1156)
          at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:356)
          at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:333)
          at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:865)
          at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:244)
          at com.oracle.ovm.mgr.api.physical.ServerProxy.executeCurrentJobOperationAction(Unknown Source)
          at com.oracle.ovm.mgr.api.job.JobEngine.operationActioner(JobEngine.java:230)
          at com.oracle.ovm.mgr.api.job.JobEngine.objectActioner(JobEngine.java:322)
          at com.oracle.ovm.mgr.api.job.InternalJobDbImpl.objectCommitter(InternalJobDbImpl.java:1340)
          at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:356)
          at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:333)
          at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:106)
          at com.oracle.odof.command.InvokeMethodCommand.process(InvokeMethodCommand.java:92)
          at com.oracle.odof.core.BasicWork.processCommand(BasicWork.java:81)
          at com.oracle.odof.core.TransactionManager.processCommand(TransactionManager.java:752)
          at com.oracle.odof.core.WorkflowManager.processCommand(WorkflowManager.java:456)
          at com.oracle.odof.core.WorkflowManager.processWork(WorkflowManager.java:514)
          at com.oracle.odof.io.AbstractClient.run(AbstractClient.java:42)
          at java.lang.Thread.run(Thread.java:662)
          Caused by: com.oracle.ovm.mgr.api.exception.FailedOperationException: OVMAPI_4010E Attempt to send command: dispatch to server: ovmsco1 failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@172.20.1.11:8899/api/3 storage_plugin_refresh oracle.generic.SCSIPlugin.GenericPlugin, Status: org.apache.xmlrpc.XmlRpcException: OSCPlugin.RefreshFailedEx:'Target refresh failed: iscsiadm: No portal found.\n' [Mon Feb 11 14:00:35 CST 2013] [Mon Feb 11 14:00:35 CST 2013]
          at com.oracle.ovm.mgr.action.ActionEngine.sendCommandToServer(ActionEngine.java:512)
          at com.oracle.ovm.mgr.action.ActionEngine.sendDispatchedServerCommand(ActionEngine.java:449)
          at com.oracle.ovm.mgr.action.ActionEngine.sendServerCommand(ActionEngine.java:383)
          at com.oracle.ovm.mgr.action.StoragePluginAction.refresh(StoragePluginAction.java:122)
          ... 27 more
          Caused by: com.oracle.ovm.mgr.api.exception.IllegalOperationException: OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@172.20.1.11:8899/api/3 storage_plugin_refresh oracle.generic.SCSIPlugin.GenericPlugin, Status: org.apache.xmlrpc.XmlRpcException: OSCPlugin.RefreshFailedEx:'Target refresh failed: iscsiadm: No portal found.\n' [Mon Feb 11 14:00:35 CST 2013]
          at com.oracle.ovm.mgr.action.ActionEngine.sendAction(ActionEngine.java:803)
          at com.oracle.ovm.mgr.action.ActionEngine.sendCommandToServer(ActionEngine.java:508)
          ... 30 more


          FailedOperationCleanup
          ----------
          Starting failed operation 'Storage Array Refresh' cleanup on object 'ovmsco1'
          Complete rollback operation 'Storage Array Refresh' cleanup on object 'ovmsco1'

          Rollbacker
          ----------
          14:00:35,477: Starting rollbacker...
          Executing rollback operation 'Storage Array Refresh' on object '31:33:37:35:32:32:53:55:45:31:35:33:52:30:4c:32 (ovmsco1)'
          Complete rollback operation 'Storage Array Refresh' completed with direction=DONE
          Executing rollback operation 'ISCSI Storage Array Discover Targets' on object '0004fb0000090000231ddadb0304bbfb (Group1)'
          Complete rollback operation 'ISCSI Storage Array Discover Targets' completed with direction=DONE
          Executing rollback operation 'ISCSI Storage Array Discover Targets' on object '0004fb0000090000894cc51c39b1b988 (Juneau)'
          Complete rollback operation 'ISCSI Storage Array Discover Targets' completed with direction=DONE
          Executing rollback operation 'ISCSI Storage Array Discover Targets' on object '0004fb0000090000f8f2530e60d3227c (Group2)'
          Complete rollback operation 'ISCSI Storage Array Discover Targets' completed with direction=DONE
          14:00:35,505: Rollbacker completed...

          Objects To Be Rolled Back
          -------------------------
          Object (IN_USE): [Server] 31:33:37:35:32:32:53:55:45:31:35:33:52:30:4c:32 (ovmsco1)
          Object (IN_USE): [IscsiStorageArray] 0004fb0000090000231ddadb0304bbfb (Group1)
          Object (IN_USE): [IscsiStorageArray] 0004fb0000090000894cc51c39b1b988 (Juneau)
          Object (IN_USE): [IscsiStorageArray] 0004fb0000090000f8f2530e60d3227c (Group2)


          Write Methods Invoked
          -------------------
          14:00:31,935 Class=InternalJobDbImpl vessel_id=4787 method=addTransactionIdentifier accessLevel=6 owningTx=1360612831935
          14:00:31,937 Class=ServerDbImpl vessel_id=512 method=refreshStorageLayer accessLevel=6 owningTx=1360612831935
          14:00:31,942 Class=IscsiStorageArrayDbImpl vessel_id=2676 method=addJobOperation accessLevel=6 owningTx=1360612831935
          14:00:31,960 Class=IscsiStorageArrayDbImpl vessel_id=2059 method=addJobOperation accessLevel=6 owningTx=1360612831935
          14:00:31,989 Class=IscsiStorageArrayDbImpl vessel_id=1871 method=addJobOperation accessLevel=6 owningTx=1360612831935
          14:00:32,007 Class=InternalJobDbImpl vessel_id=4787 method=setCompletedStep accessLevel=6 owningTx=1360612831935
          14:00:32,009 Class=InternalJobDbImpl vessel_id=4787 method=setAssociatedHandles accessLevel=6 owningTx=1360612831935
          14:00:32,191 Class=InternalJobDbImpl vessel_id=4787 method=setTuringMachineFlag accessLevel=6 owningTx=1360612831935
          14:00:32,191 Class=ServerDbImpl vessel_id=512 method=setCurrentOperationToLater accessLevel=6 owningTx=1360612831935
          14:00:32,191 Class=InternalJobDbImpl vessel_id=4787 method=setTuringMachineFlag accessLevel=6 owningTx=1360612831935
          14:00:32,197 Class=InternalJobDbImpl vessel_id=4787 method=setContext accessLevel=6 owningTx=1360612831935
          14:00:32,338 Class=InternalJobDbImpl vessel_id=4787 method=setContext accessLevel=6 owningTx=1360612831935
          14:00:32,344 Class=IscsiStorageArrayDbImpl vessel_id=1871 method=setCurrentJobOperationComplete accessLevel=6 owningTx=1360612831935
          14:00:32,344 Class=IscsiStorageArrayDbImpl vessel_id=1871 method=nextJobOperation accessLevel=6 owningTx=1360612831935
          14:00:32,346 Class=InternalJobDbImpl vessel_id=4787 method=setContext accessLevel=6 owningTx=1360612831935
          14:00:32,485 Class=InternalJobDbImpl vessel_id=4787 method=setContext accessLevel=6 owningTx=1360612831935
          14:00:32,559 Class=IscsiStorageArrayDbImpl vessel_id=2676 method=setCurrentJobOperationComplete accessLevel=6 owningTx=1360612831935
          14:00:32,559 Class=IscsiStorageArrayDbImpl vessel_id=2676 method=nextJobOperation accessLevel=6 owningTx=1360612831935
          14:00:32,561 Class=InternalJobDbImpl vessel_id=4787 method=setContext accessLevel=6 owningTx=1360612831935
          14:00:32,716 Class=InternalJobDbImpl vessel_id=4787 method=setContext accessLevel=6 owningTx=1360612831935
          14:00:32,724 Class=IscsiStorageArrayDbImpl vessel_id=2059 method=setCurrentJobOperationComplete accessLevel=6 owningTx=1360612831935
          14:00:32,735 Class=IscsiStorageArrayDbImpl vessel_id=2059 method=nextJobOperation accessLevel=6 owningTx=1360612831935
          14:00:32,755 Class=InternalJobDbImpl vessel_id=4787 method=setTuringMachineFlag accessLevel=6 owningTx=1360612831935
          14:00:32,755 Class=ServerDbImpl vessel_id=512 method=setCurrentOperationToLater accessLevel=6 owningTx=1360612831935
          14:00:32,756 Class=InternalJobDbImpl vessel_id=4787 method=setTuringMachineFlag accessLevel=6 owningTx=1360612831935
          14:00:35,393 Class=ServerDbImpl vessel_id=512 method=nextJobOperation accessLevel=6 owningTx=1360612831935
          14:00:35,393 Class=InternalJobDbImpl vessel_id=4787 method=setFailedOperation accessLevel=6 owningTx=1360612831935
          14:00:35,480 Class=ServerDbImpl vessel_id=512 method=nextJobOperation accessLevel=6 owningTx=1360612831935
          14:00:35,480 Class=IscsiStorageArrayDbImpl vessel_id=1871 method=nextJobOperation accessLevel=6 owningTx=1360612831935
          14:00:35,480 Class=IscsiStorageArrayDbImpl vessel_id=2676 method=nextJobOperation accessLevel=6 owningTx=1360612831935
          14:00:35,480 Class=IscsiStorageArrayDbImpl vessel_id=2059 method=nextJobOperation accessLevel=6 owningTx=1360612831935
          14:00:35,486 Class=ServerDbImpl vessel_id=512 method=nextJobOperation accessLevel=6 owningTx=1360612831935
          14:00:35,493 Class=IscsiStorageArrayDbImpl vessel_id=1871 method=nextJobOperation accessLevel=6 owningTx=1360612831935
          14:00:35,499 Class=IscsiStorageArrayDbImpl vessel_id=2676 method=nextJobOperation accessLevel=6 owningTx=1360612831935
          14:00:35,505 Class=IscsiStorageArrayDbImpl vessel_id=2059 method=nextJobOperation accessLevel=6 owningTx=1360612831935
          Completed Step: ROLLBACK
          Job failed commit (internal) due to OVMAPI_B000E Storage plugin command [storage_plugin_refresh] failed for storage server [0004fb0000090000894cc51c39b1b988] failed with [com.oracle.ovm.mgr.api.exception.FailedOperationException: OVMAPI_4010E Attempt to send command: dispatch to server: ovmsco1 failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@172.20.1.11:8899/api/3 storage_plugin_refresh oracle.generic.SCSIPlugin.GenericPlugin, Status: OSCPlugin.RefreshFailedEx:'Target refresh failed: iscsiadm: No portal found.\n' [Mon Feb 11 14:00:35 CST 2013] [Mon Feb 11 14:00:35 CST 2013]] OVMAPI_4010E Attempt to send command: dispatch to server: ovmsco1 failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@172.20.1.11:8899/api/3 storage_plugin_refresh oracle.generic.SCSIPlugin.GenericPlugin, Status: org.apache.xmlrpc.XmlRpcException: OSCPlugin.RefreshFailedEx:'Target refresh failed: iscsiadm: No portal found.\n' [Mon Feb 11 14:00:35 CST 2013] [Mon Feb 11 14:00:35 CST 2013] [Mon Feb 11 14:00:35 CST 2013]
          com.oracle.ovm.mgr.api.exception.FailedOperationException: OVMAPI_B000E Storage plugin command [storage_plugin_refresh] failed for storage server [0004fb0000090000894cc51c39b1b988] failed with [com.oracle.ovm.mgr.api.exception.FailedOperationException: OVMAPI_4010E Attempt to send command: dispatch to server: ovmsco1 failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@172.20.1.11:8899/api/3 storage_plugin_refresh oracle.generic.SCSIPlugin.GenericPlugin, Status: OSCPlugin.RefreshFailedEx:'Target refresh failed: iscsiadm: No portal found.\n' [Mon Feb 11 14:00:35 CST 2013] [Mon Feb 11 14:00:35 CST 2013]] OVMAPI_4010E Attempt to send command: dispatch to server: ovmsco1 failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@172.20.1.11:8899/api/3 storage_plugin_refresh oracle.generic.SCSIPlugin.GenericPlugin, Status: org.apache.xmlrpc.XmlRpcException: OSCPlugin.RefreshFailedEx:'Target refresh failed: iscsiadm: No portal found.\n' [Mon Feb 11 14:00:35 CST 2013] [Mon Feb 11 14:00:35 CST 2013] [Mon Feb 11 14:00:35 CST 2013]
          at com.oracle.ovm.mgr.action.StoragePluginAction.processException(StoragePluginAction.java:1358)
          at com.oracle.ovm.mgr.action.StoragePluginAction.refresh(StoragePluginAction.java:126)
          at com.oracle.ovm.mgr.op.physical.storage.StorageRefresh.refresh(StorageRefresh.java:59)
          at com.oracle.ovm.mgr.op.physical.storage.StorageRefresh.action(StorageRefresh.java:50)
          at com.oracle.ovm.mgr.api.collectable.ManagedObjectDbImpl.executeCurrentJobOperationAction(ManagedObjectDbImpl.java:1156)
          at sun.reflect.GeneratedMethodAccessor690.invoke(Unknown Source)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          at java.lang.reflect.Method.invoke(Method.java:597)
          at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:356)
          at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:333)
          at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:865)
          at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:244)
          at com.oracle.ovm.mgr.api.physical.ServerProxy.executeCurrentJobOperationAction(Unknown Source)
          at com.oracle.ovm.mgr.api.job.JobEngine.operationActioner(JobEngine.java:230)
          at com.oracle.ovm.mgr.api.job.JobEngine.objectActioner(JobEngine.java:322)
          at com.oracle.ovm.mgr.api.job.InternalJobDbImpl.objectCommitter(InternalJobDbImpl.java:1340)
          at sun.reflect.GeneratedMethodAccessor858.invoke(Unknown Source)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          at java.lang.reflect.Method.invoke(Method.java:597)
          at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:356)
          at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:333)
          at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:106)
          at com.oracle.odof.command.InvokeMethodCommand.process(InvokeMethodCommand.java:92)
          at com.oracle.odof.core.BasicWork.processCommand(BasicWork.java:81)
          at com.oracle.odof.core.TransactionManager.processCommand(TransactionManager.java:752)
          at com.oracle.odof.core.WorkflowManager.processCommand(WorkflowManager.java:456)
          at com.oracle.odof.core.WorkflowManager.processWork(WorkflowManager.java:514)
          at com.oracle.odof.io.AbstractClient.run(AbstractClient.java:42)
          at java.lang.Thread.run(Thread.java:662)
          Caused by: com.oracle.ovm.mgr.api.exception.FailedOperationException: OVMAPI_4010E Attempt to send command: dispatch to server: ovmsco1 failed. OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@172.20.1.11:8899/api/3 storage_plugin_refresh oracle.generic.SCSIPlugin.GenericPlugin, Status: org.apache.xmlrpc.XmlRpcException: OSCPlugin.RefreshFailedEx:'Target refresh failed: iscsiadm: No portal found.\n' [Mon Feb 11 14:00:35 CST 2013] [Mon Feb 11 14:00:35 CST 2013]
          at com.oracle.ovm.mgr.action.ActionEngine.sendCommandToServer(ActionEngine.java:512)
          at com.oracle.ovm.mgr.action.ActionEngine.sendDispatchedServerCommand(ActionEngine.java:449)
          at com.oracle.ovm.mgr.action.ActionEngine.sendServerCommand(ActionEngine.java:383)
          at com.oracle.ovm.mgr.action.StoragePluginAction.refresh(StoragePluginAction.java:122)
          ... 27 more
          Caused by: com.oracle.ovm.mgr.api.exception.IllegalOperationException: OVMAPI_4004E Server Failed Command: dispatch https://?uname?:?pwd?@172.20.1.11:8899/api/3 storage_plugin_refresh oracle.generic.SCSIPlugin.GenericPlugin, Status: org.apache.xmlrpc.XmlRpcException: OSCPlugin.RefreshFailedEx:'Target refresh failed: iscsiadm: No portal found.\n' [Mon Feb 11 14:00:35 CST 2013]
          at com.oracle.ovm.mgr.action.ActionEngine.sendAction(ActionEngine.java:803)
          at com.oracle.ovm.mgr.action.ActionEngine.sendCommandToServer(ActionEngine.java:508)
          ... 30 more


          ----------
          End of Job
          ----------