This discussion is archived
8 Replies Latest reply: Jan 25, 2013 6:33 AM by 980874 RSS

Adding an OVM server back into OVM Manager after a reload

980874 Newbie
Currently Being Moderated
We have created a sandbox environment to test Oracle virtualization and had 2 physical servers setup. One running Oracle OVM and the other Oracle Linux. The Oracle Linux box had OVM Manager installed and was using local storage for the repository.

After about a month of running the Oracle Linux box was reloaded without saving any of the OVM Manager configuration information. After reloading the Oracle Linux box and installing OVM Manager my first issue was adding the OVM server back into a pool. We were getting the error that it was already being managed by another host. After a quick search of the Oracle forum I found a post suggesting that we stop the ovs-agent on the OVM box and remove a file.

These were the commands:
# service ovs-agent stop
# rm /etc/ovs-agent/db/server
# service ovs-agent start

After doing this I was able to discover and add the server into a pool. The next issue was when I went to look at the virtual machines hosted on the OVM machine. All 6 of the virtual machines that were created now show as "orphan_004fb0000060000...". In digging around I noticed that the repository for the local storage was not setup so when I go to try add the local disk back as a repository and I select "create a data repository: select physical disk" it comes back as blank.

If I click the storage tab and look under local file systems I see the disk and it has location repository specified and server listed under presented servers.

I am thinking the mapping is somehow messed up after adding the host back to a new OVM Manager and I need to relink the storage to the repository but am a little lost as to how to accomplish this.

I am trying to avoid reloading the entire OVM server so I don't lose the work already done with the existing virtual machines.
  • 1. Re: Adding an OVM server back into OVM Manager after a reload
    user12273962 Pro
    Currently Being Moderated
    1. Right click the rep under "local file systems" and run a refresh. See if that helps.

    2. Did you have unclustered server pool? If NOT, then where is your clustered server pool mount point?
  • 2. Re: Adding an OVM server back into OVM Manager after a reload
    980874 Newbie
    Currently Being Moderated
    To answer your second question first, It is an unclustered server pool.

    When I run the refresh I get an error. Below are the details:

    Job Construction Phase
    ----------------------
    begin()
    Appended operation 'FileSystem refresh' to object '0004fb0000050000f6e5bfcdec6df114'.
    commit()
    Completed Step: COMMIT

    Objects and Operations
    ----------------------
    Object (IN_USE): [Server] 44:45:4c:4c:31:00:10:51:80:50:b1:c0:4f:4e:4d:31 (fnt1gvs001.syncreon.local)
    Object (IN_USE): [LocalFileServer] 0004fb00000900008975e0e107a39557 (Local FS fnt1gvs001.syncreon.local)
    Object (IN_USE): [LocalFileSystem] 0004fb0000050000f6e5bfcdec6df114
    Operation: FileSystem refresh

    Job Running Phase at 15:03 on Wed, Dec 19, 2012
    ----------------------------------------------
    Job Participants: [44:45:4c:4c:31:00:10:51:80:50:b1:c0:4f:4e:4d:31 (fnt1gvs001.syncreon.local)]


    Actioner
    --------
    Starting operation 'FileSystem refresh' on object '0004fb0000050000f6e5bfcdec6df114'
    Job Internal Error (Indeterminate)com.oracle.ovm.mgr.api.exception.RuleException: OVMRU_002017E Cannot perform operation. This repository: 36842b2b01874c700182161460904cc01, is not owned by this manager
    Wed Dec 19 15:03:54 EST 2012
    at com.oracle.ovm.mgr.rules.modules.api.virtual.RepositoryRules.throwErrorIfNotOwnedByThisManager(RepositoryRules.java:378)
    at com.oracle.ovm.mgr.rules.modules.api.virtual.RepositoryRules.setSimpleNamePre(RepositoryRules.java:197)
    at com.oracle.ovm.mgr.api.job.JobEngine.invokeMethod(JobEngine.java:662)
    at com.oracle.ovm.mgr.api.job.JobEngine.invokeMethod(JobEngine.java:631)
    at com.oracle.ovm.mgr.rules.RulesEngine.runRules(RulesEngine.java:190)
    at com.oracle.ovm.mgr.rules.RulesEngine.preProcess(RulesEngine.java:142)
    at com.oracle.ovm.mgr.model.ModelEngine.preValidate(ModelEngine.java:547)
    at com.oracle.ovm.mgr.model.ModelEngine.access$200(ModelEngine.java:65)
    at com.oracle.ovm.mgr.model.ModelEngine$3.notify(ModelEngine.java:352)
    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:326)
    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)
    at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:822)
    at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:245)
    at com.oracle.ovm.mgr.api.virtual.RepositoryProxy.setSimpleName(Unknown Source)
    at com.oracle.ovm.mgr.op.physical.storage.FileSystemRefresh.processRepoMetadata(FileSystemRefresh.java:338)
    at com.oracle.ovm.mgr.op.physical.storage.FileSystemRefresh.discoverOcfs2FileSystem(FileSystemRefresh.java:188)
    at com.oracle.ovm.mgr.op.physical.storage.FileSystemRefresh.action(FileSystemRefresh.java:84)
    at com.oracle.ovm.mgr.api.collectable.ManagedObjectDbImpl.executeCurrentJobOperationAction(ManagedObjectDbImpl.java:1009)
    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330)
    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)
    at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:822)
    at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:245)
    at com.oracle.ovm.mgr.api.physical.storage.LocalFileSystemProxy.executeCurrentJobOperationAction(Unknown Source)
    at com.oracle.ovm.mgr.api.job.JobEngine.operationActioner(JobEngine.java:218)
    at com.oracle.ovm.mgr.api.job.JobEngine.objectActioner(JobEngine.java:309)
    at com.oracle.ovm.mgr.api.job.InternalJobDbImpl.objectCommitter(InternalJobDbImpl.java:1140)
    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330)
    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)
    at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:136)
    at com.oracle.odof.command.InvokeMethodCommand.process(InvokeMethodCommand.java:100)
    at com.oracle.odof.core.BasicWork.processCommand(BasicWork.java:81)
    at com.oracle.odof.core.TransactionManager.processCommand(TransactionManager.java:773)
    at com.oracle.odof.core.WorkflowManager.processCommand(WorkflowManager.java:401)
    at com.oracle.odof.core.WorkflowManager.processWork(WorkflowManager.java:459)
    at com.oracle.odof.io.AbstractClient.run(AbstractClient.java:42)
    at java.lang.Thread.run(Thread.java:662)


    FailedOperationCleanup
    ----------
    Starting failed operation 'FileSystem refresh' cleanup on object '0004fb0000050000f6e5bfcdec6df114'
    Complete rollback operation 'FileSystem refresh' completed with direction=0004fb0000050000f6e5bfcdec6df114

    Rollbacker
    ----------
    Executing rollback operation 'FileSystem refresh' on object '0004fb0000050000f6e5bfcdec6df114'
    Complete rollback operation 'FileSystem refresh' completed with direction=DONE

    Objects To Be Rolled Back
    -------------------------
    Object (IN_USE): [Server] 44:45:4c:4c:31:00:10:51:80:50:b1:c0:4f:4e:4d:31 (fnt1gvs001.syncreon.local)
    Object (IN_USE): [Repository] 0004fb0000030000eac6a78cc3aa8b91 (36842b2b01874c700182161460904cc01)
    Object (IN_USE): [LocalFileServer] 0004fb00000900008975e0e107a39557 (Local FS fnt1gvs001.syncreon.local)
    Object (IN_USE): [LocalFileSystem] 0004fb0000050000f6e5bfcdec6df114


    Write Methods Invoked
    -------------------
    Class=InternalJobDbImpl vessel_id=1569 method=addTransactionIdentifier accessLevel=6
    Class=LocalFileSystemDbImpl vessel_id=930 method=refresh accessLevel=6
    Class=ServerDbImpl vessel_id=544 method=lock accessLevel=6
    Class=LocalFileServerDbImpl vessel_id=897 method=lock accessLevel=6
    Class=InternalJobDbImpl vessel_id=1569 method=setCompletedStep accessLevel=6
    Class=InternalJobDbImpl vessel_id=1569 method=setAssociatedHandles accessLevel=6
    Class=RepositoryDbImpl vessel_id=1111 method=setSimpleName accessLevel=6
    Class=LocalFileSystemDbImpl vessel_id=930 method=nextJobOperation accessLevel=6
    Class=InternalJobDbImpl vessel_id=1569 method=setFailedOperation accessLevel=6
    Class=ServerDbImpl vessel_id=544 method=nextJobOperation accessLevel=6
    Class=LocalFileServerDbImpl vessel_id=897 method=nextJobOperation accessLevel=6
    Class=LocalFileSystemDbImpl vessel_id=930 method=nextJobOperation accessLevel=6
    Class=LocalFileSystemDbImpl vessel_id=930 method=nextJobOperation accessLevel=6
    Completed Step: ROLLBACK
    Job failed commit (internal) due to com.oracle.ovm.mgr.api.exception.RuleException: OVMRU_002017E Cannot perform operation. This repository: 36842b2b01874c700182161460904cc01, is not owned by this manager
    Wed Dec 19 15:03:54 EST 2012
    at com.oracle.ovm.mgr.rules.modules.api.virtual.RepositoryRules.throwErrorIfNotOwnedByThisManager(RepositoryRules.java:378)
    at com.oracle.ovm.mgr.rules.modules.api.virtual.RepositoryRules.setSimpleNamePre(RepositoryRules.java:197)
    at com.oracle.ovm.mgr.api.job.JobEngine.invokeMethod(JobEngine.java:662)
    at com.oracle.ovm.mgr.api.job.JobEngine.invokeMethod(JobEngine.java:631)
    at com.oracle.ovm.mgr.rules.RulesEngine.runRules(RulesEngine.java:190)
    at com.oracle.ovm.mgr.rules.RulesEngine.preProcess(RulesEngine.java:142)
    at com.oracle.ovm.mgr.model.ModelEngine.preValidate(ModelEngine.java:547)
    at com.oracle.ovm.mgr.model.ModelEngine.access$200(ModelEngine.java:65)
    at com.oracle.ovm.mgr.model.ModelEngine$3.notify(ModelEngine.java:352)
    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:326)
    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)
    at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:822)
    at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:245)
    at com.oracle.ovm.mgr.api.virtual.RepositoryProxy.setSimpleName(Unknown Source)
    at com.oracle.ovm.mgr.op.physical.storage.FileSystemRefresh.processRepoMetadata(FileSystemRefresh.java:338)
    at com.oracle.ovm.mgr.op.physical.storage.FileSystemRefresh.discoverOcfs2FileSystem(FileSystemRefresh.java:188)
    at com.oracle.ovm.mgr.op.physical.storage.FileSystemRefresh.action(FileSystemRefresh.java:84)
    at com.oracle.ovm.mgr.api.collectable.ManagedObjectDbImpl.executeCurrentJobOperationAction(ManagedObjectDbImpl.java:1009)
    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330)
    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)
    at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:822)
    at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:245)
    at com.oracle.ovm.mgr.api.physical.storage.LocalFileSystemProxy.executeCurrentJobOperationAction(Unknown Source)
    at com.oracle.ovm.mgr.api.job.JobEngine.operationActioner(JobEngine.java:218)
    at com.oracle.ovm.mgr.api.job.JobEngine.objectActioner(JobEngine.java:309)
    at com.oracle.ovm.mgr.api.job.InternalJobDbImpl.objectCommitter(InternalJobDbImpl.java:1140)
    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330)
    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)
    at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:136)
    at com.oracle.odof.command.InvokeMethodCommand.process(InvokeMethodCommand.java:100)
    at com.oracle.odof.core.BasicWork.processCommand(BasicWork.java:81)
    at com.oracle.odof.core.TransactionManager.processCommand(TransactionManager.java:773)
    at com.oracle.odof.core.WorkflowManager.processCommand(WorkflowManager.java:401)
    at com.oracle.odof.core.WorkflowManager.processWork(WorkflowManager.java:459)
    at com.oracle.odof.io.AbstractClient.run(AbstractClient.java:42)
    at java.lang.Thread.run(Thread.java:662)
    ...
    Wed Dec 19 15:03:54 EST 2012
    com.oracle.ovm.mgr.api.exception.FailedOperationException: com.oracle.ovm.mgr.api.exception.RuleException: OVMRU_002017E Cannot perform operation. This repository: 36842b2b01874c700182161460904cc01, is not owned by this manager
    Wed Dec 19 15:03:54 EST 2012
    at com.oracle.ovm.mgr.rules.modules.api.virtual.RepositoryRules.throwErrorIfNotOwnedByThisManager(RepositoryRules.java:378)
    at com.oracle.ovm.mgr.rules.modules.api.virtual.RepositoryRules.setSimpleNamePre(RepositoryRules.java:197)
    at com.oracle.ovm.mgr.api.job.JobEngine.invokeMethod(JobEngine.java:662)
    at com.oracle.ovm.mgr.api.job.JobEngine.invokeMethod(JobEngine.java:631)
    at com.oracle.ovm.mgr.rules.RulesEngine.runRules(RulesEngine.java:190)
    at com.oracle.ovm.mgr.rules.RulesEngine.preProcess(RulesEngine.java:142)
    at com.oracle.ovm.mgr.model.ModelEngine.preValidate(ModelEngine.java:547)
    at com.oracle.ovm.mgr.model.ModelEngine.access$200(ModelEngine.java:65)
    at com.oracle.ovm.mgr.model.ModelEngine$3.notify(ModelEngine.java:352)
    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:326)
    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)
    at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:822)
    at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:245)
    at com.oracle.ovm.mgr.api.virtual.RepositoryProxy.setSimpleName(Unknown Source)
    at com.oracle.ovm.mgr.op.physical.storage.FileSystemRefresh.processRepoMetadata(FileSystemRefresh.java:338)
    at com.oracle.ovm.mgr.op.physical.storage.FileSystemRefresh.discoverOcfs2FileSystem(FileSystemRefresh.java:188)
    at com.oracle.ovm.mgr.op.physical.storage.FileSystemRefresh.action(FileSystemRefresh.java:84)
    at com.oracle.ovm.mgr.api.collectable.ManagedObjectDbImpl.executeCurrentJobOperationAction(ManagedObjectDbImpl.java:1009)
    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330)
    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)
    at com.oracle.odof.core.storage.Transaction.invokeMethod(Transaction.java:822)
    at com.oracle.odof.core.Exchange.invokeMethod(Exchange.java:245)
    at com.oracle.ovm.mgr.api.physical.storage.LocalFileSystemProxy.executeCurrentJobOperationAction(Unknown Source)
    at com.oracle.ovm.mgr.api.job.JobEngine.operationActioner(JobEngine.java:218)
    at com.oracle.ovm.mgr.api.job.JobEngine.objectActioner(JobEngine.java:309)
    at com.oracle.ovm.mgr.api.job.InternalJobDbImpl.objectCommitter(InternalJobDbImpl.java:1140)
    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330)
    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)
    at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:136)
    at com.oracle.odof.command.InvokeMethodCommand.process(InvokeMethodCommand.java:100)
    at com.oracle.odof.core.BasicWork.processCommand(BasicWork.java:81)
    at com.oracle.odof.core.TransactionManager.processCommand(TransactionManager.java:773)
    at com.oracle.odof.core.WorkflowManager.processCommand(WorkflowManager.java:401)
    at com.oracle.odof.core.WorkflowManager.processWork(WorkflowManager.java:459)
    at com.oracle.odof.io.AbstractClient.run(AbstractClient.java:42)
    at java.lang.Thread.run(Thread.java:662)
    ...
    Wed Dec 19 15:03:54 EST 2012
    at com.oracle.ovm.mgr.api.job.InternalJobDbImpl.objectCommitter(InternalJobDbImpl.java:1201)
    at sun.reflect.GeneratedMethodAccessor1396.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:330)
    at com.oracle.odof.core.AbstractVessel.invokeMethod(AbstractVessel.java:290)
    at com.oracle.odof.core.BasicWork.invokeMethod(BasicWork.java:136)
    at com.oracle.odof.command.InvokeMethodCommand.process(InvokeMethodCommand.java:100)
    at com.oracle.odof.core.BasicWork.processCommand(BasicWork.java:81)
    at com.oracle.odof.core.TransactionManager.processCommand(TransactionManager.java:773)
    at com.oracle.odof.core.WorkflowManager.processCommand(WorkflowManager.java:401)
    at com.oracle.odof.core.WorkflowManager.processWork(WorkflowManager.java:459)
    at com.oracle.odof.io.AbstractClient.run(AbstractClient.java:42)
    at java.lang.Thread.run(Thread.java:662)


    ----------
    End of Job
    ----------
  • 3. Re: Adding an OVM server back into OVM Manager after a reload
    user12273962 Pro
    Currently Being Moderated
    The only solution I know of is to reinstall the OVM Manager using the old VM managers UUID

    http://docs.oracle.com/cd/E27300_01/E27308/html/vmiug-manager-restoring.html

    I suggest you open a SR with oracle about this if you don't have the old UUID. I don't know of a work around. Maybe someone else does. The repo connection is looking for the old UUID of the manager.
  • 4. Re: Adding an OVM server back into OVM Manager after a reload
    980874 Newbie
    Currently Being Moderated
    Thanks for the info. This is something I was afraid of and was hoping that I could forcefully break the link and reconnect it to the new OVM Manager. I will hold out hope that someone may know another solution.
  • 5. Re: Adding an OVM server back into OVM Manager after a reload
    934842 Newbie
    Currently Being Moderated
    This happened to me not to long ago. I had an issue when deploying 3.2.1 beta and it ended up corrupting my db. I blew away my OVM manager and completely redeployed clean...I did not capture the UUID. The majority of what you need is in the OCFS2 server pool filesystem of the host.

    In the storage tab you said you can see the storage disk. In the repository tab you also stated the storage disks were presented to the pool (or server) correct?

    Do you have the OVM CLI tools on the manager? If so can you confirm the environment with the following:

    - list repository
    - show repository name=<name of repo from above command> (if no name is there change name= to id= and us the id)
    - list server
    - show server name=<name of server from list server command>

    Does it show presented?
  • 6. Re: Adding an OVM server back into OVM Manager after a reload
    945503 Newbie
    Currently Being Moderated
    Yes. The simple answer is that you have to reinstall the OVM server node.
    Any legacy configuration that the server had before you rediscovered it in your manager will still be there, and likely to give you grief.

    The more complicated answer is that you will have to manually hack the server under the hood and undo the previous configuration before you have the manager discover it.
    That's great and all if you know what configuration file in linux that the manager had changed before you dropped the server out of the manager.

    It would be really nice if there was an option to RESET and roll back to configuration on the OVM server node back to a fresh install configuration without having to get media out and reinstall the box.
    Unfortunately no such feature exists yet. (one can dream though)

    Edited by: 942500 on Dec 20, 2012 5:53 PM
  • 7. Re: Adding an OVM server back into OVM Manager after a reload
    980874 Newbie
    Currently Being Moderated
    We are currently running version 3.1.1.305 and from my understanding if I am reading correctly I need 3.1.1.365 for the CLI. I do not currently have a support contract with Oracle so I am unable to download patch ID 14227416.

    I am trying to get some assistance on this issue from our Oracle sales rep.
  • 8. Re: Adding an OVM server back into OVM Manager after a reload
    980874 Newbie
    Currently Being Moderated
    Ok I was able to find this through some digging. It appears that the UUID for your OVM Manager is also stored in the ".ovsrepo" file. The OVS_REPO_MGR_UUID is the Oracle VM Manager UUID.

    The Oracle VM Manager UUID is listed in the “.config ” file on the Oracle VM Manager host in the /u01/app/oracle/ovm-manager-3/ directory AS WELL as in each server pool' .ovsrepo file in the pool file system.

    Once I found this I reinstalled OVM Manager with the correct UUID and was able to reattach the storage repositories without losing any data or having to reload the host machine.

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points