5 Replies Latest reply: May 6, 2013 4:25 PM by 1007403 RSS

    Agent does not show up in Enterprise Manager GC after being redeployed

    user12136669
      Hi,

      We are running Grid Control 11g.

      One of my team members deployed a Grid Control agent out onto a server using the "Install Agent" functionality in Enterprise Manager GC.

      The agent deployment worked successfully. The host name of the target server appeared under the 'Hosts' screen in the GC Enterprise Manager targets tab.

      Shortly after my team member had deployed the agent, he realised he had installed the Grid Control Agent into a misspelled Agent Home on the target server. He then attempted to delete the agent. He did this by shutting down the agent and removing the agent home directory on the target server. He then realised that the removal operation was not complete, so he removed the host via the Enterprise Manager GUI. He did this on the Hosts screen in the targets tab. He selected the host and then chose 'remove'. There are no databases or listeners on the target server, so he did not have to remove those first.

      He then attempted to redeploy the agent. The agent deployment operation said it has finished successfully, however the name of the host does not appear under the 'targets' screen in EM. Besides the 'deployment history' section of EM, there are no traces of the host in the Enterprise Management console at all.

      The agent is up and running up on the target server. The expected ports are in use.

      On the target server, I tried running

      $ ./emctl status agent
      Oracle Enterprise Manager 11g Release 1 Grid Control 11.1.0.1.0
      Copyright (c) 1996, 2010 Oracle Corporation. All rights reserved.
      ---------------------------------------------------------------
      Agent Version : 11.1.0.1.0
      OMS Version : 11.1.0.1.0
      Protocol Version : 11.1.0.0.0
      Agent Home : /sv/app/oracle/gcagent/agent11g
      Agent binaries : /sv/app/oracle/gcagent/agent11g
      Agent Process ID : 15613
      Parent Process ID : 15583
      Agent URL : http://xxxxxx:xxxx/emd/main/
      Repository URL : http://xxxxxxxxxxx:xxxxx/em/upload/
      Started at : 2011-05-11 13:40:52
      Started by user : sv_ora
      Last Reload : 2011-05-11 13:40:52
      Last successful upload : (none)
      Last attempted upload : (none)
      Total Megabytes of XML files uploaded so far : 0.00
      Number of XML files pending upload : 44
      Size of XML files pending upload(MB) : 2.42
      Available disk space on upload filesystem : 86.33%
      Last attempted heartbeat to OMS : 2011-05-11 14:05:00
      Last successful heartbeat to OMS : unknown

      These two lines are of some concern:

      "Total Megabytes of XML files uploaded so far : 0.00
      Last successful heartbeat to OMS : unknown"

      I was concerned that no XML files have been uploaded, so I ran

      * <AGENT_HOME>/bin/emctl reload agent
      EMD reload completed successfully

      The 'emctl status agent' still reports 0.00 megabytes

      * I also ran <AGENT_HOME>/bin/agentca -d

      * We reran <AGENT_HOME>/root.sh on the target server, and rebooted the target server.

      I also restarted the entire OMS on our Grid Control server.

      I suspect that the original removal of the host from grid control marked the host somewhere as being "removed from grid control". I think that the redeployment of the agent has not updated this record, so it is still marked somewhere as being removed. I just do not where this is.

      Does anybody have any ideas of how to proceed from here? I am happy to upload any log files that people think might be useful.
        • 1. Re: Agent does not show up in Enterprise Manager GC after being redeployed
          user12136669
          Update.

          I've got a little further.

          I went into Setup -> Agents. I saw that the agent of my target server was blocked.
          I selected it, and clicked on unblock. It is now reporting in the setup -> agents page as unblocked.

          My emctl status agent command, that I've been running on my target server, is looking a lot happier than it was. It is now reporting a value > 0 for "Total Megabytes of XML files uploaded so far", and the "Last successful heartbeat to OMS" now has a value.

          $ ./emctl status agent
          Oracle Enterprise Manager 11g Release 1 Grid Control 11.1.0.1.0
          Copyright (c) 1996, 2010 Oracle Corporation. All rights reserved.
          ---------------------------------------------------------------
          Agent Version : 11.1.0.1.0
          OMS Version : 11.1.0.1.0
          Protocol Version : 11.1.0.0.0
          Agent Home : /sv/app/oracle/gcagent/agent11g
          Agent binaries : /sv/app/oracle/gcagent/agent11g
          Agent Process ID : 15613
          Parent Process ID : 15583
          Agent URL : http://xxxxxxxx/emd/main/
          Repository URL : http://xxxxxxxxx/em/upload/
          Started at : 2011-05-11 13:40:52
          Started by user : sv_ora
          Last Reload : 2011-05-11 15:23:36
          Last successful upload : 2011-05-11 15:24:25
          Total Megabytes of XML files uploaded so far : 0.08
          Number of XML files pending upload : 0
          Size of XML files pending upload(MB) : 0.00
          Available disk space on upload filesystem : 86.33%
          Last successful heartbeat to OMS : 2011-05-11 15:27:36
          ---------------------------------------------------------------

          HOWEVER, when I navigate to the "Targets -> Hosts" page, my target server is still not showing up in the list of hosts.

          The "Agent" page for my target server seems to be half working.
          The page reports the "Status" as Up but "Host" as unavailable.
          I was able to configure some target listeners from this page, and I successfully used the "Upload Metric Data" button.

          I then clicked on the "Agent Resynchronization" link. I chose the "Unblock agent on successful completion of agent resynchronization" option.

          The operation fails with this message:

          "Agent Operation completed with errors. For those targets that could not be saved, please go to the target's monitoring configuration page to save them. All other targets have been saved successfully. Agent has not been unblocked.
               
          Internal Repository Error Message: SQL Exception occured while syncing pdp settings | Exception: java.sql.SQLException: ORA-20206: Target does not exist: <target hostname>:host ORA-06512: at "SYSMAN.MGMT_TARGET", line 571 ORA-06512: at "SYSMAN.MGMT_CREDENTIALS_UI", line 303 ORA-06512: at line 1"

          Does anybody have any ideas how to fix this?

          Thanks
          • 2. Re: Agent does not show up in Enterprise Manager GC after being redeployed
            vivalavida
            Hi,
            please check article-ID "Grid Agent Resynchronization Results in ORA-20206: Target does not exist: agentmachine.domain:host" [ID 1273737.1]
            in MOS...

            The reason is certainly that the agent has been reinstalled on the node and that the target.xml file is not correct. You will find the file target.xml
            in the directory $AGENT_HOME/sysman/emd. It might be helpful to check that file since it contains all discovered targets.

            But try to follow the instructions of the mentioned article-ID...

            Edited by: VivaLaVida on 11.05.2011 08:12
            • 3. Re: Agent does not show up in Enterprise Manager GC after being redeployed
              user12136669
              Thanks, I followed the steps on the note you gave me to recreate targets.xml and it worked!

              * Stopped the agent (emctl stop agent)
              * Recreated targets.xml as per the instructions in the note.
              * Started the agent (emctl start agent)

              * Reran agentca -d (to reregister the targets).

              Then I went to the EM GC Gui to manually readd the agent, but it had already picked it up automatically.
              • 4. Re: Agent does not show up in Enterprise Manager GC after being redeployed
                Sanya
                Correct solution.... kudos!!!
                • 5. Re: Agent does not show up in Enterprise Manager GC after being redeployed
                  1007403
                  I don't find agentca binary, where's it?