7 Replies Latest reply: Feb 20, 2013 2:26 AM by 991195 RSS

    HP-UX Cluster deploy EM agent11g

    991195
      Hi experts,

      We would like to deploy oracle EM agent on a HP-UX cluster with Service guard.
      The main idea is that we have find that document but we are not sure if is for RAC environment or cluster environtment in general.
      http://docs.oracle.com/cd/E11857_01/install.111/e16847/install_agent_on_clstr_slnt.htm#BABIJIDD
      Must deploy 5 agent to every oracle Database ? is any way to configure the existing EM config either deploy agent on HP-UX cluster?
      Please can you help us how deploy the agent if is an cluster environtment ?
      Cheers
        • 1. Re: HP-UX Cluster deploy EM agent11g
          Courtney Llamas-Oracle
          That doc is talking about RAC Clusters. For SG clusters, what you need is http://docs.oracle.com/cd/E11857_01/em.111/e16790/ha_multi_resource.htm#autoId68
          This will tell you how to configure an agent on each node, and relocate the database target based on package location. If you have multiple packages (1 db per package) this is the recommended method.

          The alternative, is to use ORACLE_HOSTNAME=<package vip) during the install, which means you end up with 1 agent per package and is a lot of overhead and maintenance (How to Configure Grid Control Agents to Monitor Virtual Hostname in HA environments (Doc ID 406014.1)). I highly recommend the relocate target method.

          In 11g, you can also use the emctl relocate command, instead of having to install EMCLI on each target node. You first must set the agent pairs that a target can move between with emcli set_standby_agent (once).
          For information on the EMCLI Method see, Setup and Configure Target Relocate Using EMCLI 10.2 or 11.1 (Doc ID 577443.1)

          1) Make sure you have 11g Agents (or higher)
          2) Install a single Agent on each of the Cluster nodes
          3) Discover the failover targets on each of the active cluster nodes
          4) Flag the targets, so they are allowed to fail-over to another Agent
          via EMCTL. Repeat for each target:
          $ emcli set_standby_agent -src_agent="<primary>" -dest_agent="<backup>" -target_name="<name>" -target_type="<type>"

          On a 3-node (or more) cluster, simply run the EMCLI command for each standby node (changing the dest_agent parameter)
          This setup is just a one-time thing, for each target in the cluster that needs to failover to one or more nodes.

          5) Now that the ground work has been done, you have 2 ways of making the target 'move' to another Agent:

          A> You can use EMCLI to relocate the target:
          $ emcli relocate_targets -src_agent=<old agent> -dest_agent=<new agent> -target_name=<name> -target_type=<type> -copy_from_src -force=yes

          Pro:
          - Can be done 'centrally', so an admin can use this to do a manual failover

          Con:
          - If you want to automate this, using the cluster failover scripts, you will need to deploy EMCLI everywhere (more maintenance)

          B> You can use EMCTL on the 'new agent' to force a failover to that Agent:
          $ emctl relocate_target agent '<name>' '<type>'

          Pro:
          - This is the way to automate the failover in cluster scripts

          Con:
          - You need to be in the Agent home of the surviving machine to run this command (you can ONLY pull targets to the Agent you are running from)
          - An Agent can only 'assume' responsibility if it is allowed to have this target (this is the reason for running the emcli standby_agent commands)
          • 2. Re: HP-UX Cluster deploy EM agent11g
            991195
            Hi thanks for answer .

            So in a nutshel I have two ways to deploy agent for Grid Control at SG cluster of 3 nodes.

            1.- Deploy agent as single host ( 1 for each DB instance )
            2.- Deploy agent as signle host each cluster node. Then configure "Configuring Grid Control Repository in Active/Passive High Availability Environments".


            Silent installation with response files :

            RESPONSEFILE_VERSION=2.2.1.0.0
            TOPLEVEL_COMPONENT={"oracle.sysman.top.agent","11.1.0.1.0"}
            DEINSTALL_LIST={"oracle.sysman.top.agent","11.1.0.1.0"}
            COMPONENT_LANGUAGES={"en"}
            OPTIONAL_CONFIG_TOOLS={"agentca","agentsecure"}
            OMS_HOST=rhoraccc0.dom.es
            OMS_PORT=7799
            AGENT_REGISTRATION_PASSWORD=XXXXXXXXXXXXX
            b_doAgentConfig=true
            b_doDiscovery=true
            b_startAgent=true
            b_secureOMS=true
            ORACLE_AGENT_HOME_LOCATION=/oracle/
            SECURITY_UPDATES_VIA_MYORACLESUPPORT=FALSE
            DECLINE_SECURITY_UPDATES=TRUE
            FROM_LOCATION=/home/oracle/grid/linux/agent/stage/products.xml
            b_upgrade=false
            b_silentInstall=true
            oracle.sysman.top.agent:s_installType="AGENT"
            INSTALL_TYPE="Complete"
            oracle.sysman.ccr:b_doValidation=false
            ORACLE_HOSTNAME=<PHYSICAL HOST>


            Then allow targets fail-over:

            $ emcli relocate_targets -src_agent=<old agent> -dest_agent=<new agent> -target_name=<name> -target_type=<type> -copy_from_src -force=yes


            Cheers
            • 3. Re: HP-UX Cluster deploy EM agent11g
              Courtney Llamas-Oracle
              The relocate_target is only valid when you have 1 agent per host (installed on normal hostname).

              If you install with oracle_hostname, you will only be able to monitor targets within a single package on that specific VIP. If you're listener is not in the package, you'll get TNS errors, etc.
              • 4. Re: HP-UX Cluster deploy EM agent11g
                991195
                So I must install without ORACLE_HOSTNAME=<PHYSICAL HOST>. So this will :

                RESPONSEFILE_VERSION=2.2.1.0.0
                TOPLEVEL_COMPONENT={"oracle.sysman.top.agent","11.1.0.1.0"}
                DEINSTALL_LIST={"oracle.sysman.top.agent","11.1.0.1.0"}
                COMPONENT_LANGUAGES={"en"}
                OPTIONAL_CONFIG_TOOLS={"agentca","agentsecure"}
                OMS_HOST=rhoraccc0.dom.es
                OMS_PORT=7799
                AGENT_REGISTRATION_PASSWORD=XXXXXXXXXXXXX
                b_doAgentConfig=true
                b_doDiscovery=true
                b_startAgent=true
                b_secureOMS=true
                ORACLE_AGENT_HOME_LOCATION=/oracle/
                SECURITY_UPDATES_VIA_MYORACLESUPPORT=FALSE
                DECLINE_SECURITY_UPDATES=TRUE
                FROM_LOCATION=/home/oracle/grid/linux/agent/stage/products.xml
                b_upgrade=false
                b_silentInstall=true
                oracle.sysman.top.agent:s_installType="AGENT"
                INSTALL_TYPE="Complete"
                oracle.sysman.ccr:b_doValidation=false



                What do you mean a normal host?



                Cheers
                • 5. Re: HP-UX Cluster deploy EM agent11g
                  Courtney Llamas-Oracle
                  Host1 = serverA
                  Host2 = serverB
                  Host3 = serverC

                  Cluster contains PackageDB1, PackageDB2, PackageDB3
                  VIPs of PackageDB1, PackageDB2, PackageDB3

                  if you install per package:
                  ORACLE_HOSTNAME=PackageDB1
                  The target will be discovered with PackageDB1 as the Hostname, Agent, Listener Configuration. The VIP has to fail with the package. Only the Active host is monitored or can be patched/have jobs run.

                  If you Install per host and use relocate_target scripts
                  You will have a host/agent/listener for serverA, serverB, serverC. The target will reside on one of these nodes (whichever is active). Using relocate_target, you can move it with failover scripts from A to B to C, etc. Each physical server will have an agent, therefore allowing patching/OS monitoring of the server.
                  • 6. Re: HP-UX Cluster deploy EM agent11g
                    991195
                    Thanks for all your support,

                    I was finally deploy the agent and I could see the DB Instances , listeners, but there are some metric errors. But it seems that was something casual because now I cannot see any information.


                    Error Locating host destination in the repository. You may not have access to the destination or it does not exist.

                    The relocation method is for allowing the other agents to monitor other destinations, right ?

                    emcli relocate_targets -src_agent=<old agent> -dest_agent=<new agent> -target_name=<name> -target_type=<type> -copy_from_src -force=yes


                    Thanks in advance
                    • 7. Re: HP-UX Cluster deploy EM agent11g
                      991195
                      emagent.trc contains the following errors:



                      013-02-20 09:09:44,009 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:09:56,899 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:09:59,074 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:10:00,249 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:10:03,919 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:10:04,432 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:10:05,444 Thread-4 WARN http: nmehl_httpListener: signaled to exit from emctl
                      2013-02-20 09:10:06,484 Thread-4 WARN : Signalled to Exit Normally. Signaled to exit from emctl
                      2013-02-20 09:10:20,533 Thread-1 WARN diagnostics.cpu: nmegam_ActivityManager_init: per-thread cpu usage not available for this platform
                      2013-02-20 09:10:21,245 Thread-1 WARN upload: Merging leftover data file named rawdata0.dat
                      2013-02-20 09:10:21,245 Thread-1 WARN upload: Merging leftover data file named rawdata1.dat
                      2013-02-20 09:10:21,254 Thread-1 WARN upload: Merging leftover data file named rawdata5.dat
                      2013-02-20 09:10:21,254 Thread-1 WARN upload: Merging leftover data file named rawdata6.dat
                      2013-02-20 09:10:21,255 Thread-1 WARN upload: Merging leftover data file named rawdata7.dat
                      2013-02-20 09:10:21,289 Thread-1 WARN command: Job Subsystem Timeout set at 600 seconds
                      2013-02-20 09:10:21,350 Thread-1 WARN upload: Upload manager has no Failure script: disabled
                      2013-02-20 09:10:21,450 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:10:22,170 Thread-1 WARN metadata: TABLE metric umsg_imap_user_count can't define table_name: mail_imap_usercount. Filename:/oracle/agent11g/sysman/admin/metadata/imap.xml
                      2013-02-20 09:10:22,170 Thread-1 WARN metadata: TABLE metric umsg_imap_db_session_count can't define table_name: umsg_imap_dbconnections. Filename:/oracle/agent11g/sysman/admin/metadata/imap.xml
                      2013-02-20 09:10:22,171 Thread-1 WARN metadata: TABLE metric umsg_imap_socket_count can't define table_name: umsg_imap_socketcount. Filename:/oracle/agent11g/sysman/admin/metadata/imap.xml
                      2013-02-20 09:10:22,171 Thread-1 WARN metadata: TABLE metric umsg_imap_connections can't define table_name: mail_imap_connections. Filename:/oracle/agent11g/sysman/admin/metadata/imap.xml
                      2013-02-20 09:10:22,171 Thread-1 WARN metadata: TABLE metric umsg_imap_connections_lost can't define table_name: umsg_imap_connections_lost. Filename:/oracle/agent11g/sysman/admin/metadata/imap.xml
                      2013-02-20 09:10:22,171 Thread-1 WARN metadata: TABLE metric umsg_imap_connections_timeout can't define table_name: umsg_imap_connections_timeout. Filename:/oracle/agent11g/sysman/admin/metadata/imap.xml
                      2013-02-20 09:10:22,171 Thread-1 WARN metadata: TABLE metric umsg_imap_connections_total can't define table_name: umsg_imap_connections_count. Filename:/oracle/agent11g/sysman/admin/metadata/imap.xml
                      2013-02-20 09:10:22,171 Thread-1 WARN metadata: TABLE metric umsg_imap_debug_user_list can't define table_name: mail_pop_users_list. Filename:/oracle/agent11g/sysman/admin/metadata/imap.xml
                      2013-02-20 09:10:22,172 Thread-1 WARN metadata: TABLE metric umsg_imap_ds_metrics can't define table_name: mail_imap_functions. Filename:/oracle/agent11g/sysman/admin/metadata/imap.xml
                      2013-02-20 09:10:22,172 Thread-1 WARN metadata: TABLE metric umsg_imap_network can't define table_name: mail_imap_network. Filename:/oracle/agent11g/sysman/admin/metadata/imap.xml
                      2013-02-20 09:10:22,172 Thread-1 WARN metadata: TABLE metric umsg_imap_network_receive_bytes can't define table_name: umsg_imap_network_receive. Filename:/oracle/agent11g/sysman/admin/metadata/imap.xml
                      2013-02-20 09:10:22,172 Thread-1 WARN metadata: TABLE metric umsg_imap_network_transmit_bytes can't define table_name: umsg_imap_network_transmit. Filename:/oracle/agent11g/sysman/admin/metadata/imap.xml
                      2013-02-20 09:10:22,173 Thread-1 WARN metadata: TABLE metric umsg_imap_command_counts can't define table_name: mail_imap_commandcount. Filename:/oracle/agent11g/sysman/admin/metadata/imap.xml
                      2013-02-20 09:10:22,173 Thread-1 WARN metadata: TABLE metric umsg_imap_protocol_metrics can't define table_name: umsg_imap_frequencytimes. Filename:/oracle/agent11g/sysman/admin/metadata/imap.xml
                      2013-02-20 09:10:22,173 Thread-1 WARN metadata: TABLE metric umsg_sps_metrics can't define table_name: mail_imap_sps. Filename:/oracle/agent11g/sysman/admin/metadata/imap.xml
                      2013-02-20 09:10:22,173 Thread-1 WARN metadata: TABLE metric umsg_uptime can't define table_name: mail_imap_uptime. Filename:/oracle/agent11g/sysman/admin/metadata/imap.xml
                      2013-02-20 09:10:23,032 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:10:23,630 Thread-1 WARN metadata: File=file:/oracle/agent11g/sysman/admin/metadata/oracle_ovf_ivr.xml,line=399: <InstanceProperty> is not valid element, will be ignored
                      2013-02-20 09:10:24,577 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:10:24,596 Thread-1 WARN metadata: TABLE metric uptime can't define table_name: mail_pop_uptime. Filename:/oracle/agent11g/sysman/admin/metadata/pop.xml
                      2013-02-20 09:10:24,597 Thread-1 WARN metadata: TABLE metric debug_usercount can't define table_name: mail_pop_users_list. Filename:/oracle/agent11g/sysman/admin/metadata/pop.xml
                      2013-02-20 09:10:24,597 Thread-1 WARN metadata: TABLE metric connections can't define table_name: mail_pop_connections. Filename:/oracle/agent11g/sysman/admin/metadata/pop.xml
                      2013-02-20 09:10:24,597 Thread-1 WARN metadata: TABLE metric commandcount can't define table_name: mail_pop_commandcount. Filename:/oracle/agent11g/sysman/admin/metadata/pop.xml
                      2013-02-20 09:10:24,597 Thread-1 WARN metadata: TABLE metric spsstats can't define table_name: mail_pop_sps. Filename:/oracle/agent11g/sysman/admin/metadata/pop.xml
                      2013-02-20 09:10:24,597 Thread-1 WARN metadata: TABLE metric usercount can't define table_name: mail_pop_usercount. Filename:/oracle/agent11g/sysman/admin/metadata/pop.xml
                      2013-02-20 09:10:24,598 Thread-1 WARN metadata: TABLE metric socketcount can't define table_name: umsg_pop_socketcount. Filename:/oracle/agent11g/sysman/admin/metadata/pop.xml
                      2013-02-20 09:10:24,598 Thread-1 WARN metadata: TABLE metric frequency_times can't define table_name: umsg_pop_frequencytimes. Filename:/oracle/agent11g/sysman/admin/metadata/pop.xml
                      2013-02-20 09:10:24,598 Thread-1 WARN metadata: TABLE metric userlist can't define table_name: umsg_pop_userlist. Filename:/oracle/agent11g/sysman/admin/metadata/pop.xml
                      2013-02-20 09:10:24,598 Thread-1 WARN metadata: TABLE metric dbconnections can't define table_name: umsg_pop_dbconnections. Filename:/oracle/agent11g/sysman/admin/metadata/pop.xml
                      2013-02-20 09:10:24,598 Thread-1 WARN metadata: TABLE metric connections_lost can't define table_name: umsg_pop_connections_lost. Filename:/oracle/agent11g/sysman/admin/metadata/pop.xml
                      2013-02-20 09:10:24,599 Thread-1 WARN metadata: TABLE metric connections_timeout can't define table_name: umsg_pop_connections_timeout. Filename:/oracle/agent11g/sysman/admin/metadata/pop.xml
                      2013-02-20 09:10:24,599 Thread-1 WARN metadata: TABLE metric connections_total can't define table_name: umsg_pop_connections_count. Filename:/oracle/agent11g/sysman/admin/metadata/pop.xml
                      2013-02-20 09:10:24,599 Thread-1 WARN metadata: TABLE metric network_transmit can't define table_name: umsg_pop_network_transmit. Filename:/oracle/agent11g/sysman/admin/metadata/pop.xml
                      2013-02-20 09:10:24,599 Thread-1 WARN metadata: TABLE metric network_receive can't define table_name: umsg_pop_network_receive. Filename:/oracle/agent11g/sysman/admin/metadata/pop.xml
                      2013-02-20 09:10:24,599 Thread-1 WARN metadata: TABLE metric Function calls can't define table_name: mail_pop_functions. Filename:/oracle/agent11g/sysman/admin/metadata/pop.xml
                      2013-02-20 09:10:24,776 Thread-1 WARN metadata: TABLE metric uptime can't define table_name: mail_imap_health. Filename:/oracle/agent11g/sysman/admin/metadata/slist.xml
                      2013-02-20 09:10:24,784 Thread-1 WARN metadata: TABLE metric uptime can't define table_name: mail_smtpi_health. Filename:/oracle/agent11g/sysman/admin/metadata/smtp_in.xml
                      2013-02-20 09:10:24,784 Thread-1 WARN metadata: TABLE metric connections can't define table_name: mail_mta_connections. Filename:/oracle/agent11g/sysman/admin/metadata/smtp_in.xml
                      2013-02-20 09:10:24,784 Thread-1 WARN metadata: TABLE metric connections_current can't define table_name: smtpin_connections_current. Filename:/oracle/agent11g/sysman/admin/metadata/smtp_in.xml
                      2013-02-20 09:10:24,784 Thread-1 WARN metadata: TABLE metric transmit can't define table_name: mail_mta_transmit. Filename:/oracle/agent11g/sysman/admin/metadata/smtp_in.xml
                      2013-02-20 09:10:24,785 Thread-1 WARN metadata: TABLE metric receive can't define table_name: mail_mta_receive. Filename:/oracle/agent11g/sysman/admin/metadata/smtp_in.xml
                      2013-02-20 09:10:24,785 Thread-1 WARN metadata: TABLE metric messages_received can't define table_name: umsg_smtpin_messages_received. Filename:/oracle/agent11g/sysman/admin/metadata/smtp_in.xml
                      2013-02-20 09:10:24,785 Thread-1 WARN metadata: TABLE metric messages_avgsize can't define table_name: umsg_smtpin_messages_avgsize. Filename:/oracle/agent11g/sysman/admin/metadata/smtp_in.xml
                      2013-02-20 09:10:24,785 Thread-1 WARN metadata: TABLE metric messages_avgdeliverytime can't define table_name: umsg_smtpin_messages_avgtime. Filename:/oracle/agent11g/sysman/admin/metadata/smtp_in.xml
                      2013-02-20 09:10:24,786 Thread-1 WARN metadata: TABLE metric transmission_response can't define table_name: umsg_smtpin_transmission_reponse. Filename:/oracle/agent11g/sysman/admin/metadata/smtp_in.xml
                      2013-02-20 09:10:24,786 Thread-1 WARN metadata: TABLE metric network_connection_total can't define table_name: umsg_smtpin_network_connection_total. Filename:/oracle/agent11g/sysman/admin/metadata/smtp_in.xml
                      2013-02-20 09:10:24,786 Thread-1 WARN metadata: TABLE metric network_receive_bytes can't define table_name: umsg_smtpin_network_receive_bytes. Filename:/oracle/agent11g/sysman/admin/metadata/smtp_in.xml
                      2013-02-20 09:10:24,786 Thread-1 WARN metadata: TABLE metric messages_receive_dl can't define table_name: umsg_smtpin_message_receive_dl. Filename:/oracle/agent11g/sysman/admin/metadata/smtp_in.xml
                      2013-02-20 09:10:24,786 Thread-1 WARN metadata: TABLE metric messages_deferred can't define table_name: umsg_smtpin_messages_deferred. Filename:/oracle/agent11g/sysman/admin/metadata/smtp_in.xml
                      2013-02-20 09:10:24,787 Thread-1 WARN metadata: TABLE metric messages_avgreceip can't define table_name: umsg_smtpin_message_avgreceip. Filename:/oracle/agent11g/sysman/admin/metadata/smtp_in.xml
                      2013-02-20 09:10:24,808 Thread-1 WARN metadata: TABLE metric uptime can't define table_name: mail_smtpo_health. Filename:/oracle/agent11g/sysman/admin/metadata/smtp_out.xml
                      2013-02-20 09:10:24,808 Thread-1 WARN metadata: TABLE metric connections can't define table_name: mail_mta_connections. Filename:/oracle/agent11g/sysman/admin/metadata/smtp_out.xml
                      2013-02-20 09:10:24,808 Thread-1 WARN metadata: TABLE metric transmit can't define table_name: mail_mta_transmit. Filename:/oracle/agent11g/sysman/admin/metadata/smtp_out.xml
                      2013-02-20 09:10:24,809 Thread-1 WARN metadata: TABLE metric connections_current can't define table_name: umsg_smtpout_connections_current. Filename:/oracle/agent11g/sysman/admin/metadata/smtp_out.xml
                      2013-02-20 09:10:24,809 Thread-1 WARN metadata: TABLE metric messages_transmitted can't define table_name: umsg_smtpout_messages_transmitted. Filename:/oracle/agent11g/sysman/admin/metadata/smtp_out.xml
                      2013-02-20 09:10:24,809 Thread-1 WARN metadata: TABLE metric messages_avgsize can't define table_name: umsg_smtpout_messages_avgsize. Filename:/oracle/agent11g/sysman/admin/metadata/smtp_out.xml
                      2013-02-20 09:10:24,809 Thread-1 WARN metadata: TABLE metric messages_avgdeliverytime can't define table_name: umsg_smtpout_messages_avgtime. Filename:/oracle/agent11g/sysman/admin/metadata/smtp_out.xml
                      2013-02-20 09:10:24,809 Thread-1 WARN metadata: TABLE metric messages_avgdeliverytime_local can't define table_name: umsg_smtpout_messages_avgtime_local. Filename:/oracle/agent11g/sysman/admin/metadata/smtp_out.xml
                      2013-02-20 09:10:24,809 Thread-1 WARN metadata: TABLE metric messages_avgdeliverytime_remote can't define table_name: umsg_smtpout_messages_avgtime_remote. Filename:/oracle/agent11g/sysman/admin/metadata/smtp_out.xml
                      2013-02-20 09:10:24,810 Thread-1 WARN metadata: TABLE metric messages_avgrelay_time can't define table_name: umsg_smtpout_messages_avgtime_relay. Filename:/oracle/agent11g/sysman/admin/metadata/smtp_out.xml
                      2013-02-20 09:10:24,810 Thread-1 WARN metadata: TABLE metric delivery_performance can't define table_name: umsg_smtpout_delivery_performance. Filename:/oracle/agent11g/sysman/admin/metadata/smtp_out.xml
                      2013-02-20 09:10:24,810 Thread-1 WARN metadata: TABLE metric relay_performance can't define table_name: umsg_smtpout_messages_avgtime_relay. Filename:/oracle/agent11g/sysman/admin/metadata/smtp_out.xml
                      2013-02-20 09:10:24,810 Thread-1 WARN metadata: TABLE metric connections_outbound can't define table_name: umsg_smtpout_connections_outbound. Filename:/oracle/agent11g/sysman/admin/metadata/smtp_out.xml
                      2013-02-20 09:10:24,811 Thread-1 WARN metadata: TABLE metric connections_outbound_local can't define table_name: umsg_smtpout_connections_outbound_local. Filename:/oracle/agent11g/sysman/admin/metadata/smtp_out.xml
                      2013-02-20 09:10:24,811 Thread-1 WARN metadata: TABLE metric connections_outbound_remote can't define table_name: umsg_smtpout_connections_outbound_remote. Filename:/oracle/agent11g/sysman/admin/metadata/smtp_out.xml
                      2013-02-20 09:10:24,811 Thread-1 WARN metadata: TABLE metric transmitted_bytes can't define table_name: umsg_smtpout_transmitted_bytes. Filename:/oracle/agent11g/sysman/admin/metadata/smtp_out.xml
                      2013-02-20 09:10:24,811 Thread-1 WARN metadata: TABLE metric transmission_rate can't define table_name: umsg_smtpout_transmission_rate. Filename:/oracle/agent11g/sysman/admin/metadata/smtp_out.xml
                      2013-02-20 09:10:25,167 Thread-13 ERROR TargetManager: Skipping target {EC2, oracle_database}: Missing properties - password
                      2013-02-20 09:10:25,167 Thread-11 ERROR TargetManager: Skipping target {HCP, oracle_database}: Missing properties - password
                      2013-02-20 09:10:25,168 Thread-12 ERROR TargetManager: Skipping target {BPP, oracle_database}: Missing properties - password
                      2013-02-20 09:10:25,188 Thread-21 WARN fetchlets.PropsFromAssocTgt: nmefpfa_getPropsFromsAssoc: referred target {cluster_instance} does not exist
                      2013-02-20 09:10:25,222 Thread-21 WARN TargetManager: Exception in computing dynamic properties of {hpsapbe3, host },from_cluster::Could not fetch properties from associated target
                      2013-02-20 09:10:26,088 Thread-1 WARN upload: Truncating value of "SHORT_NAME" from "Average Synchronous Single-Block Read Latency (ms)" to "Average Synchronous Single-Block Read La"
                      2013-02-20 09:10:26,088 Thread-1 WARN upload: Truncating value of "SHORT_NAME" from "Average Synchronous Single-Block Read Latency (ms)" to "Average Synchronous Single-Block Read La"
                      2013-02-20 09:10:26,111 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:10:26,625 Thread-1 WARN upload: Truncating value of "COLUMN_LABEL" from "Total messages processed per queue per subscriber per minute in the last interval" to "Total messages processed per queue per subscriber per minute in "
                      2013-02-20 09:10:26,625 Thread-1 WARN upload: Truncating value of "COLUMN_LABEL" from "Total messages received per queue per subscriber per minute in the last interval" to "Total messages received per queue per subscriber per minute in t"
                      2013-02-20 09:10:26,625 Thread-1 WARN upload: Truncating value of "COLUMN_LABEL" from "Messages processed per queue (%) per subscriber per minute in the last interval" to "Messages processed per queue (%) per subscriber per minute in th"
                      2013-02-20 09:10:26,626 Thread-1 WARN upload: Truncating value of "COLUMN_LABEL" from "Age of the first message in persistent queue per subscriber (seconds)" to "Age of the first message in persistent queue per subscriber (sec"
                      2013-02-20 09:10:26,627 Thread-1 WARN upload: Truncating value of "COLUMN_LABEL" from "Age of the first message in the buffered queue per queue (seconds)" to "Age of the first message in the buffered queue per queue (second"
                      2013-02-20 09:10:26,787 Thread-1 WARN collector: CLEAR_MESSAGE_NLSID "netapp_filer_filer_capacity_Aggregate_capacity_allocated_per_clear" is too long, truncating to "netapp_filer_filer_capacity_Aggregate_capacity_allocated_per_cle"
                      2013-02-20 09:10:26,911 Thread-1 WARN collector: MESSAGE_NLSID "soainfra_mediatorRequestBreakdown_invokeOnewayPhase_averageTime_cond" is too long, truncating to "soainfra_mediatorRequestBreakdown_invokeOnewayPhase_averageTime_"
                      2013-02-20 09:10:26,911 Thread-1 WARN collector: MESSAGE_NLSID "soainfra_mediatorRequestBreakdown_transformationPhase_averageTime_cond" is too long, truncating to "soainfra_mediatorRequestBreakdown_transformationPhase_averageTim"
                      2013-02-20 09:10:26,911 Thread-1 WARN collector: MESSAGE_NLSID "soainfra_mediatorRequestBreakdown_invokeOnewayPhase_throughput_cond" is too long, truncating to "soainfra_mediatorRequestBreakdown_invokeOnewayPhase_throughput_c"
                      2013-02-20 09:10:26,911 Thread-1 WARN collector: MESSAGE_NLSID "soainfra_mediatorRequestBreakdown_transformationPhase_throughput_cond" is too long, truncating to "soainfra_mediatorRequestBreakdown_transformationPhase_throughput"
                      2013-02-20 09:10:26,911 Thread-1 WARN collector: MESSAGE_NLSID "soainfra_mediatorRequestBreakdown_invokeOnewayPhase_totalTime_cond" is too long, truncating to "soainfra_mediatorRequestBreakdown_invokeOnewayPhase_totalTime_co"
                      2013-02-20 09:10:26,911 Thread-1 WARN collector: MESSAGE_NLSID "soainfra_mediatorRequestBreakdown_transformationPhase_totalTime_cond" is too long, truncating to "soainfra_mediatorRequestBreakdown_transformationPhase_totalTime_"
                      2013-02-20 09:10:26,975 Thread-1 WARN collector: MESSAGE_NLSID "deployment_overview_ejbPoolAccessSuccess_percentage_by_server_cond" is too long, truncating to "deployment_overview_ejbPoolAccessSuccess_percentage_by_server_co"
                      2013-02-20 09:10:26,975 Thread-1 WARN collector: MESSAGE_NLSID "deployment_overview_ejbTransactionCommit_percentage_by_server_cond" is too long, truncating to "deployment_overview_ejbTransactionCommit_percentage_by_server_co"
                      2013-02-20 09:10:27,032 Thread-1 WARN collector: MESSAGE_NLSID "authentication_request_containing_allow_federation_creation_by_identity_provider" is too long, truncating to "authentication_request_containing_allow_federation_creation_by_i"
                      2013-02-20 09:10:27,032 Thread-1 WARN collector: MESSAGE_NLSID "authentication_request_containing_allow_federation_creation_by_service_provider" is too long, truncating to "authentication_request_containing_allow_federation_creation_by_s"
                      2013-02-20 09:10:27,033 Thread-1 WARN collector: MESSAGE_NLSID "federation_termination_response_sent_successfully_by_identity_provider" is too long, truncating to "federation_termination_response_sent_successfully_by_identity_pr"
                      2013-02-20 09:10:27,033 Thread-1 WARN collector: MESSAGE_NLSID "signed_federation_termination_request_received_by_identity_provider" is too long, truncating to "signed_federation_termination_request_received_by_identity_provi"
                      2013-02-20 09:10:27,033 Thread-1 WARN collector: MESSAGE_NLSID "federation_termination_response_received_successfully_by_identity_provider" is too long, truncating to "federation_termination_response_received_successfully_by_identit"
                      2013-02-20 09:10:27,033 Thread-1 WARN collector: MESSAGE_NLSID "signed_federation_termination_response_received_by_identity_provider" is too long, truncating to "signed_federation_termination_response_received_by_identity_prov"
                      2013-02-20 09:10:27,034 Thread-1 WARN collector: MESSAGE_NLSID "name_registration_response_sent_successfully_by_identity_provider" is too long, truncating to "name_registration_response_sent_successfully_by_identity_provide"
                      2013-02-20 09:10:27,034 Thread-1 WARN collector: MESSAGE_NLSID "name_registration_response_received_successfully_by_identity_provider" is too long, truncating to "name_registration_response_received_successfully_by_identity_pro"
                      2013-02-20 09:10:27,034 Thread-1 WARN collector: MESSAGE_NLSID "federation_termination_response_sent_successfully_by_service_provider" is too long, truncating to "federation_termination_response_sent_successfully_by_service_pro"
                      2013-02-20 09:10:27,034 Thread-1 WARN collector: MESSAGE_NLSID "signed_federation_termination_request_received_by_service_provider" is too long, truncating to "signed_federation_termination_request_received_by_service_provid"
                      2013-02-20 09:10:27,035 Thread-1 WARN collector: MESSAGE_NLSID "federation_termination_response_received_successfully_by_service_provider" is too long, truncating to "federation_termination_response_received_successfully_by_service"
                      2013-02-20 09:10:27,035 Thread-1 WARN collector: MESSAGE_NLSID "signed_federation_termination_response_received_by_service_provider" is too long, truncating to "signed_federation_termination_response_received_by_service_provi"
                      2013-02-20 09:10:27,035 Thread-1 WARN collector: MESSAGE_NLSID "name_registration_response_received_successfully_by_service_provider" is too long, truncating to "name_registration_response_received_successfully_by_service_prov"
                      2013-02-20 09:10:27,035 Thread-1 WARN collector: MESSAGE_NLSID "successful_federation_termination_request_received_by_service_provider" is too long, truncating to "successful_federation_termination_request_received_by_service_pr"
                      2013-02-20 09:10:27,036 Thread-1 WARN collector: MESSAGE_NLSID "successful_federation_termination_request_sent_by_service_provider" is too long, truncating to "successful_federation_termination_request_sent_by_service_provid"
                      2013-02-20 09:10:27,036 Thread-1 WARN collector: MESSAGE_NLSID "successful_name_registration_request_received_by_service_provider" is too long, truncating to "successful_name_registration_request_received_by_service_provide"
                      2013-02-20 09:10:27,036 Thread-1 WARN collector: MESSAGE_NLSID "successful_federation_termination_request_received_by_identity_provider" is too long, truncating to "successful_federation_termination_request_received_by_identity_p"
                      2013-02-20 09:10:27,036 Thread-1 WARN collector: MESSAGE_NLSID "successful_federation_termination_request_sent_by_identity_provider" is too long, truncating to "successful_federation_termination_request_sent_by_identity_provi"
                      2013-02-20 09:10:27,036 Thread-1 WARN collector: MESSAGE_NLSID "successful_name_registration_request_received_by_identity_provider" is too long, truncating to "successful_name_registration_request_received_by_identity_provid"
                      2013-02-20 09:10:27,103 Thread-1 WARN collector: MESSAGE_NLSID "cluster_app_overview_ejbPoolAccessSuccess_percentage_by_server_cond" is too long, truncating to "cluster_app_overview_ejbPoolAccessSuccess_percentage_by_server_c"
                      2013-02-20 09:10:27,104 Thread-1 WARN collector: MESSAGE_NLSID "cluster_app_overview_ejbTransactionCommit_percentage_by_server_cond" is too long, truncating to "cluster_app_overview_ejbTransactionCommit_percentage_by_server_c"
                      2013-02-20 09:10:27,682 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:10:29,242 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:10:36,078 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:10:46,143 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:10:50,661 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:11:06,132 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:11:21,327 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:11:51,969 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:12:10,883 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:12:22,620 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:12:53,258 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:13:23,887 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:13:54,539 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:14:25,173 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:14:55,807 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:15:26,595 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:15:57,743 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:16:28,704 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:16:59,730 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:17:30,461 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:18:01,107 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:18:31,770 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:19:02,421 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:19:33,064 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:20:03,732 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:20:34,431 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:20:59,981 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:21:04,595 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:21:05,095 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:21:05,808 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:21:35,753 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:22:06,380 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005
                      2013-02-20 09:22:37,048 Thread-4 ERROR http: Error in obtaining IP address, ret = 7005