7 Replies Latest reply on Jan 13, 2009 7:34 PM by bobby_saini

    sunmc log files

    800747
      Hi all,

      I have following add-ons on my system:
      Advanced System Monitoring
      Service Availability Manager
      Performance Reporting Manager
      Solaris Container Manager
      System Reliability Manager
      Sun Management Center Integration for Unicenter TNG
      Generic X86/X64 Config Reader
      Please give me a light about sunmc logfiles.
      What are the sunmc important logfiles to check above add-ons and sunmc server and agent status?
      Please state that their name and path also.
      Please help.

      Thanks.
        • 1. Re: sunmc log files
          bobby_saini
          Sun MC Log Files can be found here:
          /var/opt/SUNWsymon/log

          To view the log files, please refer to the following forum posting:
          http://forums.halcyoninc.com/showthread.php?t=167

          Depending on what your problem is, you will want to look at the appropriate log file.

          For example, if your experiencing problems with your console to server connection, look at the server.log file. If there are problems with the agent, loading modules, setting alarm thresholds, etc, then look at the agent.log.

          If you can tell me what problems your experiencing, I can tell you what log files to look at and also try and assist you in resolving your problems.

          bobby
          http://www.HalcyonInc.com
          • 2. Re: sunmc log files
            800747
            Hi bobby,

            Appreciate for your reply.

            one of my sunmc agent is now in inactive state.
            In the sunmc console,
            Tools --> Report Manager
            Data --> Data Availability
            agent is in inactive hosts tab

            but sunmc agent service is running.
            root@system1 # svcs -a | grep sunmc
            disabled       Dec_25   svc:/application/management/sunmcplatform:default
            online          2:42:01 svc:/application/management/sunmcagent:default
            Problem statement
            ---------------------------
            case1: I can't see reports (cpu utilization, overall cpuutilization, swap, memory and system uptime) in PRM for this agent.

            How can I solve this agent back to normal state?
            What is the next step do i need to check?
            my version is sunmc 4.0

            case2: I have another agent which is in active state and sunmc agent service is running.
            root@system2 # svcs -a | grep sunmc
            disabled       Dec_18   svc:/application/management/sunmcplatform:default
            online          2:34:27 svc:/application/management/sunmcagent:default
            But the CPU utilization and overall cpu utilization are not able to see in PRM, eventhough sunmc agent is loaded kernel reader (simple) module and system is running more than 24 hours.

            What should i check next?

            Please help. Thanks a lot!
            • 3. Re: sunmc log files
              mike_kirk
              Hi Yap,
              Yap wrote:
              Hi bobby,

              What should i check next?

              Please help. Thanks a lot!
              PRM is sensitive to hostname/IP issues: even if your Server can communicate with your Agent OK using it's IP for normal Agent operations... PRM will still try to contact the system by hostname to gather data. More info here:

              [http://forums.halcyoninc.com/showthread.php?t=115|http://forums.halcyoninc.com/showthread.php?t=115]

              If PRM has been giving you problems for some time now, I'd also recommend you try our Reporter and Enterprise Reporter products as an alternative. Both can be downloaded for a 30-day eval:

              [http://www.halcyoninc.com/products/EReporter/index.php|http://www.halcyoninc.com/products/EReporter/index.php]
                   [http://www.halcyoninc.com/products/Reporter/index.php|http://www.halcyoninc.com/products/Reporter/index.php]

              Regards,

              Mike.Kirk@HalcyonInc.com
              [http://www.HalcyonInc.com|http://www.HalcyonInc.com]
              New !! : [http://forums.HalcyonInc.com|http://forums.HalcyonInc.com] !!
              • 4. Re: sunmc log files
                800747
                Hi bobby,

                one of my sunmc agent is now in inactive state.
                In the sunmc console,
                Tools --> Report Manager
                Data --> Data Availability
                agent is in inactive hosts tab

                but sunmc agent service is running.
                root@system1 # svcs -a | grep sunmc
                disabled       Dec_25   svc:/application/management/sunmcplatform:default
                online          2:42:01 svc:/application/management/sunmcagent:default
                Problem statement
                ---------------------------
                case1: I can't see reports (cpu utilization, overall cpuutilization, swap, memory and system uptime) in PRM for this agent.

                How can I solve this agent back to normal state?
                What is the next step do i need to check?
                my version is sunmc 4.0

                I found some error in the agent.log. Please let me know if you have idea on what is happening on this agent node.
                log file information for this agent node from agent.log
                --------------------------------------------------------------------------

                [0010c865 0040 ]info Jan 13 10:55:18 agent loading module: perftool.........................................................................................................................................
                [0010c866 0047 ]info Jan 13 10:55:18 agent *** shutting down subagents ***..................................................................................................................................
                [0010c867 0087 ]info Jan 13 10:55:18 agent sh: Executing sh {-c} {usr/bin/sh /opt/SUNWsymon/modules/sbin/em-daemon.sh stop} process 12222..................................................................
                [0010c868 005a ]error Jan 13 10:55:18 agent syntax error in perftool.dat(1) at token '5444145'...............................................................................................................
                [0010c869 0042 ]error Jan 13 10:55:18 agent *** aborting execution ***.......................................................................................................................................
                [0010c86a 0046 ]info Jan 13 10:55:25 agent Channel already exists: syslog...................................................................................................................................
                [0010c86b 0048 ]info Jan 13 10:55:25 agent using ip mode for URL addressing.................................................................................................................................
                [0010c86c 004a ]info Jan 13 10:55:25 agent Skipping bind to interface 0.0.0.0...............................................................................................................................
                [0010c86d 003b ]info Jan 13 10:55:25 agent enabling timed jobs..............................................................................................................................................
                [0010c86e 004d ]info Jan 13 10:55:25 agent subagent registry successfully loaded............................................................................................................................
                [0010c86f 005a ]info Jan 13 10:55:25 agent loading trap actions: subagent-manager-trap-action...............................................................................................................
                [0010c870 0040 ]info Jan 13 10:55:25 agent startLegacySubagents: <>.........................................................................................................................................
                [0010c871 003e ]info Jan 13 10:55:25 agent activating MIB objects...........................................................................................................................................
                [0010c872 0052 ]info Jan 13 10:55:25 agent Setting event trap resend interval to 900s.......................................................................................................................
                [0010c873 0046 ]info Jan 13 10:55:25 agent MIB object activation complete...................................................................................................................................
                [0010c874 0085 ]warning Jan 13 10:55:26 agent SNMP Report: 10.58.3.249:164-2-1-5-espublic-0:0:4:da:40:a:3a:3:b:4:89:0 1.3.6.1.6.3.6.1.2.2.0....................................................................
                [0010c875 003b ]info Jan 13 10:55:33 agent loading module: mcp..............................................................................................................................................
                [0010c876 0046 ]info Jan 13 10:55:33 agent module mcp successfully loaded...................................................................................................................................
                [0010c877 004a ]info Jan 13 10:55:34 agent module mcp turning on as scheduled.........
                • 5. Re: sunmc log files
                  bobby_saini
                  It seems like your problematic agent is dying and automatically restarting:

                  <--SNIP-->
                  [0010c866 0047 ]info Jan 13 10:55:18 agent *** shutting down subagents ***..................................................................................................................................
                  [0010c867 0087 ]info Jan 13 10:55:18 agent sh: Executing sh {-c} {usr/bin/sh /opt/SUNWsymon/modules/sbin/em-daemon.sh stop} process 12222..................................................................
                  [0010c868 005a ]error Jan 13 10:55:18 agent syntax error in perftool.dat(1) at token '5444145'...............................................................................................................
                  [0010c869 0042 ]error Jan 13 10:55:18 agent *** aborting execution ***.......................................................................................................................................
                  [0010c86a 0046 ]info Jan 13 10:55:25 agent Channel already exists:
                  <--SNIP-->

                  In addition, the perftool module may not be loading up correctly and causing all the problems on the agent. To single out the problem, can you comment out the line containing perftool in /var/opt/SUNWsymon/cfg/base-modules-d.dat.

                  Restart the agent and see if the agent is healthy. However, note, this will not fix your PRM issues. We need the perftool module to be loaded. But 1st, lets confirm what is causing the problems.

                  You might want to try reseeding the agent:
                  /opt/SUNWsymon/sbin/es-run base-usm-seed.sh -s <SEED> -c agent

                  cheers,
                  bobby
                  • 6. Re: sunmc log files
                    800747
                    Hi bobby,

                    Thanks for your suggestion.

                    May I know what following command will do?

                    /opt/SUNWsymon/sbin/es-run base-usm-seed.sh -s <SEED> -c agent
                    • 7. Re: sunmc log files
                      bobby_saini
                      it will regenerate security files under /var/opt/SUNWsymon/cfg for server to agent communication.

                      bobby