1 Reply Latest reply: Jul 17, 2013 8:06 AM by user3183694 RSS

    Why root.sh fails in the second node?

    user3183694

      Hi

      After successful install of oracle11 grid in 2 nodes and running root.sh on
      node1, root.sh on node2 fails:

       

       

       

      [root@vmorarac2 dev]#
      /u01/app/product/11.2.0/oracle/root.sh

       

      Running Oracle 11g root.sh script...

       

       

       

      The following environment variables are set as:

       

          ORACLE_OWNER= oracle

       

          ORACLE_HOME=  /u01/app/product/11.2.0/oracle

       

       

       

      Enter the full pathname of the local bin directory:
      [/usr/local/bin]:

       

      The file "dbhome" already exists in /usr/local/bin. 
      Overwrite it? (y/n)

       

      [n]: y

       

         Copying dbhome to /usr/local/bin ...

       

      The file "oraenv" already exists in /usr/local/bin. 
      Overwrite it? (y/n)

       

      [n]: y

       

         Copying oraenv to /usr/local/bin ...

       

      The file "coraenv" already exists in /usr/local/bin. 
      Overwrite it? (y/n)

       

      [n]: y

       

         Copying coraenv to /usr/local/bin ...

       

       

       

       

       

      Creating /etc/oratab file...

       

      Entries will be added to the /etc/oratab file as needed
      by

       

      Database Configuration Assistant when a database is
      created

       

      Finished running generic part of root.sh script.

       

      Now product-specific root actions will be performed.

       

      2013-07-17 08:37:10: Parsing the host name

       

      2013-07-17 08:37:10: Checking for super user
      privileges

       

      2013-07-17 08:37:10: User has super user privileges

       

      Using configuration parameter file:
      /u01/app/product/11.2.0/oracle/crs/install/crsconfig_params

       

      Creating trace directory

       

      LOCAL ADD MODE

       

      Creating OCR keys for user 'root', privgrp 'root'..

       

      Operation successful.

       

      Adding daemon to inittab

       

      CRS-4123: Oracle High Availability Services has been
      started.

       

      ohasd is starting

       

      CRS-4402: The CSS daemon was started in exclusive mode but
      found an active CSS daemon on node vmorarac1, number 1, and is terminating

       

      CRS-2673: Attempting to stop 'ora.cssdmonitor' on
      'vmorarac2'

       

      CRS-2677: Stop of 'ora.cssdmonitor' on 'vmorarac2'
      succeeded

       

      An active cluster was found during exclusive startup,
      restarting to join the cluster

       

      CRS-2672: Attempting to start 'ora.mdnsd' on
      'vmorarac2'

       

      CRS-2676: Start of 'ora.mdnsd' on 'vmorarac2'
      succeeded

       

      CRS-2672: Attempting to start 'ora.gipcd' on
      'vmorarac2'

       

      CRS-2676: Start of 'ora.gipcd' on 'vmorarac2'
      succeeded

       

      CRS-2672: Attempting to start 'ora.gpnpd' on
      'vmorarac2'

       

      CRS-2676: Start of 'ora.gpnpd' on 'vmorarac2'
      succeeded

       

      CRS-2672: Attempting to start 'ora.cssdmonitor' on
      'vmorarac2'

       

      CRS-2676: Start of 'ora.cssdmonitor' on 'vmorarac2'
      succeeded

       

      CRS-2672: Attempting to start 'ora.cssd' on
      'vmorarac2'

       

      CRS-2672: Attempting to start 'ora.diskmon' on
      'vmorarac2'

       

      CRS-2676: Start of 'ora.diskmon' on 'vmorarac2'
      succeeded

       

      CRS-2674: Start of 'ora.cssd' on 'vmorarac2' failed

       

      CRS-2679: Attempting to clean 'ora.cssd' on
      'vmorarac2'

       

      CRS-2681: Clean of 'ora.cssd' on 'vmorarac2' succeeded

       

      CRS-2673: Attempting to stop 'ora.diskmon' on
      'vmorarac2'

       

      CRS-2677: Stop of 'ora.diskmon' on 'vmorarac2'
      succeeded

       

      CRS-4000: Command Start failed, or completed with
      errors.

       

      CRS-2672: Attempting to start 'ora.cssd' on
      'vmorarac2'

       

      CRS-2672: Attempting to start 'ora.diskmon' on
      'vmorarac2'

       

      CRS-2674: Start of 'ora.diskmon' on 'vmorarac2' failed

       

      CRS-2679: Attempting to clean 'ora.diskmon' on
      'vmorarac2'

       

      CRS-5016: Process
      "/u01/app/product/11.2.0/oracle/bin/diskmon" spawned by agent
      "/u01/app/product/11.2.0/oracle/bin/orarootagent.bin" for action "clean" failed:
      details at "(:CLSN00010:)" in
      "/u01/app/product/11.2.0/oracle/log/vmorarac2/agent/ohasd/orarootagent_root/orarootagent_root.log"

       

      CRS-2681: Clean of 'ora.diskmon' on 'vmorarac2'
      succeeded

       

      CRS-2674: Start of 'ora.cssd' on 'vmorarac2' failed

       

      CRS-2679: Attempting to clean 'ora.cssd' on
      'vmorarac2'

       

      CRS-2681: Clean of 'ora.cssd' on 'vmorarac2' succeeded

       

      CRS-4000: Command Start failed, or completed with
      errors.

       

      Command return code of 1 (256) from command:
      /u01/app/product/11.2.0/oracle/bin/crsctl start resource ora.ctssd -init -env
      USR_ORA_ENV=CTSS_REBOOT=TRUE

       

      Start of resource "ora.ctssd -init -env
      USR_ORA_ENV=CTSS_REBOOT=TRUE" failed

       

      Failed to start CTSS

       

      Failed to start Oracle Clusterware stack

       

      [root@vmorarac2 dev]#

       

      [root@vmorarac2 dev]#

       

       

       

      --------------------------------------------

       

      try again:

       

      [root@vmorarac2 bin]# ./crsctl start resource ora.ctssd
      -init -env USR_ORA_ENV=CTSS_REBOOT=TRUE

       

      CRS-2672: Attempting to start 'ora.cssd' on
      'vmorarac2'

       

      CRS-2672: Attempting to start 'ora.diskmon' on
      'vmorarac2'

       

      CRS-2674: Start of 'ora.diskmon' on 'vmorarac2' failed

       

      CRS-2679: Attempting to clean 'ora.diskmon' on
      'vmorarac2'

       

      CRS-5016: Process
      "/u01/app/product/11.2.0/oracle/bin/diskmon" spawned by agent
      "/u01/app/product/11.2.0/oracle/bin/orarootagent.bin" for action "clean" failed:
      details at "(:CLSN00010:)" in
      "/u01/app/product/11.2.0/oracle/log/vmorarac2/agent/ohasd/orarootagent_root/orarootagent_root.log"

       

      CRS-2681: Clean of 'ora.diskmon' on 'vmorarac2'
      succeeded

       

      CRS-2674: Start of 'ora.cssd' on 'vmorarac2' failed

       

      CRS-2679: Attempting to clean 'ora.cssd' on
      'vmorarac2'

       

      CRS-2681: Clean of 'ora.cssd' on 'vmorarac2' succeeded

       

      CRS-4000: Command Start failed, or completed with
      errors.

       

       

       

      -------------------------------------

       

       

       

      [root@vmorarac2 bin]# ps -ef | grep u01

       

      root      8913     1  0 08:37 ?        00:00:06
      /u01/app/product/11.2.0/oracle/bin/ohasd.bin reboot

       

      oracle   10507     1  0 08:39 ?        00:00:02
      /u01/app/product/11.2.0/oracle/bin/oraagent.bin

       

      oracle   10522     1  0 08:39 ?        00:00:00
      /u01/app/product/11.2.0/oracle/bin/mdnsd.bin

       

      oracle   10534     1  0 08:39 ?        00:00:00
      /u01/app/product/11.2.0/oracle/bin/gipcd.bin

       

      oracle   10548     1  0 08:39 ?        00:00:39
      /u01/app/product/11.2.0/oracle/bin/gpnpd.bin

       

      root     11723     1  0 11:00 ?        00:00:03
      /u01/app/product/11.2.0/oracle/bin/cssdmonitor

       

       

       

      --------------------------------------------------

       

      [oracle@vmorarac2 bin]$ ./crsctl check crs

       

      CRS-4638: Oracle High Availability Services is online

       

      CRS-4535: Cannot communicate with Cluster Ready
      Services

       

      CRS-4530: Communications failure contacting Cluster
      Synchronization Services daemon

       

      CRS-4534: Cannot communicate with Event Manager

        • 1. Re: Why root.sh fails in the second node?
          user3183694

          log:

           

           

          Oracle Database 11g Clusterware Release 11.2.0.1.0 - Production Copyright 1996, 2009 Oracle. All rights reserved.



          2013-07-17 08:38:04.485: [    AGFW][3481860944] Starting the agent: /u01/app/product/11.2.0/oracle/log/vmorarac2/agent/ohasd/orarootagent_root/



          2013-07-17 08:38:04.485: [   AGENT][3481860944] Agent framework initialized, Process Id = 10319



          2013-07-17 08:38:04.487: [ USRTHRD][3481860944] Utils::getCrsHome crsHome /u01/app/product/11.2.0/oracle



          2013-07-17 08:38:04.487: [ USRTHRD][3481860944] Process::convertPidToString pid = 10319



          2013-07-17 08:38:04.488: [    AGFW][3481860944] SERVER IPC CONNECT STR: (ADDRESS=(PROTOCOL=IPC)(KEY=OHASD_IPC_SOCKET_11))



          2013-07-17 08:38:04.488: [CLSFRAME][3481860944] Inited lsf context 0x317e9e0



          2013-07-17 08:38:04.488: [CLSFRAME][3481860944] Initing CLS Framework messaging



          2013-07-17 08:38:04.488: [CLSFRAME][3481860944] New Framework state: 2



          2013-07-17 08:38:04.488: [CLSFRAME][3481860944] M2M is starting...



          2013-07-17 08:38:04.490: [ CRSCOMM][3481860944] m_pClscCtx=0x31d1bd0m_pUgblm=0x31d5720



          2013-07-17 08:38:04.490: [ CRSCOMM][3481860944] Starting send thread



          2013-07-17 08:38:04.490: [ CRSCOMM][1119435072] clsIpc: sendWork thread started.



          2013-07-17 08:38:04.491: [ CRSCOMM][1129924928] IPC Client thread started listening



          2013-07-17 08:38:04.491: [ CRSCOMM][1129924928] init data sent from server



          2013-07-17 08:38:04.491: [CLSFRAME][3481860944] New IPC Member:{Relative|Node:0|Process:0|Type:2}:OHASD:vmorarac2



          2013-07-17 08:38:04.491: [CLSFRAME][3481860944] New process connected to us ID:{Relative|Node:0|Process:0|Type:2} Info:OHASD:vmorarac2



          2013-07-17 08:38:04.492: [CLSFRAME][3481860944] Starting thread model named: MultiThread



          2013-07-17 08:38:04.492: [CLSFRAME][3481860944] Starting thread model named: SingleThread



          2013-07-17 08:38:04.492: [CLSFRAME][3481860944] Starting thread model named: SingleThreadT



          2013-07-17 08:38:04.492: [CLSFRAME][3481860944] New Framework state: 3



          2013-07-17 08:38:04.493: [    AGFW][3481860944] Agent Framework started successfully



          2013-07-17 08:38:04.493: [    AGFW][1182374208] Agfw engine module has enabled...



          2013-07-17 08:38:04.493: [CLSFRAME][1182374208] Module Enabling is complete



          2013-07-17 08:38:04.493: [CLSFRAME][1182374208] New Framework state: 6



          2013-07-17 08:38:04.493: [    AGFW][1182374208] Agent is started with userid: root , expected user: root



          2013-07-17 08:38:04.493: [    AGFW][1182374208] Agent sending message to PE: AGENT_HANDSHAKE[Proxy] ID 20484:14



          2013-07-17 08:38:04.505: [    AGFW][1182374208] Agent received the message: RESTYPE_ADD[ora.crs.type] ID 8196:358



          2013-07-17 08:38:04.506: [    AGFW][1182374208] Added new restype: ora.crs.type



          2013-07-17 08:38:04.506: [    AGFW][1182374208] Agent sending last reply for: RESTYPE_ADD[ora.crs.type] ID 8196:358



          2013-07-17 08:38:04.506: [    AGFW][1182374208] Agent received the message: RESTYPE_ADD[ora.ctss.type] ID 8196:360



          2013-07-17 08:38:04.506: [    AGFW][1182374208] Added new restype: ora.ctss.type



          2013-07-17 08:38:04.507: [    AGFW][1182374208] Agent sending last reply for: RESTYPE_ADD[ora.ctss.type] ID 8196:360



          2013-07-17 08:38:04.516: [    AGFW][1182374208] Agent received the message: RESTYPE_ADD[ora.diskmon.type] ID 8196:362



          2013-07-17 08:38:04.516: [    AGFW][1182374208] Added new restype: ora.diskmon.type



          2013-07-17 08:38:04.517: [    AGFW][1182374208] Agent sending last reply for: RESTYPE_ADD[ora.diskmon.type] ID 8196:362



          2013-07-17 08:38:04.519: [    AGFW][1182374208] Agent received the message: RESTYPE_ADD[ora.drivers.acfs.type] ID 8196:364



          2013-07-17 08:38:04.520: [    AGFW][1182374208] Added new restype: ora.drivers.acfs.type



          2013-07-17 08:38:04.520: [    AGFW][1182374208] Agent sending last reply for: RESTYPE_ADD[ora.drivers.acfs.type] ID 8196:364



          2013-07-17 08:38:04.521: [    AGFW][1182374208] Agent received the message: RESOURCE_ADD[ora.diskmon 1 1] ID 4356:366



          2013-07-17 08:38:04.521: [    AGFW][1182374208] Added new resource: ora.diskmon 1 1 to the agfw



          2013-07-17 08:38:04.522: [    AGFW][1182374208] Agent sending last reply for: RESOURCE_ADD[ora.diskmon 1 1] ID 4356:366



          2013-07-17 08:38:04.522: [    AGFW][1182374208] Agent received the message: RESOURCE_START[ora.diskmon 1 1] ID 4098:367



          2013-07-17 08:38:04.522: [    AGFW][1182374208] Preparing START command for: ora.diskmon 1 1



          2013-07-17 08:38:04.522: [    AGFW][1182374208] ora.diskmon 1 1 state changed from: UNKNOWN to: STARTING



          2013-07-17 08:38:04.526: [    AGFW][1161394496] Executing command: start for resource: ora.diskmon 1 1



          2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] clsn_agent::start {



          2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] DaemonAgent{



          2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] }DaemonAgent



          2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] DiskmonAgent::DiskmonAgent {



          2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] InitAttrs {



          2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] __IS_HASD_AGENT=TRUE



          2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] }InitAttrs



          2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] DiskmonAgent::DiskmonAgent }



          2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] DiskmonAgent::start {



          2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] Arg Value = -d



          2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] Arg Value = -f



          2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] Total Count of Environment Variables = 3



          2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] Adding Environment Variables _ORA_AGENT_ACTION=TRUE



          2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] Adding Environment Variables __IS_HASD_AGENT=



          2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] Adding Environment variable from USR_ORA_ENV ORACLE_USER=oracle



          2013-07-17 08:38:04.527: [ora.diskmon][1161394496] [start] Utils:execCmd action = 1 flags = 5 ohome = (null) cmdname = diskmon.



          2013-07-17 08:38:04.528: [ora.diskmon][1161394496] [start] getOracleHomeAttrib: oracle_home = /u01/app/product/11.2.0/oracle



          2013-07-17 08:38:04.528: [ora.diskmon][1161394496] [start] Utils:execCmd Running the binary from /u01/app/product/11.2.0/oracle/bin/diskmon



          2013-07-17 08:38:04.531: [CRSTIMER][1091324224] Timer Thread Starting.



          2013-07-17 08:38:04.533: [ora.diskmon][1161394496] [start] execCmd ret = 0



          2013-07-17 08:38:04.533: [ora.diskmon][1161394496] [start] }DaemonAgent::start



          2013-07-17 08:38:10.534: [ora.diskmon][1161394496] [start] DiskmonAgent::connect {



          2013-07-17 08:38:10.534: [ora.diskmon][1161394496] [start] Process::convertPidToString pid = 10319



          2013-07-17 08:38:10.535: [ora.diskmon][1161394496] [start] DiskmonAgent::connect }



          2013-07-17 08:38:10.535: [ora.diskmon][1161394496] [start] DiskmonAgent::start }



          2013-07-17 08:38:10.535: [ora.diskmon][1161394496] [start] clsn_agent::start }



          2013-07-17 08:38:10.535: [    AGFW][1161394496] Command: start for resource: ora.diskmon 1 1 completed with status: SUCCESS



          2013-07-17 08:38:10.535: [    AGFW][1182374208] Agent sending reply for: RESOURCE_START[ora.diskmon 1 1] ID 4098:367



          2013-07-17 08:38:10.537: [    AGFW][1161394496] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:38:10.538: [ora.diskmon][1161394496] [check] DiskmonAgent::check {



          2013-07-17 08:38:10.538: [ora.diskmon][1161394496] [check] DiskmonAgent::check } 0



          2013-07-17 08:38:10.538: [    AGFW][1161394496] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:38:10.538: [    AGFW][1182374208] ora.diskmon 1 1 state changed from: STARTING to: ONLINE



          2013-07-17 08:38:10.538: [    AGFW][1182374208] Started implicit monitor for:ora.diskmon 1 1



          2013-07-17 08:38:10.538: [    AGFW][1182374208] Agent sending last reply for: RESOURCE_START[ora.diskmon 1 1] ID 4098:367



          2013-07-17 08:38:30.543: [    AGFW][1182374208] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:38:30.544: [    AGFW][1161394496] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:38:30.544: [ora.diskmon][1161394496] [check] DiskmonAgent::check {



          2013-07-17 08:38:30.544: [ora.diskmon][1161394496] [check] DiskmonAgent::check } 0



          2013-07-17 08:38:30.545: [    AGFW][1161394496] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:38:50.550: [    AGFW][1182374208] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:38:50.551: [    AGFW][1161394496] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:38:50.551: [ora.diskmon][1161394496] [check] DiskmonAgent::check {



          2013-07-17 08:38:50.551: [ora.diskmon][1161394496] [check] DiskmonAgent::check } 0



          2013-07-17 08:38:50.551: [    AGFW][1161394496] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:39:02.782: [    AGFW][1182374208] Agent received the message: RESOURCE_STOP[ora.diskmon 1 1] ID 4099:547



          2013-07-17 08:39:02.782: [    AGFW][1182374208] Preparing STOP command for: ora.diskmon 1 1



          2013-07-17 08:39:02.782: [    AGFW][1182374208] ora.diskmon 1 1 state changed from: ONLINE to: STOPPING



          2013-07-17 08:39:02.783: [    AGFW][1161394496] Executing command: stop for resource: ora.diskmon 1 1



          2013-07-17 08:39:02.783: [ora.diskmon][1161394496] [stop] clsn_agent::stop {



          2013-07-17 08:39:02.783: [ora.diskmon][1161394496] [stop] DiskmonAgent::stop {



          2013-07-17 08:39:02.783: [ora.diskmon][1161394496] [stop] DiskmonAgent::stop }



          2013-07-17 08:39:02.783: [ora.diskmon][1161394496] [stop] clsn_agent::stop }



          2013-07-17 08:39:02.783: [    AGFW][1161394496] Command: stop for resource: ora.diskmon 1 1 completed with status: SUCCESS



          2013-07-17 08:39:02.784: [    AGFW][1161394496] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:39:02.784: [ora.diskmon][1161394496] [check] DiskmonAgent::check {



          2013-07-17 08:39:02.784: [ora.diskmon][1161394496] [check] DiskmonAgent::check } 2



          2013-07-17 08:39:02.784: [    AGFW][1161394496] check for resource: ora.diskmon 1 1 completed with status: PLANNED_OFFLINE



          2013-07-17 08:39:02.784: [    AGFW][1182374208] Agent sending reply for: RESOURCE_STOP[ora.diskmon 1 1] ID 4099:547



          2013-07-17 08:39:02.785: [    AGFW][1182374208] ora.diskmon 1 1 state changed from: STOPPING to: PLANNED_OFFLINE



          2013-07-17 08:39:02.785: [    AGFW][1182374208] Agent sending last reply for: RESOURCE_STOP[ora.diskmon 1 1] ID 4099:547



          2013-07-17 08:39:02.785: [    AGFW][1182374208] Agent has no resources to be monitored.Sending suicide request.



          2013-07-17 08:39:02.786: [    AGFW][1182374208] Agent sending message to PE: AGENT_SUICIDE[Proxy] ID 20486:72



          2013-07-17 08:39:02.789: [    AGFW][1182374208] Agent is commiting suicide.



          2013-07-17 08:39:02.790: [    AGFW][1182374208] Agent is exiting with exit code: 1



          2013-07-17 08:39:12.176: [    AGFW][1664629584] Starting the agent: /u01/app/product/11.2.0/oracle/log/vmorarac2/agent/ohasd/orarootagent_root/



          2013-07-17 08:39:12.176: [   AGENT][1664629584] Agent framework initialized, Process Id = 10581



          2013-07-17 08:39:12.178: [ USRTHRD][1664629584] Utils::getCrsHome crsHome /u01/app/product/11.2.0/oracle



          2013-07-17 08:39:12.178: [ USRTHRD][1664629584] Process::convertPidToString pid = 10581



          2013-07-17 08:39:12.178: [    AGFW][1664629584] SERVER IPC CONNECT STR: (ADDRESS=(PROTOCOL=IPC)(KEY=OHASD_IPC_SOCKET_11))



          2013-07-17 08:39:12.178: [CLSFRAME][1664629584] Inited lsf context 0xd9309e0



          2013-07-17 08:39:12.179: [CLSFRAME][1664629584] Initing CLS Framework messaging



          2013-07-17 08:39:12.179: [CLSFRAME][1664629584] New Framework state: 2



          2013-07-17 08:39:12.179: [CLSFRAME][1664629584] M2M is starting...



          2013-07-17 08:39:12.180: [ CRSCOMM][1664629584] m_pClscCtx=0xd983bd0m_pUgblm=0xd987720



          2013-07-17 08:39:12.180: [ CRSCOMM][1664629584] Starting send thread



          2013-07-17 08:39:12.181: [ CRSCOMM][1115052352] clsIpc: sendWork thread started.



          2013-07-17 08:39:12.181: [ CRSCOMM][1125542208] IPC Client thread started listening



          2013-07-17 08:39:12.181: [ CRSCOMM][1125542208] init data sent from server



          2013-07-17 08:39:12.181: [CLSFRAME][1664629584] New IPC Member:{Relative|Node:0|Process:0|Type:2}:OHASD:vmorarac2



          2013-07-17 08:39:12.181: [CLSFRAME][1664629584] New process connected to us ID:{Relative|Node:0|Process:0|Type:2} Info:OHASD:vmorarac2



          2013-07-17 08:39:12.182: [CLSFRAME][1664629584] Starting thread model named: MultiThread



          2013-07-17 08:39:12.182: [CLSFRAME][1664629584] Starting thread model named: SingleThread



          2013-07-17 08:39:12.182: [CLSFRAME][1664629584] Starting thread model named: SingleThreadT



          2013-07-17 08:39:12.182: [CLSFRAME][1664629584] New Framework state: 3



          2013-07-17 08:39:12.182: [    AGFW][1664629584] Agent Framework started successfully



          2013-07-17 08:39:12.182: [    AGFW][1177991488] Agfw engine module has enabled...



          2013-07-17 08:39:12.183: [CLSFRAME][1177991488] Module Enabling is complete



          2013-07-17 08:39:12.183: [CLSFRAME][1177991488] New Framework state: 6



          2013-07-17 08:39:12.183: [    AGFW][1177991488] Agent is started with userid: root , expected user: root



          2013-07-17 08:39:12.183: [    AGFW][1177991488] Agent sending message to PE: AGENT_HANDSHAKE[Proxy] ID 20484:14



          2013-07-17 08:39:12.192: [    AGFW][1177991488] Agent received the message: RESTYPE_ADD[ora.crs.type] ID 8196:886



          2013-07-17 08:39:12.192: [    AGFW][1177991488] Added new restype: ora.crs.type



          2013-07-17 08:39:12.192: [    AGFW][1177991488] Agent sending last reply for: RESTYPE_ADD[ora.crs.type] ID 8196:886



          2013-07-17 08:39:12.198: [    AGFW][1177991488] Agent received the message: RESTYPE_ADD[ora.ctss.type] ID 8196:888



          2013-07-17 08:39:12.198: [    AGFW][1177991488] Added new restype: ora.ctss.type



          2013-07-17 08:39:12.199: [    AGFW][1177991488] Agent sending last reply for: RESTYPE_ADD[ora.ctss.type] ID 8196:888



          2013-07-17 08:39:12.204: [    AGFW][1177991488] Agent received the message: RESTYPE_ADD[ora.diskmon.type] ID 8196:890



          2013-07-17 08:39:12.204: [    AGFW][1177991488] Added new restype: ora.diskmon.type



          2013-07-17 08:39:12.204: [    AGFW][1177991488] Agent sending last reply for: RESTYPE_ADD[ora.diskmon.type] ID 8196:890



          2013-07-17 08:39:12.209: [    AGFW][1177991488] Agent received the message: RESTYPE_ADD[ora.drivers.acfs.type] ID 8196:892



          2013-07-17 08:39:12.209: [    AGFW][1177991488] Added new restype: ora.drivers.acfs.type



          2013-07-17 08:39:12.210: [    AGFW][1177991488] Agent sending last reply for: RESTYPE_ADD[ora.drivers.acfs.type] ID 8196:892



          2013-07-17 08:39:12.210: [    AGFW][1177991488] Agent received the message: RESOURCE_ADD[ora.diskmon 1 1] ID 4356:894



          2013-07-17 08:39:12.210: [    AGFW][1177991488] Added new resource: ora.diskmon 1 1 to the agfw



          2013-07-17 08:39:12.210: [    AGFW][1177991488] Agent sending last reply for: RESOURCE_ADD[ora.diskmon 1 1] ID 4356:894



          2013-07-17 08:39:12.210: [    AGFW][1177991488] Agent received the message: RESOURCE_START[ora.diskmon 1 1] ID 4098:895



          2013-07-17 08:39:12.211: [    AGFW][1177991488] Preparing START command for: ora.diskmon 1 1



          2013-07-17 08:39:12.211: [    AGFW][1177991488] ora.diskmon 1 1 state changed from: UNKNOWN to: STARTING



          2013-07-17 08:39:12.216: [    AGFW][1167501632] Executing command: start for resource: ora.diskmon 1 1



          2013-07-17 08:39:12.216: [ora.diskmon][1167501632] [start] clsn_agent::start {



          2013-07-17 08:39:12.216: [ora.diskmon][1167501632] [start] DaemonAgent{



          2013-07-17 08:39:12.216: [ora.diskmon][1167501632] [start] }DaemonAgent



          2013-07-17 08:39:12.216: [ora.diskmon][1167501632] [start] DiskmonAgent::DiskmonAgent {



          2013-07-17 08:39:12.216: [ora.diskmon][1167501632] [start] InitAttrs {



          2013-07-17 08:39:12.216: [ora.diskmon][1167501632] [start] __IS_HASD_AGENT=TRUE



          2013-07-17 08:39:12.216: [ora.diskmon][1167501632] [start] }InitAttrs



          2013-07-17 08:39:12.217: [ora.diskmon][1167501632] [start] DiskmonAgent::DiskmonAgent }



          2013-07-17 08:39:12.217: [ora.diskmon][1167501632] [start] DiskmonAgent::start {



          2013-07-17 08:39:12.217: [ora.diskmon][1167501632] [start] Arg Value = -d



          2013-07-17 08:39:12.217: [ora.diskmon][1167501632] [start] Arg Value = -f



          2013-07-17 08:39:12.217: [ora.diskmon][1167501632] [start] Total Count of Environment Variables = 3



          2013-07-17 08:39:12.217: [ora.diskmon][1167501632] [start] Adding Environment Variables _ORA_AGENT_ACTION=TRUE



          2013-07-17 08:39:12.217: [ora.diskmon][1167501632] [start] Adding Environment Variables __IS_HASD_AGENT=



          2013-07-17 08:39:12.217: [ora.diskmon][1167501632] [start] Adding Environment variable from USR_ORA_ENV ORACLE_USER=oracle



          2013-07-17 08:39:12.217: [ora.diskmon][1167501632] [start] Utils:execCmd action = 1 flags = 5 ohome = (null) cmdname = diskmon.



          2013-07-17 08:39:12.217: [ora.diskmon][1167501632] [start] getOracleHomeAttrib: oracle_home = /u01/app/product/11.2.0/oracle



          2013-07-17 08:39:12.217: [ora.diskmon][1167501632] [start] Utils:execCmd Running the binary from /u01/app/product/11.2.0/oracle/bin/diskmon



          2013-07-17 08:39:12.220: [CRSTIMER][1198971200] Timer Thread Starting.



          2013-07-17 08:39:12.220: [ora.diskmon][1167501632] [start] execCmd ret = 0



          2013-07-17 08:39:12.220: [ora.diskmon][1167501632] [start] }DaemonAgent::start



          2013-07-17 08:39:18.222: [ora.diskmon][1167501632] [start] DiskmonAgent::connect {



          2013-07-17 08:39:18.222: [ora.diskmon][1167501632] [start] Process::convertPidToString pid = 10581



          2013-07-17 08:39:18.222: [ora.diskmon][1167501632] [start] DiskmonAgent::connect }



          2013-07-17 08:39:18.222: [ora.diskmon][1167501632] [start] DiskmonAgent::start }



          2013-07-17 08:39:18.222: [ora.diskmon][1167501632] [start] clsn_agent::start }



          2013-07-17 08:39:18.222: [    AGFW][1167501632] Command: start for resource: ora.diskmon 1 1 completed with status: SUCCESS



          2013-07-17 08:39:18.223: [    AGFW][1177991488] Agent sending reply for: RESOURCE_START[ora.diskmon 1 1] ID 4098:895



          2013-07-17 08:39:18.223: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:39:18.223: [ora.diskmon][1167501632] [check] DiskmonAgent::check {



          2013-07-17 08:39:18.224: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0



          2013-07-17 08:39:18.224: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:39:18.225: [    AGFW][1177991488] ora.diskmon 1 1 state changed from: STARTING to: ONLINE



          2013-07-17 08:39:18.225: [    AGFW][1177991488] Started implicit monitor for:ora.diskmon 1 1



          2013-07-17 08:39:18.225: [    AGFW][1177991488] Agent sending last reply for: RESOURCE_START[ora.diskmon 1 1] ID 4098:895



          2013-07-17 08:39:38.231: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:39:38.232: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:39:38.232: [ora.diskmon][1167501632] [check] DiskmonAgent::check {



          2013-07-17 08:39:38.232: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0



          2013-07-17 08:39:38.232: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:39:58.237: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:39:58.238: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:39:58.238: [ora.diskmon][1167501632] [check] DiskmonAgent::check {



          2013-07-17 08:39:58.238: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0



          2013-07-17 08:39:58.238: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:40:12.107: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:966



          2013-07-17 08:40:18.243: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:40:18.244: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:40:18.244: [ora.diskmon][1167501632] [check] DiskmonAgent::check {



          2013-07-17 08:40:18.244: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0



          2013-07-17 08:40:18.244: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:40:38.250: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:40:38.251: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:40:38.251: [ora.diskmon][1167501632] [check] DiskmonAgent::check {



          2013-07-17 08:40:38.251: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0



          2013-07-17 08:40:38.251: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:40:42.116: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:982



          2013-07-17 08:40:58.245: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:40:58.246: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:40:58.247: [ora.diskmon][1167501632] [check] DiskmonAgent::check {



          2013-07-17 08:40:58.247: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0



          2013-07-17 08:40:58.247: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:41:12.125: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:998



          2013-07-17 08:41:18.252: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:41:18.252: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:41:18.253: [ora.diskmon][1167501632] [check] DiskmonAgent::check {



          2013-07-17 08:41:18.253: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0



          2013-07-17 08:41:18.253: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:41:38.259: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:41:38.260: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:41:38.260: [ora.diskmon][1167501632] [check] DiskmonAgent::check {



          2013-07-17 08:41:38.260: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0



          2013-07-17 08:41:38.260: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:41:58.255: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:41:58.256: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:41:58.256: [ora.diskmon][1167501632] [check] DiskmonAgent::check {



          2013-07-17 08:41:58.256: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0



          2013-07-17 08:41:58.256: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:42:12.134: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:1026



          2013-07-17 08:42:18.261: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:42:18.262: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:42:18.262: [ora.diskmon][1167501632] [check] DiskmonAgent::check {



          2013-07-17 08:42:18.262: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0



          2013-07-17 08:42:18.262: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:42:38.268: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:42:38.269: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:42:38.269: [ora.diskmon][1167501632] [check] DiskmonAgent::check {



          2013-07-17 08:42:38.269: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0



          2013-07-17 08:42:38.269: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:42:58.265: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:42:58.266: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:42:58.266: [ora.diskmon][1167501632] [check] DiskmonAgent::check {



          2013-07-17 08:42:58.267: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0



          2013-07-17 08:42:58.267: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:43:12.144: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:1054



          2013-07-17 08:43:18.272: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:43:18.272: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:43:18.273: [ora.diskmon][1167501632] [check] DiskmonAgent::check {



          2013-07-17 08:43:18.273: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0



          2013-07-17 08:43:18.273: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:43:38.278: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:43:38.278: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:43:38.279: [ora.diskmon][1167501632] [check] DiskmonAgent::check {



          2013-07-17 08:43:38.279: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0



          2013-07-17 08:43:38.279: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:43:42.154: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:1070



          2013-07-17 08:43:58.284: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:43:58.285: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:43:58.285: [ora.diskmon][1167501632] [check] DiskmonAgent::check {



          2013-07-17 08:43:58.285: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0



          2013-07-17 08:43:58.285: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:44:12.154: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:1086



          2013-07-17 08:44:18.291: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:44:18.292: [    AGFW][1167501632] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:44:18.292: [ora.diskmon][1167501632] [check] DiskmonAgent::check {



          2013-07-17 08:44:18.292: [ora.diskmon][1167501632] [check] DiskmonAgent::check } 0



          2013-07-17 08:44:18.292: [    AGFW][1167501632] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:44:38.296: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:44:38.297: [    AGFW][1157011776] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:44:38.297: [ora.diskmon][1157011776] [check] DiskmonAgent::check {



          2013-07-17 08:44:38.297: [ora.diskmon][1157011776] [check] DiskmonAgent::check } 0



          2013-07-17 08:44:38.297: [    AGFW][1157011776] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:44:42.163: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:1102



          2013-07-17 08:44:58.302: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:44:58.303: [    AGFW][1157011776] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:44:58.303: [ora.diskmon][1157011776] [check] DiskmonAgent::check {



          2013-07-17 08:44:58.304: [ora.diskmon][1157011776] [check] DiskmonAgent::check } 0



          2013-07-17 08:44:58.304: [    AGFW][1157011776] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:45:12.174: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:1118



          2013-07-17 08:45:18.309: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:45:18.309: [    AGFW][1157011776] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:45:18.310: [ora.diskmon][1157011776] [check] DiskmonAgent::check {



          2013-07-17 08:45:18.310: [ora.diskmon][1157011776] [check] DiskmonAgent::check } 0



          2013-07-17 08:45:18.310: [    AGFW][1157011776] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:45:38.315: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:45:38.316: [    AGFW][1157011776] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:45:38.316: [ora.diskmon][1157011776] [check] DiskmonAgent::check {



          2013-07-17 08:45:38.316: [ora.diskmon][1157011776] [check] DiskmonAgent::check } 0



          2013-07-17 08:45:38.316: [    AGFW][1157011776] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:45:42.183: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:1134



          2013-07-17 08:45:58.312: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:45:58.313: [    AGFW][1157011776] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:45:58.313: [ora.diskmon][1157011776] [check] DiskmonAgent::check {



          2013-07-17 08:45:58.313: [ora.diskmon][1157011776] [check] DiskmonAgent::check } 0



          2013-07-17 08:45:58.313: [    AGFW][1157011776] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:46:12.192: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:1150



          2013-07-17 08:46:18.318: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:46:18.319: [    AGFW][1157011776] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:46:18.319: [ora.diskmon][1157011776] [check] DiskmonAgent::check {



          2013-07-17 08:46:18.319: [ora.diskmon][1157011776] [check] DiskmonAgent::check } 0



          2013-07-17 08:46:18.319: [    AGFW][1157011776] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:46:38.325: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:46:38.326: [    AGFW][1157011776] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:46:38.326: [ora.diskmon][1157011776] [check] DiskmonAgent::check {



          2013-07-17 08:46:38.326: [ora.diskmon][1157011776] [check] DiskmonAgent::check } 0



          2013-07-17 08:46:38.326: [    AGFW][1157011776] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:46:42.203: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:1166



          2013-07-17 08:46:58.332: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:46:58.333: [    AGFW][1157011776] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:46:58.333: [ora.diskmon][1157011776] [check] DiskmonAgent::check {



          2013-07-17 08:46:58.333: [ora.diskmon][1157011776] [check] DiskmonAgent::check } 0



          2013-07-17 08:46:58.333: [    AGFW][1157011776] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:47:12.203: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:1182



          2013-07-17 08:47:18.338: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:47:18.339: [    AGFW][1157011776] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:47:18.339: [ora.diskmon][1157011776] [check] DiskmonAgent::check {



          2013-07-17 08:47:18.339: [ora.diskmon][1157011776] [check] DiskmonAgent::check } 0



          2013-07-17 08:47:18.340: [    AGFW][1157011776] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:47:38.345: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:47:38.345: [    AGFW][1157011776] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:47:38.346: [ora.diskmon][1157011776] [check] DiskmonAgent::check {



          2013-07-17 08:47:38.346: [ora.diskmon][1157011776] [check] DiskmonAgent::check } 0



          2013-07-17 08:47:38.346: [    AGFW][1157011776] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:47:42.211: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:1198



          2013-07-17 08:47:58.351: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:47:58.352: [    AGFW][1157011776] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:47:58.352: [ora.diskmon][1157011776] [check] DiskmonAgent::check {



          2013-07-17 08:47:58.352: [ora.diskmon][1157011776] [check] DiskmonAgent::check } 0



          2013-07-17 08:47:58.352: [    AGFW][1157011776] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:48:12.220: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:1214



          2013-07-17 08:48:18.358: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:48:18.359: [    AGFW][1157011776] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:48:18.359: [ora.diskmon][1157011776] [check] DiskmonAgent::check {



          2013-07-17 08:48:18.359: [ora.diskmon][1157011776] [check] DiskmonAgent::check } 0



          2013-07-17 08:48:18.359: [    AGFW][1157011776] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:48:38.365: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:48:38.366: [    AGFW][1157011776] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:48:38.366: [ora.diskmon][1157011776] [check] DiskmonAgent::check {



          2013-07-17 08:48:38.366: [ora.diskmon][1157011776] [check] DiskmonAgent::check } 0



          2013-07-17 08:48:38.366: [    AGFW][1157011776] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:48:42.230: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:1230



          2013-07-17 08:48:58.370: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:48:58.371: [    AGFW][1157011776] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:48:58.371: [ora.diskmon][1157011776] [check] DiskmonAgent::check {



          2013-07-17 08:48:58.371: [ora.diskmon][1157011776] [check] DiskmonAgent::check } 0



          2013-07-17 08:48:58.371: [    AGFW][1157011776] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:49:12.238: [    AGFW][1177991488] Agent received the message: AGENT_HB[Engine] ID 12293:1246



          2013-07-17 08:49:18.377: [    AGFW][1177991488] CHECK initiated by timer for: ora.diskmon 1 1



          2013-07-17 08:49:18.378: [    AGFW][1157011776] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:49:18.378: [ora.diskmon][1157011776] [check] DiskmonAgent::check {



          2013-07-17 08:49:18.379: [ora.diskmon][1157011776] [check] DiskmonAgent::check } 0



          2013-07-17 08:49:18.379: [    AGFW][1157011776] check for resource: ora.diskmon 1 1 completed with status: ONLINE



          2013-07-17 08:49:23.262: [    AGFW][1177991488] Agent received the message: RESOURCE_STOP[ora.diskmon 1 1] ID 4099:1310



          2013-07-17 08:49:23.262: [    AGFW][1177991488] Preparing STOP command for: ora.diskmon 1 1



          2013-07-17 08:49:23.262: [    AGFW][1177991488] ora.diskmon 1 1 state changed from: ONLINE to: STOPPING



          2013-07-17 08:49:23.263: [    AGFW][1157011776] Executing command: stop for resource: ora.diskmon 1 1



          2013-07-17 08:49:23.263: [ora.diskmon][1157011776] [stop] clsn_agent::stop {



          2013-07-17 08:49:23.263: [ora.diskmon][1157011776] [stop] DiskmonAgent::stop {



          2013-07-17 08:49:23.264: [ora.diskmon][1157011776] [stop] DiskmonAgent::stop }



          2013-07-17 08:49:23.264: [ora.diskmon][1157011776] [stop] clsn_agent::stop }



          2013-07-17 08:49:23.264: [    AGFW][1157011776] Command: stop for resource: ora.diskmon 1 1 completed with status: SUCCESS



          2013-07-17 08:49:23.264: [    AGFW][1157011776] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:49:23.264: [    AGFW][1177991488] Agent sending reply for: RESOURCE_STOP[ora.diskmon 1 1] ID 4099:1310



          2013-07-17 08:49:23.264: [ora.diskmon][1157011776] [check] DiskmonAgent::check {



          2013-07-17 08:49:23.264: [ora.diskmon][1157011776] [check] DiskmonAgent::check } 2



          2013-07-17 08:49:23.264: [    AGFW][1157011776] check for resource: ora.diskmon 1 1 completed with status: PLANNED_OFFLINE



          2013-07-17 08:49:23.265: [    AGFW][1177991488] ora.diskmon 1 1 state changed from: STOPPING to: PLANNED_OFFLINE



          2013-07-17 08:49:23.265: [    AGFW][1177991488] Agent sending last reply for: RESOURCE_STOP[ora.diskmon 1 1] ID 4099:1310



          2013-07-17 08:49:23.266: [    AGFW][1177991488] Agent has no resources to be monitored.Sending suicide request.



          2013-07-17 08:49:23.267: [    AGFW][1177991488] Agent sending message to PE: AGENT_SUICIDE[Proxy] ID 20486:330



          2013-07-17 08:49:23.269: [    AGFW][1177991488] Agent is commiting suicide.



          2013-07-17 08:49:23.270: [    AGFW][1177991488] Agent is exiting with exit code: 1



          2013-07-17 08:49:28.542: [    AGFW][2996173648] Starting the agent: /u01/app/product/11.2.0/oracle/log/vmorarac2/agent/ohasd/orarootagent_root/



          2013-07-17 08:49:28.542: [   AGENT][2996173648] Agent framework initialized, Process Id = 10940



          2013-07-17 08:49:28.544: [ USRTHRD][2996173648] Utils::getCrsHome crsHome /u01/app/product/11.2.0/oracle



          2013-07-17 08:49:28.544: [ USRTHRD][2996173648] Process::convertPidToString pid = 10940



          2013-07-17 08:49:28.544: [    AGFW][2996173648] SERVER IPC CONNECT STR: (ADDRESS=(PROTOCOL=IPC)(KEY=OHASD_IPC_SOCKET_11))



          2013-07-17 08:49:28.544: [CLSFRAME][2996173648] Inited lsf context 0x1339b9e0



          2013-07-17 08:49:28.544: [CLSFRAME][2996173648] Initing CLS Framework messaging



          2013-07-17 08:49:28.545: [CLSFRAME][2996173648] New Framework state: 2



          2013-07-17 08:49:28.545: [CLSFRAME][2996173648] M2M is starting...



          2013-07-17 08:49:28.546: [ CRSCOMM][2996173648] m_pClscCtx=0x133eebd0m_pUgblm=0x133f2720



          2013-07-17 08:49:28.546: [ CRSCOMM][2996173648] Starting send thread



          2013-07-17 08:49:28.546: [ CRSCOMM][1116977472] clsIpc: sendWork thread started.



          2013-07-17 08:49:28.547: [ CRSCOMM][1127467328] IPC Client thread started listening



          2013-07-17 08:49:28.547: [ CRSCOMM][1127467328] init data sent from server



          2013-07-17 08:49:28.548: [CLSFRAME][2996173648] New IPC Member:{Relative|Node:0|Process:0|Type:2}:OHASD:vmorarac2



          2013-07-17 08:49:28.548: [CLSFRAME][2996173648] New process connected to us ID:{Relative|Node:0|Process:0|Type:2} Info:OHASD:vmorarac2



          2013-07-17 08:49:28.548: [CLSFRAME][2996173648] Starting thread model named: MultiThread



          2013-07-17 08:49:28.549: [CLSFRAME][2996173648] Starting thread model named: SingleThread



          2013-07-17 08:49:28.549: [CLSFRAME][2996173648] Starting thread model named: SingleThreadT



          2013-07-17 08:49:28.549: [CLSFRAME][2996173648] New Framework state: 3



          2013-07-17 08:49:28.549: [    AGFW][2996173648] Agent Framework started successfully



          2013-07-17 08:49:28.549: [    AGFW][1169426752] Agfw engine module has enabled...



          2013-07-17 08:49:28.549: [CLSFRAME][1169426752] Module Enabling is complete



          2013-07-17 08:49:28.549: [CLSFRAME][1169426752] New Framework state: 6



          2013-07-17 08:49:28.549: [    AGFW][1169426752] Agent is started with userid: root , expected user: root



          2013-07-17 08:49:28.550: [    AGFW][1169426752] Agent sending message to PE: AGENT_HANDSHAKE[Proxy] ID 20484:14



          2013-07-17 08:49:28.560: [    AGFW][1169426752] Agent received the message: RESTYPE_ADD[ora.crs.type] ID 8196:1419



          2013-07-17 08:49:28.560: [    AGFW][1169426752] Added new restype: ora.crs.type



          2013-07-17 08:49:28.560: [    AGFW][1169426752] Agent sending last reply for: RESTYPE_ADD[ora.crs.type] ID 8196:1419



          2013-07-17 08:49:28.562: [    AGFW][1169426752] Agent received the message: RESTYPE_ADD[ora.ctss.type] ID 8196:1421



          2013-07-17 08:49:28.563: [    AGFW][1169426752] Added new restype: ora.ctss.type



          2013-07-17 08:49:28.563: [    AGFW][1169426752] Agent sending last reply for: RESTYPE_ADD[ora.ctss.type] ID 8196:1421



          2013-07-17 08:49:28.573: [    AGFW][1169426752] Agent received the message: RESTYPE_ADD[ora.diskmon.type] ID 8196:1423



          2013-07-17 08:49:28.573: [    AGFW][1169426752] Added new restype: ora.diskmon.type



          2013-07-17 08:49:28.574: [    AGFW][1169426752] Agent sending last reply for: RESTYPE_ADD[ora.diskmon.type] ID 8196:1423



          2013-07-17 08:49:28.579: [    AGFW][1169426752] Agent received the message: RESTYPE_ADD[ora.drivers.acfs.type] ID 8196:1425



          2013-07-17 08:49:28.579: [    AGFW][1169426752] Added new restype: ora.drivers.acfs.type



          2013-07-17 08:49:28.580: [    AGFW][1169426752] Agent sending last reply for: RESTYPE_ADD[ora.drivers.acfs.type] ID 8196:1425



          2013-07-17 08:49:28.580: [    AGFW][1169426752] Agent received the message: RESOURCE_ADD[ora.diskmon 1 1] ID 4356:1427



          2013-07-17 08:49:28.580: [    AGFW][1169426752] Added new resource: ora.diskmon 1 1 to the agfw



          2013-07-17 08:49:28.582: [    AGFW][1169426752] Agent sending last reply for: RESOURCE_ADD[ora.diskmon 1 1] ID 4356:1427



          2013-07-17 08:49:28.582: [    AGFW][1169426752] Agent received the message: RESOURCE_START[ora.diskmon 1 1] ID 4098:1428



          2013-07-17 08:49:28.582: [    AGFW][1169426752] Preparing START command for: ora.diskmon 1 1



          2013-07-17 08:49:28.582: [    AGFW][1169426752] ora.diskmon 1 1 state changed from: UNKNOWN to: STARTING



          2013-07-17 08:49:28.589: [    AGFW][1158936896] Executing command: start for resource: ora.diskmon 1 1



          2013-07-17 08:49:28.589: [ora.diskmon][1158936896] [start] clsn_agent::start {



          2013-07-17 08:49:28.589: [ora.diskmon][1158936896] [start] DaemonAgent{



          2013-07-17 08:49:28.589: [ora.diskmon][1158936896] [start] }DaemonAgent



          2013-07-17 08:49:28.589: [ora.diskmon][1158936896] [start] DiskmonAgent::DiskmonAgent {



          2013-07-17 08:49:28.589: [ora.diskmon][1158936896] [start] InitAttrs {



          2013-07-17 08:49:28.589: [ora.diskmon][1158936896] [start] __IS_HASD_AGENT=TRUE



          2013-07-17 08:49:28.589: [ora.diskmon][1158936896] [start] }InitAttrs



          2013-07-17 08:49:28.589: [CRSTIMER][1190406464] Timer Thread Starting.



          2013-07-17 08:49:28.589: [ora.diskmon][1158936896] [start] DiskmonAgent::DiskmonAgent }



          2013-07-17 08:49:28.590: [ora.diskmon][1158936896] [start] DiskmonAgent::start {



          2013-07-17 08:49:28.590: [ora.diskmon][1158936896] [start] Arg Value = -d



          2013-07-17 08:49:28.590: [ora.diskmon][1158936896] [start] Arg Value = -f



          2013-07-17 08:49:28.590: [ora.diskmon][1158936896] [start] Total Count of Environment Variables = 3



          2013-07-17 08:49:28.590: [ora.diskmon][1158936896] [start] Adding Environment Variables _ORA_AGENT_ACTION=TRUE



          2013-07-17 08:49:28.590: [ora.diskmon][1158936896] [start] Adding Environment Variables __IS_HASD_AGENT=



          2013-07-17 08:49:28.590: [ora.diskmon][1158936896] [start] Adding Environment variable from USR_ORA_ENV CTSS_REBOOT=TRUE



          2013-07-17 08:49:28.590: [ora.diskmon][1158936896] [start] Utils:execCmd action = 1 flags = 5 ohome = (null) cmdname = diskmon.



          2013-07-17 08:49:28.590: [ora.diskmon][1158936896] [start] getOracleHomeAttrib: oracle_home = /u01/app/product/11.2.0/oracle



          2013-07-17 08:49:28.590: [ora.diskmon][1158936896] [start] Utils:execCmd Running the binary from /u01/app/product/11.2.0/oracle/bin/diskmon



          2013-07-17 08:49:28.593: [ora.diskmon][1158936896] [start] execCmd ret = 0



          2013-07-17 08:49:28.593: [ora.diskmon][1158936896] [start] }DaemonAgent::start



          2013-07-17 08:49:34.594: [ora.diskmon][1158936896] [start] DiskmonAgent::connect {



          2013-07-17 08:50:28.475: [    AGFW][1169426752] Agent received the message: AGENT_HB[Engine] ID 12293:1487



          2013-07-17 08:50:28.601: [    AGFW][1148447040]Created alert : (:CRSAGF00113:) :  Aborting the command: start for resource: ora.diskmon 1 1



          2013-07-17 08:50:28.601: [ora.diskmon][1148447040] [start] clsn_agent::abort {



          2013-07-17 08:50:28.601: [ora.diskmon][1148447040] [start] abort {



          2013-07-17 08:50:28.601: [ora.diskmon][1148447040] [start] abort command: start



          2013-07-17 08:50:28.601: [ora.diskmon][1148447040] [start] tryActionLock {



          2013-07-17 08:50:28.761: [ora.diskmon][1158936896] [start] DiskmonAgent::connect: skgznp_connect failed with error 56815 and the timeout expired



          2013-07-17 08:50:28.761: [ora.diskmon][1158936896] [start] (null) category: 56815, operation: connect, loc: skgznpcon6, OS error: 2, other:



          2013-07-17 08:50:28.761: [ora.diskmon][1158936896] [start] DiskmonAgent::connect } error



          2013-07-17 08:50:28.761: [ora.diskmon][1158936896] [start] DiskmonAgent::start }



          2013-07-17 08:50:28.761: [ora.diskmon][1158936896] [start] clsn_agent::start }



          2013-07-17 08:50:32.604: [ora.diskmon][1148447040] [start] got lock



          2013-07-17 08:50:32.604: [ora.diskmon][1148447040] [start] tryActionLock }



          2013-07-17 08:50:32.604: [ora.diskmon][1148447040] [start] abort  }



          2013-07-17 08:50:32.604: [ora.diskmon][1148447040] [start] clsn_agent::abort }



          2013-07-17 08:50:32.604: [    AGFW][1148447040] Command: start for resource: ora.diskmon 1 1 completed with status: TIMEDOUT



          2013-07-17 08:50:32.605: [    AGFW][1148447040] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:50:32.605: [ora.diskmon][1148447040] [check] DiskmonAgent::check {



          2013-07-17 08:50:32.605: [ora.diskmon][1148447040] [check] DiskmonAgent::connect {



          2013-07-17 08:50:32.605: [ora.diskmon][1148447040] [check] DiskmonAgent::connect: skgznp_connect failed with error 56815 and the timeout expired



          2013-07-17 08:50:32.605: [ora.diskmon][1148447040] [check] (null) category: 56815, operation: connect, loc: skgznpcon6, OS error: 2, other:



          2013-07-17 08:50:32.605: [ora.diskmon][1148447040] [check] DiskmonAgent::connect } error



          2013-07-17 08:50:32.605: [ora.diskmon][1148447040] [check] DiskmonAgent::check } 5



          2013-07-17 08:50:32.605: [    AGFW][1148447040] check for resource: ora.diskmon 1 1 completed with status: FAILED



          2013-07-17 08:50:32.606: [    AGFW][1169426752] Agent sending reply for: RESOURCE_START[ora.diskmon 1 1] ID 4098:1428



          2013-07-17 08:50:32.606: [    AGFW][1169426752] ora.diskmon 1 1 state changed from: STARTING to: FAILED



          2013-07-17 08:50:32.606: [    AGFW][1169426752] Agent sending last reply for: RESOURCE_START[ora.diskmon 1 1] ID 4098:1428



          2013-07-17 08:50:32.607: [    AGFW][1169426752] Agent has no resources to be monitored.Sending suicide request.



          2013-07-17 08:50:32.607: [    AGFW][1169426752] Agent sending message to PE: AGENT_SUICIDE[Proxy] ID 20486:42



          2013-07-17 08:50:32.612: [    AGFW][1169426752] Agent is commiting suicide.



          2013-07-17 08:50:32.612: [    AGFW][1169426752] Agent is exiting with exit code: 1



          2013-07-17 08:50:32.699: [    AGFW][2907843408] Starting the agent: /u01/app/product/11.2.0/oracle/log/vmorarac2/agent/ohasd/orarootagent_root/



          2013-07-17 08:50:32.700: [   AGENT][2907843408] Agent framework initialized, Process Id = 11009



          2013-07-17 08:50:32.701: [ USRTHRD][2907843408] Utils::getCrsHome crsHome /u01/app/product/11.2.0/oracle



          2013-07-17 08:50:32.701: [ USRTHRD][2907843408] Process::convertPidToString pid = 11009



          2013-07-17 08:50:32.702: [    AGFW][2907843408] SERVER IPC CONNECT STR: (ADDRESS=(PROTOCOL=IPC)(KEY=OHASD_IPC_SOCKET_11))



          2013-07-17 08:50:32.702: [CLSFRAME][2907843408] Inited lsf context 0x42439e0



          2013-07-17 08:50:32.702: [CLSFRAME][2907843408] Initing CLS Framework messaging



          2013-07-17 08:50:32.702: [CLSFRAME][2907843408] New Framework state: 2



          2013-07-17 08:50:32.702: [CLSFRAME][2907843408] M2M is starting...



          2013-07-17 08:50:32.704: [ CRSCOMM][2907843408] m_pClscCtx=0x4296bd0m_pUgblm=0x429a720



          2013-07-17 08:50:32.704: [ CRSCOMM][2907843408] Starting send thread



          2013-07-17 08:50:32.705: [ CRSCOMM][1096681792] clsIpc: sendWork thread started.



          2013-07-17 08:50:32.705: [ CRSCOMM][1119566144] IPC Client thread started listening



          2013-07-17 08:50:32.705: [ CRSCOMM][1119566144] init data sent from server



          2013-07-17 08:50:32.705: [CLSFRAME][2907843408] New IPC Member:{Relative|Node:0|Process:0|Type:2}:OHASD:vmorarac2



          2013-07-17 08:50:32.705: [CLSFRAME][2907843408] New process connected to us ID:{Relative|Node:0|Process:0|Type:2} Info:OHASD:vmorarac2



          2013-07-17 08:50:32.706: [CLSFRAME][2907843408] Starting thread model named: MultiThread



          2013-07-17 08:50:32.706: [CLSFRAME][2907843408] Starting thread model named: SingleThread



          2013-07-17 08:50:32.706: [CLSFRAME][2907843408] Starting thread model named: SingleThreadT



          2013-07-17 08:50:32.706: [CLSFRAME][2907843408] New Framework state: 3



          2013-07-17 08:50:32.706: [    AGFW][2907843408] Agent Framework started successfully



          2013-07-17 08:50:32.707: [    AGFW][1172015424] Agfw engine module has enabled...



          2013-07-17 08:50:32.707: [CLSFRAME][1172015424] Module Enabling is complete



          2013-07-17 08:50:32.707: [CLSFRAME][1172015424] New Framework state: 6



          2013-07-17 08:50:32.707: [    AGFW][1172015424] Agent is started with userid: root , expected user: root



          2013-07-17 08:50:32.707: [    AGFW][1172015424] Agent sending message to PE: AGENT_HANDSHAKE[Proxy] ID 20484:14



          2013-07-17 08:50:32.717: [    AGFW][1172015424] Agent received the message: RESTYPE_ADD[ora.crs.type] ID 8196:1527



          2013-07-17 08:50:32.717: [    AGFW][1172015424] Added new restype: ora.crs.type



          2013-07-17 08:50:32.717: [    AGFW][1172015424] Agent sending last reply for: RESTYPE_ADD[ora.crs.type] ID 8196:1527



          2013-07-17 08:50:32.722: [    AGFW][1172015424] Agent received the message: RESTYPE_ADD[ora.ctss.type] ID 8196:1529



          2013-07-17 08:50:32.723: [    AGFW][1172015424] Added new restype: ora.ctss.type



          2013-07-17 08:50:32.723: [    AGFW][1172015424] Agent sending last reply for: RESTYPE_ADD[ora.ctss.type] ID 8196:1529



          2013-07-17 08:50:32.728: [    AGFW][1172015424] Agent received the message: RESTYPE_ADD[ora.diskmon.type] ID 8196:1531



          2013-07-17 08:50:32.730: [    AGFW][1172015424] Added new restype: ora.diskmon.type



          2013-07-17 08:50:32.730: [    AGFW][1172015424] Agent sending last reply for: RESTYPE_ADD[ora.diskmon.type] ID 8196:1531



          2013-07-17 08:50:32.732: [    AGFW][1172015424] Agent received the message: RESTYPE_ADD[ora.drivers.acfs.type] ID 8196:1533



          2013-07-17 08:50:32.732: [    AGFW][1172015424] Added new restype: ora.drivers.acfs.type



          2013-07-17 08:50:32.733: [    AGFW][1172015424] Agent sending last reply for: RESTYPE_ADD[ora.drivers.acfs.type] ID 8196:1533



          2013-07-17 08:50:32.733: [    AGFW][1172015424] Agent received the message: RESOURCE_ADD[ora.diskmon 1 1] ID 4356:1535



          2013-07-17 08:50:32.733: [    AGFW][1172015424] Added new resource: ora.diskmon 1 1 to the agfw



          2013-07-17 08:50:32.734: [    AGFW][1172015424] Agent sending last reply for: RESOURCE_ADD[ora.diskmon 1 1] ID 4356:1535



          2013-07-17 08:50:32.734: [    AGFW][1172015424] Agent received the message: RESOURCE_CLEAN[ora.diskmon 1 1] ID 4100:1536



          2013-07-17 08:50:32.734: [    AGFW][1172015424] Preparing CLEAN command for: ora.diskmon 1 1



          2013-07-17 08:50:32.734: [    AGFW][1172015424] ora.diskmon 1 1 state changed from: UNKNOWN to: CLEANING



          2013-07-17 08:50:32.739: [    AGFW][1151035712] Executing command: clean for resource: ora.diskmon 1 1



          2013-07-17 08:50:32.740: [ora.diskmon][1151035712] [clean] clsn_agent::clean {



          2013-07-17 08:50:32.740: [ora.diskmon][1151035712] [clean] DaemonAgent{



          2013-07-17 08:50:32.740: [ora.diskmon][1151035712] [clean] }DaemonAgent



          2013-07-17 08:50:32.740: [ora.diskmon][1151035712] [clean] DiskmonAgent::DiskmonAgent {



          2013-07-17 08:50:32.740: [ora.diskmon][1151035712] [clean] InitAttrs {



          2013-07-17 08:50:32.740: [ora.diskmon][1151035712] [clean] __IS_HASD_AGENT=TRUE



          2013-07-17 08:50:32.740: [ora.diskmon][1151035712] [clean] }InitAttrs



          2013-07-17 08:50:32.740: [ora.diskmon][1151035712] [clean] DiskmonAgent::DiskmonAgent }



          2013-07-17 08:50:32.740: [ora.diskmon][1151035712] [clean] DiskmonAgent::clean {



          2013-07-17 08:50:32.740: [ora.diskmon][1151035712] [clean] Arg Value = -d



          2013-07-17 08:50:32.740: [ora.diskmon][1151035712] [clean] Arg Value = -z



          2013-07-17 08:50:32.740: [ora.diskmon][1151035712] [clean] Arg Value = 8



          2013-07-17 08:50:32.740: [ora.diskmon][1151035712] [clean] Total Count of Environment Variables = 3



          2013-07-17 08:50:32.740: [ora.diskmon][1151035712] [clean] Adding Environment Variables _ORA_AGENT_ACTION=TRUE



          2013-07-17 08:50:32.740: [ora.diskmon][1151035712] [clean] Adding Environment Variables __IS_HASD_AGENT=



          2013-07-17 08:50:32.740: [ora.diskmon][1151035712] [clean] Adding Environment variable from USR_ORA_ENV CTSS_REBOOT=TRUE



          2013-07-17 08:50:32.740: [ora.diskmon][1151035712] [clean] Utils:execCmd action = 4 flags = 6 ohome = (null) cmdname = diskmon.



          2013-07-17 08:50:32.741: [ora.diskmon][1151035712] [clean] getOracleHomeAttrib: oracle_home = /u01/app/product/11.2.0/oracle



          2013-07-17 08:50:32.741: [ora.diskmon][1151035712] [clean] Utils:execCmd Running the binary from /u01/app/product/11.2.0/oracle/bin/diskmon



          2013-07-17 08:50:32.746: [CRSTIMER][1192995136] Timer Thread Starting.



          2013-07-17 08:50:32.847: [ora.diskmon][1151035712] [clean] (:CLSN00010:)Utils:execCmd scls_process_join() uret 1



           



          2013-07-17 08:50:32.848: [ora.diskmon][1151035712] [clean] Exception type=2 string=CRS-5016: Process "/u01/app/product/11.2.0/oracle/bin/diskmon" spawned by agent "/u01/app/product/11.2.0/oracle/bin/orarootagent.bin" for action "clean" failed: details at "(:CLSN00010:)" in "/u01/app/product/11.2.0/oracle/log/vmorarac2/agent/ohasd/orarootagent_root/orarootagent_root.log"



           



          2013-07-17 08:50:32.848: [    AGFW][1151035712] sending status msg [CRS-5016: Process "/u01/app/product/11.2.0/oracle/bin/diskmon" spawned by agent "/u01/app/product/11.2.0/oracle/bin/orarootagent.bin" for action "clean" failed: details at "(:CLSN00010:)" in "/u01/app/product/11.2.0/oracle/log/vmorarac2/agent/ohasd/orarootagent_root/orarootagent_root.log"



          ] for clean for resource: ora.diskmon 1 1



          2013-07-17 08:50:32.848: [    AGFW][1172015424] Agent sending reply for: RESOURCE_CLEAN[ora.diskmon 1 1] ID 4100:1536



          2013-07-17 08:50:32.850: [ora.diskmon][1151035712] [clean] execCmd ret = 1



          2013-07-17 08:50:32.850: [ora.diskmon][1151035712] [clean] DiskmonAgent::clean } nopipe



          2013-07-17 08:50:32.850: [ora.diskmon][1151035712] [clean] clsn_agent::clean }



          2013-07-17 08:50:32.850: [    AGFW][1151035712] Command: clean for resource: ora.diskmon 1 1 completed with status: SUCCESS



          2013-07-17 08:50:32.850: [    AGFW][1161525568] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 08:50:32.850: [CLSFRAME][1130056000] TM [MultiThread] is changing desired thread # to 3. Current # is 2



          2013-07-17 08:50:32.851: [    AGFW][1172015424] Agent sending reply for: RESOURCE_CLEAN[ora.diskmon 1 1] ID 4100:1536



          2013-07-17 08:50:32.851: [ora.diskmon][1161525568] [check] DiskmonAgent::check {



          2013-07-17 08:50:32.851: [ora.diskmon][1161525568] [check] DiskmonAgent::connect {



          2013-07-17 08:50:32.851: [ora.diskmon][1161525568] [check] DiskmonAgent::connect: skgznp_connect failed with error 56815 and the timeout expired



          2013-07-17 08:50:32.851: [ora.diskmon][1161525568] [check] (null) category: 56815, operation: connect, loc: skgznpcon6, OS error: 2, other:



          2013-07-17 08:50:32.851: [ora.diskmon][1161525568] [check] DiskmonAgent::connect } error



          2013-07-17 08:50:32.851: [ora.diskmon][1161525568] [check] DiskmonAgent::check } 2



          2013-07-17 08:50:32.851: [    AGFW][1161525568] check for resource: ora.diskmon 1 1 completed with status: PLANNED_OFFLINE



          2013-07-17 08:50:32.852: [    AGFW][1172015424] ora.diskmon 1 1 state changed from: CLEANING to: PLANNED_OFFLINE



          2013-07-17 08:50:32.852: [    AGFW][1172015424] Agent sending last reply for: RESOURCE_CLEAN[ora.diskmon 1 1] ID 4100:1536



          2013-07-17 08:50:32.852: [    AGFW][1172015424] Agent has no resources to be monitored.Sending suicide request.



          2013-07-17 08:50:32.852: [    AGFW][1172015424] Agent sending message to PE: AGENT_SUICIDE[Proxy] ID 20486:40



          2013-07-17 08:50:32.855: [    AGFW][1172015424] Agent is commiting suicide.



          2013-07-17 08:50:32.856: [    AGFW][1172015424] Agent is exiting with exit code: 1



          2013-07-17 10:50:02.795: [    AGFW][812591952] Starting the agent: /u01/app/product/11.2.0/oracle/log/vmorarac2/agent/ohasd/orarootagent_root/



          2013-07-17 10:50:02.796: [   AGENT][812591952] Agent framework initialized, Process Id = 11577



          2013-07-17 10:50:02.797: [ USRTHRD][812591952] Utils::getCrsHome crsHome /u01/app/product/11.2.0/oracle



          2013-07-17 10:50:02.798: [ USRTHRD][812591952] Process::convertPidToString pid = 11577



          2013-07-17 10:50:02.798: [    AGFW][812591952] SERVER IPC CONNECT STR: (ADDRESS=(PROTOCOL=IPC)(KEY=OHASD_IPC_SOCKET_11))



          2013-07-17 10:50:02.798: [CLSFRAME][812591952] Inited lsf context 0x11c249e0



          2013-07-17 10:50:02.798: [CLSFRAME][812591952] Initing CLS Framework messaging



          2013-07-17 10:50:02.798: [CLSFRAME][812591952] New Framework state: 2



          2013-07-17 10:50:02.798: [CLSFRAME][812591952] M2M is starting...



          2013-07-17 10:50:02.800: [ CRSCOMM][812591952] m_pClscCtx=0x11c77bd0m_pUgblm=0x11c7b720



          2013-07-17 10:50:02.800: [ CRSCOMM][812591952] Starting send thread



          2013-07-17 10:50:02.800: [ CRSCOMM][1102891328] clsIpc: sendWork thread started.



          2013-07-17 10:50:02.801: [ CRSCOMM][1113381184] IPC Client thread started listening



          2013-07-17 10:50:02.801: [ CRSCOMM][1113381184] init data sent from server



          2013-07-17 10:50:02.801: [CLSFRAME][812591952] New IPC Member:{Relative|Node:0|Process:0|Type:2}:OHASD:vmorarac2



          2013-07-17 10:50:02.801: [CLSFRAME][812591952] New process connected to us ID:{Relative|Node:0|Process:0|Type:2} Info:OHASD:vmorarac2



          2013-07-17 10:50:02.802: [CLSFRAME][812591952] Starting thread model named: MultiThread



          2013-07-17 10:50:02.803: [CLSFRAME][812591952] Starting thread model named: SingleThread



          2013-07-17 10:50:02.803: [CLSFRAME][812591952] Starting thread model named: SingleThreadT



          2013-07-17 10:50:02.803: [CLSFRAME][812591952] New Framework state: 3



          2013-07-17 10:50:02.803: [    AGFW][812591952] Agent Framework started successfully



          2013-07-17 10:50:02.803: [    AGFW][1165830464] Agfw engine module has enabled...



          2013-07-17 10:50:02.803: [CLSFRAME][1165830464] Module Enabling is complete



          2013-07-17 10:50:02.803: [CLSFRAME][1165830464] New Framework state: 6



          2013-07-17 10:50:02.803: [    AGFW][1165830464] Agent is started with userid: root , expected user: root



          2013-07-17 10:50:02.804: [    AGFW][1165830464] Agent sending message to PE: AGENT_HANDSHAKE[Proxy] ID 20484:14



          2013-07-17 10:50:02.812: [    AGFW][1165830464] Agent received the message: RESTYPE_ADD[ora.crs.type] ID 8196:3568



          2013-07-17 10:50:02.812: [    AGFW][1165830464] Added new restype: ora.crs.type



          2013-07-17 10:50:02.812: [    AGFW][1165830464] Agent sending last reply for: RESTYPE_ADD[ora.crs.type] ID 8196:3568



          2013-07-17 10:50:02.819: [    AGFW][1165830464] Agent received the message: RESTYPE_ADD[ora.ctss.type] ID 8196:3570



          2013-07-17 10:50:02.820: [    AGFW][1165830464] Added new restype: ora.ctss.type



          2013-07-17 10:50:02.820: [    AGFW][1165830464] Agent sending last reply for: RESTYPE_ADD[ora.ctss.type] ID 8196:3570



          2013-07-17 10:50:02.820: [    AGFW][1165830464] Agent received the message: RESTYPE_ADD[ora.diskmon.type] ID 8196:3572



          2013-07-17 10:50:02.821: [    AGFW][1165830464] Added new restype: ora.diskmon.type



          2013-07-17 10:50:02.821: [    AGFW][1165830464] Agent sending last reply for: RESTYPE_ADD[ora.diskmon.type] ID 8196:3572



          2013-07-17 10:50:02.829: [    AGFW][1165830464] Agent received the message: RESTYPE_ADD[ora.drivers.acfs.type] ID 8196:3574



          2013-07-17 10:50:02.829: [    AGFW][1165830464] Added new restype: ora.drivers.acfs.type



          2013-07-17 10:50:02.830: [    AGFW][1165830464] Agent sending last reply for: RESTYPE_ADD[ora.drivers.acfs.type] ID 8196:3574



          2013-07-17 10:50:02.830: [    AGFW][1165830464] Agent received the message: RESOURCE_ADD[ora.diskmon 1 1] ID 4356:3576



          2013-07-17 10:50:02.830: [    AGFW][1165830464] Added new resource: ora.diskmon 1 1 to the agfw



          2013-07-17 10:50:02.831: [    AGFW][1165830464] Agent sending last reply for: RESOURCE_ADD[ora.diskmon 1 1] ID 4356:3576



          2013-07-17 10:50:02.832: [    AGFW][1165830464] Agent received the message: RESOURCE_START[ora.diskmon 1 1] ID 4098:3577



          2013-07-17 10:50:02.832: [    AGFW][1165830464] Preparing START command for: ora.diskmon 1 1



          2013-07-17 10:50:02.832: [    AGFW][1165830464] ora.diskmon 1 1 state changed from: UNKNOWN to: STARTING



          2013-07-17 10:50:02.842: [    AGFW][1144850752] Executing command: start for resource: ora.diskmon 1 1



          2013-07-17 10:50:02.842: [ora.diskmon][1144850752] [start] clsn_agent::start {



          2013-07-17 10:50:02.843: [ora.diskmon][1144850752] [start] DaemonAgent{



          2013-07-17 10:50:02.843: [ora.diskmon][1144850752] [start] }DaemonAgent



          2013-07-17 10:50:02.843: [ora.diskmon][1144850752] [start] DiskmonAgent::DiskmonAgent {



          2013-07-17 10:50:02.843: [ora.diskmon][1144850752] [start] InitAttrs {



          2013-07-17 10:50:02.843: [ora.diskmon][1144850752] [start] __IS_HASD_AGENT=TRUE



          2013-07-17 10:50:02.843: [ora.diskmon][1144850752] [start] }InitAttrs



          2013-07-17 10:50:02.843: [ora.diskmon][1144850752] [start] DiskmonAgent::DiskmonAgent }



          2013-07-17 10:50:02.843: [ora.diskmon][1144850752] [start] DiskmonAgent::start {



          2013-07-17 10:50:02.843: [ora.diskmon][1144850752] [start] Arg Value = -d



          2013-07-17 10:50:02.843: [ora.diskmon][1144850752] [start] Arg Value = -f



          2013-07-17 10:50:02.843: [ora.diskmon][1144850752] [start] Total Count of Environment Variables = 3



          2013-07-17 10:50:02.843: [ora.diskmon][1144850752] [start] Adding Environment Variables _ORA_AGENT_ACTION=TRUE



          2013-07-17 10:50:02.843: [ora.diskmon][1144850752] [start] Adding Environment Variables __IS_HASD_AGENT=



          2013-07-17 10:50:02.843: [ora.diskmon][1144850752] [start] Adding Environment variable from USR_ORA_ENV CTSS_REBOOT=TRUE



          2013-07-17 10:50:02.843: [ora.diskmon][1144850752] [start] Utils:execCmd action = 1 flags = 5 ohome = (null) cmdname = diskmon.



          2013-07-17 10:50:02.844: [ora.diskmon][1144850752] [start] getOracleHomeAttrib: oracle_home = /u01/app/product/11.2.0/oracle



          2013-07-17 10:50:02.844: [ora.diskmon][1144850752] [start] Utils:execCmd Running the binary from /u01/app/product/11.2.0/oracle/bin/diskmon



          2013-07-17 10:50:02.846: [CRSTIMER][1186810176] Timer Thread Starting.



          2013-07-17 10:50:02.847: [ora.diskmon][1144850752] [start] execCmd ret = 0



          2013-07-17 10:50:02.847: [ora.diskmon][1144850752] [start] }DaemonAgent::start



          2013-07-17 10:50:08.849: [ora.diskmon][1144850752] [start] DiskmonAgent::connect {



          2013-07-17 10:51:02.726: [    AGFW][1165830464] Agent received the message: AGENT_HB[Engine] ID 12293:3636



          2013-07-17 10:51:02.857: [    AGFW][1155340608]Created alert : (:CRSAGF00113:) :  Aborting the command: start for resource: ora.diskmon 1 1



          2013-07-17 10:51:02.857: [ora.diskmon][1155340608] [start] clsn_agent::abort {



          2013-07-17 10:51:02.857: [ora.diskmon][1155340608] [start] abort {



          2013-07-17 10:51:02.857: [ora.diskmon][1155340608] [start] abort command: start



          2013-07-17 10:51:02.857: [ora.diskmon][1155340608] [start] tryActionLock {



          2013-07-17 10:51:03.010: [ora.diskmon][1144850752] [start] DiskmonAgent::connect: skgznp_connect failed with error 56815 and the timeout expired



          2013-07-17 10:51:03.010: [ora.diskmon][1144850752] [start] (null) category: 56815, operation: connect, loc: skgznpcon6, OS error: 2, other:



          2013-07-17 10:51:03.011: [ora.diskmon][1144850752] [start] DiskmonAgent::connect } error



          2013-07-17 10:51:03.011: [ora.diskmon][1144850752] [start] DiskmonAgent::start }



          2013-07-17 10:51:03.011: [ora.diskmon][1144850752] [start] clsn_agent::start }



          2013-07-17 10:51:06.861: [ora.diskmon][1155340608] [start] got lock



          2013-07-17 10:51:06.861: [ora.diskmon][1155340608] [start] tryActionLock }



          2013-07-17 10:51:06.861: [ora.diskmon][1155340608] [start] abort  }



          2013-07-17 10:51:06.861: [ora.diskmon][1155340608] [start] clsn_agent::abort }



          2013-07-17 10:51:06.861: [    AGFW][1155340608] Command: start for resource: ora.diskmon 1 1 completed with status: TIMEDOUT



          2013-07-17 10:51:06.861: [    AGFW][1165830464] Agent sending reply for: RESOURCE_START[ora.diskmon 1 1] ID 4098:3577



          2013-07-17 10:51:06.862: [    AGFW][1144850752] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 10:51:06.862: [ora.diskmon][1144850752] [check] DiskmonAgent::check {



          2013-07-17 10:51:06.862: [ora.diskmon][1144850752] [check] DiskmonAgent::connect {



          2013-07-17 10:51:06.862: [ora.diskmon][1144850752] [check] DiskmonAgent::connect: skgznp_connect failed with error 56815 and the timeout expired



          2013-07-17 10:51:06.862: [ora.diskmon][1144850752] [check] (null) category: 56815, operation: connect, loc: skgznpcon6, OS error: 2, other:



          2013-07-17 10:51:06.862: [ora.diskmon][1144850752] [check] DiskmonAgent::connect } error



          2013-07-17 10:51:06.862: [ora.diskmon][1144850752] [check] DiskmonAgent::check } 5



          2013-07-17 10:51:06.862: [    AGFW][1144850752] check for resource: ora.diskmon 1 1 completed with status: FAILED



          2013-07-17 10:51:06.862: [    AGFW][1165830464] ora.diskmon 1 1 state changed from: STARTING to: FAILED



          2013-07-17 10:51:06.863: [    AGFW][1165830464] Agent sending last reply for: RESOURCE_START[ora.diskmon 1 1] ID 4098:3577



          2013-07-17 10:51:06.863: [    AGFW][1165830464] Agent has no resources to be monitored.Sending suicide request.



          2013-07-17 10:51:06.863: [    AGFW][1165830464] Agent sending message to PE: AGENT_SUICIDE[Proxy] ID 20486:42



          2013-07-17 10:51:06.866: [    AGFW][1165830464] Agent is commiting suicide.



          2013-07-17 10:51:06.866: [    AGFW][1165830464] Agent is exiting with exit code: 1



          2013-07-17 10:51:06.954: [    AGFW][551844688] Starting the agent: /u01/app/product/11.2.0/oracle/log/vmorarac2/agent/ohasd/orarootagent_root/



          2013-07-17 10:51:06.955: [   AGENT][551844688] Agent framework initialized, Process Id = 11643



          2013-07-17 10:51:06.956: [ USRTHRD][551844688] Utils::getCrsHome crsHome /u01/app/product/11.2.0/oracle



          2013-07-17 10:51:06.957: [ USRTHRD][551844688] Process::convertPidToString pid = 11643



          2013-07-17 10:51:06.957: [    AGFW][551844688] SERVER IPC CONNECT STR: (ADDRESS=(PROTOCOL=IPC)(KEY=OHASD_IPC_SOCKET_11))



          2013-07-17 10:51:06.957: [CLSFRAME][551844688] Inited lsf context 0x1941c9e0



          2013-07-17 10:51:06.957: [CLSFRAME][551844688] Initing CLS Framework messaging



          2013-07-17 10:51:06.957: [CLSFRAME][551844688] New Framework state: 2



          2013-07-17 10:51:06.957: [CLSFRAME][551844688] M2M is starting...



          2013-07-17 10:51:06.959: [ CRSCOMM][551844688] m_pClscCtx=0x1946fbd0m_pUgblm=0x19473720



          2013-07-17 10:51:06.959: [ CRSCOMM][551844688] Starting send thread



          2013-07-17 10:51:06.960: [ CRSCOMM][1117034816] clsIpc: sendWork thread started.



          2013-07-17 10:51:06.960: [ CRSCOMM][1127524672] IPC Client thread started listening



          2013-07-17 10:51:06.960: [ CRSCOMM][1127524672] init data sent from server



          2013-07-17 10:51:06.960: [CLSFRAME][551844688] New IPC Member:{Relative|Node:0|Process:0|Type:2}:OHASD:vmorarac2



          2013-07-17 10:51:06.960: [CLSFRAME][551844688] New process connected to us ID:{Relative|Node:0|Process:0|Type:2} Info:OHASD:vmorarac2



          2013-07-17 10:51:06.961: [CLSFRAME][551844688] Starting thread model named: MultiThread



          2013-07-17 10:51:06.961: [CLSFRAME][551844688] Starting thread model named: SingleThread



          2013-07-17 10:51:06.962: [CLSFRAME][551844688] Starting thread model named: SingleThreadT



          2013-07-17 10:51:06.962: [CLSFRAME][551844688] New Framework state: 3



          2013-07-17 10:51:06.962: [    AGFW][551844688] Agent Framework started successfully



          2013-07-17 10:51:06.962: [    AGFW][1179973952] Agfw engine module has enabled...



          2013-07-17 10:51:06.962: [CLSFRAME][1179973952] Module Enabling is complete



          2013-07-17 10:51:06.962: [CLSFRAME][1179973952] New Framework state: 6



          2013-07-17 10:51:06.962: [    AGFW][1179973952] Agent is started with userid: root , expected user: root



          2013-07-17 10:51:06.962: [    AGFW][1179973952] Agent sending message to PE: AGENT_HANDSHAKE[Proxy] ID 20484:14



          2013-07-17 10:51:06.972: [    AGFW][1179973952] Agent received the message: RESTYPE_ADD[ora.crs.type] ID 8196:3676



          2013-07-17 10:51:06.973: [    AGFW][1179973952] Added new restype: ora.crs.type



          2013-07-17 10:51:06.973: [    AGFW][1179973952] Agent sending last reply for: RESTYPE_ADD[ora.crs.type] ID 8196:3676



          2013-07-17 10:51:06.980: [    AGFW][1179973952] Agent received the message: RESTYPE_ADD[ora.ctss.type] ID 8196:3678



          2013-07-17 10:51:06.980: [    AGFW][1179973952] Added new restype: ora.ctss.type



          2013-07-17 10:51:06.980: [    AGFW][1179973952] Agent sending last reply for: RESTYPE_ADD[ora.ctss.type] ID 8196:3678



          2013-07-17 10:51:06.986: [    AGFW][1179973952] Agent received the message: RESTYPE_ADD[ora.diskmon.type] ID 8196:3680



          2013-07-17 10:51:06.986: [    AGFW][1179973952] Added new restype: ora.diskmon.type



          2013-07-17 10:51:06.987: [    AGFW][1179973952] Agent sending last reply for: RESTYPE_ADD[ora.diskmon.type] ID 8196:3680



          2013-07-17 10:51:06.987: [    AGFW][1179973952] Agent received the message: RESTYPE_ADD[ora.drivers.acfs.type] ID 8196:3682



          2013-07-17 10:51:06.987: [    AGFW][1179973952] Added new restype: ora.drivers.acfs.type



          2013-07-17 10:51:06.988: [    AGFW][1179973952] Agent sending last reply for: RESTYPE_ADD[ora.drivers.acfs.type] ID 8196:3682



          2013-07-17 10:51:06.988: [    AGFW][1179973952] Agent received the message: RESOURCE_ADD[ora.diskmon 1 1] ID 4356:3684



          2013-07-17 10:51:06.988: [    AGFW][1179973952] Added new resource: ora.diskmon 1 1 to the agfw



          2013-07-17 10:51:06.989: [    AGFW][1179973952] Agent sending last reply for: RESOURCE_ADD[ora.diskmon 1 1] ID 4356:3684



          2013-07-17 10:51:06.989: [    AGFW][1179973952] Agent received the message: RESOURCE_CLEAN[ora.diskmon 1 1] ID 4100:3685



          2013-07-17 10:51:06.989: [    AGFW][1179973952] Preparing CLEAN command for: ora.diskmon 1 1



          2013-07-17 10:51:06.989: [    AGFW][1179973952] ora.diskmon 1 1 state changed from: UNKNOWN to: CLEANING



          2013-07-17 10:51:06.999: [CRSTIMER][1097541952] Timer Thread Starting.



          2013-07-17 10:51:06.999: [    AGFW][1158994240] Executing command: clean for resource: ora.diskmon 1 1



          2013-07-17 10:51:06.999: [ora.diskmon][1158994240] [clean] clsn_agent::clean {



          2013-07-17 10:51:06.999: [ora.diskmon][1158994240] [clean] DaemonAgent{



          2013-07-17 10:51:06.999: [ora.diskmon][1158994240] [clean] }DaemonAgent



          2013-07-17 10:51:06.999: [ora.diskmon][1158994240] [clean] DiskmonAgent::DiskmonAgent {



          2013-07-17 10:51:06.999: [ora.diskmon][1158994240] [clean] InitAttrs {



          2013-07-17 10:51:06.999: [ora.diskmon][1158994240] [clean] __IS_HASD_AGENT=TRUE



          2013-07-17 10:51:06.999: [ora.diskmon][1158994240] [clean] }InitAttrs



          2013-07-17 10:51:07.000: [ora.diskmon][1158994240] [clean] DiskmonAgent::DiskmonAgent }



          2013-07-17 10:51:07.000: [ora.diskmon][1158994240] [clean] DiskmonAgent::clean {



          2013-07-17 10:51:07.000: [ora.diskmon][1158994240] [clean] Arg Value = -d



          2013-07-17 10:51:07.000: [ora.diskmon][1158994240] [clean] Arg Value = -z



          2013-07-17 10:51:07.000: [ora.diskmon][1158994240] [clean] Arg Value = 8



          2013-07-17 10:51:07.000: [ora.diskmon][1158994240] [clean] Total Count of Environment Variables = 3



          2013-07-17 10:51:07.000: [ora.diskmon][1158994240] [clean] Adding Environment Variables _ORA_AGENT_ACTION=TRUE



          2013-07-17 10:51:07.000: [ora.diskmon][1158994240] [clean] Adding Environment Variables __IS_HASD_AGENT=



          2013-07-17 10:51:07.000: [ora.diskmon][1158994240] [clean] Adding Environment variable from USR_ORA_ENV CTSS_REBOOT=TRUE



          2013-07-17 10:51:07.000: [ora.diskmon][1158994240] [clean] Utils:execCmd action = 4 flags = 6 ohome = (null) cmdname = diskmon.



          2013-07-17 10:51:07.000: [ora.diskmon][1158994240] [clean] getOracleHomeAttrib: oracle_home = /u01/app/product/11.2.0/oracle



          2013-07-17 10:51:07.000: [ora.diskmon][1158994240] [clean] Utils:execCmd Running the binary from /u01/app/product/11.2.0/oracle/bin/diskmon



          2013-07-17 10:51:07.105: [ora.diskmon][1158994240] [clean] (:CLSN00010:)Utils:execCmd scls_process_join() uret 1



           



          2013-07-17 10:51:07.106: [ora.diskmon][1158994240] [clean] Exception type=2 string=CRS-5016: Process "/u01/app/product/11.2.0/oracle/bin/diskmon" spawned by agent "/u01/app/product/11.2.0/oracle/bin/orarootagent.bin" for action "clean" failed: details at "(:CLSN00010:)" in "/u01/app/product/11.2.0/oracle/log/vmorarac2/agent/ohasd/orarootagent_root/orarootagent_root.log"



           



          2013-07-17 10:51:07.106: [    AGFW][1158994240] sending status msg [CRS-5016: Process "/u01/app/product/11.2.0/oracle/bin/diskmon" spawned by agent "/u01/app/product/11.2.0/oracle/bin/orarootagent.bin" for action "clean" failed: details at "(:CLSN00010:)" in "/u01/app/product/11.2.0/oracle/log/vmorarac2/agent/ohasd/orarootagent_root/orarootagent_root.log"



          ] for clean for resource: ora.diskmon 1 1



          2013-07-17 10:51:07.107: [    AGFW][1179973952] Agent sending reply for: RESOURCE_CLEAN[ora.diskmon 1 1] ID 4100:3685



          2013-07-17 10:51:07.108: [ora.diskmon][1158994240] [clean] execCmd ret = 1



          2013-07-17 10:51:07.108: [ora.diskmon][1158994240] [clean] DiskmonAgent::clean } nopipe



          2013-07-17 10:51:07.108: [ora.diskmon][1158994240] [clean] clsn_agent::clean }



          2013-07-17 10:51:07.108: [    AGFW][1158994240] Command: clean for resource: ora.diskmon 1 1 completed with status: SUCCESS



          2013-07-17 10:51:07.109: [    AGFW][1158994240] Executing command: check for resource: ora.diskmon 1 1



          2013-07-17 10:51:07.109: [    AGFW][1179973952] Agent sending reply for: RESOURCE_CLEAN[ora.diskmon 1 1] ID 4100:3685



          2013-07-17 10:51:07.111: [ora.diskmon][1158994240] [check] DiskmonAgent::check {



          2013-07-17 10:51:07.111: [ora.diskmon][1158994240] [check] DiskmonAgent::connect {



          2013-07-17 10:51:07.111: [ora.diskmon][1158994240] [check] DiskmonAgent::connect: skgznp_connect failed with error 56815 and the timeout expired



          2013-07-17 10:51:07.111: [ora.diskmon][1158994240] [check] (null) category: 56815, operation: connect, loc: skgznpcon6, OS error: 2, other:



          2013-07-17 10:51:07.111: [ora.diskmon][1158994240] [check] DiskmonAgent::connect } error



          2013-07-17 10:51:07.111: [ora.diskmon][1158994240] [check] DiskmonAgent::check } 2



          2013-07-17 10:51:07.111: [    AGFW][1158994240] check for resource: ora.diskmon 1 1 completed with status: PLANNED_OFFLINE



          2013-07-17 10:51:07.112: [    AGFW][1179973952] ora.diskmon 1 1 state changed from: CLEANING to: PLANNED_OFFLINE



          2013-07-17 10:51:07.112: [    AGFW][1179973952] Agent sending last reply for: RESOURCE_CLEAN[ora.diskmon 1 1] ID 4100:3685



          2013-07-17 10:51:07.112: [    AGFW][1179973952] Agent has no resources to be monitored.Sending suicide request.



          2013-07-17 10:51:07.112: [    AGFW][1179973952] Agent sending message to PE: AGENT_SUICIDE[Proxy] ID 20486:40



          2013-07-17 10:51:07.116: [    AGFW][1179973952] Agent is commiting suicide.



          2013-07-17 10:51:07.116: [    AGFW][1179973952] Agent is exiting with exit code: 1