2 Replies Latest reply: May 31, 2013 8:51 AM by user8792751 RSS

    root.sh failed when i installing grid infrastructure

    user8792751
      please find the below messages when i run root.sh in node1.
      RS-4256: Updating the profile
      CRS-4266: Voting file(s) successfully replaced
      ##  STATE    File Universal Id                File Name Disk group
      --  -----    -----------------                --------- ---------
       1. ONLINE   06512028491d4f94bf6e54e85805c172 (ORCL:DISK1) [DG1]
      Located 1 voting disk(s).
      CRS-2673: Attempting to stop 'ora.crsd' on 'rac1'
      CRS-2677: Stop of 'ora.crsd' on 'rac1' succeeded
      CRS-2673: Attempting to stop 'ora.asm' on 'rac1'
      CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded
      CRS-2673: Attempting to stop 'ora.ctssd' on 'rac1'
      CRS-2677: Stop of 'ora.ctssd' on 'rac1' succeeded
      CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'rac1'
      CRS-2677: Stop of 'ora.cssdmonitor' on 'rac1' succeeded
      CRS-2673: Attempting to stop 'ora.cssd' on 'rac1'
      CRS-2677: Stop of 'ora.cssd' on 'rac1' succeeded
      CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac1'
      CRS-2677: Stop of 'ora.gpnpd' on 'rac1' succeeded
      CRS-2673: Attempting to stop 'ora.gipcd' on 'rac1'
      CRS-2677: Stop of 'ora.gipcd' on 'rac1' succeeded
      CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac1'
      CRS-2677: Stop of 'ora.mdnsd' on 'rac1' succeeded
      CRS-2672: Attempting to start 'ora.mdnsd' on 'rac1'
      CRS-2676: Start of 'ora.mdnsd' on 'rac1' succeeded
      CRS-2672: Attempting to start 'ora.gipcd' on 'rac1'
      CRS-2676: Start of 'ora.gipcd' on 'rac1' succeeded
      CRS-2672: Attempting to start 'ora.gpnpd' on 'rac1'
      CRS-2676: Start of 'ora.gpnpd' on 'rac1' succeeded
      CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac1'
      CRS-2676: Start of 'ora.cssdmonitor' on 'rac1' succeeded
      CRS-2672: Attempting to start 'ora.cssd' on 'rac1'
      CRS-2672: Attempting to start 'ora.diskmon' on 'rac1'
      CRS-2676: Start of 'ora.diskmon' on 'rac1' succeeded
      CRS-2676: Start of 'ora.cssd' on 'rac1' succeeded
      CRS-2672: Attempting to start 'ora.ctssd' on 'rac1'
      CRS-2676: Start of 'ora.ctssd' on 'rac1' succeeded
      CRS-2672: Attempting to start 'ora.asm' on 'rac1'
      CRS-2676: Start of 'ora.asm' on 'rac1' succeeded
      CRS-2672: Attempting to start 'ora.crsd' on 'rac1'
      CRS-2676: Start of 'ora.crsd' on 'rac1' succeeded
      CRS-2672: Attempting to start 'ora.evmd' on 'rac1'
      CRS-2676: Start of 'ora.evmd' on 'rac1' succeeded
      
      ACRS-2672: Attempting to start 'ora.asm' on 'rac1'
      CRS-2676: Start of 'ora.asm' on 'rac1' succeeded
      CRS-2672: Attempting to start 'ora.DG1.dg' on 'rac1'
      CRS-4535: Cannot communicate with Cluster Ready Services
      CRS-4000: Command Create failed, or completed with errors.
      create diskgroup DG1 ... failed
      CRS-4535: Cannot communicate with Cluster Ready Services
      CRS-4000: Command Add failed, or completed with errors.
      Command return code of 1 (256) from command: /u01/app/11.2.0/grid/bin/crsctl add type ora.registry.acfs.type -basetype ora.local_resource.type -file /u01/app/11.2.0/grid/crs/template/registry.acfs.type
      add ora.registry.acfs.type ... failed
      PRCR-1070 : Failed to check if resource ora.net1.network is registered
      Cannot communicate with crsd
      add scan=rac-scan ... failed
      Preparing packages for installation...
      cvuqdisk-1.0.7-1
      Configure Oracle Grid Infrastructure for a Cluster ... failed
      Updating inventory properties for clusterware
      Starting Oracle Universal Installer...
      
      Checking swap space: must be greater than 500 MB.   Actual 1011 MB    Passed
      The inventory pointer is located at /etc/oraInst.loc
      The inventory is located at /u01/app/oraInventory
      'UpdateNodeList' was successful.
      [root@rac1 ~]# 
      Edited by: user8792751 on May 31, 2013 3:07 AM
        • 1. Re: root.sh failed when i installing grid infrastructure
          user8792751
          please find the output of alertrac1.log
          Oracle Database 11g Clusterware Release 11.2.0.1.0 - Production Copyright 1996, 2009 Oracle. All rights reserved.
          2013-05-30 16:14:57.184
          [client(9589)]CRS-2106:The OLR location /u01/app/11.2.0/grid/cdata/rac1.olr is inaccessible. Details in /u01/app/11.2.0/grid/log/rac1/client/ocrconfig_9589.log.
          2013-05-30 16:14:57.333
          [client(9589)]CRS-2101:The OLR was formatted using version 3.
          2013-05-30 16:15:44.746
          [ohasd(9903)]CRS-2112:The OLR service started on node rac1.
          2013-05-30 16:15:45.276
          [ohasd(9903)]CRS-2772:Server 'rac1' has been assigned to pool 'Free'.
          2013-05-30 16:16:57.454
          [ohasd(9903)]CRS-2302:Cannot get GPnP profile. Error CLSGPNP_NO_DAEMON (GPNPD daemon is not running). 
          2013-05-30 16:17:06.586
          [cssd(11105)]CRS-1713:CSSD daemon is started in exclusive mode
          2013-05-30 16:17:08.177
          [cssd(11105)]CRS-1709:Lease acquisition failed for node rac1 because no voting file has been configured; Details at (:CSSNM00031:) in /u01/app/11.2.0/grid/log/rac1/cssd/ocssd.log
          2013-05-30 16:17:26.200
          [cssd(11105)]CRS-1601:CSSD Reconfiguration complete. Active nodes are rac1 .
          2013-05-30 16:18:27.097
          [ctssd(11162)]CRS-2407:The new Cluster Time Synchronization Service reference node is host rac1.
          2013-05-30 16:18:27.650
          [ctssd(11162)]CRS-2401:The Cluster Time Synchronization Service started on host rac1.
          [client(11364)]CRS-10001:ACFS-9327: Verifying ADVM/ACFS devices.
          [client(11369)]CRS-10001:ACFS-9322: done.
          2013-05-30 16:22:33.918
          [client(11413)]CRS-1006:The OCR location +DG1 is inaccessible. Details in /u01/app/11.2.0/grid/log/rac1/client/ocrconfig_11413.log.
          2013-05-30 16:22:37.393
          [client(11413)]CRS-1001:The OCR was formatted using version 3.
          2013-05-30 16:23:05.852
          [crsd(11498)]CRS-1012:The OCR service started on node rac1.
          2013-05-30 16:23:11.879
          [cssd(11105)]CRS-1605:CSSD voting file is online: ORCL:DISK1; details in /u01/app/11.2.0/grid/log/rac1/cssd/ocssd.log.
          2013-05-30 16:23:13.099
          [cssd(11105)]CRS-1626:A Configuration change request completed successfully
          2013-05-30 16:23:13.226
          [cssd(11105)]CRS-1601:CSSD Reconfiguration complete. Active nodes are rac1 .
          2013-05-30 16:24:23.989
          [ctssd(11162)]CRS-2405:The Cluster Time Synchronization Service on host rac1 is shutdown by user
          2013-05-30 16:24:40.074
          [cssd(11105)]CRS-1603:CSSD on node rac1 shutdown by user.
          2013-05-30 16:25:08.005
          [cssd(12085)]CRS-1713:CSSD daemon is started in clustered mode
          2013-05-30 16:25:43.091
          [cssd(12085)]CRS-1707:Lease acquisition for node rac1 number 1 completed
          2013-05-30 16:25:43.152
          [cssd(12085)]CRS-1605:CSSD voting file is online: ORCL:DISK1; details in /u01/app/11.2.0/grid/log/rac1/cssd/ocssd.log.
          2013-05-30 16:26:01.207
          [cssd(12085)]CRS-1601:CSSD Reconfiguration complete. Active nodes are rac1 .
          2013-05-30 16:26:59.801
          [ctssd(12166)]CRS-2407:The new Cluster Time Synchronization Service reference node is host rac1.
          2013-05-30 16:26:59.921
          [ctssd(12166)]CRS-2401:The Cluster Time Synchronization Service started on host rac1.
          2013-05-30 16:29:55.266
          [crsd(12305)]CRS-1012:The OCR service started on node rac1.
          2013-05-30 16:30:32.476
          [crsd(12305)]CRS-1201:CRSD started on node rac1.
          2013-05-30 16:30:55.311
          [crsd(12305)]CRS-2772:Server 'rac1' has been assigned to pool 'Free'.
          2013-05-30 16:38:02.547
          [/u01/app/11.2.0/grid/bin/orarootagent.bin(11088)]CRS-5818:Aborted command 'check for resource: ora.crsd 1 1' for resource 'ora.crsd'. Details at (:CRSAGF00113:) in /u01/app/11.2.0/grid/log/rac1/agent/ohasd/orarootagent_root/orarootagent_root.log.
          2013-05-30 16:38:31.302
          [ohasd(9903)]CRS-2765:Resource 'ora.crsd' has failed on server 'rac1'.
          2013-05-30 16:39:21.713
          [crsd(12753)]CRS-1012:The OCR service started on node rac1.
          2013-05-30 16:39:26.844
          [crsd(12753)]CRS-1201:CRSD started on node rac1.
          2013-05-30 16:40:43.215
          [/u01/app/11.2.0/grid/bin/orarootagent.bin(12845)]CRS-5822:Agent '/u01/app/11.2.0/grid/bin/orarootagent_root' disconnected from server. Details at (:CRSAGF00117:) in /u01/app/11.2.0/grid/log/rac1/agent/crsd/orarootagent_root/orarootagent_root.log.
          2013-05-30 16:40:46.557
          [ohasd(9903)]CRS-2765:Resource 'ora.crsd' has failed on server 'rac1'.
          2013-05-30 16:41:07.865
          [crsd(12892)]CRS-1012:The OCR service started on node rac1.
          2013-05-30 16:41:12.533
          [crsd(12892)]CRS-1201:CRSD started on node rac1.
          2013-05-30 16:42:06.567
          [ohasd(9903)]CRS-2765:Resource 'ora.crsd' has failed on server 'rac1'.
          2013-05-30 16:42:09.637
          [/u01/app/11.2.0/grid/bin/orarootagent.bin(12978)]CRS-5822:Agent '/u01/app/11.2.0/grid/bin/orarootagent_root' disconnected from server. Details at (:CRSAGF00117:) in /u01/app/11.2.0/grid/log/rac1/agent/crsd/orarootagent_root/orarootagent_root.log.
          2013-05-30 16:42:26.606
          [crsd(13024)]CRS-1012:The OCR service started on node rac1.
          2013-05-30 16:42:32.439
          [crsd(13024)]CRS-1201:CRSD started on node rac1.
          2013-05-30 16:43:47.863
          [/u01/app/11.2.0/grid/bin/orarootagent.bin(13112)]CRS-5822:Agent '/u01/app/11.2.0/grid/bin/orarootagent_root' disconnected from server. Details at (:CRSAGF00117:) in /u01/app/11.2.0/grid/log/rac1/agent/crsd/orarootagent_root/orarootagent_root.log.
          2013-05-30 16:43:48.254
          [ohasd(9903)]CRS-2765:Resource 'ora.crsd' has failed on server 'rac1'.
          2013-05-30 16:44:01.739
          [crsd(13154)]CRS-1012:The OCR service started on node rac1.
          2013-05-30 16:44:07.780
          [crsd(13154)]CRS-1201:CRSD started on node rac1.
          2013-05-30 16:45:36.961
          [/u01/app/11.2.0/grid/bin/orarootagent.bin(13239)]CRS-5822:Agent '/u01/app/11.2.0/grid/bin/orarootagent_root' disconnected from server. Details at (:CRSAGF00117:) in /u01/app/11.2.0/grid/log/rac1/agent/crsd/orarootagent_root/orarootagent_root.log.
          2013-05-30 16:46:32.732
          [/u01/app/11.2.0/grid/bin/orarootagent.bin(11088)]CRS-5818:Aborted command 'check for resource: ora.crsd 1 1' for resource 'ora.crsd'. Details at (:CRSAGF00113:) in /u01/app/11.2.0/grid/log/rac1/agent/ohasd/orarootagent_root/orarootagent_root.log.
          2013-05-30 16:46:55.506
          [ohasd(9903)]CRS-2765:Resource 'ora.crsd' has failed on server 'rac1'.
          2013-05-30 16:47:06.433
          [crsd(13294)]CRS-1012:The OCR service started on node rac1.
          2013-05-30 16:47:11.813
          [crsd(13294)]CRS-1201:CRSD started on node rac1.
          2013-05-30 16:48:23.770
          [/u01/app/11.2.0/grid/bin/orarootagent.bin(13376)]CRS-5822:Agent '/u01/app/11.2.0/grid/bin/orarootagent_root' disconnected from server. Details at (:CRSAGF00117:) in /u01/app/11.2.0/grid/log/rac1/agent/crsd/orarootagent_root/orarootagent_root.log.
          2013-05-30 16:48:24.560
          [ohasd(9903)]CRS-2765:Resource 'ora.crsd' has failed on server 'rac1'.
          2013-05-30 16:48:38.484
          [crsd(13423)]CRS-1012:The OCR service started on node rac1.
          2013-05-30 16:48:42.307
          [crsd(13423)]CRS-1201:CRSD started on node rac1.
          2013-05-30 16:49:57.938
          [/u01/app/11.2.0/grid/bin/orarootagent.bin(13508)]CRS-5822:Agent '/u01/app/11.2.0/grid/bin/orarootagent_root' disconnected from server. Details at (:CRSAGF00117:) in /u01/app/11.2.0/grid/log/rac1/agent/crsd/orarootagent_root/orarootagent_root.log.
          2013-05-30 16:50:54.589
          [/u01/app/11.2.0/grid/bin/orarootagent.bin(11088)]CRS-5818:Aborted command 'check for resource: ora.crsd 1 1' for resource 'ora.crsd'. Details at (:CRSAGF00113:) in /u01/app/11.2.0/grid/log/rac1/agent/ohasd/orarootagent_root/orarootagent_root.log.
          2013-05-30 16:51:00.484
          [ohasd(9903)]CRS-2765:Resource 'ora.crsd' has failed on server 'rac1'.
          2013-05-30 16:51:18.659
          [crsd(13561)]CRS-1012:The OCR service started on node rac1.
          2013-05-30 16:51:24.623
          [crsd(13561)]CRS-1201:CRSD started on node rac1.
          2013-05-30 16:52:37.107
          [ohasd(9903)]CRS-2765:Resource 'ora.crsd' has failed on server 'rac1'.
          2013-05-30 16:52:37.849
          [/u01/app/11.2.0/grid/bin/orarootagent.bin(13645)]CRS-5822:Agent '/u01/app/11.2.0/grid/bin/orarootagent_root' disconnected from server. Details at (:CRSAGF00117:) in /u01/app/11.2.0/grid/log/rac1/agent/crsd/orarootagent_root/orarootagent_root.log.
          2013-05-30 16:53:37.234
          [/u01/app/11.2.0/grid/bin/orarootagent.bin(11088)]CRS-5818:Aborted command 'check for resource: ora.crsd 1 1' for resource 'ora.crsd'. Details at (:CRSAGF00113:) in /u01/app/11.2.0/grid/log/rac1/agent/ohasd/orarootagent_root/orarootagent_root.log.
          please let me know if you have any inputs.
          • 2. Re: root.sh failed when i installing grid infrastructure
            user8792751
            Hi all,

            Please let me know what was the issue in my case....

            Iam posting the orarootagentroot.log

            Oracle Database 11g Clusterware Release 11.2.0.1.0 - Production Copyright 1996, 2009 Oracle. All rights reserved.
            2013-05-30 16:17:05.651: [    AGFW][3045746384] Starting the agent: /u01/app/11.2.0/grid/log/rac1/agent/ohasd/orarootagent_root/
            2013-05-30 16:17:05.652: [   AGENT][3045746384] Agent framework initialized, Process Id = 11088
            2013-05-30 16:17:05.730: [ USRTHRD][3045746384] Utils::getCrsHome crsHome /u01/app/11.2.0/grid
            2013-05-30 16:17:05.731: [ USRTHRD][3045746384] Process::convertPidToString pid = 11088
            2013-05-30 16:17:05.732: [    AGFW][3045746384] SERVER IPC CONNECT STR: (ADDRESS=(PROTOCOL=IPC)(KEY=OHASD_IPC_SOCKET_11))
            2013-05-30 16:17:05.732: [CLSFRAME][3045746384] Inited lsf context 0xa22ad78
            2013-05-30 16:17:05.748: [CLSFRAME][3045746384] Initing CLS Framework messaging
            2013-05-30 16:17:05.749: [CLSFRAME][3045746384] New Framework state: 2
            2013-05-30 16:17:05.749: [CLSFRAME][3045746384] M2M is starting...
            2013-05-30 16:17:05.756: [ CRSCOMM][3045746384] m_pClscCtx=0xa254b48m_pUgblm=0xa257420
            2013-05-30 16:17:05.756: [ CRSCOMM][3045746384] Starting send thread
            2013-05-30 16:17:05.756: [ CRSCOMM][3034651536] clsIpc: sendWork thread started.
            2013-05-30 16:17:05.763: [ CRSCOMM][3024161680] IPC Client thread started listening
            2013-05-30 16:17:05.764: [ CRSCOMM][3024161680] init data sent from server
            2013-05-30 16:17:05.764: [CLSFRAME][3045746384] New IPC Member:{Relative|Node:0|Process:0|Type:2}:OHASD:rac1
            2013-05-30 16:17:05.765: [CLSFRAME][3045746384] New process connected to us ID:{Relative|Node:0|Process:0|Type:2} Info:OHASD:rac1
            2013-05-30 16:17:05.765: [CLSFRAME][3045746384] Starting thread model named: MultiThread
            2013-05-30 16:17:05.766: [CLSFRAME][3045746384] Starting thread model named: SingleThread
            2013-05-30 16:17:05.766: [CLSFRAME][3045746384] Starting thread model named: SingleThreadT
            2013-05-30 16:17:05.766: [CLSFRAME][3045746384] New Framework state: 3
            2013-05-30 16:17:05.766: [    AGFW][3045746384] Agent Framework started successfully
            2013-05-30 16:17:05.770: [    AGFW][2971712400] Agfw engine module has enabled...
            2013-05-30 16:17:05.771: [CLSFRAME][2971712400] Module Enabling is complete
            2013-05-30 16:17:05.771: [CLSFRAME][2971712400] New Framework state: 6
            2013-05-30 16:17:05.772: [    AGFW][2971712400] Agent is started with userid: root , expected user: root
            2013-05-30 16:17:05.777: [    AGFW][2971712400] Agent sending message to PE: AGENT_HANDSHAKE[Proxy] ID 20484:14
            2013-05-30 16:17:05.820: [    AGFW][2971712400] Agent received the message: RESTYPE_ADD[ora.crs.type] ID 8196:381
            2013-05-30 16:17:05.822: [    AGFW][2971712400] Added new restype: ora.crs.type
            2013-05-30 16:17:05.822: [    AGFW][2971712400] Agent sending last reply for: RESTYPE_ADD[ora.crs.type] ID 8196:381
            2013-05-30 16:17:05.838: [    AGFW][2971712400] Agent received the message: RESTYPE_ADD[ora.ctss.type] ID 8196:383
            2013-05-30 16:17:05.838: [    AGFW][2971712400] Added new restype: ora.ctss.type
            2013-05-30 16:17:05.839: [    AGFW][2971712400] Agent sending last reply for: RESTYPE_ADD[ora.ctss.type] ID 8196:383
            2013-05-30 16:17:05.849: [    AGFW][2971712400] Agent received the message: RESTYPE_ADD[ora.diskmon.type] ID 8196:385
            2013-05-30 16:17:05.849: [    AGFW][2971712400] Added new restype: ora.diskmon.type
            2013-05-30 16:17:05.850: [    AGFW][2971712400] Agent sending last reply for: RESTYPE_ADD[ora.diskmon.type] ID 8196:385
            2013-05-30 16:17:05.861: [    AGFW][2971712400] Agent received the message: RESTYPE_ADD[ora.drivers.acfs.type] ID 8196:387
            2013-05-30 16:17:05.862: [    AGFW][2971712400] Added new restype: ora.drivers.acfs.type
            2013-05-30 16:17:05.862: [    AGFW][2971712400] Agent sending last reply for: RESTYPE_ADD[ora.drivers.acfs.type] ID 8196:387
            2013-05-30 16:17:05.863: [    AGFW][2971712400] Agent received the message: RESOURCE_ADD[ora.diskmon 1 1] ID 4356:389
            2013-05-30 16:17:05.863: [    AGFW][2971712400] Added new resource: ora.diskmon 1 1 to the agfw
            2013-05-30 16:17:05.865: [    AGFW][2971712400] Agent sending last reply for: RESOURCE_ADD[ora.diskmon 1 1] ID 4356:389
            2013-05-30 16:17:05.872: [    AGFW][2971712400] Agent received the message: RESOURCE_START[ora.diskmon 1 1] ID 4098:390
            2013-05-30 16:17:05.872: [    AGFW][2971712400] Preparing START command for: ora.diskmon 1 1
            2013-05-30 16:17:05.872: [    AGFW][2971712400] ora.diskmon 1 1 state changed from: UNKNOWN to: STARTING
            2013-05-30 16:17:05.877: [    AGFW][2982202256] Executing command: start for resource: ora.diskmon 1 1
            2013-05-30 16:17:05.878: [ora.diskmon][2982202256] [start] clsn_agent::start {
            2013-05-30 16:17:05.878: [ora.diskmon][2982202256] [start] DaemonAgent{
            2013-05-30 16:17:05.878: [ora.diskmon][2982202256] [start] }DaemonAgent
            2013-05-30 16:17:05.878: [ora.diskmon][2982202256] [start] DiskmonAgent::DiskmonAgent {
            2013-05-30 16:17:05.880: [CRSTIMER][2950732688] Timer Thread Starting.
            2013-05-30 16:17:05.901: [ora.diskmon][2982202256] [start] InitAttrs {
            2013-05-30 16:17:05.902: [ora.diskmon][2982202256] [start] __IS_HASD_AGENT=TRUE
            2013-05-30 16:17:05.902: [ora.diskmon][2982202256] [start] }InitAttrs
            2013-05-30 16:17:05.902: [ora.diskmon][2982202256] [start] DiskmonAgent::DiskmonAgent }
            2013-05-30 16:17:05.902: [ora.diskmon][2982202256] [start] DiskmonAgent::start {
            2013-05-30 16:17:05.903: [ora.diskmon][2982202256] [start] Arg Value = -d
            2013-05-30 16:17:05.903: [ora.diskmon][2982202256] [start] Arg Value = -f
            2013-05-30 16:17:05.903: [ora.diskmon][2982202256] [start] Total Count of Environment Variables = 3
            2013-05-30 16:17:05.903: [ora.diskmon][2982202256] [start] Adding Environment Variables _ORA_AGENT_ACTION=TRUE
            2013-05-30 16:17:05.903: [ora.diskmon][2982202256] [start] Adding Environment Variables __IS_HASD_AGENT=
            2013-05-30 16:17:05.903: [ora.diskmon][2982202256] [start] Adding Environment variable from USR_ORA_ENV ORACLE_USER=oracle
            2013-05-30 16:17:05.903: [ora.diskmon][2982202256] [start] Utils:execCmd action = 1 flags = 5 ohome = (null) cmdname = diskmon. 
            2013-05-30 16:17:05.903: [ora.diskmon][2982202256] [start] getOracleHomeAttrib: oracle_home = /u01/app/11.2.0/grid
            2013-05-30 16:17:05.906: [ora.diskmon][2982202256] [start] Utils:execCmd Running the binary from /u01/app/11.2.0/grid/bin/diskmon
            2013-05-30 16:17:05.945: [ora.diskmon][2982202256] [start] execCmd ret = 0
            2013-05-30 16:17:05.950: [ora.diskmon][2982202256] [start] }DaemonAgent::start
            2013-05-30 16:17:11.954: [ora.diskmon][2982202256] [start] DiskmonAgent::connect {
            2013-05-30 16:17:11.959: [ora.diskmon][2982202256] [start] Process::convertPidToString pid = 11088
            2013-05-30 16:17:11.961: [ora.diskmon][2982202256] [start] DiskmonAgent::connect }
            2013-05-30 16:17:11.961: [ora.diskmon][2982202256] [start] DiskmonAgent::start }
            2013-05-30 16:17:11.962: [ora.diskmon][2982202256] [start] clsn_agent::start }
            2013-05-30 16:17:11.962: [    AGFW][2982202256] Command: start for resource: ora.diskmon 1 1 completed with status: SUCCESS
            2013-05-30 16:17:11.963: [    AGFW][2982202256] Executing command: check for resource: ora.diskmon 1 1
            2013-05-30 16:17:11.964: [ora.diskmon][2982202256] [check] DiskmonAgent::check {
            2013-05-30 16:17:11.965: [ora.diskmon][2982202256] [check] DiskmonAgent::check } 0
            2013-05-30 16:17:11.965: [    AGFW][2982202256] check for resource: ora.diskmon 1 1 completed with status: ONLINE
            2013-05-30 16:17:11.967: [CLSFRAME][3013671824] TM [MultiThread] is changing desired thread # to 3. Current # is 2
            2013-05-30 16:17:11.967: [CLSFRAME][3013671824] TM [MultiThread] is changing desired thread # to 4. Current # is 3
            2013-05-30 16:17:11.969: [    AGFW][2971712400] Agent sending reply for: RESOURCE_START[ora.diskmon 1 1] ID 4098:390
            2013-05-30 16:17:11.969: [    AGFW][2971712400] ora.diskmon 1 1 state changed from: STARTING to: ONLINE
            2013-05-30 16:17:11.970: [    AGFW][2971712400] Started implicit monitor for:ora.diskmon 1 1
            2013-05-30 16:17:11.970: [    AGFW][2971712400] Agent sending last reply for: RESOURCE_START[ora.diskmon 1 1] ID 4098:390
            2013-05-30 16:17:30.054: [    AGFW][2971712400] Agent received the message: RESOURCE_ADD[ora.ctssd 1 1] ID 4356:432
            2013-05-30 16:17:30.058: [    AGFW][2971712400] Added new resource: ora.ctssd 1 1 to the agfw
            2013-05-30 16:17:30.059: [    AGFW][2971712400] Agent sending last reply for: RESOURCE_ADD[ora.ctssd 1 1] ID 4356:432
            2013-05-30 16:17:30.070: [    AGFW][2971712400] Agent received the message: RESOURCE_START[ora.ctssd 1 1] ID 4098:435
            2013-05-30 16:17:30.070: [    AGFW][2971712400] Preparing START command for: ora.ctssd 1 1
            2013-05-30 16:17:30.071: [    AGFW][2971712400] ora.ctssd 1 1 state changed from: UNKNOWN to: STARTING
            2013-05-30 16:17:30.073: [    AGFW][2992692112] Executing command: start for resource: ora.ctssd 1 1
            2013-05-30 16:17:30.074: [ora.ctssd][2992692112] [start] clsn_agent::start {
            2013-05-30 16:17:30.074: [ora.ctssd][2992692112] [start] DaemonAgent{
            2013-05-30 16:17:30.074: [ora.ctssd][2992692112] [start] }DaemonAgent
            2013-05-30 16:17:30.086: [ora.ctssd][2992692112] [start] InitAttrs {
            2013-05-30 16:17:30.086: [ora.ctssd][2992692112] [start] __IS_HASD_AGENT=TRUE
            2013-05-30 16:17:30.086: [ora.ctssd][2992692112] [start] }InitAttrs
            2013-05-30 16:17:30.087: [ora.ctssd][2992692112] [start] Total Count of Environment Variables = 3
            2013-05-30 16:17:30.087: [ora.ctssd][2992692112] [start] Adding Environment Variables _ORA_AGENT_ACTION=TRUE
            2013-05-30 16:17:30.087: [ora.ctssd][2992692112] [start] Adding Environment Variables __IS_HASD_AGENT=
            2013-05-30 16:17:30.087: [ora.ctssd][2992692112] [start] Skipping empty Env variable from USR_ORA_ENV 
            2013-05-30 16:17:30.087: [ora.ctssd][2992692112] [start] Utils:execCmd action = 1 flags = 5 ohome = (null) cmdname = octssd. 
            2013-05-30 16:17:30.087: [ora.ctssd][2992692112] [start] getOracleHomeAttrib: oracle_home = /u01/app/11.2.0/grid
            2013-05-30 16:17:30.088: [ora.ctssd][2992692112] [start] Utils:execCmd Running the binary from /u01/app/11.2.0/grid/bin/octssd
            2013-05-30 16:17:30.172: [ora.ctssd][2992692112] [start] execCmd ret = 0
            2013-05-30 16:17:31.202: [ COMMCRS][2919263120]clsc_connect: (0xa2f0b00) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:17:31.204: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:17:31.985: [    AGFW][2971712400] CHECK initiated by timer for: ora.diskmon 1 1
            2013-05-30 16:17:31.988: [    AGFW][2982202256] Executing command: check for resource: ora.diskmon 1 1
            2013-05-30 16:17:31.988: [ora.diskmon][2982202256] [check] DiskmonAgent::check {
            2013-05-30 16:17:31.990: [ora.diskmon][2982202256] [check] DiskmonAgent::check } 0
            2013-05-30 16:17:31.990: [    AGFW][2982202256] check for resource: ora.diskmon 1 1 completed with status: ONLINE
            2013-05-30 16:17:32.262: [ COMMCRS][2919263120]clsc_connect: (0xa2f0af8) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:17:32.264: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:17:33.268: [ COMMCRS][2919263120]clsc_connect: (0xa2f0b08) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:17:33.269: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:17:34.273: [ COMMCRS][2919263120]clsc_connect: (0xa2f0b08) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:17:34.275: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:17:35.278: [ COMMCRS][2919263120]clsc_connect: (0xa2f0b08) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:17:35.280: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:17:36.284: [ COMMCRS][2919263120]clsc_connect: (0xa2f0b08) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:17:36.286: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:17:37.296: [ COMMCRS][2919263120]clsc_connect: (0xa2f0b08) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:17:37.297: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:17:38.305: [ COMMCRS][2919263120]clsc_connect: (0xa2f0b08) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:17:38.306: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:17:39.311: [ COMMCRS][2919263120]clsc_connect: (0xa2f0b08) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:17:39.313: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:17:40.318: [ COMMCRS][2919263120]clsc_connect: (0xa2f0b08) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:17:40.321: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:17:41.325: [ COMMCRS][2919263120]clsc_connect: (0xa2f0b08) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:17:41.327: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:17:42.332: [ COMMCRS][2919263120]clsc_connect: (0xa2f0b08) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:17:42.334: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:17:43.339: [ COMMCRS][2919263120]clsc_connect: (0xa2f0b08) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:17:43.341: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:17:44.346: [ COMMCRS][2919263120]clsc_connect: (0xa2f0b08) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:17:44.348: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:17:45.353: [ COMMCRS][2919263120]clsc_connect: (0xa2f0b08) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:17:45.354: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:17:46.358: [ COMMCRS][2919263120]clsc_connect: (0xa2f0b08) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:17:46.360: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:17:47.367: [ COMMCRS][2919263120]clsc_connect: (0xa2f0b08) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:17:47.368: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:17:48.373: [ COMMCRS][2919263120]clsc_connect: (0xa2f0b08) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:17:48.375: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:17:49.381: [ COMMCRS][2919263120]clsc_connect: (0xa2f0b08) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:17:49.382: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:17:50.388: [ COMMCRS][2919263120]clsc_connect: (0xa2f0b08) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:17:50.389: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:17:51.393: [ COMMCRS][2919263120]clsc_connect: (0xa2f0b08) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:17:51.395: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:17:51.999: [    AGFW][2971712400] CHECK initiated by timer for: ora.diskmon 1 1
            2013-05-30 16:17:52.001: [    AGFW][2982202256] Executing command: check for resource: ora.diskmon 1 1
            2013-05-30 16:17:52.001: [ora.diskmon][2982202256] [check] DiskmonAgent::check {
            2013-05-30 16:17:52.003: [ora.diskmon][2982202256] [check] DiskmonAgent::check } 0
            2013-05-30 16:17:52.003: [    AGFW][2982202256] check for resource: ora.diskmon 1 1 completed with status: ONLINE
            2013-05-30 16:17:52.399: [ COMMCRS][2919263120]clsc_connect: (0xa2f0af8) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:17:52.401: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:17:53.406: [ COMMCRS][2919263120]clsc_connect: (0xa2f0b08) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:17:53.408: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:17:54.412: [ COMMCRS][2919263120]clsc_connect: (0xa2f0b08) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:17:54.414: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:17:55.419: [ COMMCRS][2919263120]clsc_connect: (0xa2f0b08) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:17:55.421: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:17:56.425: [ COMMCRS][2919263120]clsc_connect: (0xa2f0b08) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:17:56.428: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:17:57.435: [ COMMCRS][2919263120]clsc_connect: (0xa2f0b08) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:17:57.437: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:17:58.441: [ COMMCRS][2919263120]clsc_connect: (0xa2f0b08) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:17:58.442: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:17:59.447: [ COMMCRS][2919263120]clsc_connect: (0xa2f0b08) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:17:59.449: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:18:00.088: [CLSFRAME][3013671824] TM [MultiThread] is changing desired thread # to 3. Current # is 4
            2013-05-30 16:18:00.090: [CLSFRAME][2940242832] Worker thread is exiting in TM [MultiThread] to meet the desired count of 3. New count is 3
            2013-05-30 16:18:00.091: [    AGFW][2982202256] Timer request to reduce thread count received, current cnt=3
            2013-05-30 16:18:00.453: [ COMMCRS][2919263120]clsc_connect: (0xa2f0878) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:18:00.455: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:18:01.458: [ COMMCRS][2919263120]clsc_connect: (0xa2f0878) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:18:01.460: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:18:02.467: [ COMMCRS][2919263120]clsc_connect: (0xa2f0878) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:18:02.469: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:18:02.644: [    AGFW][2971712400] Agent received the message: AGENT_HB[Engine] ID 12293:453
            2013-05-30 16:18:03.474: [ COMMCRS][2919263120]clsc_connect: (0xa2f0878) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:18:03.476: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:18:04.482: [ COMMCRS][2919263120]clsc_connect: (0xa2f0878) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:18:04.483: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:18:05.487: [ COMMCRS][2919263120]clsc_connect: (0xa2f0878) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:18:05.488: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:18:06.492: [ COMMCRS][2919263120]clsc_connect: (0xa2f0878) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:18:06.494: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:18:07.499: [ COMMCRS][2919263120]clsc_connect: (0xa2f0878) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:18:07.500: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:18:08.503: [ COMMCRS][2919263120]clsc_connect: (0xa2f0878) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:18:08.504: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:18:09.509: [ COMMCRS][2919263120]clsc_connect: (0xa2f0878) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:18:09.511: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:18:10.516: [ COMMCRS][2919263120]clsc_connect: (0xa2f0878) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:18:10.517: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:18:11.522: [ COMMCRS][2919263120]clsc_connect: (0xa2f0878) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:18:11.523: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:18:12.009: [    AGFW][2971712400] CHECK initiated by timer for: ora.diskmon 1 1
            2013-05-30 16:18:12.012: [    AGFW][2929752976] Executing command: check for resource: ora.diskmon 1 1
            2013-05-30 16:18:12.013: [ora.diskmon][2929752976] [check] DiskmonAgent::check {
            2013-05-30 16:18:12.014: [ora.diskmon][2929752976] [check] DiskmonAgent::check } 0
            2013-05-30 16:18:12.014: [    AGFW][2929752976] check for resource: ora.diskmon 1 1 completed with status: ONLINE
            2013-05-30 16:18:12.530: [ COMMCRS][2919263120]clsc_connect: (0xa2f0878) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:18:12.531: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:18:13.536: [ COMMCRS][2919263120]clsc_connect: (0xa2f0878) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:18:13.538: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:18:14.544: [ COMMCRS][2919263120]clsc_connect: (0xa2f0878) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:18:14.545: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:18:15.551: [ COMMCRS][2919263120]clsc_connect: (0xa2f0878) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:18:15.553: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:18:16.557: [ COMMCRS][2919263120]clsc_connect: (0xa2f0878) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:18:16.559: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:18:17.565: [ COMMCRS][2919263120]clsc_connect: (0xa2f0878) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:18:17.566: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:18:18.571: [ COMMCRS][2919263120]clsc_connect: (0xa2f0878) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:18:18.573: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:18:19.579: [ COMMCRS][2919263120]clsc_connect: (0xa2f0878) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:18:19.581: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:18:20.585: [ COMMCRS][2919263120]clsc_connect: (0xa2f0878) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:18:20.588: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:18:21.594: [ COMMCRS][2919263120]clsc_connect: (0xa2f0878) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:18:21.595: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:18:22.602: [ COMMCRS][2919263120]clsc_connect: (0xa2f0878) no listener at (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD))
            
            [  clsdmc][2992692112]Fail to connect (ADDRESS=(PROTOCOL=ipc)(KEY=rac1DBG_CTSSD)) with status 9
            2013-05-30 16:18:22.603: [ora.ctssd][2992692112] [start] Error = error 9 encountered when connecting to CTSSD
            2013-05-30 16:18:23.608: [ COMMCRS][29