4 Replies Latest reply: Mar 25, 2013 1:01 PM by Gvashist RSS

    Oracle 10g RAC crs_stat -t  State is UNKNOW / OFFLINE after Reboot Nodes

    Gvashist
      Hello Expert

      i have a Issue .


      on my Oracle 10g R2 RAC Rhel 5.5 vmware workstation 9_ .
      All my Resource are ONLINE state & RAC machine working fine .

      After Reboot Both Nodes , i found some Issues on crs_stat -t output

      [oracle@gvrac1 ~]$ crs_stat -t
      Name Type Target State Host
      ------------------------------------------------------------
      ora.gvrac1.gsd application ONLINE ONLINE gvrac1
      ora.gvrac1.ons application ONLINE ONLINE gvrac1
      ora.gvrac1.vip application ONLINE ONLINE gvrac1
      ora.gvrac2.gsd application    ONLINE    UNKNOWN   gvrac2
      ora.gvrac2.ons application    ONLINE    UNKNOWN   gvrac2
      ora.gvrac2.vip application ONLINE ONLINE gvrac2


      ocrcheck , crsctl check crs fine on All nodes .

      Pls suggest me to Solve this issue . its my Daily task for Fix this Issue .
      Daily i try SRVCTL cmd to Stop nodesapps & Restart All nodeapps again to fix these .



      But now i need to Fix it .


      Thanks
        • 1. Re: Oracle 10g RAC crs_stat -t  State is UNKNOW / OFFLINE after Reboot Nodes
          P.Forstmann
          Did you look in Clusterware log files in $CRS_HOME/log/<node> directory tree ?
          Is there any error messages for these resources ?

          You can also try to a couple of crs_stop/crs_start on the full resource name displayed by crs_stat: this could also help.
          • 2. Re: Oracle 10g RAC crs_stat -t  State is UNKNOW / OFFLINE after Reboot Nodes
            Gvashist
            Thanks for Reply

            1. i check Log

            i found Error Msg


            [oracle@gvrac1 ~]$ tail -100 /u01/app/oracle/product/10.2.0/crs_1/log/gvrac1/crsd/crsd.log
            2013-03-24 18:47:05.728: [  CRSEVT][2698685328]0CAAMonitorHandler :: 0:Action Script /u01/app/oracle/product/10.2.0/crs_1/bin/racgwrap(check) timed out for ora.gvrac1.vip! (timeout=60)
            2013-03-24 18:47:05.729: [  CRSAPP][2698685328]0CheckResource error for ora.gvrac1.vip error code = -2
            2013-03-24 18:47:05.899: [  CRSEVT][2698685328]0CAAMonitorHandler :: 0:Action Script /u01/app/oracle/product/10.2.0/crs_1/bin/racgwrap(check) timed out for ora.gvrac1.vip! (timeout=60)
            2013-03-24 18:47:05.899: [  CRSAPP][2698685328]0CheckResource error for ora.gvrac1.vip error code = -2
            2013-03-24 18:47:06.058: [  CRSEVT][2698685328]0CAAMonitorHandler :: 0:Action Script /u01/app/oracle/product/10.2.0/crs_1/bin/racgwrap(check) timed out for ora.gvrac1.vip! (timeout=60)
            2013-03-24 18:47:06.058: [  CRSAPP][2698685328]0CheckResource error for ora.gvrac1.vip error code = -2
            2013-03-24 18:47:06.489: [  CRSEVT][2698685328]0CAAMonitorHandler :: 0:Action Script /u01/app/oracle/product/10.2.0/crs_1/bin/racgwrap(check) timed out for ora.gvrac1.vip! (timeout=60)
            2013-03-24 18:47:06.489: [  CRSAPP][2698685328]0CheckResource error for ora.gvrac1.vip error code = -2
            2013-03-24 18:47:07.100: [  CRSEVT][2698685328]0CAAMonitorHandler :: 0:Action Script /u01/app/oracle/product/10.2.0/crs_1/bin/racgwrap(check) timed out for ora.gvrac1.vip! (timeout=60)
            2013-03-24 18:47:07.100: [  CRSAPP][2698685328]0CheckResource error for ora.gvrac1.vip error code = -2
            2013-03-24 18:47:07.865: [  CRSEVT][2677705616]0CAAMonitorHandler :: 0:Action Script /u01/app/oracle/product/10.2.0/crs_1/bin/racgwrap(check) timed out for ora.gvrac1.vip! (timeout=60)
            2013-03-24 18:47:07.866: [  CRSAPP][2677705616]0CheckResource error for ora.gvrac1.vip error code = -2
            2013-03-24 18:47:08.970: [  CRSEVT][2698685328]0CAAMonitorHandler :: 0:Action Script /u01/app/oracle/product/10.2.0/crs_1/bin/racgwrap(check) timed out for ora.gvrac1.vip! (timeout=60)
            2013-03-24 18:47:08.970: [  CRSAPP][2698685328]0CheckResource error for ora.gvrac1.vip error code = -2
            2013-03-24 18:47:10.000: [  CRSEVT][2698685328]0CAAMonitorHandler :: 0:Action Script /u01/app/oracle/product/10.2.0/crs_1/bin/racgwrap(check) timed out for ora.gvrac1.vip! (timeout=60)
            2013-03-24 18:47:10.001: [  CRSAPP][2698685328]0CheckResource error for ora.gvrac1.vip error code = -2
            2013-03-24 18:47:10.970: [  CRSEVT][2677705616]0CAAMonitorHandler :: 0:Action Script /u01/app/oracle/product/10.2.0/crs_1/bin/racgwrap(check) timed out for ora.gvrac1.vip! (timeout=60)
            2013-03-24 18:47:10.970: [  CRSAPP][2677705616]0CheckResource error for ora.gvrac1.vip error code = -2
            2013-03-24 18:47:11.999: [  CRSEVT][2698685328]0CAAMonitorHandler :: 0:Action Script /u01/app/oracle/product/10.2.0/crs_1/bin/racgwrap(check) timed out for ora.gvrac1.vip! (timeout=60)
            2013-03-24 18:47:11.999: [  CRSAPP][2698685328]0CheckResource error for ora.gvrac1.vip error code = -2
            2013-03-24 18:47:13.101: [  CRSEVT][2677705616]0CAAMonitorHandler :: 0:Action Script /u01/app/oracle/product/10.2.0/crs_1/bin/racgwrap(check) timed out for ora.gvrac1.vip! (timeout=60)
            2013-03-24 18:47:13.101: [  CRSAPP][2677705616]0CheckResource error for ora.gvrac1.vip error code = -2
            2013-03-24 18:47:13.884: [  CRSEVT][2677705616]0CAAMonitorHandler :: 0:Action Script /u01/app/oracle/product/10.2.0/crs_1/bin/racgwrap(check) timed out for ora.gvrac1.vip! (timeout=60)
            2013-03-24 18:47:13.885: [  CRSAPP][2677705616]0CheckResource error for ora.gvrac1.vip error code = -2
            2013-03-24 18:47:15.194: [  CRSEVT][2698685328]0CAAMonitorHandler :: 0:Action Script /u01/app/oracle/product/10.2.0/crs_1/bin/racgwrap(check) timed out for ora.gvrac1.vip! (timeout=60)
            2013-03-24 18:47:15.194: [  CRSAPP][2698685328]0CheckResource error for ora.gvrac1.vip error code = -2
            2013-03-24 18:47:15.923: [  CRSEVT][2698685328]0CAAMonitorHandler :: 0:Action Script /u01/app/oracle/product/10.2.0/crs_1/bin/racgwrap(check) timed out for ora.gvrac1.vip! (timeout=60)
            2013-03-24 18:47:15.923: [  CRSAPP][2698685328]0CheckResource error for ora.gvrac1.vip error code = -2
            2013-03-24 18:47:16.987: [  CRSEVT][2698685328]0CAAMonitorHandler :: 0:Action Script /u01/app/oracle/product/10.2.0/crs_1/bin/racgwrap(check) timed out for ora.gvrac1.vip! (timeout=60)
            2013-03-24 18:47:16.987: [  CRSAPP][2698685328]0CheckResource error for ora.gvrac1.vip error code = -2
            2013-03-24 18:47:17.332: [  CRSEVT][2698685328]0CAAMonitorHandler :: 0:Action Script /u01/app/oracle/product/10.2.0/crs_1/bin/racgwrap(check) timed out for ora.gvrac1.vip! (timeout=60)
            2013-03-24 18:47:17.332: [  CRSAPP][2698685328]0CheckResource error for ora.gvrac1.vip error code = -2
            2013-03-24 18:47:26.415: [  OCRSRV][3044850576]th_select_handler: Failed to retrieve procctx from ht. constr = [166629080] retval lht [-27] Signal CV.


            2. i am New on RAC, so i m not able to find out the Issue from log .


            Thanks
            • 3. Re: Oracle 10g RAC crs_stat -t  State is UNKNOW / OFFLINE after Reboot Nodes
              998870
              Did you try metalink? You might be hitting bug 7614171.
              • 4. Re: Oracle 10g RAC crs_stat -t  State is UNKNOW / OFFLINE after Reboot Nodes
                Gvashist
                Thanks for Reply


                Checked MOS Doc. Nothing for Me.

                well i also Configure Hangcheck as MOS Suggest.

                If i Upgrade this Machine to 10.2.0.4 , My issue resolve ?