8 Replies Latest reply: Mar 17, 2014 5:16 AM by 1809668 RSS

    Unable to start one RAC instance

    1809668

      Hi,

       

      We have two nodes Oracle RAC(11.2.0.3) and one instance got crashed. Unable to start the instance and gave the below command two start the instance.

       

      srvctl start instance -d accela -i ACCELA1

       

      getting below error

       

      PRCR-1013: Failed to start resource ora.accela.db

      PRCR-1064: Failed to start resource ora.accela.db on node adbcprora01

      CRS-2527: Unable to start 'ora.accela.db' because it has a 'hard' dependency on 'ora.DG_DATAFILES.dg'

      CRS-2503: Resource 'ora.DG_DATAFILES.dg' is in unknown state and must be stopped first.

        • 1. Re: Unable to start one RAC instance
          Baris Yildirim

          maybe problem on asm disk mount

           

          try to start instance using sqlplus to find out the problem easier

           

          export ORACLE_SID=SET_YOUR_DB_SID

          export ORACLE_HOME=SET_YOUR_DB_HOME

           

          sqlplus / as sysdba

           

          startup;

           

          and post the error

           

          Regards

          • 2. Re: Unable to start one RAC instance
            Vandana B -Oracle

            Hi,

             

            Try the following once done please let us know if the same resolves your issue

             

            ++ srvctl stop diskgroup -g DG_DATAFILES -n "<node1 or node name where the instance is not coming up>" -f

            ++ srvctl start diskgroup -g DG_DATAFILES -n "<node1 or node name where the instance is not coming up>" -f

            ++ srvctl start instance -d accela -i ACCELA1

             

            Regards,

            Vandana - Oracle

            • 3. Re: Unable to start one RAC instance
              Billy~Verreynne

              I would check the status of the ASM diskgroups first - as the o/s grid user, use sqlplus "/ as sysasm", and run the following query to show details about disks seen by ASM, and the status of the diskgroup.

               

              col name format a20
              col path format a30
              col diskgroup format a15
              break on diskgroup skip 1
              select
                      nvl(g.name, '<not mounted>')    as "DISKGROUP",
                      d.name,
                      d.path,
                      d.mount_status,
                      d.header_status,
                      d.mode_status,
                      d.state,
                      d.redundancy,
                      d.total_mb,
                      d.free_mb,
                      d.failgroup
              from    v$asm_disk d,
                      v$asm_diskgroup g
              where   d.group_number = g.group_number (+ )
              and     'MOUNTED' = g.state (+ )
              order by
                      g.name,
                      d.failgroup,
                      d.name;

              • 4. Re: Unable to start one RAC instance
                1809668

                Now all diskgroups are in mount state. And when I start the particular instance I am getting below error.

                 

                C:\Users\ORAinsttemp>srvctl start instance -d accela -i ACCELA1

                PRCR-1013 : Failed to start resource ora.accela.db

                PRCR-1064 : Failed to start resource ora.accela.db on node adbcprora01

                CRS-5014: Agent "D:\app\11.2.0\grid\bin\oraagent.exe" timed out starting process

                "D:\app\11.2.0\grid\bin\lsnrctl.exe" for action "clean": details at "(:CLSN0000

                9:)" in "D:\app\11.2.0\grid\log\adbcprora01\agent\crsd\oraagent\oraagent.log"

                CRS-5017: The resource action "ora.LISTENER.lsnr clean" encountered the followin

                g error:

                (:CLSN00009:)Utils:execCmd aborted. For details refer to "(:CLSN00106:)" in "D:\

                app\11.2.0\grid\log\adbcprora01\agent\crsd\oraagent\oraagent.log".

                 

                CRS-5014: Agent "D:\app\11.2.0\grid\bin\oraagent.exe" timed out starting process

                "D:\app\11.2.0\grid\bin\lsnrctl.exe" for action "check": details at "(:CLSN0000

                9:)" in "D:\app\11.2.0\grid\log\adbcprora01\agent\crsd\oraagent\oraagent.log"

                CRS-5017: The resource action "ora.LISTENER.lsnr check" encountered the followin

                g error:

                (:CLSN00009:)Utils:execCmd aborted. For details refer to "(:CLSN00109:)" in "D:\

                app\11.2.0\grid\log\adbcprora01\agent\crsd\oraagent\oraagent.log".

                 

                CRS-2680: ▌╘ß ╩Σ┘φ▌ 'ora.LISTENER.lsnr' ▌φ 'adbcprora01'

                ORA-21561: OID generation failed

                CRS-5017: The resource action "ora.accela.db start" encountered the following er

                ror:

                ORA-21561: OID generation failed

                . For details refer to "(:CLSN00107:)" in "D:\app\11.2.0\grid\log\adbcprora01\ag

                ent\crsd\oraagent\oraagent.log".

                 

                CRS-2674: ▌╘ß ╚╧┴ 'ora.accela.db' ┌ß∞ 'adbcprora01'

                ORA-21561: OID generation failed

                 

                 

                Also crs_stat -t status as below

                 

                C:\Users\ORAinsttemp>crs_stat -t

                Name           Type           Target    State     Host

                ------------------------------------------------------------

                ora....LOGS.dg ora....up.type ONLINE    ONLINE    adbcprora01

                ora....ILES.dg ora....up.type ONLINE    ONLINE    adbcprora01

                ora....DISK.dg ora....up.type ONLINE    ONLINE    adbcprora01

                ora....SK_M.dg ora....up.type ONLINE    ONLINE    adbcprora01

                ora....ER.lsnr ora....er.type ONLINE    ONLINE    adbcprora02

                ora....N1.lsnr ora....er.type ONLINE    ONLINE    adbcprora01

                ora....N2.lsnr ora....er.type ONLINE    ONLINE    adbcprora02

                ora....N3.lsnr ora....er.type ONLINE    ONLINE    adbcprora02

                ora.accela.db  ora....se.type ONLINE    ONLINE    adbcprora02

                ora....SM1.asm application    ONLINE    ONLINE    adbcprora01

                ora....01.lsnr application    ONLINE    UNKNOWN   adbcprora01

                ora....a01.gsd application    OFFLINE   OFFLINE

                ora....a01.ons application    ONLINE    ONLINE    adbcprora01

                ora....a01.vip ora....t1.type ONLINE    ONLINE    adbcprora01

                ora....SM2.asm application    ONLINE    ONLINE    adbcprora02

                ora....02.lsnr application    ONLINE    ONLINE    adbcprora02

                ora....a02.gsd application    OFFLINE   OFFLINE

                ora....a02.ons application    ONLINE    ONLINE    adbcprora02

                ora....a02.vip ora....t1.type ONLINE    ONLINE    adbcprora02

                ora.asm        ora.asm.type   ONLINE    ONLINE    adbcprora01

                ora.cvu        ora.cvu.type   ONLINE    ONLINE    adbcprora02

                ora.gsd        ora.gsd.type   OFFLINE   OFFLINE

                ora....network ora....rk.type ONLINE    ONLINE    adbcprora01

                ora.oc4j       ora.oc4j.type  ONLINE    ONLINE    adbcprora02

                ora.ons        ora.ons.type   ONLINE    ONLINE    adbcprora01

                ora....ry.acfs ora....fs.type ONLINE    ONLINE    adbcprora01

                ora.scan1.vip  ora....ip.type ONLINE    ONLINE    adbcprora01

                ora.scan2.vip  ora....ip.type ONLINE    ONLINE    adbcprora02

                ora.scan3.vip  ora....ip.type ONLINE    ONLINE    adbcprora02

                • 5. Re: Unable to start one RAC instance
                  Vandana B -Oracle

                  Hi,

                   

                  Could you try stopping and starting the database?

                   

                  ++ srvctl stop database -d accela

                  ++ srvctl start database -d accela

                   

                  Regards,

                  Vandana - Oracle

                  • 6. Re: Unable to start one RAC instance
                    1809668

                    Thanks Vandana. Now it both the instances are working.

                     

                    But in crs_stat -t, still show

                    ora....01.lsnr applicationONLINEUNKNOWN   adbcprora01

                    and rest of the resources state is ONLINE

                    • 7. Re: Unable to start one RAC instance
                      Vandana B -Oracle

                      Hi,

                       

                      If the status of a resource becomes "UNKNOWN" then the only solution for changing the status to "ONLINE" is by retstarting the resource

                       

                      Regards,

                      Vandana - Oracle

                      • 8. Re: Unable to start one RAC instance
                        1809668

                        Hi Vandana,

                         

                        How do I restart the below resource as its state in UNKNOWN.

                         

                        ora....01.lsnr applicationONLINEUNKNOWN   adbcprora01