13 Replies Latest reply: Nov 15, 2012 9:10 AM by 9233598 RSS

    RAC nodes reboot, ASM/crs all restarted but database

    9233598
      OS version: linux redhat 5
      DB version: 11.2.0.3
      2 node RAC

      It used all working fine. I just configured/implemented hugepages lately, and rebooted boxes.

      everything else such as ASM, listener, crs all restarted, but db not.

      What should I check to be sure oracle auto restart working?
      Thanks.
        • 1. Re: RAC nodes reboot, ASM/crs all restarted but database
          damorgan
          What in the docs led you to move to hugepages? Please post the link and how it relates to your specific database. And where did you do it? Cache fusion interconnect? Storage? Public? Would you like us to guess? Post the commands you used to make the change ... every one, in order, the exact syntax. Also the specific syntax you used to alter the spfile for hugepages.

          Finally: Writing "the DB does not start" is as meaningful a statement as saying "my car doesn't start." I can no more tell from thousands of miles away in an internet forum why your database doesn't start than I can tell whether the battery is dead or someone stole the fuel pump. Assume your database was a car ... what would your mechanic ask you? What sound do you hear when you turn the key? Well I'll ask "what did the database write to your alert log?" And please don't describe it ... post it.

          Along with all of the other relevant information if you want further help.
          • 2. Re: RAC nodes reboot, ASM/crs all restarted but database
            JohnWatson
            Check how the database is registered in the OCR, for example,
            [oracle@iron1 ~]$ crsctl status resource ora.qsx.db -p
            NAME=ora.qsx.db
            TYPE=ora.database.type
            ACL=owner:oracle:rwx,pgrp:oinstall:rwx,other::r--
            ACTION_FAILURE_TEMPLATE=
            ACTION_SCRIPT=
            ACTIVE_PLACEMENT=1
            AGENT_FILENAME=%CRS_HOME%/bin/oraagent%CRS_EXE_SUFFIX%
            AUTO_START=restore
            CARDINALITY=2
            note the value of the AUTO_START attribute, in the example above it is "restore". That means that the clusterware will restart it only if it was running when the clusterware was shutdown.
            • 3. Re: RAC nodes reboot, ASM/crs all restarted but database
              9233598
              Thank you for your reply.

              Here is the information per your command:

              crsctl status resource ora.plt.db -p
              NAME=ora.plt.db
              TYPE=ora.database.type
              ACL=owner:oracle:rwx,pgrp:oinstall:rwx,other::r--
              ACTION_FAILURE_TEMPLATE=
              ACTION_SCRIPT=
              ACTIVE_PLACEMENT=1
              AGENT_FILENAME=%CRS_HOME%/bin/oraagent%CRS_EXE_SUFFIX%
              AUTO_START=restore
              CARDINALITY=2
              CHECK_INTERVAL=1
              CHECK_TIMEOUT=30
              CLUSTER_DATABASE=true
              DATABASE_TYPE=RAC
              DB_UNIQUE_NAME=PLT
              DEFAULT_TEMPLATE=PROPERTY(RESOURCE_CLASS=database) PROPERTY(DB_UNIQUE_NAME= CONCAT(PARSE(%NAME%, ., 2), %USR_ORA_DOMAIN%, .)) ELEMENT(INSTANCE_NAME= %GEN_USR_ORA_INST_NAME%) ELEMENT(DATABASE_TYPE= %DATABASE_TYPE%)
              DEGREE=1
              DESCRIPTION=Oracle Database resource
              ENABLED=1
              FAILOVER_DELAY=0
              FAILURE_INTERVAL=60
              FAILURE_THRESHOLD=1
              GEN_AUDIT_FILE_DEST=/u01/app/oracle/admin/PLT/adump
              GEN_START_OPTIONS=
              GEN_START_OPTIONS@SERVERNAME(oracle09)=open
              GEN_START_OPTIONS@SERVERNAME(oracle10)=open
              GEN_USR_ORA_INST_NAME=
              GEN_USR_ORA_INST_NAME@SERVERNAME(oracle09)=PLT1
              GEN_USR_ORA_INST_NAME@SERVERNAME(oracle10)=PLT2
              HOSTING_MEMBERS=
              INSTANCE_FAILOVER=0
              LOAD=1
              LOGGING_LEVEL=1
              MANAGEMENT_POLICY=AUTOMATIC
              NLS_LANG=
              NOT_RESTARTING_TEMPLATE=
              OFFLINE_CHECK_INTERVAL=0
              ONLINE_RELOCATION_TIMEOUT=0
              ORACLE_HOME=/u01/app/oracle/product/11.2.0/dbhome_1
              ORACLE_HOME_OLD=
              PLACEMENT=restricted
              PROFILE_CHANGE_TEMPLATE=
              RESTART_ATTEMPTS=2
              ROLE=PRIMARY
              SCRIPT_TIMEOUT=60
              SERVER_POOLS=ora.PLT
              SPFILE=+DAT/PLT/spfilePLT.ora
              START_DEPENDENCIES=hard(ora._DAT.dg,ora._FRA.dg) weak(type:ora.listener.type,global:type:ora.scan_listener.type,uniform:ora.ons,global:ora.gns) pullup(ora._DAT.dg,ora._FRA.dg)


              Are they looking right?

              Thanks
              • 4. Re: RAC nodes reboot, ASM/crs all restarted but database
                585179
                Hi,

                How did you stop the database before the server reboot? If you stopped it by using srvctl stop database then it is normal that the database is not coming back automated when the server rebooted. If you stop everything using crsctl command then the database will be back automated. Otherwise check you cluster related log files


                Hope it helps

                Cheers
                • 5. Re: RAC nodes reboot, ASM/crs all restarted but database
                  9233598
                  I stopped by srvctl stop db first, then using crsctl as root to stop crs.

                  Is this the right way to do? I thought I have to stop db first before crs.
                  • 6. Re: RAC nodes reboot, ASM/crs all restarted but database
                    585179
                    9233598 wrote:
                    I stopped by srvctl stop db first, then using crsctl as root to stop crs.

                    Is this the right way to do? I thought I have to stop db first before crs.
                    Hi,

                    I have seen some environments stop database with srvctl first but I also have seen some other environments stop everything using crsctl. Both command will work without problem. crsctl will stop all resources managed by clusterware.


                    Hope it helps

                    Cheers
                    • 7. Re: RAC nodes reboot, ASM/crs all restarted but database
                      Shivananda Rao
                      Hello,
                      crsctl status resource ora.plt.db -p
                      The above command when executed in your previous post shows that the database AUTO_START is set to "restore". This would mean that the database state after server reboot would be restored to the state it was before the server reboot. So, in your case, you had the database down before you rebooted the server and hence the database did not come up after the reboot activity.

                      What does the "srvctl config database -d plt" show ? Just make sure that the POLICY MANAGEMENT is set to AUTOMATIC and if so then I do not consider there to be any issue.

                      Just a note: With 11gR2, crsctl is not supported to modify or edit resources with prefix ora.* .
                      • 8. Re: RAC nodes reboot, ASM/crs all restarted but database
                        9233598
                        Here is the result from srvctl config database:
                        rvctl config database -d plt
                        Database unique name: PLT
                        Database name: PLT
                        Oracle home: /u01/app/oracle/product/11.2.0/dbhome_1
                        Oracle user: oracle
                        Spfile: +DAT/PLT/spfilePLT.ora
                        Domain:
                        Start options: open
                        Stop options: immediate
                        Database role: PRIMARY
                        Management policy: AUTOMATIC
                        Server pools: PLT
                        Database instances: PLT1,PLT2
                        Disk Groups: DAT,FRA
                        Mount point paths:
                        Services: plt_ro_jdbc,plt_ro_oci,plt_rw_jdbc ,plt_rw_ociType: RAC
                        Database is administrator managed
                        • 9. Re: RAC nodes reboot, ASM/crs all restarted but database
                          Sebastian Solbach -Dba Community-Oracle
                          Hi,

                          if you stopped the db with "srvctl stop database", then this is recorded in OCR. And when the clusterstack restarts it does not restart the database (since you told clusterware it should be stopped).

                          Works as designed.

                          If you want that the cluster restarts database simply shutdown the clusterware (crsctl stop crs).

                          Regards
                          Sebastian
                          • 10. Re: RAC nodes reboot, ASM/crs all restarted but database
                            698658
                            Hi Sebastian,
                            whem You do crsctl stop crs when DB is up and running this is considered shutdown abort , right ?
                            Regards
                            GregG
                            • 11. Re: RAC nodes reboot, ASM/crs all restarted but database
                              JohnWatson
                              GregG wrote:
                              Hi Sebastian,
                              whem You do crsctl stop crs when DB is up and running this is considered shutdown abort , right ?
                              Regards
                              GregG
                              Only if you cobnfigured it that way. If you look at the post above, it is clear:
                              srvctl config database -d plt
                              Database unique name: PLT
                              Database name: PLT
                              Oracle home: /u01/app/oracle/product/11.2.0/dbhome_1
                              Oracle user: oracle
                              Spfile: +DAT/PLT/spfilePLT.ora
                              Domain:
                              Start options: open
                              Stop options: immediate
                              Database role: PRIMARY
                              Management policy: AUTOMATIC
                              Server pools: PLT
                              Database instances: PLT1,PLT2
                              Disk Groups: +DAT,+FRA
                              Mount point paths:
                              Services: plt_ro_jdbc,plt_ro_oci,plt_rw_jdbc ,plt_rw_ociType: RAC
                              Database is administrator managed
                              Can you see the attribute "Stop options"?
                              • 12. Re: RAC nodes reboot, ASM/crs all restarted but database
                                698658
                                Thats clear, thank You.
                                Regards
                                GregG
                                • 13. Re: RAC nodes reboot, ASM/crs all restarted but database
                                  9233598
                                  Thanks, That answered my concerns.

                                  Appreciated your help.