13 Replies Latest reply: Jul 29, 2011 9:36 PM by 878567 RSS

    The observer only reinstate the old primary database after startup it twice

    878567
      I use Oracle 11g r2 configure a data guard and fast_start fail over (using data guard broker) environment on linux.
      The data guard works fine, and the fail over too.
      The problem is that after fail over, the observer can't reinstate the old primary database to new standby database after start up it to mount, and what strange is that if shutdown (immediate) it again, and start up it to mount again, the reinstating can be completed automatically and successfully.
      I use 'shutdown abort' to simulate the database crash.
      Any one have some advice? Thanks in advance!

      Edited by: user1665703 on 2011-7-27 下午4:33
        • 1. Re: The observer only reinstate the old primary database after startup it twice
          878567
          There are some strange log in drcorcl.log,

          2011-07-27 16:30:57.702 RSM0: HEALTH CHECK ERROR: ORA-16808: primary database is not open
          2011-07-27 16:30:57.844 RSM Warning: cannot find the destination setting in v$archive_dest for database 'orclSt
          d'.
          2011-07-27 16:30:57.844 RSM0: HEALTH CHECK WARNING: ORA-16728: consistency check for property LogXptMode found
          ORA-16777 error
          2011-07-27 16:30:57.845 RSM0: HEALTH CHECK WARNING: ORA-16777: unable to find the destination entry of a standb
          y database in V$ARCHIVE_DEST
          2011-07-27 16:30:57.845 RSM Warning: cannot find the destination setting in v$archive_dest for database 'orclSt
          d'.
          2011-07-27 16:30:57.845 RSM0: HEALTH CHECK WARNING: ORA-16728: consistency check for property DelayMins found O
          RA-16777 error
          2011-07-27 16:30:57.845 RSM0: HEALTH CHECK WARNING: ORA-16777: unable to find the destination entry of a standb
          y database in V$ARCHIVE_DEST
          2011-07-27 16:30:57.846 RSM Warning: cannot find the destination setting in v$archive_dest for database 'orclSt
          d'.
          2011-07-27 16:30:57.846 RSM0: HEALTH CHECK WARNING: ORA-16728: consistency check for property Binding found ORA
          -16777 error
          2011-07-27 16:30:57.846 RSM0: HEALTH CHECK WARNING: ORA-16777: unable to find the destination entry of a standb
          y database in V$ARCHIVE_DEST
          2011-07-27 16:30:57.847 RSM Warning: cannot find the destination setting in v$archive_dest for database 'orclSt
          d'.
          2011-07-27 16:30:57.847 RSM0: HEALTH CHECK WARNING: ORA-16728: consistency check for property MaxFailure found
          ORA-16777 error
          2011-07-27 16:30:57.847 RSM0: HEALTH CHECK WARNING: ORA-16777: unable to find the destination entry of a standb
          y database in V$ARCHIVE_DEST
          2011-07-27 16:30:57.848 RSM Warning: cannot find the destination setting in v$archive_dest for database 'orclSt
          d'.
          2011-07-27 16:30:57.848 RSM0: HEALTH CHECK WARNING: ORA-16728: consistency check for property MaxConnections fo
          und ORA-16777 error
          2011-07-27 16:30:57.848 RSM0: HEALTH CHECK WARNING: ORA-16777: unable to find the destination entry of a standb
          y database in V$ARCHIVE_DEST
          2011-07-27 16:30:57.849 RSM Warning: cannot find the destination setting in v$archive_dest for database 'orclSt
          d'.
          2011-07-27 16:30:57.849 RSM0: HEALTH CHECK WARNING: ORA-16728: consistency check for property ReopenSecs found
          ORA-16777 error
          2011-07-27 16:30:57.849 RSM0: HEALTH CHECK WARNING: ORA-16777: unable to find the destination entry of a standb
          y database in V$ARCHIVE_DEST
          2011-07-27 16:30:57.849 RSM Warning: cannot find the destination setting in v$archive_dest for database 'orclSt
          d'.
          2011-07-27 16:30:57.850 RSM0: HEALTH CHECK WARNING: ORA-16728: consistency check for property NetTimeout found
          ORA-16777 error
          2011-07-27 16:30:57.850 RSM0: HEALTH CHECK WARNING: ORA-16777: unable to find the destination entry of a standb
          y database in V$ARCHIVE_DEST
          2011-07-27 16:30:57.850 RSM Warning: cannot find the destination setting in v$archive_dest for database 'orclSt
          d'.
          2011-07-27 16:30:57.850 RSM0: HEALTH CHECK WARNING: ORA-16728: consistency check for property RedoCompression f
          ound ORA-16777 error
          2011-07-27 16:30:57.851 RSM0: HEALTH CHECK WARNING: ORA-16777: unable to find the destination entry of a standb
          • 2. Re: The observer only reinstate the old primary database after startup it twice
            878567
            And Oracle always keep logging follow to the drcorcl.log (before the second time start up the old primary database),


            2011-07-27 16:31:00.783 01001000 697064046 DMON: Entered rfm_get_chief_lock() for CTL_QUIESCE, reason 0
            2011-07-27 16:31:00.784 00000000 697064048 DMON: CTL_GET_STATUS operation completed
            2011-07-27 16:31:04.837 CLSR: CRS not configured, config = 2
            .
            2011-07-27 16:31:05.838 01001000 697064046 DMON: Entered rfm_get_chief_lock() for CTL_QUIESCE, reason 0
            2011-07-27 16:31:05.838 00000000 697064049 DMON: CTL_GET_STATUS operation completed
            2011-07-27 16:31:08.029 01001000 697064046 DMON: Entered rfm_get_chief_lock() for CTL_QUIESCE, reason 0
            2011-07-27 16:31:08.907 CLSR: CRS not configured, config = 2
            .
            2011-07-27 16:31:09.957 01001000 697064046 DMON: Entered rfm_get_chief_lock() for CTL_QUIESCE, reason 0
            2011-07-27 16:31:09.957 00000000 697064050 DMON: CTL_GET_STATUS operation completed
            2011-07-27 16:31:13.008 CLSR: CRS not configured, config = 2
            .
            2011-07-27 16:31:14.009 01001000 697064046 DMON: Entered rfm_get_chief_lock() for CTL_QUIESCE, reason 0
            2011-07-27 16:31:14.009 00000000 697064051 DMON: CTL_GET_STATUS operation completed
            2011-07-27 16:31:17.100 CLSR: CRS not configured, config = 2

            Edited by: user1665703 on 2011-7-27 下午4:39
            • 3. Re: The observer only reinstate the old primary database after startup it twice
              mseberg
              I think this is because the location of broker configuration files is not set the same common location in the primary.

              Check out - 3.2 Setting Up the Broker Configuration Files :

              http://download.oracle.com/docs/cd/B19306_01/server.102/b14230/configure.htm#i1021661

              Best Regards

              mseberg
              • 4. Re: The observer only reinstate the old primary database after startup it twice
                CKPT
                I use Oracle 11g r2 configure a data guard and fast_start fail over (using data guard broker) environment on linux.
                The data guard works fine, and the fail over too.
                The problem is that after fail over, the observer can't reinstate the old primary database to new standby database after start up it to mount, and what strange is that if shutdown (immediate) it again, and start up it to mount again, the reinstating can be completed automatically and successfully.
                I use 'shutdown abort' to simulate the database crash.
                Any one have some advice? Thanks in advance!
                As Mseberg said, it might due to wron configuraiton, You can again reconfiugre it or drop & recreate the broker configuration.

                *Step By Step How to Recreate Dataguard Broker Configuration [ID 808783.1]*
                • 5. Re: The observer only reinstate the old primary database after startup it twice
                  878567
                  I changed the configuration file of data guard broker both on primary and standby, but the problem still exist.
                  I set the value of configuration file on primary and standby to different file name, is this correct?


                  On primary
                  SQL> show parameter DG_BROKER_CONFIG_FILE

                  NAME TYPE VALUE
                  ------------------------------------ ----------- ------------------------------
                  dg_broker_config_file1 string /home/oracle/app/oracle/dgbrok
                  er/dr1orclPri.dat
                  dg_broker_config_file2 string /home/oracle/app/oracle/dgbrok
                  er/dr2orclPri.dat
                  SQL>

                  On standby
                  SQL> show parameter DG_BROKER_CONFIG_FILE

                  NAME TYPE VALUE
                  ------------------------------------ ----------- ------------------------------
                  dg_broker_config_file1 string /home/oracle/app/oracle/dgbrok
                  er/dr1orclStd.dat
                  dg_broker_config_file2 string /home/oracle/app/oracle/dgbrok
                  er/dr2orclStd.dat
                  • 6. Re: The observer only reinstate the old primary database after startup it twice
                    mseberg
                    Great!

                    What does :
                    SELECT DEST_NAME, DESTINATION FROM V$ARCHIVE_DEST WHERE DEST_NAME='STANDBY_ARCHIVE_DEST' ;
                    Looks like the standby database does not have the archive destination set.

                    In broker :

                    EDIT DATABASE '<standby_db_uniquename>' set PROPERTY STANDBYARCHIVELOCATION='<correct location>'

                    mseberg
                    • 7. Re: The observer only reinstate the old primary database after startup it twice
                      878567
                      Yes, the primary database doesn't has 'STANDBY_ARCHIVE_DEST' in view V$ARCHIVE_DEST, I don't know why.
                      but after I set STANDBYARCHIVELOCATION in broker, the problem still exits,

                      show database verbose  'orclPri'
                      ...
                      StandbyArchiveLocation = '/home/oracle/app/oracle/dataguard'
                      ...


                      show database verbose 'orclStd'
                      ...
                      StandbyArchiveLocation = 'USE_DB_RECOVERY_FILE_DEST'
                      ...
                      • 8. Re: The observer only reinstate the old primary database after startup it twice
                        mseberg
                        I would remove broker. Fix in SQL, and then put broker back.

                        Somewhere the wires are crossed and this is probably the fastest out.

                        This note may help if you decided to go this way :

                        ORA-16504 when the Data Guard Configuration has already been removed [ID 246685.1]

                        Best Regards

                        mseberg
                        • 9. Re: The observer only reinstate the old primary database after startup it twice
                          878567
                          mseberg, thanks,
                          But I am not very clearly understand what you said, can you please give more details what I need to do?

                          Do you mean delete the broker configuration and then recreate it?

                          Edited by: user1665703 on 2011-7-29 上午11:31
                          • 10. Re: The observer only reinstate the old primary database after startup it twice
                            mseberg
                            Do you mean delete the broker configuration and then recreate it?
                            Correct. And sorry if I was unclear.

                            The note I posted before has some details.

                            Use This

                            How to Safely Remove a Data Guard Broker Configuration [ID 261336.1]


                            Also consider searching the Data Guard forum for "broker", I'm sure you will find some setup examples. Make sure to adjust the default 90 days as needed.

                            mseberg

                            Edited by: mseberg on Jul 29, 2011 2:11 PM
                            • 11. Re: The observer only reinstate the old primary database after startup it twice
                              878567
                              I recreated the broker configuration, problem still exist...
                              By the way, what does ' [ID 261336.1]' means?
                              • 12. Re: The observer only reinstate the old primary database after startup it twice
                                mseberg
                                Its the Oracle Support note on removing broker.

                                If you done all this and are still having an issue its time for Oracle Support to enter the picture.

                                We cannot just keep trying things and hope for different results.

                                If you and or Oracle Support solve this please consider posting the solution here.

                                Best Regards

                                mseberg
                                • 13. Re: The observer only reinstate the old primary database after startup it twice
                                  878567
                                  oh,oh, I think I don't have the Oracle Support license, so....
                                  Maybe it's caused by my environment, I used Linux redhat 6. Formerly, I did those things successfully on Windows XP platform.