4 Replies Latest reply: Mar 20, 2014 9:40 AM by BQ-OC RSS

    Physical standby databases gets disabled in dgbroker configuration when shuting down one physical database

    BQ-OC

      Hi!

      I have 8 physical standby and when i shutdown one the rest gets disabled in configuration.

      DGMGRL> show configuration

      Configuration - GESS1_BROKER

        Protection Mode: MaxPerformance
        Databases:

          GESS1  - Primary database
          GCOD1A - Physical standby database
          GCOD1B - Physical standby database
          GCOD2B - Physical standby database
          GCOD3A - Physical standby database
          GCOD3B - Physical standby database
          GCOD4A - Physical standby database
          GCOD4B - Physical standby database
          GCOD2A - Physical standby database

       

      Fast-Start Failover: DISABLED

      Configuration Status:
      SUCCESS

       

      then when i shutdown GCOD4A the other physical standby databases gets disabled one by one as you see below

       

      Databases:
          GESS1  - Primary database
            Error: ORA-16778: redo transport error for one or more databases

          GCOD1A - Physical standby database (disabled)
          GCOD1B - Physical standby database
          GCOD2B - Physical standby database
          GCOD3A - Physical standby database
          GCOD3B - Physical standby database
          GCOD4A - Physical standby database
            Error: ORA-01034: ORACLE not available

          GCOD4B - Physical standby database
          GCOD2A - Physical standby database

       

      GESS1  - Primary database
            Error: ORA-16778: redo transport error for one or more databases

          GCOD1A - Physical standby database (disabled)
          GCOD1B - Physical standby database (disabled)
          GCOD2B - Physical standby database
          GCOD3A - Physical standby database
          GCOD3B - Physical standby database
          GCOD4A - Physical standby database
            Error: ORA-01034: ORACLE not available

          GCOD4B - Physical standby database
          GCOD2A - Physical standby database (disabled)

       

       

      GESS1  - Primary database
            Error: ORA-16778: redo transport error for one or more databases

          GCOD1A - Physical standby database (disabled)
          GCOD1B - Physical standby database (disabled)
          GCOD2B - Physical standby database (disabled)
          GCOD3A - Physical standby database (disabled)
          GCOD3B - Physical standby database
          GCOD4A - Physical standby database
            Error: ORA-01034: ORACLE not available

          GCOD4B - Physical standby database
          GCOD2A - Physical standby database (disabled)

       

        Databases:

          GESS1  - Primary database

          GCOD1A - Physical standby database (disabled)

          GCOD1B - Physical standby database (disabled)

          GCOD2B - Physical standby database (disabled)

          GCOD3A - Physical standby database (disabled)

          GCOD3B - Physical standby database (disabled)

          GCOD4A - Physical standby database (disabled)

          GCOD4B - Physical standby database

          GCOD2A - Physical standby database (disabled)

       

      I also see that log_archive_dest_state for the physical standbys which are disabled in primary database GESS1 gets value RESET.

       

      SQL> sho parameter log_archive_dest_state

      log_archive_dest_state_2             string      RESET

      log_archive_dest_state_3             string      RESET

      log_archive_dest_state_4             string      RESET

      log_archive_dest_state_5             string      RESET

      log_archive_dest_state_6             string      RESET

      log_archive_dest_state_7             string      RESET

      log_archive_dest_state_8             string      RESET

      log_archive_dest_state_9             string      ENABLE

       

      Thankfull for some ideas on what could be wrong.

      Regards

      BQ

        • 1. Re: Physical standby databases gets disabled in dgbroker configuration when shuting down one physical database
          mseberg

          Hello;

           

          I would remove and recreate:

           

          How to Safely Remove a Data Guard Broker Configuration (Doc ID 261336.1)

           

          Step by Step How to Create Dataguard Broker Configuration (Doc ID 984622.1)

           

          Best Regards

           

          mseberg

          • 2. Re: Physical standby databases gets disabled in dgbroker configuration when shuting down one physical database
            CKPT

            Have you tried to disable database which you have performed and shutdown.

            DGMGRL> disable database 'GCOD4A';

            DGMGRL> show configuration;

             

            - Only with DG Broker status we should not come to conclusion, You got chance to look at DG Broker log files and also alert log.

            • 3. Re: Physical standby databases gets disabled in dgbroker configuration when shuting down one physical database
              BQ-OC

              Hi!

              I'm going to recreate the configuration from scratch and see if the problem show again.

              This is what logs says

               

              When  shut down GCOD4A alertlog on primary GESS1 says:

              (You can se the RESET of the other databases starts coming)

               

              Tue Mar 18 08:37:40 2014

              LNS: Attempting destination LOG_ARCHIVE_DEST_8 network reconnect (3135)

              LNS: Destination LOG_ARCHIVE_DEST_8 network reconnect abandoned

              Error 3135 for archive log file 2 to 'GCOD4A'

              Errors in file /opt/oracle/diag/rdbms/gess1/GESS1/trace/GESS1_nsa8_21431.trc:

              ORA-03135: connection lost contact

              LNS: Failed to archive log 2 thread 1 sequence 536 (3135)

              Tue Mar 18 08:37:44 2014

              ALTER SYSTEM SET log_archive_dest_state_2='RESET' SCOPE=BOTH;

              ALTER SYSTEM SET log_archive_dest_state_3='RESET' SCOPE=BOTH;

              Error 1034 received logging on to the standby

              PING[ARC2]: Heartbeat failed to connect to standby 'GCOD4A'. Error is 1034.

               

              GESS1_nsa8_21431.trc shows

              *** 2014-03-18 08:37:40.425

              krsb_stream_send: Error 3135 sending stream IOV

              RFS network connection lost at host 'GCOD4A' error 3135

              Error 3135 writing standby archive log file at host 'GCOD4A'

              *** 2014-03-18 08:37:40.426 4320 krsh.c

              LNS: Attempting destination LOG_ARCHIVE_DEST_8 network reconnect (3135)

              *** 2014-03-18 08:37:40.426 4320 krsh.c

              LNS: Destination LOG_ARCHIVE_DEST_8 network reconnect abandoned

              *** 2014-03-18 08:37:40.426 4320 krsh.c

              Error 3135 for archive log file 2 to 'GCOD4A'

              *** 2014-03-18 08:37:40.426 2932 krsi.c

              krsi_dst_fail: dest:8 err:3135 force:0 blast:1

              ORA-03135: connection lost contact

              Closing Redo Read Context

               

               

              drcGESS1.log (primary) shows, There is no information at all at that time in local dg_broker log of GCOD1A etc.

              03/18/2014 08:40:36

              Failed to connect to remote database GCOD4A. Error is ORA-01034

              Failed to send message to site GCOD4A. Error code is ORA-01034.

              Redo transport problem detected: redo transport for database GCOD4A has the following error:

                ORA-03135: connection lost contact

              Failed to connect to remote database GCOD4A. Error is ORA-01034

              Failed to send message to site GCOD4A. Error code is ORA-01034.

              Database GCOD2B has been shut down

              Failed to connect to remote database GCOD4A. Error is ORA-01034

              Metadata Resync failed. Status = ORA-01034

              Data Guard Broker Status Summary:

                Type                        Name                             Severity  Status

                Configuration               GESS1_BROKER                      Warning  ORA-16607

                Primary Database            GESS1                               Error  ORA-16778

                Physical Standby Database   GCOD1A                            Warning  ORA-16631

                Physical Standby Database   GCOD1B                            Warning  ORA-16631

                Physical Standby Database   GCOD2A                            Warning  ORA-16631

                Physical Standby Database   GCOD2B                            Warning  ORA-16631

                Physical Standby Database   GCOD3A                            Success  ORA-00000

                Physical Standby Database   GCOD3B                            Success  ORA-00000

                Physical Standby Database   GCOD4A                              Error  ORA-01034

                Physical Standby Database   GCOD4B                            Success  ORA-00000

               

              Regards

              /Johan

              • 4. Re: Physical standby databases gets disabled in dgbroker configuration when shuting down one physical database
                BQ-OC

                Hi!

                I rebuild the configuration and then shut down several of the physical standby databases and the other databases didnt get disabled.

                Then I set databases into Snapshot Standby mode and back to physical standby mode.

                (In my project I'm going to have 8 standby databases, 4 of them will be snapshot standby (active) and 4 physical standby (inactive) and I'll switch them twice a day for "update" from primary database)

                And when I shut down one of the physical standby databases the other physical standby databases got disabled again.

                Somehow this seems to have something to do with Snapshot standby.

                I 've created an SR but will appreciate if someone have any idea about the problem.

                Regards

                Johan