4 Replies Latest reply on Jan 24, 2020 5:35 AM by Beauty_and_dBest

    EBS 12.1.3 Standby Db

    Beauty_and_dBest

      EBS 12.1.3

      11.2.0.4 Database

      OL6

       

       

      Hi ALL,

       

      We have configured dataguard in our database.

      It has been running for more than a year.

      But lately, I got report that it is not syncing from primary to the standby.

       

      The report I run is below , which is run at the primary database:

       

      SQL> ALTER SESSION SET nls_date_format='DD-MON-YYYY HH24:MI:SS';

      Session altered.

      SQL> SELECT sequence#, first_time, next_time,archived,applied,deleted FROM   v$archived_log where first_time > sysdate -1

      ORDER BY sequence#;

       

      SEQUENCE# FIRST_TIME                    NEXT_TIME                     ARC APPLIED   DEL                                                                           

      ---------- ----------------------------- ----------------------------- --- --------- ---                                                                           

           13280 23-JAN-2020 11:48:00          23-JAN-2020 14:03:18          YES NO        NO                                                                            

           13280 23-JAN-2020 11:48:00          23-JAN-2020 14:03:18          YES NO        NO                                                                            

           13281 23-JAN-2020 14:03:18          23-JAN-2020 14:31:40          YES NO        NO                                                                            

           13281 23-JAN-2020 14:03:18          23-JAN-2020 14:31:40          YES NO        NO                                                                            

           13282 23-JAN-2020 14:31:40          23-JAN-2020 14:34:21          YES NO        NO                                                                            

           13283 23-JAN-2020 14:34:21          23-JAN-2020 14:38:58          YES NO        NO                                                                            

           13284 23-JAN-2020 14:38:58          23-JAN-2020 14:42:54          YES NO        NO                                                                            

           13285 23-JAN-2020 14:42:54          23-JAN-2020 14:49:09          YES NO        NO                                                                            

           13286 23-JAN-2020 14:49:09          23-JAN-2020 14:54:50          YES NO        NO                                                                            

           13287 23-JAN-2020 14:54:50          23-JAN-2020 16:01:38          YES NO        NO                                                                            

           13288 23-JAN-2020 16:01:38          23-JAN-2020 18:58:15          YES NO        NO                                                                            

           13289 23-JAN-2020 18:58:15          23-JAN-2020 21:54:34          YES NO        NO                                                                            

           13290 23-JAN-2020 21:54:34          23-JAN-2020 23:31:16          YES NO        NO                                                                            

           13291 23-JAN-2020 23:31:16          24-JAN-2020 01:00:07          YES NO        NO                                                                            

           13291 23-JAN-2020 23:31:16          24-JAN-2020 01:00:07          YES NO        NO                                                                            

           13292 24-JAN-2020 01:00:07          24-JAN-2020 01:17:25          YES NO        NO                                                                            

           13292 24-JAN-2020 01:00:07          24-JAN-2020 01:17:25          YES NO        NO                                                                            

           13293 24-JAN-2020 01:17:25          24-JAN-2020 01:17:42          YES NO        NO                                                                            

           13293 24-JAN-2020 01:17:25          24-JAN-2020 01:17:42          YES NO        NO                                                                            

           13294 24-JAN-2020 01:17:42          24-JAN-2020 04:09:03          YES NO        NO                                                                            

           13294 24-JAN-2020 01:17:42          24-JAN-2020 04:09:03          YES NO        NO                                                                            

           13295 24-JAN-2020 04:09:03          24-JAN-2020 06:32:30          YES NO        NO                                                                            

           13296 24-JAN-2020 06:32:30          24-JAN-2020 08:12:33          YES NO        NO                                                                            

           13297 24-JAN-2020 08:12:33          24-JAN-2020 10:07:07          YES NO        NO                                                                            

       

      24 rows selected.

       

       

      But if I run below on both  primary and  standby , the archivelogs list is sync or updated.

       

      $ sqlplus sys/manager@PROD as sysdba

       

      SQL*Plus: Release 11.2.0.3.0 Production on Fri Jan 24 10:48:31 2020

      Copyright (c) 1982, 2011, Oracle.  All rights reserved.

       

      Connected to:

      Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production

      With the Partitioning, OLAP, Data Mining and Real Application Testing options

       

      SQL> archive log list;

      Database log mode              Archive Mode

      Automatic archival             Enabled

      Archive destination            /oracle/PROD/db/apps_st/data/archive

      Oldest online log sequence     13297

      Next log sequence to archive   13298

      Current log sequence           13298

      SQL>

       

       

      $ sqlplus sys/manager@STBY as sysdba

       

      SQL*Plus: Release 11.2.0.3.0 Production on Fri Jan 24 10:49:58 2020

      Copyright (c) 1982, 2011, Oracle.  All rights reserved.

       

      Connected to:

      Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production

      With the Partitioning, OLAP, Data Mining and Real Application Testing options

       

      SQL> archive log list;

      Database log mode              Archive Mode

      Automatic archival             Enabled

      Archive destination            /oracle/PROD/db/apps_st/data/archive

      Oldest online log sequence     13296

      Next log sequence to archive   0

      Current log sequence           13297

      SQL>

       

       

       

      Please help why I got wrong report?

       

       

      Kind regards,

      jc

        • 1. Re: EBS 12.1.3 Standby Db
          SMOOTHEDATH

          Hello BnB ,

           

          Could you please check below note ?

           

           

          Thanks & Regards

          Sohin

          • 2. Re: EBS 12.1.3 Standby Db
            Beauty_and_dBest

            Hi Sohin,

             

            Additional input in alert log.

             

             

            RFS[266]: Assigned to RFS process 30633

            RFS[266]: Opened log for thread 1 sequence 13289 dbid 343873040 branch 972901858

            Fri Jan 24 11:22:30 2020

            RFS[263]: Possible network disconnect with primary database

            Fri Jan 24 11:22:30 2020

            RFS[267]: Assigned to RFS process 30631

            RFS[267]: Opened log for thread 1 sequence 13290 dbid 343873040 branch 972901858

            Fri Jan 24 11:47:36 2020

            RFS[260]: Possible network disconnect with primary database

             

            Incremental checkpoint up to RBA [0x33f2.3dcf4.0], current log tail at RBA [0x33f2.4db68.0]

            Fri Jan 24 10:56:46 2020

            Immediate Kill Session#: 574, Serial#: 34343

            Immediate Kill Session: sess: 0x29ad40a50  OS pid: 18836

            Fri Jan 24 10:57:08 2020

            Immediate Kill Session#: 1650, Serial#: 175

            Immediate Kill Session: sess: 0x29b800cb0  OS pid: 26810

            Fri Jan 24 10:57:09 2020

            SMON: Parallel transaction recovery tried

            Fri Jan 24 10:57:49 2020

            Immediate Kill Session#: 597, Serial#: 30959

            Immediate Kill Session: sess: 0x29acfc238  OS pid: 24144

            Fri Jan 24 11:00:12 2020

            Incremental checkpoint up to RBA [0x33f2.f9bea.0], current log tail at RBA [0x33f2.11215d.0]

            Fri Jan 24 11:16:45 2020

            WARN: ARC1: Terminating pid 24807 hung on an I/O operation

            krsv_proc_kill: Killing 1 processes (Process by index)

            ARC1: Error 16198 due to hung I/O operation to LOG_ARCHIVE_DEST_2

            ARC1: Detected ARCH process failure

            ARC1: STARTING ARCH PROCESSES

            Fri Jan 24 11:16:47 2020

            ARC0 started with pid=149, OS id=29179

            ARC0: Archival started

            ARC1: STARTING ARCH PROCESSES COMPLETE

            Reclaiming FAL entry from dead process [pid 24807]

            Fri Jan 24 11:20:20 2020

            Incremental checkpoint up to RBA [0x33f2.152741.0], current log tail at RBA [0x33f2.16301f.0]

            Fri Jan 24 11:21:48 2020

            WARN: ARC1: Terminating pid 3314 hung on an I/O operation

            WARN: ARC1: Terminating pid 10297 hung on an I/O operation

            krsv_proc_kill: Killing 1 processes (Process by index)

            krsv_proc_kill: Killing 1 processes (Process by index)

            ARC1: Detected ARCH process failure

            ARC1: Detected ARCH process failure

            ARC1: STARTING ARCH PROCESSES

            Fri Jan 24 11:21:52 2020

            ARC2 started with pid=145, OS id=31275

            Fri Jan 24 11:21:52 2020

            ARC3 started with pid=146, OS id=31277

            ARC2: Archival started

            ARC2: Becoming the heartbeat ARCH

            ARC3: Archival started

            ARC1: STARTING ARCH PROCESSES COMPLETE

            Reclaiming FAL entry from dead process [pid 10297]

            Reclaiming FAL entry from dead process [pid 3314]

            Fri Jan 24 11:21:57 2020

            Thread 1 cannot allocate new log, sequence 13299

            Private strand flush not complete

              Current log# 2 seq# 13298 mem# 0: /oracle/PROD/db/apps_st/data/log02a.dbf

              Current log# 2 seq# 13298 mem# 1: /oracle/PROD/db/apps_st/data/log02b.dbf

            Beginning log switch checkpoint up to RBA [0x33f3.2.10], SCN: 5970238491168

            Thread 1 advanced to log sequence 13299 (LGWR switch)

              Current log# 1 seq# 13299 mem# 0: /oracle/PROD/db/apps_st/data/log01a.dbf

              Current log# 1 seq# 13299 mem# 1: /oracle/PROD/db/apps_st/data/log01b.dbf

            Fri Jan 24 11:22:15 2020

            Archived Log entry 34108 added for thread 1 sequence 13298 ID 0x16ee8c48 dest 1:

            Fri Jan 24 11:27:03 2020

            Completed checkpoint up to RBA [0x33f3.2.10], SCN: 5970238491168

            Fri Jan 24 11:40:43 2020

            Incremental checkpoint up to RBA [0x33f3.30aa5.0], current log tail at RBA [0x33f3.40dfe.0]

            Fri Jan 24 11:46:53 2020

            WARN: ARC1: Terminating pid 22537 hung on an I/O operation

            krsv_proc_kill: Killing 1 processes (Process by index)

            Fri Jan 24 12:00:57 2020

            Incremental checkpoint up to RBA [0x33f3.8276e.0], current log tail at RBA [0x33f3.98a1c.0]

             

             

             

            Please help...

            • 3. Re: EBS 12.1.3 Standby Db
              SMOOTHEDATH

              Hi BnB

               

              Please check below note

               

              ARCH: Krsv_hung_process_check: Error 16198 (Doc ID 2598884.1)

               

              Thanks & Regards

              Sohin

              • 4. Re: EBS 12.1.3 Standby Db
                Beauty_and_dBest

                Hi ALL,

                 

                SQL>  select process ,block#,sequence#,status from v$managed_standby;

                 

                PROCESS       BLOCK#  SEQUENCE# STATUS

                --------- ---------- ---------- ------------

                ARCH         1800192      13268 CLOSING

                ARCH               0          0 CONNECTED

                ARCH         1955840      13261 CLOSING

                ARCH         1017856      13256 CLOSING

                RFS              612      13300 RECEIVING

                MRP0               0      13260 WAIT_FOR_GAP

                RFS           389121      13290 RECEIVING

                RFS           387073      13295 RECEIVING

                RFS           456705      13289 RECEIVING

                 

                9 rows selected.

                 

                 

                Does this mean we miss the 13260 archive log? WAIT_FOR_GAP ?

                Do I need to copy it from backup?

                 

                Please help...

                 

                 

                Kind regards,