2 Replies Latest reply: Sep 13, 2012 3:48 AM by 961879 RSS

    Problem during ingestion: KCF: read, write or open error, block=0x32a0ac on

    961879
      Hi,
      I started a big ingestion yesterday but during the night I received this error:

      Thu Sep 13 05:57:49 2012
      KCF: read, write or open error, block=0x32a0ac online=1
      file=7 '+DATAREP/dpctrdb/datafile/mdbe2es3.274.789498073'
      error=17510 txt: ''
      Automatic datafile offline due to write error on
      file 7: +DATAREP/dpctrdb/datafile/mdbe2es3.274.789498073
      Thu Sep 13 05:57:50 2012
      KCF: read, write or open error, block=0x32a917 online=1
      file=7 '+DATAREP/dpctrdb/datafile/mdbe2es3.274.789498073'
      error=17510 txt: ''
      Automatic datafile offline due to write error on
      file 7: +DATAREP/dpctrdb/datafile/mdbe2es3.274.789498073
      Thu Sep 13 05:57:52 2012
      Reconfiguration started (old inc 2, new inc 4)
      List of instances:
      3 (myinst: 3)
      Global Resource Directory frozen
      * dead instance detected - domain 0 invalid = TRUE
      Communication channels reestablished
      Master broadcasted resource hash value bitmaps
      Non-local Process blocks cleaned out
      Thu Sep 13 05:57:52 2012
      LMS 2: 1 GCS shadows cancelled, 0 closed, 0 Xw survived
      Thu Sep 13 05:57:52 2012
      LMS 1: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
      Thu Sep 13 05:57:52 2012
      LMS 0: 1 GCS shadows cancelled, 0 closed, 0 Xw survived
      NOTE: delay file unidentification for gn=3 fn=274 cnt=74
      Set master node info
      Submitted all remote-enqueue requests
      Dwn-cvts replayed, VALBLKs dubious
      All grantable enqueues granted
      Post SMON to start 1st pass IR
      Submitted all GCS remote-cache requests
      Post SMON to start 1st pass IR
      Fix write in gcs resources
      Thu Sep 13 05:57:53 2012
      Instance recovery: looking for dead threads
      Beginning instance recovery of 2 threads
      Reconfiguration complete
      Thu Sep 13 05:57:53 2012
      NOTE: completing the delayed file unidentification for gn=3 fn=274
      KCF: read, write or open error, block=0x32a551 online=0
      KCF: read, write or open error, block=0x32a550 online=0
      file=7 'MISSING0'
      error=17510 txt: ''
      file=7 'MISSING0'
      error=17510 txt: ''
      KCF: read, write or open error, block=0x32540e online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x32540a online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x32542d online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x325421 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x325402 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x325300 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x32207e online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x32207a online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x322076 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x32203e online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x321e74 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x321d35 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x1307b9 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x1307a9 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x1138fd online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x32530c online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x11389c online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x32546f online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x325473 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x32546b online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x3253fe online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x3253fa online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x3253f2 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x3253ee online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x3253e2 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x3253df online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x3253db online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x3253d3 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x3252fb online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x3252f3 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x3252ef online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x3252d3 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x3252cf online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x3252b1 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x3252e3 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x3249cd online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x3249bd online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x321ffc online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x3252df online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x3253cb online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x3253a2 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x32539e online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x321db4 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x321df6 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x3220b3 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x325392 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x3220b7 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x325381 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x325380 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x32538c online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x3220bb online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x3220bf online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x3220f6 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x329a13 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x329a17 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x329a1e online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x32a039 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x3220fe online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x3220fa online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x17b70f online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x325552 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x325502 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x325512 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x325524 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x11391d online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x322e16 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x325542 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x321eb4 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x321ef4 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x322db6 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x322dc6 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x322dd6 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x329cd8 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x325494 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x3254d3 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x3254de online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x3254fe online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      KCF: read, write or open error, block=0x3254e2 online=0
      file=7 'MISSING0'
      error=15079 txt: ''
      parallel recovery started with 31 processes
      Started redo scan
      Thu Sep 13 05:58:06 2012
      Completed redo scan
      read 1197829 KB redo, 31941 data blocks need recovery
      Started redo application at
      Thread 1: logseq 596, block 1227268
      Thread 2: logseq 295, block 96
      Recovery of Online Redo Log: Thread 1 Group 2 Seq 596 Reading mem 0
      Mem# 0: +DATAREP/dpctrdb/onlinelog/group_2.258.788789245
      Mem# 1: +RECOVERYREP/dpctrdb/onlinelog/group_2.258.788789257
      Thu Sep 13 05:58:13 2012
      ALTER SYSTEM SET service_names='DPCTRDBService3','DPCTRDBService','DPCTRDBService1' SCOPE=MEMORY SID='DPCTRDB3';
      Thu Sep 13 05:58:13 2012
      Shutting down instance (abort)
      License high water mark = 63
      USER (ospid: 1101): terminating the instance
      Recovery of Online Redo Log: Thread 2 Group 3 Seq 295 Reading mem 0
      Mem# 0: +DATAREP/dpctrdb/onlinelog/group_3.270.788792613
      Mem# 1: +RECOVERYREP/dpctrdb/onlinelog/group_3.263.788792623
      Thu Sep 13 05:58:13 2012
      opiodr aborting process unknown ospid (27826) as a result of ORA-1092
      Instance terminated by USER, pid = 1101
      Thu Sep 13 05:58:14 2012
      Instance shutdown complete
      Thu Sep 13 05:58:28 2012
      Adjusting the default value of parameter parallel_max_servers
      from 320 to 285 due to the value of parameter processes (300)
      Starting ORACLE instance (normal)
      Thu Sep 13 05:58:39 2012

      Does someone know the cause of the problem?
      It's a serious problem for me.
      Thanks, bye.
        • 1. Re: Problem during ingestion: KCF: read, write or open error, block=0x32a0ac on
          kuljeet singh -
          KCF: read, write or open error, block=0x32a0ac online=1
          file=7 '+DATAREP/dpctrdb/datafile/mdbe2es3.274.789498073'
          error=17510 txt: ''
          Automatic datafile offline due to write error on
          oracle is automatically offline the datafile due to write error.
          can you check is +DATAREP diskgroup mounted.
          Also check asm log for any error
          • 2. Re: Problem during ingestion: KCF: read, write or open error, block=0x32a0ac on
            961879
            The enterprise manager said the the datafile needs recovery but when I execute the recover I receive:
            638.1497.793881221 thread=3 sequence=638
            RMAN-00571: ===========================================================
            RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
            RMAN-00571: ===========================================================
            RMAN-03002: failure of recover command at 09/13/2012 10:45:57
            ORA-00283: recovery session canceled due to errors
            RMAN-11003: failure during parse/execution of SQL statement: alter database recover logfile '+RECOVERYREP/dpctrdb/archivelog/2012_09_13/thread_3_seq_638.1497.793881221'
            ORA-00283: recovery session canceled due to errors
            ORA-10562: Error occurred while applying redo to data block (file# 7, block# 3282782)
            ORA-10564: tablespace MDBE2ES3TBS
            ORA-01110: data file 7: '+DATAREP/dpctrdb/datafile/mdbe2es3.274.789498073'
            ORA-10560: block type '0'
            ORA-00600: internal error code, arguments: [4553], [2], [0], [], [], [], [], [], [], [], [], []