This discussion is archived
7 Replies Latest reply: Jan 27, 2013 12:13 AM by user2042891 RSS

Corrupt Datafiles following "Successful" Clone Process

user1175247 Newbie
Currently Being Moderated
Looking for help with the following problem/issue:

We attempted to clone our production database to our development environment using RMAN, following a process successfully used on multiple occasions in the past.

Both databases are running on UNIX and are versoin 11.2.0.2.7 (with JUL2012 PSU applied).

The RMAN cloning operation completed WITHOUT errors AND, the new database can be shutdown, started, and accessed via SYS logoin, but all attempts by users to access the databsae returns "corrupt datafile" message on most datafiles.

Below is what we are seeing in the alert log:

Tue Nov 06 19:52:39 2012
Hex dump of (file 230, block 26214385) in trace file /oracle/app/oracle/diag/rdbms/IDB/trace/IDB_j000_16046.trc
Corrupt block relative dba: 0x018ffff1 (file 230, block 26214385)
Completely zero block found during buffer read
Reading datafile '/oradata/IDB/undotbs_02.dbf' for corruption at rdba:
0x018ffff1 (file 230, bock 26214385)
Reread (file 230, block 26214385) found same corrupt data (no logical check)
Tue Nov 06 19:52:39 2012
Corrupt Block Found
TSN = 31, TSNAME = UNDOTBS_02
RFN = 1024, BLK = 26214385, RDBA = 26214385
OBJN = .1, OBJD = -1, OBJECT = , SUBONJECT =
SEGMENT_OWNER = , SEGMENT_TYPE =
Errors in file /oracle/app/oracle/diag/rdbms/IDB/trace/IDB_j000_16046.trc (incident 36257):
ORA-01578: ORACLE data block corrupted (file # 230, block # 26214385)
ORA-01110: data file 230: '/oradata/IDB/undotbs_02.dbf'
Incident details in /oracle/app/oracle/dia/rdbms/IDB/incident/incdir_36257/IDB_j000_16046_i36257.trc

Trace File Content:

Hex dump of (file 230, block 26214385)
Dump of memory from 0x00000004877BE000 to 0x00000004877C0000
4877BE000 00A20000 018FFFF1 00000000 00000101 [................]
4877BE010 00000000 00000000 00000000 00000000 [................]
Repeat 509 times
4877BFFF0 00000000 00000000 00000000 00000001 [................]
Corrupt block relative dba: 0x018ffff1 (file 230, block 26214385)
Completely zero block found during buffer read
Reading datafile /oradata/IDB/undotbs_02.dbf' for corruptoin at rdba:
0x018ffff1 (file 230, block 26214385)
Reread (file 230, block 26214385) found same corrupt data (no logical check)
DDE: Problem Key 'ORA 1110' was flood controlled (0x5) (no incdent)
ORA-01110: data file 230: '/oradata/IDB/undotbs_02.dbf'
Byte offset to file# 230 block 26214385 is 4294844416

This is the FIRST attempt to clone the production database since the installation of the JUL2012 patch -- and this is the ONLY change to all servers since the last successful cloning.

We have since attempted cloning between different environments (to rule out server specific issue) and cloning to new mount points (to rule out h/w issue), but eah attempt results in the same errors.

If anyone has experienced this same issue/problem, any insight is welcome. Anyone with viable options, ideas also welcome.

Our next approach will be to uninstall patches since our last "good" cloning and attempt again.

Thanks!

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points