This discussion is archived
3 Replies Latest reply: Jan 31, 2013 6:40 AM by 652072 RSS

RMAN backup failure

GeetaM Newbie
Currently Being Moderated
Hi Guru

RMAN backup is failing daily, i have googled it but still didnt find helpful, could you please assist me, below is the log file of RMAN -
+ /bin/sh -x bin/rman_dbf_bck.sh ***** ARCH INCR RAC_NO 
PARAMETER=
+ [ -z ***** ] 
PARAMETER= ***** 
+ shift 
+ [ -z ARCH ] 
PARAMETER= *****  ARCH 
+ shift 
+ [ -z INCR ] 
PARAMETER= *****  ARCH  INCR 
+ shift 
+ [ -z RAC_NO ] 
PARAMETER= *****  ARCH  INCR  RAC_NO 
+ shift 
+ [ -z  ] 
+ sudo -u oracle /export/home/oracle/source/Oracle/scripts/control-M/rman_dbf_bck.sh ***** ARCH INCR RAC_NO 

Tue Jan 29 19:00:13 EST 2013 ----------------Beginning of Script------------
Script name: rman_dbf_bck.sh

ORACLE_SID:  *****
HOST NAME:   dukeinfs01
ORACLE_HOME: /opt/u01/app/oracle/product/11.2.0.3/DB
DB_MODE:     ARCH
RMAN_LV:     INCR
RAC_YORN:    RAC_NO

/export/home/oracle/source/Oracle/scripts/control-M/rman_dbf_bck.sh: [[: not found
/export/home/oracle/source/Oracle/scripts/control-M/rman_dbf_bck.sh: [[: not found
/export/home/oracle/source/Oracle/scripts/control-M/rman_dbf_bck.sh: [[: not found
/export/home/oracle/source/Oracle/scripts/control-M/rman_dbf_bck.sh: [[: not found
COLD database backup requested for *****

%rman catalog cmdfile '/export/home/oracle/source/Oracle/scripts/control-M/rman_noarch_rac.cmd'

/export/home/oracle/source/Oracle/scripts/control-M/rman_dbf_bck.sh: [[: not found
RMAN> 2> 3> 4> 5> 6> 7> 8> 9> 10> 11> 12> 13> 14> 15> 16> 17> 18> 
Recovery Manager: Release 11.2.0.3.0 - Production on Tue Jan 29 19:00:13 2013

Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.

RMAN> connect target *
2> sql "alter session set optimizer_mode=RULE";
3> run {
4> 
5>      allocate channel t1 type 'sbt_tape';
6>      allocate channel t2 type 'sbt_tape';
7>      allocate channel t3 type 'sbt_tape';
8>      allocate channel t4 type 'sbt_tape';
9>      send 'NB_ORA_POLICY=RMAN_DATA,NB_ORA_SCHED=Full';
10> 
11>      backup as compressed backupset format 'FULL_%u_%d_%s' tag '%TAG' database;
12>      
13>      release channel t1;
14>      release channel t2;
15>      release channel t3;
16>      release channel t4;
17>     }
18> 
connected to target database: ***** (DBID=2975797407)

using target database control file instead of recovery catalog
sql statement: alter session set optimizer_mode=RULE

allocated channel: t1
channel t1: SID=987 device type=SBT_TAPE
channel t1: Veritas NetBackup for Oracle - Release 7.1 (2011020313)

allocated channel: t2
channel t2: SID=1093 device type=SBT_TAPE
channel t2: Veritas NetBackup for Oracle - Release 7.1 (2011020313)

allocated channel: t3
channel t3: SID=1388 device type=SBT_TAPE
channel t3: Veritas NetBackup for Oracle - Release 7.1 (2011020313)

allocated channel: t4
channel t4: SID=1486 device type=SBT_TAPE
channel t4: Veritas NetBackup for Oracle - Release 7.1 (2011020313)

sent command to channel: t1
sent command to channel: t2
sent command to channel: t3
sent command to channel: t4

Starting backup at 29-JAN-13
channel t1: starting compressed full datafile backup set
channel t1: specifying datafile(s) in backup set
input datafile file number=00002 name=/opt/u01/oradata/*****/sysaux01.dbf
input datafile file number=00021 name=/opt/u03/oradata/*****/pm_db01.dbf
input datafile file number=00018 name=/opt/u03/oradata/*****/pm_indx01.dbf
input datafile file number=00006 name=/opt/u03/oradata/*****/ems_indx02.dbf
input datafile file number=00009 name=/opt/u03/oradata/*****/ems_db02.dbf
input datafile file number=00010 name=/opt/u03/oradata/*****/ems_db03.dbf
channel t1: starting piece 1 at 29-JAN-13
channel t2: starting compressed full datafile backup set
channel t2: specifying datafile(s) in backup set
input datafile file number=00003 name=/opt/u01/oradata/*****/undotbs01.dbf
input datafile file number=00020 name=/opt/u04/oradata/*****/pm_db03.dbf
input datafile file number=00017 name=/opt/u02/oradata/*****/pm_indx03.dbf
input datafile file number=00005 name=/opt/u02/oradata/*****/ems_indx01.dbf
input datafile file number=00016 name=/opt/u02/oradata/*****/emsevent_db02.dbf
input datafile file number=00011 name=/opt/u02/oradata/*****/emsevent_indx03.dbf
channel t2: starting piece 1 at 29-JAN-13
channel t3: starting compressed full datafile backup set
channel t3: specifying datafile(s) in backup set
input datafile file number=00022 name=/opt/u02/oradata/*****/pm_db02.dbf
input datafile file number=00019 name=/opt/u04/oradata/*****/pm_indx02.dbf
input datafile file number=00008 name=/opt/u04/oradata/*****/ems_db01.dbf
input datafile file number=00012 name=/opt/u04/oradata/*****/emsevent_indx01.dbf
input datafile file number=00015 name=/opt/u04/oradata/*****/emsevent_db01.dbf
input datafile file number=00007 name=/opt/u04/oradata/*****/ems_indx03.dbf
channel t3: starting piece 1 at 29-JAN-13
channel t4: starting compressed full datafile backup set
channel t4: specifying datafile(s) in backup set
input datafile file number=00001 name=/opt/u01/oradata/*****/system01.dbf
input datafile file number=00004 name=/opt/u01/oradata/*****/users01.dbf
input datafile file number=00013 name=/opt/u03/oradata/*****/emsevent_indx02.dbf
input datafile file number=00014 name=/opt/u03/oradata/*****/emsevent_db03.dbf
channel t4: starting piece 1 at 29-JAN-13
*RMAN-03009: failure of backup command on t3 channel at 01/29/2013 19:01:36*
*ORA-19506: failed to create sequential file, name="FULL_48o0lcqe_*****_3208", parms=""*
*ORA-27028: skgfqcre: sbtbackup returned error*
*ORA-19511: Error received from media manager layer, error text:*
   *VxBSACreateObject: Failed with error:*
   *Server Status:  problems encountered during setup of shared memory*
*channel t3 disabled, job failed on it will be run on another channel*
*RMAN-03009: failure of backup command on t4 channel at 01/29/2013 19:01:36*
*ORA-19506: failed to create sequential file, name="FULL_49o0lcqf_*****_3209", parms=""*
*ORA-27028: skgfqcre: sbtbackup returned error*
*ORA-19511: Error received from media manager layer, error text:*
   *VxBSACreateObject: Failed with error:*
   *Server Status:  problems encountered during setup of shared memory*
*channel t4 disabled, job failed on it will be run on another channel*
channel t2: finished piece 1 at 29-JAN-13
piece handle=FULL_47o0lcqe_*****_3207 tag=%TAG comment=API Version 2.0,MMS Version 5.0.0.0
channel t2: backup set complete, elapsed time: 01:59:23
channel t2: starting compressed full datafile backup set
channel t2: specifying datafile(s) in backup set
input datafile file number=00022 name=/opt/u02/oradata/*****/pm_db02.dbf
input datafile file number=00019 name=/opt/u04/oradata/*****/pm_indx02.dbf
input datafile file number=00008 name=/opt/u04/oradata/*****/ems_db01.dbf
input datafile file number=00012 name=/opt/u04/oradata/*****/emsevent_indx01.dbf
input datafile file number=00015 name=/opt/u04/oradata/*****/emsevent_db01.dbf
input datafile file number=00007 name=/opt/u04/oradata/*****/ems_indx03.dbf
channel t2: starting piece 1 at 29-JAN-13
channel t1: finished piece 1 at 29-JAN-13
piece handle=FULL_46o0lcqe_*****_3206 tag=%TAG comment=API Version 2.0,MMS Version 5.0.0.0
channel t1: backup set complete, elapsed time: 03:13:51
channel t1: starting compressed full datafile backup set
channel t1: specifying datafile(s) in backup set
input datafile file number=00001 name=/opt/u01/oradata/*****/system01.dbf
input datafile file number=00004 name=/opt/u01/oradata/*****/users01.dbf
input datafile file number=00013 name=/opt/u03/oradata/*****/emsevent_indx02.dbf
input datafile file number=00014 name=/opt/u03/oradata/*****/emsevent_db03.dbf
channel t1: starting piece 1 at 29-JAN-13
channel t2: finished piece 1 at 29-JAN-13
piece handle=FULL_4ao0ljqa_*****_3210 tag=%TAG comment=API Version 2.0,MMS Version 5.0.0.0
channel t2: backup set complete, elapsed time: 01:20:53
channel t2: starting compressed full datafile backup set
channel t2: specifying datafile(s) in backup set
including current control file in backup set
channel t2: starting piece 1 at 29-JAN-13
channel t2: finished piece 1 at 29-JAN-13
piece handle=FULL_4co0loi0_*****_3212 tag=%TAG comment=API Version 2.0,MMS Version 5.0.0.0
channel t2: backup set complete, elapsed time: 00:00:25
channel t2: starting compressed full datafile backup set
channel t2: specifying datafile(s) in backup set
including current SPFILE in backup set
channel t2: starting piece 1 at 29-JAN-13
channel t2: finished piece 1 at 29-JAN-13
piece handle=FULL_4do0loj1_*****_3213 tag=%TAG comment=API Version 2.0,MMS Version 5.0.0.0
channel t2: backup set complete, elapsed time: 00:00:25
channel t1: finished piece 1 at 29-JAN-13
piece handle=FULL_4bo0lo5u_*****_3211 tag=%TAG comment=API Version 2.0,MMS Version 5.0.0.0
channel t1: backup set complete, elapsed time: 00:11:24
Finished backup at 29-JAN-13

released channel: t1

released channel: t2

released channel: t3
*released channel: t4*
*RMAN-00571: ===========================================================*
*RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============*
*RMAN-00571: ===========================================================*
*RMAN-03002: failure of release command at 01/29/2013 22:25:47*
*RMAN-06012: channel: t3 not allocated*

Recovery Manager complete.

no rows selected

*ORA-27028: skgfqcre: sbtbackup returned error*
*ORA-19511: Error received from media manager layer, error text:*
   *VxBSACreateObject: Failed with error:*
*ORA-27028: skgfqcre: sbtbackup returned error*
*ORA-19511: Error received from media manager layer, error text:*
   *VxBSACreateObject: Failed with error:*
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============

Tue Jan 29 22:25:53 EST 2013 ----------------End of Script------------------

exit 0


Remote Host connection details
==============================
Control-M/Agent: ********.corp.*****.com
Protocol: SSH
SSH Port: 00022
SSH Algorithm: BLOWFISH
Compression: N
  • 1. Re: RMAN backup failure
    clementeOTN Explorer
    Currently Being Moderated
    Hi
    It seems you have a problem with netbackup integration, especially with drives t3 and t4 .
    Have you seen any aplication/operating system error in the netbackup server ?

    Errors must be readed from "down to up",so, it seems that RMAN tells you he had a memory problem reported by netbackup ( Error received from media manager layer,)

    oooo--oo--oo--oo--oo--oo--oo--oo--oo--oo--oo--oo--oo--oo--oo--oo--oo
    RMAN-03009: failure of backup command on t3 channel at 01/29/2013 19:01:36
    ORA-19506: failed to create sequential file, name="FULL_48o0lcqe_****_3208", parms=""*
    *ORA-27028: skgfqcre: sbtbackup returned error*
    *ORA-19511: Error received from media manager layer, error text:*
    *VxBSACreateObject: Failed with error:*
    *Server Status: problems encountered during setup of shared memory*

    --oo--oo--oo--oo--oo--oo--oo--oo--oo--oo--oo--oo--oo--oo--oo--oo--oo--oo
  • 2. Re: RMAN backup failure
    GeetaM Newbie
    Currently Being Moderated
    Thank you for your quick response, let me check with my netbackup team and see what they have to say on it.
  • 3. Re: RMAN backup failure
    652072 Newbie
    Currently Being Moderated
    Hello, This is a shared memory issue.. Increasing the shared memory should fix the problem.... Contact your Tape Library Vendor Team...

    Regards,
    Naga

    Edited by: nagnrik on Jan 31, 2013 5:39 PM

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points