This discussion is archived
11 Replies Latest reply: Mar 7, 2013 11:22 PM by thodi RSS

RMAN + Veritas NetBackup: Backups are marked as EXPIRED immediately

thodi Newbie
Currently Being Moderated
Hi,

I may be missing something obvious here. I am taking RMAN backups using Veritas NetBackup, the NetBackup policy uses "inifinity" retention period, RMAN is configured to use a recovery windows fo 31 days (this is a NetBackup best practice for managing expiry). Backups run fine and after they're done, I see lots of reclaimable space in my FRA, but as soon as I run "crosscheck backup", every backup I have ever done is marked EXPIRED.

RMAN settings:
RMAN> show all;

starting full resync of recovery catalog
full resync complete
RMAN configuration parameters for database with db_unique_name BOPA are:
CONFIGURE RETENTION POLICY TO RECOVERY WINDOW OF 31 DAYS;
CONFIGURE BACKUP OPTIMIZATION ON;
CONFIGURE DEFAULT DEVICE TYPE TO DISK; # default
CONFIGURE CONTROLFILE AUTOBACKUP OFF;
CONFIGURE CONTROLFILE AUTOBACKUP FORMAT FOR DEVICE TYPE SBT_TAPE TO '%F'; # default
CONFIGURE CONTROLFILE AUTOBACKUP FORMAT FOR DEVICE TYPE DISK TO '%F'; # default
CONFIGURE DEVICE TYPE 'SBT_TAPE' PARALLELISM 3 BACKUP TYPE TO BACKUPSET;
CONFIGURE DEVICE TYPE DISK PARALLELISM 1 BACKUP TYPE TO BACKUPSET; # default
CONFIGURE DATAFILE BACKUP COPIES FOR DEVICE TYPE SBT_TAPE TO 1; # default
CONFIGURE DATAFILE BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
CONFIGURE ARCHIVELOG BACKUP COPIES FOR DEVICE TYPE SBT_TAPE TO 1; # default
CONFIGURE ARCHIVELOG BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
CONFIGURE MAXSETSIZE TO UNLIMITED; # default
CONFIGURE ENCRYPTION FOR DATABASE OFF; # default
CONFIGURE ENCRYPTION ALGORITHM 'AES128'; # default
CONFIGURE COMPRESSION ALGORITHM 'BZIP2'; # default
CONFIGURE ARCHIVELOG DELETION POLICY TO NONE; # default
CONFIGURE SNAPSHOT CONTROLFILE NAME TO '/u01/app/oracle/product/11.1.0/db_1/dbs/snapcf_bopa2.f'; # default
Crosscheck:
RMAN> crosscheck backup;
[...]
crosschecked backup piece: found to be 'EXPIRED'
backup piece handle=al_2298_1_809254750 RECID=2269 STAMP=809254789
Crosschecked 19 objects
The backup with handle al_2298_1_809254750 was taken last night, for example, with the RMAN settings listed above.

Any ideas? My FRA has grown to 1.5TB, I have disabled automatic crosscheck for now -- but I will need to reenable it.

Oracle 11.1.0.7 SE, 2 node Linux RAC, NetBackup 7.5
  • 1. Re: RMAN + Veritas NetBackup: Backups are marked as EXPIRED immediately
    mseberg Guru
    Currently Being Moderated
    Hello;

    Its looks like your "DEFAULT DEVICE TYPE" is Disk. RMAN will mark "Expired" any backup no longer exists on disk, but is still listed in the repository.

    CROSSCHECK command updates the repository status for a backup to EXPIRED when it cannot locate it at the location to which it was backed up.

    So if a backup was deleted from disk that you have on tape this would occur.

    Don't overuse CROSSCHECK. CROSSCHECK compares the RMAN catalog entries. This is useful when RMAN thinks a file is missing. If you need confirm recovery catalog is synchronized with corresponding data on disk or in the media then use it. Otherwise if you have no mismatched objects all you do is change the status of files to "EXPIRED".

    That said you still want to run CROSSCHECK frequently enough so that the LIST EXPIRED BACKUP command is still useful. If you manually delete any kind of RMAN related backup file at the OS level, then you will need to use CROSSCHECK at some point.

    If you use a backup retention policy and an archive log deletion policy with FRA maintenance becomes almost automatic and then there's no need to constantly execute CROSSCHECK to verify your backups.

    Best Regards

    mseberg
  • 2. Re: RMAN + Veritas NetBackup: Backups are marked as EXPIRED immediately
    Stefan Abraham Newbie
    Currently Being Moderated
    Hi

    Seems you have a confusion between "obsolete" and "Expired"
    check with report obsolete.
    It will show the backups available according to your retention policy.

    Thanks
    Stefan
  • 3. Re: RMAN + Veritas NetBackup: Backups are marked as EXPIRED immediately
    Hemant K Chitale Oracle ACE
    Currently Being Moderated
    Backups run fine and after they're done, I see lots of reclaimable space in my FRA
    Are the backups going to FRA (on disk) or tape directly ? Does the reclaimable space in the FRA change when you run a backup ?


    Are you running the CROSSCHECK command when connected to NetBackup (with the NetBackup Environment parameters configured) ? Else it would not find backuppieces written to NetBackup if it is not connected to NetBackup.


    Hemant K Chitale
  • 4. Re: RMAN + Veritas NetBackup: Backups are marked as EXPIRED immediately
    thodi Newbie
    Currently Being Moderated
    FYI: I won't be back at the office until tomorrow. Thank you for all the insight so far, I'll get back to you.
  • 5. Re: RMAN + Veritas NetBackup: Backups are marked as EXPIRED immediately
    thodi Newbie
    Currently Being Moderated
    Hemant K Chitale wrote:
    Backups run fine and after they're done, I see lots of reclaimable space in my FRA
    Are the backups going to FRA (on disk) or tape directly ? Does the reclaimable space in the FRA change when you run a backup ?
    The backups are going directly to tape. After running a backup, I see reclaimable space à la:
    FILE_TYPE                        PERCENT_SPACE_USED    GB_USED GB_RECLAIMABLE NUMBER_OF_FILES
    -------------------------------- ------------------ ---------- -------------- ---------------
    ARCHIVED LOG                                  35.83      716.6          709.6           10390
    BACKUP PIECE                                  22.85        457              0              70
    IMAGE COPY                                    18.42      368.4            214              16
    TOTAL                                          77.1       1542          923.6           10476
    But after running the crosscheck, all reclaimable columns are back to zero.
    Are you running the CROSSCHECK command when connected to NetBackup (with the NetBackup Environment parameters configured) ? Else it would not find backuppieces written to NetBackup if it is not connected to NetBackup.
    I ran 'allocate channel for maintenance type sbt_tape;' before doing the crosscheck.
  • 6. Re: RMAN + Veritas NetBackup: Backups are marked as EXPIRED immediately
    thodi Newbie
    Currently Being Moderated
    Stefan Abraham wrote:
    Seems you have a confusion between "obsolete" and "Expired"
    check with report obsolete.
    It will show the backups available according to your retention policy.
    report obsolete returns that no backups are obsolete at the moment, you are right. I guess I will have to wait what happens after 31 days :-)
  • 7. Re: RMAN + Veritas NetBackup: Backups are marked as EXPIRED immediately
    mseberg Guru
    Currently Being Moderated
    Hello Again;

    This might be worth a look too.

    'Crosscheck Backup' incorrectly marks 'EXPIRED' eventhough backup files are physically present in TAPE drive [ID 1124254.1]

    Best Regards

    mseberg
  • 8. Re: RMAN + Veritas NetBackup: Backups are marked as EXPIRED immediately
    thodi Newbie
    Currently Being Moderated
    mseberg wrote:
    Hello;

    Its looks like your "DEFAULT DEVICE TYPE" is Disk. RMAN will mark "Expired" any backup no longer exists on disk, but is still listed in the repository.

    CROSSCHECK command updates the repository status for a backup to EXPIRED when it cannot locate it at the location to which it was backed up.
    I ran 'allocate channel for maintenance type sbt_tape;' before the crosscheck, I was under the impression that this would connect me to the NetBackup database. I forgot to mention that in my initial post.

    Don't overuse CROSSCHECK. CROSSCHECK compares the RMAN catalog entries. This is useful when RMAN thinks a file is missing. If you need confirm recovery catalog is synchronized with corresponding data on disk or in the media then use it. Otherwise if you have no mismatched objects all you do is change the status of files to "EXPIRED".

    That said you still want to run CROSSCHECK frequently enough so that the LIST EXPIRED BACKUP command is still useful. If you manually delete any kind of RMAN related backup file at the OS level, then you will need to use CROSSCHECK at some point.

    If you use a backup retention policy and an archive log deletion policy with FRA maintenance becomes almost automatic and then there's no need to constantly execute CROSSCHECK to verify your backups.
    I think I may have had a misconception about corsscheck. I thought it was the way to actually expire backup images. I will have to reread a lot, I guess. The funny thing is this scheme worked flawlessly in another environment.
  • 9. Re: RMAN + Veritas NetBackup: Backups are marked as EXPIRED immediately
    thodi Newbie
    Currently Being Moderated
    mseberg wrote:
    This might be worth a look too.

    'Crosscheck Backup' incorrectly marks 'EXPIRED' eventhough backup files are physically present in TAPE drive [ID 1124254.1]
    Thanks, I will add a "show channel" command to my backup script and see what it gives. Running it in rman by hand gives
    RMAN> show channel;
    
    RMAN configuration parameters for database with db_unique_name BOPA are:
    RMAN configuration has no stored or default parameters
    
    RMAN> 
  • 10. Re: RMAN + Veritas NetBackup: Backups are marked as EXPIRED immediately
    Hemant K Chitale Oracle ACE
    Currently Being Moderated
    I ran 'allocate channel for maintenance type sbt_tape;' before doing the crosscheck.
    So, in theory, you are running the CROSSCHECK against Tape.
    But after running the crosscheck, all reclaimable columns are back to zero
    However, the Reclaimable Space on Disk seems to be getting cleared.
    After running a backup, I see reclaimable space
    I presume that you do not have a DELETE INPUT in your BACKUP ARCHIVELOG command ?


    I think you shouldn't be running CROSSCHECK against the tape.
    You might run CROSSCHECK against disk.


    Hemant K Chitale
  • 11. Re: RMAN + Veritas NetBackup: Backups are marked as EXPIRED immediately
    thodi Newbie
    Currently Being Moderated
    Hemant K Chitale wrote:
    I ran 'allocate channel for maintenance type sbt_tape;' before doing the crosscheck.
    So, in theory, you are running the CROSSCHECK against Tape.
    That's what I thought as well.
    But after running the crosscheck, all reclaimable columns are back to zero
    However, the Reclaimable Space on Disk seems to be getting cleared.
    No, the space remains allocated, it's just not reclaimable:

    Before cosscheck:
    FILE_TYPE                        PERCENT_SPACE_USED    GB_USED GB_RECLAIMABLE NUMBER_OF_FILES
    -------------------------------- ------------------ ---------- -------------- ---------------
    ARCHIVED LOG                                  44.83     672.45         667.35            9810
    BACKUP PIECE                                  27.96      419.4          418.2              58
    IMAGE COPY                                    24.54      368.1          368.1              16
    TOTAL                                         97.33    1459.95        1453.65            9884
    After crosscheck:
    FILE_TYPE                        PERCENT_SPACE_USED    GB_USED GB_RECLAIMABLE NUMBER_OF_FILES
    -------------------------------- ------------------ ---------- -------------- ---------------
    ARCHIVED LOG                                  44.91     673.65              0            9830
    BACKUP PIECE                                  27.96      419.4              0              58
    IMAGE COPY                                    24.56      368.4              0              16
    TOTAL                                         97.43    1461.45              0            9904
    After running a backup, I see reclaimable space
    I presume that you do not have a DELETE INPUT in your BACKUP ARCHIVELOG command ?
    Nope. I want as little interference with FRA contents as possible :-)
    I think you shouldn't be running CROSSCHECK against the tape.
    You might run CROSSCHECK against disk.
    I shall refrain from running crosscheck and just see what "report obsolete" tells me after 31 days of backups.

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points