Forum Stats

  • 3,825,934 Users
  • 2,260,581 Discussions
  • 7,896,738 Comments

Discussions

How does RMAN react when backup volume is out of space

4187794
4187794 Member Posts: 23
edited Feb 17, 2020 11:33AM in Recovery Manager (RMAN)

Our Lead DBA has fallen ill, and I'm having to get my learn on fast.. I am currently seeing a retention policy or redunancy 1, which I'm not thinking I'm a big fan of, but anyway - i'm trying to just understand a couple things before I delete obsolete -( because we are completely out of space.) What does RMAN do when the volume it is trying to backup to is full? Does it overwrite oldest data or not backup at all or something else?

Thank you,

EdStevens4187794evamcdaz

Best Answer

  • Dude!
    Dude! Member Posts: 22,829 Black Diamond
    edited Feb 12, 2020 4:58PM Answer ✓
    d82406a4-26a5-4471-bb0d-60b49ea94b11 wrote:Our Lead DBA has fallen ill, and I'm having to get my learn on fast.. 

    Oracle and RMAN is way to complex to get anything learn fast.

    What happens when you run out of space depends on whether you have configured a Fast Recovery Area (FRA). I cannot guess your configuration. However, RMAN does not overwrite existing data.

«134

Answers

  • Dude!
    Dude! Member Posts: 22,829 Black Diamond
    edited Feb 12, 2020 4:58PM Answer ✓
    d82406a4-26a5-4471-bb0d-60b49ea94b11 wrote:Our Lead DBA has fallen ill, and I'm having to get my learn on fast.. 

    Oracle and RMAN is way to complex to get anything learn fast.

    What happens when you run out of space depends on whether you have configured a Fast Recovery Area (FRA). I cannot guess your configuration. However, RMAN does not overwrite existing data.

  • 4187794
    4187794 Member Posts: 23
    edited Feb 12, 2020 5:13PM

    Right, I know you can't guess, but an if this then that would have done fine.  And I'm aware of the complexities of all this, my eyes are bleeding from a week of constant reading every waking hour and maintaining things the best I can.  I have to start somewhere, we have no choice - he is in a coma and this is week 3. I'm doing well and resolved all issues and we've had so far.

    Anyway, thanks..

  • EdStevens
    EdStevens Member Posts: 28,778 Gold Crown
    edited Feb 12, 2020 5:24PM
    d82406a4-26a5-4471-bb0d-60b49ea94b11 wrote:Our Lead DBA has fallen ill, and I'm having to get my learn on fast.. I am currently seeing a retention policy or redunancy 1, which I'm not thinking I'm a big fan of, but anyway - i'm trying to just understand a couple things before I delete obsolete -( because we are completely out of space.) What does RMAN do when the volume it is trying to backup to is full? Does it overwrite oldest data or not backup at all or something else? Thank you,

    As already stated, rman will not overwrite something simply because it is out of space.  Just like any other write operation, it will fail and throw an error message.  You seem hesitant about issuing

    rman> delete obsolete;

    You should not be afraid of that.  Rman will delete all backup pieces that it deems are no longer needed to enforce the retention policy, though that may not be the same files as you think.  Just trust rman to know best what it might need in case of recovery.  (I agree, I am not a fan of redundancy-based retention policy.  Much better, in my estimation, in the vast majority of cases, to have recovery window of 'n' days.

    You are in a very difficult position.  It would help us to help you if you would share:

    1) a little of your technical background, training, experience - to set expectations.

    2) your version and edition of oracle

    3) the OS oracle is running on

    4) your current backup script and schedule

    5) if the database is in archivelog mode or not

    6) the result of SQL> show parameter recovery

  • Dude!
    Dude! Member Posts: 22,829 Black Diamond
    edited Feb 12, 2020 5:28PM

    FRA or Oracle Fast Recovery Area is an Oracle managed space that will provide several measures to avoid problems under disk space pressure, such as, automatically removing obsolete data. Typically, FRA may cause your database to halt any operation when it runs out of space to archive redo logs. If you are not using FRA, changes are backup will fail when running out of backup space and you have to intervene to fix the problem.

    4187794
  • Dude!
    Dude! Member Posts: 22,829 Black Diamond
    edited Feb 12, 2020 5:43PM

    Btw... RMAN by default does not check for logical corruption. A redundancy of 1 and deleting obsolete data can increase your inability to fix potential data corruption - unless you also have flashback logs using Flashback archive or database.

    If you don't know what can be removed, perhaps you can move the backup data, or some of it, to a different storage media. RMAN will still have recordings of the metadata until you run a RMAN crosscheck.

    Also keep in mind that the default control_file_record_keep_time is 7 days by default, which defines the minimum, after which time reusable RMAN records (medatada) are overwritten, hence no longer known, although still on disk, and no longer subject to your retention policy. Perhaps before doing a RMAN obsolete, do RMAN crosscheck first.

  • 4187794
    4187794 Member Posts: 23
    edited Feb 12, 2020 6:01PM

    1) a little of your technical background, training, experience - to set expectations.

    Eh, been in IT about 15 years, through a number of positions starting at Desktop Support and ending with Info Security Analyst until recently. I work a little with a whole lot of technologies, linux and powershell more recently (that I at least enjoy). Have a Networking degree, security and networking plus, blah blah blah. I'm just a jack of all trades guy.

    2) your version and edition of oracle

    19.3 in this case, we do have some stuff still running on 11.  Enterprise Edition

    3) the OS oracle is running on

    RHEL

    4) your current backup script and schedule

    Incremental, not sure the exact schedule

    5) if the database is in archivelog mode or not

    it is, automatic enabled

    6) the result of SQL> show parameter recovery

    NAME                                 TYPE        VALUE

    ------------------------------------ ----------- ------------------------------

    db_recovery_file_dest                string      /f5/app/oracle/fast_recovery_area

    db_recovery_file_dest_size           big integer 17271M

    recovery_parallelism                 integer     0

    remote_recovery_file_dest            string

    ....as for delete obsolete - yes I've been waiting all day to pull the trigger on that on our couple / few biggest contributors.  It finds a lot. Even when I change it to recovery window of say, 30 days, and crosscheck it still reports a number of full backups that could go. I have to do something, (that), today though, as we filled up our volumne and I added 500gb more and it filled up that night (last night). So, its time to do some cleanup.

    We have a few CDBs and a few PDBs. learning how to work with them all is interesting. One thing I do find frustrating is when I connect to one and rman says its connected but (not started). The database indeed is started and functioning fine, Toad and sql developer lot in fine and our applications are fine.

    I don't know why we are on redundancy 1. I don't like it, and want to change to 30 days so we're at least a lot clearer on what that means and what we can do for customers.

    This all started (for me) when databases went down and noone knew what to do so I dove in. We ended up cancelling our support ticket and I've been on it ever since. Only been a good week or week and a half, but I'm talking 16 hour days just.. getting my hands dirty and researching. If nothing else, i've learned a ton about our infrastructure and can at least identify a few things that are worth thinking about. Like turning optimization on, for example.

  • 4187794
    4187794 Member Posts: 23
    edited Feb 12, 2020 6:06PM

    I was hoping to find proof that he was going in regularly and clearning out obsolete,  I don't see how it could have been any other way, but I filled up his bash history trying to get myself the rights I needed to this and other tasks when this first started and I was sorting through just getting INTO sqlplus and rman.

  • Dude!
    Dude! Member Posts: 22,829 Black Diamond
    edited Feb 12, 2020 6:41PM

    Redundancy 1 is the RMAN configuraiton default to have basic functions available. I see that you are using FRA and have specified a maximum size of 17 GB. How large is the database? It's a common misconceptions to have FRA too small. FRA should be at least 3 times the size of a full database backup to get reasonable results, and to allow Oracle to maintain the FRA space and automatically purge obsolete data under space pressure.

    Don't confuse Oracle FRA with RMAN though. RMAN never deletes any data unless you instruct it to do so, for example, RMAN delete obsolete, or delete expired. The later only deletes RMAN metadata, in case physical data is no longer available.

  • 4187794
    4187794 Member Posts: 23
    edited Feb 12, 2020 6:51PM

    SELECT SUM (bytes) / 1024 / 1024 / 1024 AS GB FROM dba_data_files;

    SELECT SUM (bytes)/1024/1024/1024 AS GB FROM dba_segments;

            GB

    ----------

    1680.06079

            GB

    ----------

    1573.35852

    For this (largest) database

  • Dude!
    Dude! Member Posts: 22,829 Black Diamond
    edited Feb 12, 2020 9:10PM

    Your database is 1.7 TB and FRA is 17 GB ?!

    Please show the following:

    [[email protected] ~]$ source oraenv <<< ORCL

    [[email protected] ~]$ rman target /

    RMAN> report schema;

    If the information is correct, then your backups are not going to FRA, but elsewhere, in which case Oracle space management of FRA doe not apply to backups. Check your backup scripts where you store backups.