This discussion is archived
1 2 Previous Next 16 Replies Latest reply: Feb 6, 2013 10:50 PM by Dude! RSS

Linux OS Backup with Oracle DB Installed

987680 Newbie
Currently Being Moderated
Hi Gurus,

I currently have an Oracle Linux machine with Oracle 11g Database installed. I am currently trying to come up with a backup strategy. For the DB, I have already set up RMAN backups, data pump exports and Archive log backups (using Flash Recovery Area).

My question is regarding the Linux OS backup. I plan to write a script to carry out the OS level backup (which will just run a tar), however, not sure if I need to include the Oracle database folder as well.

1. The reason I don't want to include the Oracle database folder is that when the tar command runs, and if Oracle is up and running, this might cause an issue as log files may be getting updated during that time, so the backup script will take a long time to complete (The DB will always be up and running, as I do not plan to do a 'cold backup' scenario)

2. As per point 1, if I don't include the oracle database folder, then, while restoring to a new server, I'll need to re-install the Oracle DB and then restore the DB using RMAN

So essentially, the above becomes a 3 point restore - restore the OS, re-install DB and restore DB backup using RMAN.

What I would like to achieve using the OS level backup script is upon restoring the OS backup, the OS and DB are in place and I just need to restore the DB.

To achieve that, what folders should I backup OR this is not achievable?

The Oracle DB is not using ASM.

Thanks to all those who respond, in advance.

Regards.

Edited by: 984677 on Jan 28, 2013 7:22 PM

Edited by: 984677 on Jan 30, 2013 10:41 PM
  • 1. Re: Linux OS Backup with Oracle DB Installed
    Dude! Guru
    Currently Being Moderated
    There are many more aspects that make up a reliable backup and recovery strategy than the few points you outlined. Before you can implement a backup strategy you need to determine your restore and recovery requirements. It is not a good idea to work out a backup strategy and then to worry about the possible recovery options or requirements later.

    For instance, do you need to archive files, do you need disaster recovery or partial recovery? Do you have a dedicated backup administrator? What are the most important files you really need to backup? Perhaps all you really need are OS system audit, log files and the /home directory? etc., etc...

    Complete system backups are a waste of resources. If you are an experienced and careful system administrator there is usually no need to restore the operating system. If your server fails in a couple of years, you may end up with new hardware that requires to reinstall and upgrade the system. You need documentation to rebuild the server from scratch, which is usually much faster than a restore, and also a clean solution. If you restore the system from an existing backup it can also duplicate existing problems. The same applies also to your ORACLE_BASE directory. Perhaps make some archive backups prior to major installation changes, which you should do anyway.

    I recommend the following:

    - Use only server grade hardware, such as enterprise SATA or SCSI hard drives. Desktop hardware isn't 24x7.
    - Mirror the system disks using a hardware RAID-1 storage controller.
    - Document the server configuration and keep it up to date as much as possible.

    To backup your Oracle Database you will most likely want to run it in Archivelog mode and use RMAN for backup. There are many options in regard to mirroring critical components and preparing for database recovery scenarios. This topic is very complex and cannot be explained here.

    Keep it simple. Avoid anything fancy and too sophisticated. Don't rely on 3rd party backup solutions unless you are familiar with it and know it will work in the future. Disaster usually strikes when the least expected.
  • 2. Re: Linux OS Backup with Oracle DB Installed
    987680 Newbie
    Currently Being Moderated
    Hi Dude,

    Thanks for your suggestions. Regarding Oracle DB backup, as I mentioned in my question, I have already set it up and running using RMAN (with archive log enabled).

    I agree with you on 3rd party backup softwares, hence I am going to script the backup.

    I should have mentioned earlier but the server is a VM (so snapshot is a possibility, but not daily), my bad.

    My query is that how do I backup the Linux OS, so that in case if I need to restore the whole server, the Oracle DB is also restored, and then I need to apply only RMAN restore to bring the DB upto the latest stage?

    Thanks.
  • 3. Re: Linux OS Backup with Oracle DB Installed
    Dude! Guru
    Currently Being Moderated
    Like I tried to explain in the first 2 paragraphs, I would not backup the complete operating system. ;-)

    If you still need a complete disaster solution to restore a system disk you need to create a complete binary copy or image of the system disk, or at least copy the boot sector, copy the partition layout, etc. and some tar archive. What kind of recovery do you need?

    If this is a virtual machine, simply shutdown the system and then duplicate the disk image files to have a complete redundant backup.

    Forget about VM snapshots as a recovery option. A snapshot is not a backup. A snapshot marks the time when changes are written separate from the source, so you can trash the changes to turn the VM to a previous state. However, if you loose the VM, your snapshots are useless as you are only left with data blocks that changed since your previous snapshot.

    Edited by: Dude on Jan 28, 2013 10:21 PM
  • 4. Re: Linux OS Backup with Oracle DB Installed
    987680 Newbie
    Currently Being Moderated
    Hi Dude,

    Based on some further research and what you have suggested, I propose to do the following -

    1. Backup /boot, /var and /usr OS folders.
    2. For Oracle, backup the recovery_area, oradata and admin folders (admin contains the datapump export) - This is for OS level only. RMAN backups are already running seperately for the database.

    This should give me enough to bring back the server and database and should suffice from a 'Backup Strategy'.

    Backing up the whole server would probably start bordering towards DR, which is not a requirement at the moment.

    Let me know your thoughts.

    Thanks.
  • 5. Re: Linux OS Backup with Oracle DB Installed
    Dude! Guru
    Currently Being Moderated
    1. Backup /boot, /var and /usr OS folders.
    What are you trying to accomplish? Sorry I think this is the wrong solution, and certainly not what I was recommending.
  • 6. Re: Linux OS Backup with Oracle DB Installed
    987680 Newbie
    Currently Being Moderated
    Sorry for the delay in response. Your last comment suggested that what I have proposed is not correct, so did some further research. Essentially what I want to be able to do is in case of server loss, I can re-install the OS and be able to do a full system restore.

    So I guess I'll need to back up the following folders - /etc, /home, /var, and /srv. I'll also include the /boot to compare the old and new config, just in case.

    Should this suffice for a linux os backup?
  • 7. Re: Linux OS Backup with Oracle DB Installed
    Dude! Guru
    Currently Being Moderated
    I still think you are pursuing the wrong track.

    To create a backup that you can use in order to completely recover your system is a complex task, involves a block based image backup, and as I tried to explain is usually not worth the effort and maintenance. I think it is certainly a bad idea to re-install the system and then to overwrite or add core system files from an existing backup, which has the potential to break a good installation and duplicate exiting problems. Instead you may just do a regular backup of some specific files for archiving, e.g. audit, security logs, if you need to.

    Should you bother to create a complete system backup, there is no need to exclude core system files since the files are typically static, not substantial in size and needed for system integrity. Sorry, I won't go into more details, because I'd be sitting here all day explaining the do's and don'ts. Please see my previous replies.
  • 8. Re: Linux OS Backup with Oracle DB Installed
    987680 Newbie
    Currently Being Moderated
    This is a linux server with Oracle Database installed. I do not have any specific requirement, hence I am inquiring as to what is a normal backup strategy for backing up linux servers. In case I need to rebuild the server, what sort of backup would be ideal so as to allow the new server to be up and running without requiring too much effort? E.g. normally you would have things like email configured, snmp setup, etc. I would like to be able to get this config recovered, so what would be the best way to do it?

    I understand your response regarding not backing up the whole server, so all I am after is the important folders that would normally be backed up.

    The folders that I have listed above, are they sufficient or some other folders should also be added?

    Thanks.
  • 9. Re: Linux OS Backup with Oracle DB Installed
    Dude! Guru
    Currently Being Moderated
    There exists no other universal disaster recovery strategy than to backup the whole enchilada, which is either inefficient or practically useless, usually both. You cannot simply duplicate a running system and copy open files or stuff that resides in memory.

    If you plan to do a selective backup, you need to do your homework and figure out what information you need to recover. No one can do that for you. To pick some directory and hope that it contains the information you may need for a recovery is a bad strategy. Your idea to reinstall the system and then to restore some directories from a backup is a bad strategy for the reasons already mentioned. You may backup directories, but you do not want restore and overwrite existing directories and only copy the files that you really need.

    Your most efficient OS backup strategy is a pen and a piece of paper to write down how you need to configure the server. What you need to backup for Oracle from your system disk depends on your configuration, but generally you want to keep a copy of /etc/oratab and /etc/oraInst.loc. Most of your system configuration stuff will be /etc, system log files in /var/log and email in /var/spool/mail.

    In your case the solution might be to simply shutdown the system and to copy the system's disk image file as already mentioned.
  • 10. Re: Linux OS Backup with Oracle DB Installed
    BillyVerreynne Oracle ACE
    Currently Being Moderated
    984677 wrote:
    This is a linux server with Oracle Database installed. I do not have any specific requirement, hence I am inquiring as to what is a normal backup strategy for backing up linux servers.
    I agree with Dude - there is no "normal" backup strategy for the o/s on Linux/Windows servers. To perform such a backup, is not the norm.

    The last time I did o/s level backups were in the 80's on mainframes. Where we made an o/s backup to tape, creating a boot image to boot (and restore) the o/s from tape. Only ever used that option once when a sysadmin accidentally did a factory start with the initial process loader and wiped the entire mainframe clean.

    The backup we use today is a 2 drive bay server, h/w mirroring the drives using the onboard RAID controller. This means the root/system drive is redundant (and hot swappable).
    In case I need to rebuild the server, what sort of backup would be ideal so as to allow the new server to be up and running without requiring too much effort?
    The original installation media. Takes less than 30 minutes to install. Does not care if the h/w is not exactly the same as that of the crashed server - which is what an o/s backup expects... and will likely fail when this is not the case.
    E.g. normally you would have things like email configured, snmp setup, etc. I would like to be able to get this config recovered, so what would be the best way to do it?
    I have a scp cronjob that runs nightly and grabbed the important config files from servers I deem high-profile enough for quick disaster recovery. E.g. if a server runs a custom build NAT firewall, Apache server with mod_plsql, and so on - these are not simplistic configurations, and worth backing up.

    In a VM environment - even easier. Just make a copy of the VM's system drive.

    There's also the option of building your own bare metal kickstart and making it available via bootp. (there are some high volume server sites, like ISPs, that use this method extensively)
    I understand your response regarding not backing up the whole server, so all I am after is the important folders that would normally be backed up.
    That depends on what the server runs, and what is not the norm, config wise. Files like hosts, ntp.conf, resolv.conf and so on, are pretty standard across servers. A single copy of these should suffice. However, multipath.conf will be unique as it deals with SAN LUN WWIDs. We use a lot of ssh - so private and public keys, together with authorised and known keys would be important when not wanting to redo ssh trusted connectivity.
    The folders that I have listed above, are they sufficient or some other folders should also be added?
    I would not backup folders with the intention of simply restoring the folder tree on a new server. I would be selective as to which files to use from such a backup.
  • 11. Re: Linux OS Backup with Oracle DB Installed
    Dude! Guru
    Currently Being Moderated
    … which confirms all I tried to explain, but hopefully the OP will appreciate another independent evaluation. ;-)
  • 12. Re: Linux OS Backup with Oracle DB Installed
    987680 Newbie
    Currently Being Moderated
    Hi Dude and Billy,

    Thanks again for informative responses. This has given me enough guidance to design a backup strategy and then test it out.

    H/w mirroring, RAID, etc. was my first suggestion, however, the infrastructure in the company is not sufficient enough to accommodate this. Basically we are a software consultancy and as such most of the work is done on client sites. The company only hosts demo software for presentation / demo to potential clients, and as such are not keen to invest in superior infrastructure.

    vmdk backup was also suggested, however, as suggested above, on the SAN we have, there is not enough space to backup all the VMs, hence my question on this forum.

    In case of needing to rebuild a server, the intention was always to install from media and then restore appropriate folders / files to recover the config. As I mentioned in my last post, what I wanted to backup was the required folders that would give me all / most of the setup, and allow me to get the server up and running as quickly as possible.

    Regarding the required folders to backup, the intention was never to just restore the folders as is but rather back them up so as to restore the necessary files as required. The benefit of backing up the whole folder means I have got all the files, including the required ones, so I can reference them for information.


    Dude, I understand I have to do the homework to get the information I require, and the point of this question was not to be spoonfed i.e. here is the list of folders, go back up, however, I also need to ascertain, if apart from the files that I require, is there anything else that should be backed up which would assist in quicker recovery. There is a lot of information on google, but the point of asking question in this forum, is to get an informative and authoritative reply from experienced professionals, as compared to blindly following something.


    Thanks.
  • 13. Re: Linux OS Backup with Oracle DB Installed
    BillyVerreynne Oracle ACE
    Currently Being Moderated
    984677 wrote:

    vmdk backup was also suggested, however, as suggested above, on the SAN we have, there is not enough space to backup all the VMs, hence my question on this forum.
    Root drives should not be that sizeable... The o/s needs less than 4GB. That should also compress quite nicely.

    Besides, is this not exactly why one uses virtualisation? Makes little sense to me to discard major features and benefits provided by virtualisation.
    In case of needing to rebuild a server, the intention was always to install from media and then restore appropriate folders / files to recover the config. As I mentioned in my last post, what I wanted to backup was the required folders that would give me all / most of the setup, and allow me to get the server up and running as quickly as possible.
    In that case, I'll grab the entire +/etc+ tree as a backup. It will include all the important config files - from users and passwords, to networking and name resolution, and to multipath and even ocfs2.
    however, I also need to ascertain, if apart from the files that I require, is there anything else that should be backed up which would assist in quicker recovery.
    If you want Oracle s/w as a backup, then the entire $ORACLE_BASE path - which needs to include Oracle Inventory and the one or more Oracle Homes. The Oracle o/s config files are in +/etc+.

    However, unless you are prepare to test this type of recovery and restore, i.e.
    1) install o/s
    2) copy files/dirs from backup
    3) reboot

    ..I would think that a clean install (of all s/w) would be faster. Hunting which files to copy from backup to get a valid Oracle s/w config on the recovered system, will be slow and by trial. So if you have not done this before, and scripted/automated this part of the recovery from backup, it will be a faster (and more robust and less risky) to simply reinstall Oracle.
  • 14. Re: Linux OS Backup with Oracle DB Installed
    Dude! Guru
    Currently Being Moderated
    Considering your last response I think you have a bit of a strange setup. You have a SAN without mirroring and not enough space to store a backup?

    There seems to be no clear picture about your actual recovery needs, available resources and lack of experience. Where are you actually planning to store your backup? What is your file system layout? What is your required recovery window? You are apparently trying to implement a sophisticated backup strategy, but want to make it as general as possible. I think you are over-engineering the task. How about the following:

    Buy a couple of 8 GB USB sticks to store your system disk backup and rotate between them. Then use the following command to perform a full backup when you think it was a reasonable time to do:
    lsscsi
    USB=/dev/sdx
    parted -s \$USB mklabel msdos mkpart primary 1 -- -1
    mkdir -p /mnt/usb
    mount \${USB}1 /mnt/usb
    cd /
    find . -type s > /tmp/$$
    tar -cf - . -X /tmp/$$ | { cd /mnt/usb; tar -xf -; }
    umount /mnt/usb
    The above will take care of your root file system. To include /boot, simply use tar and store it somewhere on your root system. In case you have /var or /home or other directories on separate partitions, you need to back them up separately. When you need to restore then simply reinstall your system from scratch, mount the USB stick and copy what you need.
1 2 Previous Next

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points