Hello - I am relatively new to the Oracle VM world, as I do not find the product anywhere near the quality or stability of VMware. I'm being forced to implement it for the Oracle licensing savings, supposedly. Building the servers and manager has not been a problem, outside of the fact that they are not including Xsigo drivers and we were trying to implement on Infiniband. Short story long, I am curious as to what other OVM users are doing for backups. Our daily backup and DR strategy with VMware includes full image snapshots which makes recovery very easy. I know Xen doesn't include snapshot technology. I know you can hot-clone in OVM, but nothing I have found seems to help automate the process from the command line. The CLI shell requires an expect script and from there it gets rather complicated.
We are looking to get the vm.cfg and associates *.img files backed up using an enterprise scheduler. Is anyone else doing something like this? Any and all ideas are helpful.
Oracle published a 'backup' "guide", which basically said, "you're on your own".
Oracle only supports Oracle products on Oracle VM. It is true, that if you try to use VMWare on multi core systems to run Oracle Databases, then you must license every core in the server when it comes to Database cores. Oracle VM allows CPU pinning, that restricts cores from being used and its the only method that allows you you restrict cores for Oracle Databases. Unless, you truly have a hardware partitioned system.
Oracle VM runs very well when used with Oracle Products. Even better than VMWare. Backups are an issue to some degree but you can use any agent based backs that you normal would on the VM guests. Its no different than backing up any other non VM environment. For databases, you should still what every method you're using now. Even in a virtual environment. Whether is FLASHBACK, RMAN, Dataguard or etc.
For everything eles, I personally use PXE with custom kickstart scripts to quickly recover the VM server itself. I store my VM guest on storage that allows me to do rolling snapshots at the storage level. Rolling storage snapshots are much better to use than creating a complete clone of all the data associated with each repository. In fact, if you're going to take that approach why not use replication on the storage side to reproduce your LUNS/Diskgroups on a device to use for backups?
The future of backups is really centered around centralized storage with deduplication and compression. Oracle ZFS appliances work good for this.
Welcome to my hell.
While we're not yet in production, I've been periodically shutting down all the guests and backing up the repositories via NFS for later recovery.
Otherwise, we're looking at backing them up from the VMs as though they are old-school physical servers as we switch to production mode.
I would disagree with the assertion that the product runs well for most folks. I don't seem to run in to people running OVM well unless they're on an Oracle stack from top to bottom (ZFS storage appliance, Oracle servers, Oracle OS, Oracle software running there). I think a majority of the folks that are happy with OVM haven't used VMWare. In general Oracle VM is fairly unstable and support slow to resolve issues. We're running on three Oracle servers against EMC storage over Fibre Channel, ourselves.
user12273962 - I understand the Oracle licensing / hardware partitioning, and you're making a blanket opinion statement about how "well" it performs. My experience with Oracle Support, from basic requests along with them telling me one thing, and then not following through (Xsigo drivers case-in-point), has been awful.
I'm not really trying to discuss anything other than a backup strategy. Oracle touts OVM as Enterprise level and "better than VMware", yet they completely disregard image-level backups, backups in general, and a DR strategy that doesn't involve a whole lot of SAN work. There are no scripts even offered as examples. With today's technologies, it's unacceptable to require cold backups.
We do utilize native backup solutions per product. We are using RMAN for databases and TSM for OS/system data, but I do not want to have to Bare Metal restore systems. Anyone else script out a clone > backup > delete script?
987423 - I don't use backup scripts from the CLI so I can't help you. I understand they can be done via CLI and to do a "live clone" your Repo either has to be FC or iSCSI based. Other than that, I can't help you. That's one of the reasons I never got into it. I use NFS based repos and I understand you also have to use NFS to share repos across the VM environment with SPARC based servers and Oracle VM 3.2.1. I use my storage to provide adequate backups.
Personally, I have never trusted most any backup solution enough as to not plan for bare bones restores, even if they are a last resort. As long as I have the data, I can have VM guest online in a matter of minutes using kickstart scripts. That's one of the things I love about Linux.
Rob's assessment is accurate on all accounts.
user12273962 - What are you doing / using for backups? You said you use storage to provide backups? Please elaborate.
My strategy doesn't need to necessarily be a clone, but it was one avenue I have been researching.
To you point on Rob assessment.... I have used VMware and like it. It just cost too much. Running Oracle software products is just as fast or faster on Oracle VM. In fact, most of it has to do with running Oracle Linux. Oracle Linux runs very well as VM guest and you can't beat its performance.
AS far as support, I've had interaction with Oracle Support for more than 15 years. You get good techs and bad techs. Same with EMC or VMware. I personally hate EMC support more than Oracle Support. No need to get into the details. Oracle VM support isn't that great. First level support really is difficult. IF they can't find it as a known issue, then you're hurting. If you get put in "development" then you're in no-man's land. The same is true with EMC. VMware is a little better... and you PAY FOR IT. YOU PAY ALOT and then some. I wish Oracle VM support was better. I'm hoping it gets better. The one thing its got going for it is that Oracle itself uses Oracle VM in many different areas. That is why if you generally use Oracle Hardware... then it gets attention internally. I was told that Oracle is replacing all their internal system with Oracle/Sun equipment. ZFS appliances are replacing a lot of Netapp filers that Oracle use to run.
I backup by scheduling snapshots and then I export them as shares. I then backup the shares to tape or alternate disks. It just depends on what it is. I do cold backups of my database oracle homes periodically. Regardless of what you use, I always expect a problem trying to recreate a running oracle home from a hot backup. I have plans to replicate my repos for HA but I haven't got there yet. I need to get some more hardware. :)
I have outlined the steps for the CLI in a blog post: http://portrix-systems.de/blog/brost/taking-hot-backups-with-oracle-vm/
The other way would be to manually create an OCFS snapshot and write that to tape or another destination. It seems like most people are doing backups with one of these methods. It is a shame there is no real, proper supported method or product for this out there
I can schedule ZFS snapshots with my Oracle's ZFS product. I do have to refresh my shares once I roll them to tape. I wish I could automate this part of it.
I have tested it several times. I can schedule a snapshot of a repo with running VM guests. I can then shutdown my guests. (to simulate a failure) and then do a roll back of the snapshot. When I start my VM guests, they will see the disks as flagged as dirty and do a fsck. Every time I've tested it, it comes back clean and everything works fine. I've only done this with Linux guests. I assume windows guests would react differently.
I assume you mean VM or disk image clones when you say OCFS snaphots?
Are the clones consistent across a disk with IO activity happening on the disk?
Are the clones consistent across multiple disks assigned to a single VM, again with IO happening?
I really wish I could find more information in Oracle's documentation about this topic...
I spoke with an Oracle Product Manager on Friday, who said the backups are crash-consistent on a per-disk basis. A multi-disk clone won't be crash consistent between disks... so if I/O isn't suspended, it's not a valid snapshot.
yes. But when doing manual OCFS snapshots it is recommended to suspend the VM anyway to write a truly consistent snapshot (including memory contents). I guess that would also take care of getting a consistent snaphot across multiple disks
We use commvault and backup at least nightly like any other physical machine & it uses RMAN for our DBs. We have only had to restore once from commvault in production so far (we actually restored many copies over a 55 day period to find the source of data corruption in the app db) & it was flawless. For some VMs that I can shut down on weekends I try to at least monthly do cold backups of the virtual disks & config files but unfortunately it is all command line at the OVM host level as our Production SAN has no features for anything like that...
I put up a new ZFS server in our Dev environment and my initial tests of ZFS shapshots seem promising (I have recovered from changes multiple times already and the VMs are fine).
Currently the ZFS are presented as NFS & one thing I started doing is creating a separate repository for each VM or set of related VMs so I could so snapshots for a single VM or group of VMs. They are actually in the same zpool but just logically seperated & it allows me to easier see the disk usage by VMs or groups of VMs. I am not sure if this is a good idea or not but I did this because I did not like the idea of having to restore a full repository of unrelated VMs and don't want to have to mount the ZFS snaphots in another ZFS pool just to get the files... Now I can backup via snapshots near instantaneously.
These should be easy to send to a remote ZFS server in the background. As of at least 3.2.1 I believe the CLI is now officially supported so the clone method mentioned in Bjoern Rost's blog may be the best way to do a hot clone backup in an officially supported manner. It would be great of Oracle put in a management UI to automate those command lines...
Edited by: user11391721 on Apr 1, 2013 8:52 AM CLI support & misc comments...