Oracle technologists, database architects, database administrators, and senior database application developers are all very curious people by nature – even more curious than most people. So naturally database oriented issues evoke debates similar to “tastes great, less filling” from the famous beer commercial. You might be surprised by the scope and passion of such database debates. One area getting lots of discussion of recent is database virtualization – where many seem opposed just for the sake of opposition. I‘ve written a few articles why fighting against virtualizing databases is a losing proposition.

 

 

I’ve also written several articles on how to approach successful testing of database virtualization testing on notebooks and preparation for actual deployments.

 

 

Yet the skeptics remain. So be it. Hence I’m going to focus this article on comparing some common virtualization solutions Oracle technologists can use to test drive virtualized databases on their workstations – namely VMware Workstation ($189) vs. Oracle Virtual Box (free for personal use, $50 for commercial use). For testing these tools are great, for real deployments you would of course choose their industrial strength offerings: VMware vSphere and Oracle VM, respectively. But it’s my hopes that comparing the workstation versions, plus doing so on Windows vs. Linux as the host, will glean some useful information for peoples’ curiosity regarding virtualization and databases.

For testing I’ll be using Quest’s Benchmark Factory version 6.7 (which I’ll be referring to as BMF for short) to run the industry standard TPC-C benchmark. The databases will be fairly stock installs of Oracle 11g R2 with a few initialization parameter modifications, but otherwise pretty much the default or “out of the box”. The Linux and Windows hosts and VM’s too are pretty much stock, with just a file system modification to improve database IO performance:

 

  • For Windows:
    •   HKEY_LOCAL_MACHINE\System\CurrentControlSet\ Control\FileSystem\NtfsDisableLastAccessUpdate = 1
  • For Linux:
    • Edit /etc/fstab file and add the “NOATIME” attribute
    • Example: /dev/sda2        / ext3 defaults,noatime    1 1

 

The following hardware (two identical boxes) and software was used for all reported testing:

 

  • Windows Host
    • AMD Phenom II X4 955, 3.2 GHz CPU, Quad Core
    • 16 GB RAM DDR-3 RAM
    • RAID-0 (striped), Dual 7200 RPM SATA-II Disks
    • Windows Server 2003 Enterprise R2 64-bit with SP2
    • VMware Workstation for Windows 8.02 64-bit
    • Oracle Virtual Box 4.12 for Windows 64-bit
  • Linux Host
    • AMD Phenom II X4 955, 3.2 GHz CPU, Quad Core
    • 16 GB RAM DDR-3 RAM
    • RAID-0 (striped), Dual 7200 RPM SATA-II Disks
    • Ubuntu Desktop Linux 10.04 (lucid) 64-bit
    • VMware Workstation for Linux 8.02 64-bit
    • Oracle Virtual Box 4.12 for Linux 64-bit

 

The following virtual machines were tested on both hosts and under each virtualization platform:

 

  • Windows VM
    • 2 CPU with 2 threads per CPU = 4 CPU’s
    • 8 GB RAM
    • Windows Server 2003 Enterprise R2 64-bit with SP2
    • Oracle 11g R2 (11.2.0.2) 64-bit
  • Linux VM
    • 2 CPU with 2 threads per CPU = 4 CPU’s
    • 8 GB RAM
    • Redhat Enterprise Linux 5.4 64-bit
    • Oracle 11g R2 (11.2.0.2) 64-bit

 

Due to technical difficulties such as not being able to open the Windows VM’s .VMDK file under Virtual Box due to Windows issues with storage driver properties settings (i.e. SCSI vs. IDE) and not being able to configure Linux for running two virtual machine platforms both requiring hardware virtualization support at the same time – the matrix of test scenarios was reduced to as shown in the following table:

 

 

VM OS

Virtualization Platform

VMware

Virtual Box

 

Host = Windows

Linux

Run-1

Run-3

Windows

Run-2

X

 

Host = Linux

Linux

Run-4

X

Windows

Run-5

X

 

Figure 1 below displays the BMF TPC-C benchmark results for transactions per second, often simply abbreviated and referred to as TPS. Unfortunately although commonly favored, TPS is a generally misunderstood and largely misinterpreted metric.

 

bbb1.png

Figure 1: Transactions / Second

 

Figure 1 clearly demonstrates two very key findings. First, that the various host operating systems, virtualization platforms, and VM operating systems combinations do not make a substantial difference. In other words the transactions per second results are essentially the same across all the various combinations – meaning that no one solution is better or worse than the others. The second and more important result is that the transaction per second or TPS rate rises with the concurrent user load. So at 120 concurrent users the score of 6.3 TPS is triple that at 40 concurrent users of 2.1 TPS. Of course that’s the expected result – i.e. the more concurrent users the higher the TPS rate until some threshold is reached breaking that trend. Since Figure 1 results have not yet reached that threshold, we could deduce that we can keep growing the concurrent user load even higher until the inflection point is finally reached. That however would be where we’d make the common and devastating mistake – and why TPS by itself is so dangerous.

 

Figure 2 below displays the BMF TPC-C benchmark results for average response time (in milliseconds), the one metric that all business end-users know quite well and often judge acceptable database application performance by – often even stating service level agreements or SLA’s in terms of average response time maximums. Unfortunately although the best understood and simplest metric to rate, average response time is quite often overlooked by technical people doing benchmarks in lieu of TPS and corresponding calculable physical IO rates such as IOPS. In fact these same technical people will often lower or remove the benchmark’s “keying & thinking” time in order to inflate those numbers. In essence they are more interested in engine’s tachometer (i.e. RPM’s) than the speedometer (i.e. MPH).

 

bbb2.png

Figure 2: Average Response Time (in milliseconds)

 

Figure 2 also clearly demonstrates two very key findings – so key in fact that they are the crux of the actual knowable results from this benchmark testing. First, the various host operating systems, virtualization platforms, and VM operating systems do vary in terms of performance. The various combinations are not in fact all equal. Some choices such as Run-4 (Linux host, VMware and Linux guest) performs markedly worse across the board. While other choices such as Run-2 (Windows host, VMware and Windows guest) perform somewhat inconsistently – i.e. worse at lower concurrent user loads, but gradually improving as load increases. However second and the more critical finding is that assuming a common two second response time SLA requirement, we have quite a ways to increase the user load before we’re likely to cross that threshold. Note that TPS give us no clue as to where we are in relation to any limits, just that the numbers are getting better until they don’t. Average response time clearly indicates to us where we are in relation to that limit. In fact taking Run-1 (Windows host, VMware and Linux guest) and increasing the concurrent user load until that 2 second limit on average response time was reached took over 1,000 concurrent users. So MPH do in fact tell you when you’ll arrive at your destination more so than RPM’s.

 

In closing let me clearly state that I do not intend these results to suggest nor pick any winners nor losers. Merely that these benchmark results show clear evidence for me personally to make my own conclusions in some very specific testing scenarios. There are no universal truths indicated here regarding those choices. Hence I am not trying to suggest which host OS, virtualization platform or guest OS is best. Rather to me they are all just fine and hence they are all on my tool belt. What I do suggest is that virtualization is here to stay, it works and works well, so quit swimming against the current and embrace it – and when doing so proper benchmarks and their correct results interpretations can lead to very successful virtualized database deployments.