This discussion is archived
8 Replies Latest reply: Dec 7, 2012 3:17 PM by Dude! RSS

Poor SSD disk IO speed in Oracle Linux 6.3 (Windows migration)

978527 Newbie
Currently Being Moderated
Hello,

I am trying to migrate from Windows to Oracle Linux, but I'm seeing very poor disk IO speeds. It's probably a tuning thing, but I'm relatively new to Oracle Linux and could use some detailed advice.

I took one physical server and migrated it from Windows 2008R2 to Oracle Linux 6.3 while maintaing the same Oracle version (11.2.0.3 Enterprise with ASM) and the same hardware (quad CPU 48 core HP DL585 G7 with 128GB RAM, 7 LSI 9200-8e HBAs, 28 Samsung SSD Drives). Disk IO performance, as measured using Oracle IO Calibration, was ~7,800MB/Second and 440K IOPS under windows but fell to ~2,400MB/Second and 250K IOPS under Linux.

Oracle Linux and the DB were installed using default values. The Oracle tools seem to have done a great job setting all of the obvious IO tuning parameters like the scheduler, but I figure that there are other important IO-related OS or DB parameters and that I have failed to configure the system properly.

My goal for the migration is sequential read IO speed and I would have bet money that Linux would provide better performance than Windows. I still think that it should. What basic IO tuning should I do for Oracle Linux using ASM and SSD drives?

Thank you!

Some details:
Oracle DB 11.2.0.3 enterprise installed via the GUI with the "Data Warehousing" template
ASM - single disk group, 28 SSD disks, AU=4MB
Oracle memory: Automatic memory management, 64GB allocated
Non-default Oracle params: filesystemio_options=setall, disk_asynch_io=true

Edited by: 975524 on Dec 7, 2012 8:56 AM
  • 1. Re: Poor SSD disk IO speed in Oracle Linux 6.3 (Windows migration)
    Dude! Guru
    Currently Being Moderated
    I cannot find information in your post to verify that you have setup and installed the system correctly or how you determined the performance results. Based on your input, I recommend to check a couple of previous posts and setup the system to use Kernel Hugepages and noop I/O scheduler.

    /dev/shm on Oracle Linux 6.x to run Oracle 11g R2 - manual configuration?
    /dev/shm on Oracle Linux 6.x to run Oracle 11g R2 - manual configuration?

    Understanding kernel hugepages and Oracle 11g AMM incompatibility
    Understanding kernel hugepages and Oracle 11g AMM incompatibility

    I/O scheduler in Oracle Linux 5.7
    I/O scheduler in Oracle Linux 5.7
  • 2. Re: Poor SSD disk IO speed in Oracle Linux 6.3 (Windows migration)
    rukbat Guru Moderator
    Currently Being Moderated
    Moderator Comment and Action:

    @ O.P.,
    Your duplicate post in the Database forum has been deleted.
    For your future participation in these OTN forums, do NOT ever multipost.
    That is poor forum etiquette.

    Additionally, should you ever consider cross-posting to multiple forum web sites across the Internet, it is polite to state that fact in your post and paste a link to such other cross-posts.
  • 3. Re: Poor SSD disk IO speed in Oracle Linux 6.3 (Windows migration)
    800381 Explorer
    Currently Being Moderated
    7800 GByte/sec?

    Is that a typo? Because the max transfer rate of 28 drives, assuming 6 gbit/sec per drive, is going to be about 21 Gbyte/sec.

    If it's really reporting 7,800 GByte/sec, it's lying.
  • 4. Re: Poor SSD disk IO speed in Oracle Linux 6.3 (Windows migration)
    978527 Newbie
    Currently Being Moderated
    Thanks "dude" for the advice. Unfortunately, I am still seeing low IO speeds.
    The default scheduler for OEL 6.3 with the DB pre-install package is deadline, which seemed like a far better choice than CFS. Based on your advice, I tried noop this morning and got the same results. I also tested with and without hugepages and saw only a small difference - at least in IO speed - I did not test overall DB performance. Lastly, I understand the /dev/shm issue, but even with the default configuration I'm getting 64MB allocated to Oracle, which is far more than is needed to test sequential IO - in fact I can get better results by using less RAM.

    To answer your questions, I am testing using Oracle IO Calibration, which is an IO testing feature of the Oracle DB that is similar to the standalone Oracle Orion tool. I also performed a few tests using IOMeter, but found that the Linux version of that product was not giving me consistent data. The overall trend was the same however - IO on the Linux version far lower than the same hardware running Windows. The system is functioning very well, so I assume that everything has been installed correctly, but I do not think that it was installed optimally - thus my cry for help.

    I am so surprised that Linux is showing slow IO!

    Edited by: 975524 on Dec 7, 2012 9:22 AM
  • 5. Re: Poor SSD disk IO speed in Oracle Linux 6.3 (Windows migration)
    978527 Newbie
    Currently Being Moderated
    Rukbat, you have my apology for the mostly duplicative posts. The questions were a bit different - one focused on Oracle Linux and the other on Oracle DB - but I agree that they were substantially the same issue about IO speed.

    Edited by: 975524 on Dec 7, 2012 9:20 AM

    Edited by: 975524 on Dec 7, 2012 9:21 AM
  • 6. Re: Poor SSD disk IO speed in Oracle Linux 6.3 (Windows migration)
    Dude! Guru
    Currently Being Moderated
    Performance issues are usually complex and I'm afraid there is no way to determine from here whether or not your system has been installed or setup correctly. Perhaps you have a driver, firmware, configuration or machine specific issue.

    There is certainly not enough evidence to conclude that I/O performance under Linux is much less than MS-Windows. Perhaps your findings are based on wrong information. The performance comparison you showed looks questionable.

    I suggest to perform separate performance tests at the OS and database level to see if there are obvious differences.
  • 7. Re: Poor SSD disk IO speed in Oracle Linux 6.3 (Windows migration)
    978527 Newbie
    Currently Being Moderated
    I am looking for guidance: Are there specific Linux or OEL6.3 configuration settings for SSD drives and/or for Oracle ASM with ~28 disks and/or for multiple LSI HBA systems - settings that are not the default settings since using that many HBAs and SSD drives is not common? It's not that I am getting terrible performance - 2.4GB/Second isn't bad - it's just that I am not getting the performance that I expect based on my Windows testing and I have no reason to believe that OEL can't do excellent disk IO. There is probably a setting or set of settings that would yield better performance, but I do not know where to look.
  • 8. Re: Poor SSD disk IO speed in Oracle Linux 6.3 (Windows migration)
    Dude! Guru
    Currently Being Moderated
    Are the performance readings under Windows correct? What do you expect?

    Chapter 23. Solid-State Disk Deployment Guidelines

    https://linux.oracle.com/documentation/OL6/Red_Hat_Enterprise_Linux-6-Storage_Administration_Guide-en-US.pdf

    Other things you could try is to start the system using the RHEL kernel form the Grub boot menu to see if it makes any difference, and check if HP or the storage subsystem vendor provides any special adapter or device drivers.

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points