This discussion is archived
8 Replies Latest reply: Dec 13, 2012 12:10 AM by Nicolas Wipfli - Oracle RSS

Storage Array 2530 - 36 @ 2TB HDD -  RAID6

971208 Newbie
Currently Being Moderated
Hello!

I have my Array up and running using the CAM software but I have a concern and I have little experience and trouble finding documentation on the issues..

I have a 36 SATA disk 2TB setup, with 3 hot spares. 1x3

My first issue is when creating the Volume/Virtual Disk Etc. It only seems to allow me to use 30 disk drives even though the other 3 are listed as Unassigned.. Is there a limitation to Raid 6 I am unaware of?

My second concern is the initialization time of about 510 Hours.. the job is increasing about 3% per day, does this seem right? The array is not reporting any issues and the firmware was updated.

Again, your support and help is always appreciated.

Regards and Thanks.

Ash

REHL 6.2 RD240 64 bit
  • 1. Re: Storage Array 2530 - 36 @ 2TB HDD -  RAID6
    Nicolas Wipfli - Oracle Expert
    Currently Being Moderated
    Hello,

    The firmware is designed to limit 30 drives in your vdisk, regardless the RAID level. This is for the 2500 series.
    The initialization increasing 3% a day is definitively not normal, however I do not know what is the issue. Such investigation would require debug data.

    Regards
    Nicolas
  • 2. Re: Storage Array 2530 - 36 @ 2TB HDD -  RAID6
    971208 Newbie
    Currently Being Moderated
    Thanks For the Reply!

    Well that def answers my first question.

    Noob here, how do I get this debug data?

    Cheers,
  • 3. Re: Storage Array 2530 - 36 @ 2TB HDD -  RAID6
    Nicolas Wipfli - Oracle Expert
    Currently Being Moderated
    See the document "Collecting Sun Storage Common Array Manager Array Support Data (Doc ID 1002514.1)" on My Oracle Support. It explains how to collect a supportdata which will need to be analyzed by an Oracle support engineer.

    Regards
    Nicolas
  • 4. Re: Storage Array 2530 - 36 @ 2TB HDD -  RAID6
    800381 Explorer
    Currently Being Moderated
    OK, a bit off-topic, but a 36 drive RAID volume?!?!!?

    Your performance is going to stink. Google "RAID read-modify-write" for just one reason why that's an utterly horrible idea.

    And as you're seeing, RAID initialization and therefore rebuild times are huge. So you have a huge window for multiple drive failures happening before a rebuild is finished, thereby rendering your 30-drive array permanently lost. What kind of backups do you have to restore 60+ TB of data from?
  • 5. Re: Storage Array 2530 - 36 @ 2TB HDD -  RAID6
    971208 Newbie
    Currently Being Moderated
    We need to archive many TBs worth of .wav files which is increasing daily. We had chosen RAID6 for the 2 drive fail redundancy. I understand what your saying, what direction would you go in to store the data on the array?
  • 6. Re: Storage Array 2530 - 36 @ 2TB HDD -  RAID6
    800381 Explorer
    Currently Being Moderated
    On a file system that big?

    Use a volume manager (LVM, ZFS, etc) to spread the file system across multiple RAID arrays. 10-drive RAID-6 arrays are good - 8 data, 2 parity. Pick a segment size for each array such that the RAID stripe size (segment size times number of data drives in array) is no larger than the underlying block size of the file or volume management system you plan to put on it. Multiple underlying RAID arrays will also allow both controllers of the 2530 (assuming it has two) to be used for processing IO, doubling both the effective processing power of the 2530 and the bandwidth to the device since your server will be able to actively use SAN paths to both controllers.

    For example, if you use ZFS with its 128k block size, on 8+2 RAID-6 arrays, pick a 16k segment size so you get a 128k RAID stripe width. If you're using ext3 on top of LVM, select a segment size, calculate your stripe width, then use the -datalignment option of "pvcreate" to match your stripe width. Build an even number of RAID arrays, with an equal number on each controller.

    If you manually have to partition/label each LUN, make sure you start the disk partition you're going to be writing data to on a RAID stripe boundary. For example, note that ZFS, when given a whole disk/LUN to use, puts an EFI label on the disk and starts its data partition at segment 256, which is at the first 128k boundary, assuming a 512-byte segment size.

    On a file system this big, if data availability is critical, I'd also be tempted to use software RAID on top of the multiple RAID arrays. You'll lose a small bit of performance, but it might be worth it. In my experience, block size, segment size, and RAID stripe calculations are nowhere near as important for software RAID as they are for hardware RAID.

    It's all trade-offs. The more drives you wind up using for parity, the more available your data will be.

    Don't mess around with multiple LUNs off any physical RAID array in your 2530. Just fill the entire RAID array with a single LUN.

    To get the best possible availability, if you're using Oracle's CAM to set up the RAID array, don't let it magically select which physical drives to put in each RAID array you build. If you have enough expansion trays, you want to minimize the number of drives you select from each tray. If you have a total of 5 trays and are building 10-drive RAID-6 arrays, for example, only pick two drives from each tray for each RAID array you build. That way, if an entire tray fails (someone goes into the server room and unplugs the wrong device...), you'll minimize the impact and won't lose access to your file system. If you don't have enough expansion trays to build your RAID arrays that way, try to build each RAID array in as few trays as possible. That way a tray failure will have a minimum impact on your file system (hopefully).

    And don't forget - RAID is for availability. Backups are for integrity. RAID can't protect you from someone making a mistake and running "rm -f -r ." in the wrong place.
  • 7. Re: Storage Array 2530 - 36 @ 2TB HDD -  RAID6
    979613 Newbie
    Currently Being Moderated
    How can I get this "Collecting Sun Storage Common Array Manager Array Support Data (Doc ID 1002514.1)" doc on My oracle Support?

    Can you send it to me straight way?
  • 8. Re: Storage Array 2530 - 36 @ 2TB HDD -  RAID6
    Nicolas Wipfli - Oracle Expert
    Currently Being Moderated
    Hi,

    I cannot send you this document, you need to go to https://support.oracle.com and login with your MOS account if you have one.
    If you do not have an account, please follow the steps below to collect a supportdata, however you will need to have the 2530 under contract in order to log a SR with Oracle:

    1. Go to CAM.
    2. Select the array.
    3. Click on "Service Advisor" no the top right in CAM.
    4. Go to "Collecting Support Data" and follow the steps.

    Regards
    Nicolas

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points