This discussion is archived
1 2 Previous Next 22 Replies Latest reply: Nov 15, 2012 2:26 PM by Dude! RSS

ASM Setup Questions

user10437903 Newbie
Currently Being Moderated
I am looking to see what situation is better regarding my ASM setup and what the striping behavior would be. I am using OEL 5 update 2 x86-64 and Oracle Database 11gR2 and ASM 11gR2.

Setup 1
4 Physical HD RAID 5
4 100 Gb LUNS
1 ASM Disk for each LUN
2 ASM Disk Groups (DATA and FRA)
2 ASM Disks per ASM Disk Group

Setup 2
4 Physical HD RAID 5
2 200 Gb LUNS
1 ASM Disk for each LUN
2 ASM Disk Groups (DATA and FRA)
1 ASM Disk per ASM Disk Group

Question 1:
In Setup 1, since ASM Disks for Data Disk Group are on the same physical disks, would this cause significant hot spots. I understand there would undoubttedly be contention.

Question 2:
Setup 2, ASM would not fail since it can't stripe in this setup with only one ASM Disk. In other words, ASM would work but simply wouldn't stripe even though the underlining Physical HD are being striped via RAID 5.

Question 3:
Which setup is better in terms of availability and performance?


Thanks.
  • 1. Re: ASM Setup Questions
    phaeus Pro
    Currently Being Moderated
    Hello,
    one thought, you should prevent the double stripe. ASM Stripes every IO, and if your storage has 1 Lun spread over more disks it will also stripe. In this case when the database write same io you double it on the storage layer because your stripe the things 2 times.

    regards
    Peter
  • 2. Re: ASM Setup Questions
    rcc50886 Journeyer
    Currently Being Moderated
    in either way you will get the same i/o throughput as the underlying physical disks are the same.
  • 3. Re: ASM Setup Questions
    user10437903 Newbie
    Currently Being Moderated
    Yes. It is the contention that I'm asking about (see question 1) and its effect on overloading particular disk with I/O. Typically, your striping across LUNs on a different set of Physical Disks. I understand that ASM is spreading the extents from files over different ASM Disk but in this case, its the same physical disk. I think you lose performance benefits of the ASM when you don't stripe across different Physical Disk. If anyone gets a chance can they take shot at the questions below:

    Question 1:
    In Setup 1, since ASM Disks for Data Disk Group are on the same physical disks, would this cause significant HOT SPOTS. I understand there would undoubttedly be CONTENTION.

    Question 2:
    Setup 2, ASM would not fail since it can't stripe in this setup with only one ASM Disk. In other words, ASM would work but simply wouldn't stripe even though the underlining Physical HD are being striped via RAID 5.

    Question 3:
    Which setup is better in terms of availability and performance?
  • 4. Re: ASM Setup Questions
    Sebastian Solbach (DBA Community) Guru
    Currently Being Moderated
    Hi,

    neither of them.

    Unfortunately both writers above have forgotten an important point: While it physically ends up on the same disks, the queueing of the operating system will make a difference.
    In different tests it has been found that a diskgroup containing of at least of 4 disks you will get better I/O performance (even if the underlying storage system and luns are the same).
    This has to do with the capability of the O/S how to queue the I/O via. ASM.

    Furthermore RAID 5 is never a good idea for a database system. So if you want the best availability and performance I recommend either:

    a.) Setting up your storage system to use RAID 10.
    b.) Remove the RAID on the storage system level and use ASM mirroring with normal redundancy.

    BTW: Double striping must not be necessarily bad. In normal cases (since stripe sizes on storage and ASM level are different) it really does not matter....

    In any case make sure your disk alignment (storage to OS to partition header) is set correctly. If you make an error here you will recognize this far more than any "double striping".

    Regards
    Sebastian
  • 5. Re: ASM Setup Questions
    LaserSoft Journeyer
    Currently Being Moderated
    Hi

    1. RAID-5 is not recommended, because it will impact performance for write-intensive workloads
    2. Oracle Recommends, Use external redundancy disk groups when using h/w mirroring techniques to reduce the overhead.
    3. If you combine ASM Striping and RAID-0 offers good performance too. (Stripe-On-Stripe)
    4. Minimizing number of LUNs, so that managing is very easy for future.

    Thanks
    LaserSoft
  • 6. Re: ASM Setup Questions
    Levi-Pereira Guru
    Currently Being Moderated
    @user10437903:

    About RAID options:

    Consider moving from RAID-5 to RAID-10 for your write-intensive volumes, environment with low or medium I/O write can be a good choice RAID-5, because CACHE WRITE may be efficient as RAID-10 perfomance another thing is that RAID-10 is expensive ($$$). Then configure only RAID-10 for some environments is a privilege.

    RAID 10 vs. RAID 5
    For read operations both are nearly equal. For write iops RAID 10 is better. With RAID5 the subsystem has to read the hole stripe, recalculate it and write it again and RAID 10 is just updating one block.

    Recommend you read it:
    http://en.wikipedia.org/wiki/RAID#RAID_10_versus_RAID_5_in_relational_databases

    In some environments that I manage this type (RAID stuff) of configuration is transparent. The technologies are growing up so we will not need to worry about the type of RAID configuration.

    Like "Easy Tier" of IBM (released 2010):
    Easy Tier is a function that responds to the presence of solid-state drives (SSDs) in a storage pool that also contains hard disk drives (HDDs). The system automatically and nondisruptively moves frequently accessed data from HDD MDisks to SSD MDisks, thus placing such data in a faster tier of storage.

    Easy Tier eliminates manual intervention when assigning highly active data on volumes to faster responding storage. In this dynamically tiered environment, data movement is seamless to the host application regardless of the storage tier in which the data resides.
  • 7. Re: ASM Setup Questions
    Dude! Guru
    Currently Being Moderated
    As the previous poster outlined, RAID-5 is generally known for bad write performance. RAID-5/6 is also slow to recover should a disk failure occur. RAID-10 will give the best performance and redundancy, but it is the most expensive setup.

    However, it all depends on the total number of disks and the abilities of your RAID controller. Some RAID controllers have optimized hardware for RAID 5 and 6 sequential write workloads. I suggest to benchmark various RAID levels in your configuration. Whether you opt for performance or economic reasons should depend on your expectations and requirements, not just max. performance.

    ASM is not RAID. Although it provides obviously similar features in terms of data striping and mirroring, the underlying technology is very different. ASM works on the bases of file extents and free disk space on disk failure groups. For instance you can add ASM disks to increase capacity without having to rebuild your volumes, which you cannot do with a typical RAID system unless you use JBOD.

    With ASM I would not focus so much on the underlying hardware RAID level, but rather research the options to provide physical controller and device redundancy. Meaning to create ASM disk failure groups that do not have a single point of failure. Or, alternatively, if you prefer to let your underlying SAN or RAID controller deal with data redundancy, setup ASM disks groups with external redundancy.
  • 8. Re: ASM Setup Questions
    baskar.l Pro
    Currently Being Moderated
    HI @user10437903 sorry to use your thread for asking this question.

    How to find the RAID level for currently running ASM and its disks?

    Thank you,
    baskar.l
  • 9. Re: ASM Setup Questions
    Levi-Pereira Guru
    Currently Being Moderated
    baskar.l wrote:
    HI @user10437903 sorry to use your thread for asking this question.

    How to find the RAID level for currently running ASM and its disks?
    Does not exists "RAID" on ASM.
    RAID can be implemented as a software component of a computer system or as software operating inside of a storage array.
    RAID operates on the physical presentation of storage as opposed to Oracle ASM mirroring and striping at the logical file level.

    As Dude mentioned on previous post:
    ASM is not RAID. Although it provides obviously similar features in terms of data striping and mirroring, the underlying technology is very different. ASM works on the bases of file extents and free disk space on disk failure groups. For instance you can add ASM disks to increase capacity without having to rebuild your volumes, which you cannot do with a typical RAID system unless you use JBOD.
    As Oracle ASM use "REDUNDANCY LEVEL" instead "RAID": It can be EXTERNAL,NORMAL or HIGH

    You can check what "REDUNDANCY LEVEL" is configured on your DISKGROUP issuing command below:
    as grid installation owner:
    $ asmcmd lsdg |awk '{print $13,$20,$2}'
    To know if there is a RAID configured under ASM. You should check on OS or Storage Array
  • 10. Re: ASM Setup Questions
    user10437903 Newbie
    Currently Being Moderated
    I am limited to the following setups due to $$$$$ and availlable hardware. I got a tremendous amount of information that was very good to know however, I didn't get my questions answered. I do understand that my setup is far from optimal. The only other Setup that wasn't included is the use of ASM redundancy instead of RAID 5. Thanks guys for the very useful ASM and RAID setup info. I would appreciate it if some could take a shot at my specific questions.


    I am looking to see what situation is better regarding my ASM setup and what the striping behavior would be. I am using OEL 5 update 2 x86-64 and Oracle Database 11gR2 and ASM 11gR2.

    Setup 1
    4 Physical HD RAID 5
    4 100 Gb LUNS
    1 ASM Disk for each LUN
    2 ASM Disk Groups (DATA and FRA)
    2 ASM Disks per ASM Disk Group

    Setup 2
    4 Physical HD RAID 5
    2 200 Gb LUNS
    1 ASM Disk for each LUN
    2 ASM Disk Groups (DATA and FRA)
    1 ASM Disk per ASM Disk Group

    Question 1:
    In Setup 1, since ASM Disks for Data Disk Group are on the same physical disks, would this cause significant hot spots. I understand there would undoubttedly be contention?

    Question 2:
    Setup 2, ASM would not fail since it can't stripe in this setup with only one ASM Disk. In other words, ASM would work but simply wouldn't stripe even though the underlining Physical HD are being striped via RAID 5. Please confirm?

    Question 3:
    Which setup is better in terms of availability and performance?

    Question 4:
    Can I slice a partition out of several disks for ASM with ASM Redundancy and use the free portion of the disks for other mount points for the OS? Does ASM have to use the entire Physical Disk when using ASM Redundancy.?
  • 11. Re: ASM Setup Questions
    Levi-Pereira Guru
    Currently Being Moderated
    user10437903 wrote:
    I am limited to the following setups due to $$$$$ and availlable hardware. I got a tremendous amount of information that was very good to know however, I didn't get my questions answered. I do understand that my setup is far from optimal. The only other Setup that wasn't included is the use of ASM redundancy instead of RAID 5. Thanks guys for the very useful ASM and RAID setup info. I would appreciate it if some could take a shot at my specific questions.


    I am looking to see what situation is better regarding my ASM setup and what the striping behavior would be. I am using OEL 5 update 2 x86-64 and Oracle Database 11gR2 and ASM 11gR2.

    Setup 1
    4 Physical HD RAID 5
    4 100 Gb LUNS
    1 ASM Disk for each LUN
    2 ASM Disk Groups (DATA and FRA)
    2 ASM Disks per ASM Disk Group

    Setup 2
    4 Physical HD RAID 5
    2 200 Gb LUNS
    1 ASM Disk for each LUN
    2 ASM Disk Groups (DATA and FRA)
    1 ASM Disk per ASM Disk Group
    Question 1:
    In Setup 1, since ASM Disks for Data Disk Group are on the same physical disks, would this cause significant hot spots. I understand there would undoubttedly be contention?
    The awnser is YES. It's possible since you are using same array to store RECOVER FILES (such as ARCHIVELOGS,BAKCUP,and so on) and Datafiles.

    You need to know what is the I/O characteristic of your database. e.g How many IOPS your storage achieves vs How many IOPS you really need.
    Question 2:
    Setup 2, ASM would not fail since it can't stripe in this setup with only one ASM Disk. In other words, ASM would work but simply wouldn't stripe even though the underlining Physical HD are being striped via RAID 5. Please confirm?
    Yes..ASM Stripes only with 2 ASMDISK or above. ASM will work with one ASMDISK. If you need add another ASMDISK on the future you must keep mind that this new ASMDISK must have same SIZE (i.e 200GB).
    Question 3:
    Which setup is better in terms of availability and performance?
    How many controller you have? In this setup is considered a small env.

    I recommend you create:
    4 Physical HD RAID 5
    8 50 Gb LUNS
    2 ASM Disk Groups (DATA and FRA)
    4 ASM Disks per ASM Disk Group
    Since you are using EXTERNAL REDUNDANCY perfomance and availability depend only of STORAGE ARRAY, not from ASM or it's configuration.

    Recommend small Luns because you can move/add ASMDISK from one DISKGROUP to another more easily.
    Question 4:
    Can I slice a partition out of several disks for ASM with ASM Redundancy and use the free portion of the disks for other mount points for the OS? Does ASM have to use the entire Physical Disk when using ASM Redundancy.?
    Don't do that. It's a poor configuration and it can cause a lot problem of HA and Performance.


    Oracle Support on the note “Lun Size And Performance Impact With Asm [ID 373242.1]” and on the note “How to Prepare Storage for ASM [ID 452924.1]” say that in different things:

    How to Prepare Storage for ASM
    - Maximize the number of disks in a disk group for maximum data distribution and higher I/O bandwidth.
    - Create large LUNs to reduce LUN management overhead.

    Lun Size And Performance Impact With ASM
    Size alone should not affect the performance of a LUN. The underlying hardware is what counts. There is no magic number for the LUN size. Seek the advice of the storage vendor to recommend the best configuration from a raid 1 or 5 perspective for performance and availability since this may vary between vendors
    For larger LUNs it is recommended using a large ALLOCATION UNIT.

    This should be analyzed and applied the best option for your environment.
  • 12. Re: ASM Setup Questions
    Dude! Guru
    Currently Being Moderated
    Whether you use 4 x 100 GB or 2 x 200 GB for your setup is all just lipstick as far as data availability and performance concerns. All your I/O and availability boils down to one data path, one controller, and one and the same underlying physical hardware. For this reason, ASM won't make any difference for performance and redundancy and there is no reason for you to worry about it.

    For instance, if you have 1 hard drive and split it into 2 partitions, then use RAID-0 to stripe the 2 partitions, it will give you the same performance as using the same drive with 1 partition. It might even be slower because of the unnecessary I/O overhead. You don't gain performance from striping because of striping, but because of putting the workload onto two or more physical devices instead of 1. The same goes for data redundancy. What could possibly be the point to split a disk into two partitions and then mirror the two partitions?

    Keep in mind that with your underlying RAID-5 setup, only one of the 4 disks can die, otherwise you loose all data. ASM won't fix that. I recommend you do some reading to understand what RAID is all about, and also what ASM is all about. To answer your questions, I suggest you setup ASM disk groups with external redundancy. There is no reason or performance gain if you partition or create logical drives on the underlying RAID controller. You might as well assign one drive, two drives or 4 drives, it's just cosmetics and does not matter in your setup.
  • 13. Re: ASM Setup Questions
    Sebastian Solbach (DBA Community) Guru
    Currently Being Moderated
    Hi Dude,

    as I have written above, having one lun or 4 luns does indeed make a difference performance wide.
    I agree that you end up on the same disks, which the same heads and the same speed over the same wire. However the way ASM works for the O/S it seems that the direct I/O which the O/S does to an ASM disk does indeed serialize if you only have one lun instead of 4.
    This effect is recognizable performance wide up to 4 luns. After that it really does not change. That is where I base my recommendations on, that a diskgroup should at least contain 4 luns, irrespecitively of the storage.

    Also it does have another advantage, if you need to grow. Online resize of a lun/ASM disk, while technically available online, sometimes poses diffulties in getting it through 100% online . So it is easier to add a lun, than to extend it (note: I did not say online resize is not possible, just more complex). And if you add more disks to an ASM diskgroup, they should be of the same size. So if you have a diskgroup consistent of 1 lun only, you have a big "jump" in increase, than if your diskgroup would have been consistent of 4 disks, and you add one disks with the same size.

    However more important than the luns itself, should be the alignment of the disks, and the stripe size chosen at the storage level.
    Ideally the striping of the storage should be the same as the AU size. If this is not possible, than a multitude, last a dividend. If you adhere to this, then double striping is of no issue and does not hurt.

    Regards
    Sebastian
  • 14. Re: ASM Setup Questions
    Dude! Guru
    Currently Being Moderated
    From the perspective of native or raw disk I/O performance there should be no performance gain due to a partitioned or logical device setup. The I/O performance of logical devices may however be different depending on the physical locations on spinning media. The outer sectors (beginning of a disk) are typically substantially faster.

    If ASM shows a better performance in a 4 LUN vs. a 1 LUN setup that uses the same physical storage media, then there is probably a positive affect due to multiple process threads and better CPU utilization. However, if the system I/O resources are exhausted, multiple threads and processes won't make a difference. It depends on the system, OS, etc. But I admit there is no reason to deny such an advantage by a limited setup. So creating 4 LUN's is therefore a good idea.

    Data redundancy features of ASM in this setup are doomed, since no disk failure group is on independent media. Using multiple LUN's may give the false impression of security. Regarding resizing, I think the OP will be limited by the RAID-5 setup anyway. The OP cannot add space without reinitializing the underlying RAID or adding an additional RAID controller.
1 2 Previous Next

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points