7 Replies Latest reply: Jul 25, 2013 1:38 AM by Billy~Verreynne RSS

    multiple disk group pros/cons

    user13454469

      hello all,

       

      This is with regards to 11.2.0.3 DB(RAC) on RHEL 6

       

      i am trying to identify the pro's/con's of using multiple ASM Diskgroup.  I understand oracle recommends/best practice is to have 2 DG (one data and one flash) and you can place multiple copies of control files/online redo logs(and thats the way i want to go).  But would that same be true if i use different set of DISK.  For example we have multiple RAID 10 devices and multiple of SSD devices for us that we can use for this ASM instance.  And i was thinking to create 2 more Disk group (call it DG_SYS1 and DG_SYS2)  and use that to put my online redo logs, control file and temp and system table space there. 

       

      i understand in a standalone system(where regular file system is being used), they(online redo/control file) are usually on there own drives, but with ASM when i am already using external RAID 10 config + ASM striping i assume the IO would faster or am i better of using the SSD that i can have for my redo/control?  What would be the pro's/cons of it (besides managing multiple DG)..

        • 1. Re: multiple disk group pros/cons
          Aman....

          Reason that Oracle suggests to have two disk groups is because the very idea of ASM is the storage consolidation and to take the best advantage of that storage for all the databases. But having two dg's is not a norm. If you have different kinds of databases, if you have different capacity disks, you probably should have more dg's. Also, I am not sure why you are using RAID 0 along with ASM striping?

          user13454469 wrote:

           

          hello all,

           

          This is with regards to 11.2.0.3 DB(RAC) on RHEL 6

           

          i am trying to identify the pro's/con's of using multiple ASM Diskgroup.  I understand oracle recommends/best practice is to have 2 DG (one data and one flash) and you can place multiple copies of control files/online redo logs(and thats the way i want to go).  But would that same be true if i use different set of DISK.  For example we have multiple RAID 10 devices and multiple of SSD devices for us that we can use for this ASM instance.  And i was thinking to create 2 more Disk group (call it DG_SYS1 and DG_SYS2)  and use that to put my online redo logs, control file and temp and system table space there.

           

          i understand in a standalone system(where regular file system is being used), they(online redo/control file) are usually on there own drives, but with ASM when i am already using external RAID 10 config + ASM striping i assume the IO would faster or am i better of using the SSD that i can have for my redo/control?  What would be the pro's/cons of it (besides managing multiple DG)..

          Aman....

          • 2. Re: multiple disk group pros/cons
            user13454469

            hi Aman, Thanks for the info.  i know having multiple 2 DG is not the norm, but in my case i have options of using different type of underlying disk(RAID 10 and SSD), so you are saying i am better of creating those additional DG and place my redo/control files on it....

             

            Also, you bring up a good point.  Do i really need striping at SAN level.  Maybe i am just wrong.  But would having striping at SAN level and also stripping at ASM level = faster IO (because the data will be so randomly spread accross).  Or am i just better using RAID 1 and let ASM doing the striping?  what are the pro/con on this?

            • 3. Re: multiple disk group pros/cons
              jgarry

              There are so many types of SSD and so many variants of application usage all you can say is It Depends.  Also, whether you can set up redo so that it posts as a complete write when the first of a group reports success varies by version and hardware, and that determines whether or not you should put redo on SSD.  That usually means no on a low end system.

               

              The original SAME idea came about because someone figured out that you can get the best performance by spreading I/O among the most spindles of spinning rust.  With SSD's, you wind up having a scarce resource of faster I/O, so you can no longer make such a generalization.  Specifying different levels of I/O speed has been around for ages (I remember watching an IBM robot pluck hexagonal drive boxes in and out of an array, and watching a tape robot zip up and down rows of tape drives, with a few dropped tapes scattered about, 30 years ago).

               

              In the end, it boils down to understanding what is critical I/O, what is sequential I/O, what is random I/O, and how to make them not bump into each other.  Here's a link to something I didn't even read: Solid Choices for Oracle Tuning on Solid State Disk  DBA Kevlar

              • 4. Re: multiple disk group pros/cons
                Aman....

                user13454469 wrote:

                 

                hi Aman, Thanks for the info.  i know having multiple 2 DG is not the norm, but in my case i have options of using different type of underlying disk(RAID 10 and SSD), so you are saying i am better of creating those additional DG and place my redo/control files on it....

                 

                Also, you bring up a good point.  Do i really need striping at SAN level.  Maybe i am just wrong.  But would having striping at SAN level and also stripping at ASM level = faster IO (because the data will be so randomly spread accross).  Or am i just better using RAID 1 and let ASM doing the striping?  what are the pro/con on this?

                IMO letting ASM do the handling of striping on its own and the h/w manage the mirroring is a better solution.

                 

                Aman....

                • 5. Re: multiple disk group pros/cons
                  user13454469

                  Any specific reason as to why?  Wouldn't having stripping at hardware level and then stripping again by ASM(double stripping) reduce the IO ?

                  • 6. Re: multiple disk group pros/cons
                    JohnWatson

                    You should not mix different types (or sizes) of device in one disk group. So you'll need separate groups(s) for your SSD devices. Your suggested layout looks fine to me: have your datafiles on one disk-based group, your FRA on another, and your online logfiles and controlfiles on two SSD-based groups.

                    You certainly want to use RAID 0 striping: that way you will be presenting high performing LUNs to ASM as ASM disks, and ASM can intelligently stripe its file extents across them. RAID 10 is a different matter: I would not like to say whether it is better to rely on AM to do mirroring or let your hardware do it. That is really a matter of policy: who is responsible for fault tolerance, the DBAs or the SAs?

                    • 7. Re: multiple disk group pros/cons
                      Billy~Verreynne

                      There are always exceptions.

                       

                      I often use multiple diskgroups. Reasons include different SAN RAID levels, different SAN LUNs on same server I/O fabric layer, different LUN sizes and so on - and you do not want to create diskgroups with "odd" disks. Other reasons include diskgroup usage. Single large RAC, n number of policy managed RAC databases - ideally at least 1 diskgroup per database to separate storage usage and effectively manage storage footprint. You may want a dedicated diskgroup for Grid (as high redundancy and not normal redundancy) . You may want a dedicated external redundancy diskgroup for use as a cluster file system. Etc.

                       

                      So do not take the recommendation-for-the-norm from Oracle as an absolute. As what is normal is very relative in today's IT environments and infrastructures.