5 Replies Latest reply on Apr 30, 2014 10:09 PM by J.A.

    Disk groups configuration

    Pierre Timmermans



      I am preparing a 2 nodes RAC with Oracle 12c for evaluation purpose and I am looking for some advise on how I should prepare the storage.


      I have a storage array with 16 disks of 600GB each.


      My requirements are

      - follow best-practices (up to a point, not if it is too costly)

      - ease of management is more important than to get the last bit of performance.


      The more I read documentations, forums, blogs the more confused I become about how I should create my ASM disk groups.


      I have two scenarios in mind. I did implement the first one in my evaluation but then I realized that it might not fulfill the best-practice requirement.


      1. create two luns at the SAN, each LUN with 8 disks in RAID0 10. Then create two asm disk groups: +DATA and +FRA. Then put the voting files on +DATA, put the OCR files on both disk groups, the db on +DATA and the fra on +FRA. It all seemed logical and easy to do. But then I start to find contractictory information that Oracle recommends to have 3 voting files and have them on separate disks.


      2. I have a red-had "Best-practice" white paper (deploying Oracle 11g on RAC). In their reference architecture they create much more LUN's at the storage level, specially they create 3 LUN's dedicated to the voting files. Each of those 3 LUNs contains two disks in mirror. From the 3 LUNS  they then create 1 asm disk group in normal redundancy. But this seems to me a huge waste of space in my case, because I would lose 6 of my disks to do that set-up. I could do the same without the external redundancy so losing only 3 disks (no external redundancy, only the ASM redundancy). But it is still a big waste of space considering that will be allocating 3x600G while the voting file is only 1G in size !!


      Can someone advise if scenario 1 is acceptable or not ? I can't really understanding what is the point of mirroring the voting files like crazy, because if I loose my +DATA disk group i lose my RAC anyway, not ? Or maybe there is another alternative that I did not realize ?


      Also I was reading something strange in the recommandation section of the ASM admin guide: they say, "The number of LUNs (Oracle ASM disks) for each disk group should be at least equal to four times the number of active I/O paths. For example, if a disk group has two active I/O paths, then minimum of eight LUNs should be used. The LUNs should be of equal size and performance for each disk group."

      Does this means I should not ask the storage admin to create 2 big LUN with 8 disks each but instead ask him to create 8 LUN (8 times 2 disks) ? It seems that it would add quite some management overhead to me ?


      I hope that someone can advise on which way I should follow or point me to a good documentation/discussion


      Thanks and regards, Pierre

        • 1. Re: Disk groups configuration



          I recommend you this configuration:

          1 Diskgroup --> +DATA             --> For your DB                       --> Number of DIsks and Sizes depends of your DB requirements:  Example:  2 Disks of 50GB

          1 DiskGroup --> +FRA              -->  For Fast Recovery Area.     --> Number of DIsks and Sizes depends of your DB requirements   Example:  2 Disks of 50GB

          1 Diskgroup -->  +OCR_VOTE   --> OCR and Voting Disk           --> 3 Disks of 1 GB (Its necessary to use 3 disks at least)


          If you have external redundancy on your storage you can create the DiskGroup as External, If you don't have external redundancy you can create the diskGroup as Normal.


          Never mix the OCR/Voting Disk with your DB Diskgroup.


          If you choose to put the voting disks into an Oracle ASM disk group, then Oracle ASM ensures the configuration of multiple voting disks if you use a normal or high redundancy disk group


          I hope this help you:

          Managing Oracle Cluster Registry and Voting Disks




          • 2. Re: Disk groups configuration
            Pierre Timmermans



            Thanks for the answer and also for the pdf, looks very interesting I will check in depth.


            I get the point that I should not put the OCR/Voting disks on the same ASM disk group as the +DATA or the +FRA. I will put the voting file and the OCR on ASM because I saw that it is strongly recommended (and it eases management)


            I can create a +OCR_VOTE disk group, with 3 disks. What is annoying for me is that my disks are all 600G, I don't have disks of smaller capacity, so I will allocate 3x600G (and even 6x600 if I use hardware mirroring) to the OCR and vote files. Or maybe I am missing something and I can create disks of 1G (partitions ?) even though the physical disks are 600GB ?


            Another question for the +DATA disk group : is it OK to create one big lun at the SAN level and so create the disk group based on one LUN or is it better to create multiple LUN's at the SAN level and combine them in the disk group ? Note that in any case I will use external redundancy (RAID 10)


            Thanks again and Regards, P

            • 3. Re: Disk groups configuration

              Well its going to be impractical to use 3 disks of 600Gb :-/  . So maybe you can check with you sysadmin or hardware admin to create smaller Luns. 


              If you can´t have small Luns, then you can try to take one Luns of 600GB and make different partitions and then with that new partitions create ASM Disk and then create your Diskgroups. I think this is not a certificate configuration by Oracle but its only for your evaluation purpose.


              On the other hand I prefer to have "Small" Luns for my Diskgroups for Example: +DATA --> 4 Disks of 100 GB .  If my DB need more space I only add another disk of 100GB, so I only use what I really need on Storage and I like to leave 35% at least for free space on each Diskgroup.


              I hope this help you



              • 4. Re: Disk groups configuration
                Pierre Timmermans

                Thanks, I will try to get some support from a storage engineer, until now I got minimal support and I am left on my own to do everything (which is fine to learn stuff but at the same time I may be missing obvious thinks :-)). I followed a good white papers from Red-hat (http://www.redhat.com/resourcelibrary/reference-architectures/deploying-oracle-rac-11g-r2-database-on-red-hat-enterprise…) but their disk allocation to volume groups is strange : they allocate 6 physical disks for the OCR+VOTE disk group) and I have seen nowhere that it is a requirement from Oracle. It is not even clear to me why I can't just put the voting disk on my +DATA group (performance or reliability ?)


                It is not clear for me if having a lun of 600G and then partitioning it so that I can allocate smaller chunks to multiple disk groups makes sense or not.


                Thanks again and regards, P

                • 5. Re: Disk groups configuration



                  the best practice is to create the OCR Vote on one dedicated diskGroup because of the performance, reliability and Rac administration.


                  For Example I did this on one Rac for testing purpose:

                  11g R2 on Oracle Linux 5.9

                  1 Lun of 600Gb with three partitions

                  1 partition of 250 for +DATA

                  1 partition of 347 for +FRA

                  1 partition of 3GB for OCR_VOTE


                  Each partition its a ASM disk with AsmLib, with this configuration I only use one disk (one partition of 3GB) on one diskgroup for the OCR_Voting disk.


                  I thinks this is not the best configuration and I think its not supported by Oracle, but maybe can help you.