14 Replies Latest reply: Oct 9, 2013 1:04 PM by Levi Pereira RSS

    can ocr and voting files reside on same disk?

    SaikatBanerjee

      Hi,

       

      We are using 11.2.0.3 on linux.

       

      When I created OCRVOTE diskgroup with 5 raw disks, oracle by default created 3 voting disks and used rest 2  disks to keep OCR files.

      But , when I created OCRVOTE diskgroup with 5 raw disks, I found that all the 3 disks are votingdisks and OCR also located in the same diskgroup.


      How can it be possible?..i.e, is it possible  for OCR files to be placed on disks marked as vote-disk?

       

       

      Regards,

      Saikat

        • 1. Re: can ocr and voting files reside on same disk?
          Flávio Soares

          Yes, it is entirely possible from 11g Release 2.

          • 2. Re: can ocr and voting files reside on same disk?
            SaikatBanerjee

            ok.... when I used 5 disks to create the OCR_VOTE diskgroup, then my output was as below...

             

            [grid@VM-PAGRAC2 cdata]$ ocrcheck
            Status of Oracle Cluster Registry is as follows :
                     Version                  :          3
                     Total space (kbytes)     :     262120
                     Used space (kbytes)      :       2900
                     Available space (kbytes) :     259220
                     ID                       :  767211899
                     Device/File Name         :  +OCR_VOTE
                                                Device/File integrity check succeeded
            
            
                                                Device/File not configured
            
            
                                                Device/File not configured
            
            
                                                Device/File not configured
            
            
                                                Device/File not configured
            
            
                     Cluster registry integrity check succeeded
            
            
                     Logical corruption check bypassed due to non-privileged user
            
            
            [grid@VM-PAGRAC2 cdata]$ crsctl query css votedisk
            ##  STATE    File Universal Id                File Name Disk group
            --  -----    -----------------                --------- ---------
             1. ONLINE   7aa6082f0efd4fc2bf775b3b84effa73 (/dev/raw/raw1) [OCR_VOTE]
             2. ONLINE   a3259f016e374f16bf40baf38c355f2b (/dev/raw/raw2) [OCR_VOTE]
             3. ONLINE   1f6de5f98a004fb2bfe353210668cc44 (/dev/raw/raw3) [OCR_VOTE]
            Located 3 voting disk(s).
            [grid@VM-PAGRAC2 cdata]$
            
            
            

             

            Here votedisks are clearly the three disks raw1,raw2,raw3 . But for OCR location only diskgroup name +OCR_VOTE is shown. So, does the ocr files resides on rest 2 disks in the diskgroup? ..or, they are striped to all the five disks in the OCR_VOTE diskgroup?

             

            Regards,

            Saikat

            • 3. Re: can ocr and voting files reside on same disk?
              Flávio Soares

              Yes, that its correct!

               

              When you use normal redundancy disk groups for Oracle Clusterware, its provide 3 voting disk files, 1 ocr. You can have 2 ocr only when you use the OCR MIRROR, that add one more disk used for the OCR.

               

              Check the document: Configuring Storage for Grid Infrastructure for a Cluster and Oracle Real Application Clusters (Oracle RAC)

              • 4. Re: can ocr and voting files reside on same disk?
                SaikatBanerjee
                When you use normal redundancy disk groups for Oracle Clusterware, its provide 3 voting disk files, 1 ocr

                -->But I have selected external redundancy while creating OCR-VOTE diskgroup, not normal redundancy. Still got 3 voting disks

                ---> What do you mean by 1 ocr?..the OCR      contents are striped accross all the members (5 disks) of OCR_VOTE diskgroup, including votedisks....and thats why ocrcheck does not show individual disks..rather the diskgroup??....Am I correct?

                 

                Thanks,

                Saikat

                • 5. Re: can ocr and voting files reside on same disk?
                  JohnWatson

                  SaikatBanerjee wrote:

                   

                  <snip>

                   

                  Here votedisks are clearly the three disks raw1,raw2,raw3 . But for OCR location only diskgroup name +OCR_VOTE is shown. So, does the ocr files resides on rest 2 disks in the diskgroup? ..or, they are striped to all the five disks in the OCR_VOTE diskgroup?

                   

                  Regards,

                  Saikat

                  Your mistake is to think that the vote disks are the raw devices. They are not. Even though you call them "vote disk" they are in fact files residing in the disc group, so they can co-exist with any number of any other files in the group. Unlike other file, thy are not striped.

                  • 6. Re: can ocr and voting files reside on same disk?
                    SaikatBanerjee

                    Hi John,

                     

                    Thanks, now my doubt is almost cleared.

                     

                    Can you please respond to the 4th reply  in this thread...that will make my concept clearer.

                     

                    Thanks,

                    Saikat

                    • 7. Re: can ocr and voting files reside on same disk?
                      Pradeepcmst

                      yes thats possible.. Even its good to have both VOTE and OCR files in a same group to avoid crashing or failures..

                       

                       

                       

                      Regards,

                      Pradeep. V

                      • 8. Re: can ocr and voting files reside on same disk?
                        SaikatBanerjee

                        Hi Pradeep,

                         

                        thanks,

                         

                        Can you please tell:--

                         

                        1.the OCR      contents are striped accross all the members (5 disks) of OCR_VOTE diskgroup, including votedisks....and thats why ocrcheck does not show individual disks..rather the diskgroup??....Am I correct?

                         

                        2.minimum no of votedisk in 11gr2

                        3.default no of votedisk when we create OCRVOTE diskgroup during grid installation.

                         

                         

                        From my experience I have found the answers of 2 and 3 to be 3...am i correct?

                         

                        Regards,

                        Saikat

                        • 9. Re: can ocr and voting files reside on same disk?
                          Pradeepcmst

                          Hi Saikat,

                           

                           

                           

                          1)  Voting disks are stored on individual disks Vs OCR which are stored as files in the ASM diskgroup. So voting disks will not span multiple disks i.e. they are confined in single disk. However, its recommended to store both Voting and OCR files in a same diskgroup.. In 11gr2, they made a small change in disk requirement as fas as this voting and ocr storage is concerned. The disk selection should be in odd number. Since Voting disks are stored on individual disks, we cannot store in different diskgroup like other ASM files which provides mirrored copies.For this Voting disk, the mirroring concept is within the diskgroup, which means, if you lose one disk within the diskgroup, you lose one copy of voting disk file.    

                          1 disk for a diskgroup - External Redundancy

                          3 disks for a diskgroup - Normal Redundancy

                          5 disks for a diskgroup - High Redundancy

                           

                           

                           

                          2) Minimum one disk is enough for vodedisk in 11gr2 which is External Redundancy. You use this method only if the OS level Raid redundancy is available to avoid crash or failures

                           

                          3) When i install the grid, it dint show any default number of votedisk when creating OCRVOTE diskgroup. It just asked whats the redundancy we want to use to store OCRVOTE diskgroup. Select the disks according to the redundancy what we select.

                           

                           

                           

                           

                           

                          Hope this helps.

                           

                           

                           

                           

                           

                           

                          Regards,

                          Pradeep. V

                          • 10. Re: can ocr and voting files reside on same disk?
                            Levi Pereira

                            SaikatBanerjee wrote:

                             

                            Hi Pradeep,

                             

                            thanks,

                             

                            Can you please tell:--

                             

                            1.the OCR      contents are striped accross all the members (5 disks) of OCR_VOTE diskgroup, including votedisks....and thats why ocrcheck does not show individual disks..rather the diskgroup??....Am I correct?

                             

                            2.minimum no of votedisk in 11gr2

                            3.default no of votedisk when we create OCRVOTE diskgroup during grid installation.

                             

                             

                            From my experience I have found the answers of 2 and 3 to be 3...am i correct?

                             

                            Regards,

                            Saikat

                            1. Correct.

                            2. Minimum is one, and you must have always an odd number of voting disk.

                            3. Is ASM Diskgroup configuration dependency (EXTERNAL/NORMAL/HIGH), using ASM max number of voting disk is 5, non-ASM storage option the max number is 15.

                             

                            Check this link:

                            Explaining: How to store OCR, Voting disks and ASM SPFILE on ASM Diskgroup (RAC or RAC Extended) | Levi Pereira

                            • 11. Re: can ocr and voting files reside on same disk?
                              SaikatBanerjee

                              Thanks Levi,and pradeep,

                               

                              The link Explaining: How to store OCR, Voting disks and ASM SPFILE on ASM Diskgroup (RAC or RAC Extended) | Levi Pereira is very much helpful.

                               

                              But one doubt:-

                               

                              The link mentions :-"The OCR is totally different from Voting Disk. Oracle Clusterware rely on ASM to access the OCR and SPFILE. The OCR and SPFILE are stored similar to how Oracle Database files are stored. The extents are spread across all the disks in the diskgroup and the redundancy (which is at the extent level) is based on the redundancy of the disk group. For this reason you can only have one OCR in a diskgroup."

                               

                              But in the figure Voting Disk and OCR in 11gR2: Some changes | The Oracle Instructor it is showing the extents are spread accross only 2 disks among the 8 disks.

                               

                              Can you please explain?

                               

                              Regards,

                              Saikat

                              • 12. Re: can ocr and voting files reside on same disk?
                                Levi Pereira

                                Hi,

                                Check: RAC: Frequently Asked Questions (Doc ID 220970.1)

                                How is the Oracle Cluster Registry (OCR) stored when I use ASM?

                                https://support.oracle.com/epmos/faces/DocContentDisplay?id=220970.1

                                 

                                 

                                >>The Oracle Instructor it is showing the extents are spread accross only 2 disks among the 8 disks.

                                Far as I can understand:

                                What Oracle Instructor is showing a example how is stored the mirror of extent where OCR was stored, he used a simple example using only 2 extent showing a extent and extent mirror on different failgroup, does not means wich OCR used only 2 extent or use only 2 asmdisk.

                                On that case Oracle will use all failgroups placing a extent (primary) in each failgroup and storing a extent copy (mirror) in a different failgroup.

                                Concern about redundancy (that was the point):

                                Votedisk you have a voting in each asmdisk.

                                Store OCR using a diskgroup with normal redundancy you have a extent mirrored in a different failgroup. (just it)

                                 

                                 

                                How Oracle Store OCR extent?

                                It spread OCR file across all disks if ASM Diskgroup configuration allow do it.

                                If you have  a OCR with 300MB and you store it in a Diskgroup with external redundancy using  AU_SIZE of 1MB and the Extent Size is 1MB then ASM will spread extent accross all disk of that Diskgroup storing the extents in all asmdisk maintaining balance.


                                But we should consider the size of the OCR file and configuration of ASM Diskgroup

                                 

                                The contents of ASM files are stored in a disk group as a set, or collection, of data extents that are stored on individual disks within disk groups. Each extent resides on an individual disk. Extents consist of one or more allocation units (AU).


                                When you create a disk group, you can set the Oracle ASM allocation unit size with the AU_SIZE disk group attribute. The values can be 1, 2, 4, 8, 16, 32, or 64 MB


                                So, If you create a diskgroup external redundancy with 8 ASMDISK using AU_SIZE of 64MB, EXTENT_SIZE is 64MB and OCR size is 100MB Oracle will use only two ASMDISK using 2 extent of 64 (128MB) does not matter if you have 8 ASM DISK. 


                                Regards,

                                Levi Pereira

                                 

                                 


                                • 13. Re: can ocr and voting files reside on same disk?
                                  SaikatBanerjee

                                  Hi,

                                   

                                  Thanks a lot for your reply and explanation.

                                   

                                  But, I would like to know why minimum 5 disks are required for creating OCRVOTE  diskgroup with high redundancy?..It should work with 3 disks right?..since we can have 2 copies of OCR as per asm high redudancy..thus total 3  disks should work..right?..But it is showing error minimum asm disks required.

                                   

                                  Regards,

                                  Saikat

                                  • 14. Re: can ocr and voting files reside on same disk?
                                    Levi Pereira

                                    But, I would like to know why minimum 5 disks are required for creating OCRVOTE  diskgroup with high redundancy?..It should work with 3 disks right?..since we can have 2 copies of OCR as per asm high redudancy..thus total 3  disks should work..right?..But it is showing error minimum asm disks required.

                                    Voting disk is placed directly on ASM DISK. CSSD does not rely on ASM to access voting disk, it's accessed directly at LUN/ASMDISK level.

                                    Differently OCR is placed on DISKGROUP. CRSD rely on ASM (extent spread out over ASM DISK) to access OCR.


                                    Placing Voting Disk in a Diskgroup with HIGH REDUNDANCY require minimum of 5 ASM DISK, because CSS check redundancy of DISKGROUP and try to create 5 Voting disk.

                                    EXTERNAL REDUNDANCY - CSSD choose one ASM DISK and  create one (1) voting disk.

                                    NORMAL REDUNCANCY - CSSD choose 3 ASM DISK from different FAILGROUP and create 3 VOTING DISK, one in each ASM DISK/FAILGROUP

                                    HIGH REDUNDANCY - CSSD choose 5 ASM DISK from different FAILGROUP and create 5 VOTING DISK, one in each ASM DISK/FAILGROUP.

                                    So, if you are trying move Voting Disk to a diskgroup with HIGH REDUNDANCY, CSSD will try to create 5 VOTING, as it's placed directly on ASM DISK it requires minimum 5 ASM Disk.