9 Replies Latest reply: Jan 29, 2013 1:54 PM by Jcgo-Oracle RSS

    Only ONE voting disk created on EXTERNAL redundancy

    Jcgo-Oracle
      Hi All,

      I am wondering why, when configuring 5 LUNs, then creating one DG with these 5 LUNS...
      then creating OCR/Vote disk , GRID installer only creates one voting disk?
      Of course, I can always create another voting disk in a separate DG...

      any ideas>?
      of course, when I try " ocrconfig -add +OCR_DG" i get a message that says OCR is already configured....

      thanks
        • 1. Re: Only ONE voting disk created on EXTERNAL redundancy
          Sebastian Solbach -Dba Community-Oracle
          Hi JCGO,

          you are mixing up voting disks and OCR.
          Voting disks are placed on the disk header of an ASM diskgroup. The number of voting disks is based on the redundancy of the diskgroup:
          External (no mirroring) = 1; Normal = 3; High redundancy (triple mirroring) = 5;

          The OCR however is saved as a normal file (like any datafile of a tablespace) inside ASM and therefore "inherits" the redundancy of the diskgroup.
          So while you only see 1 file, the blocks of the file are mirrored correspondingly (either double or triple (High)).

          Voting disks can only reside in one diskgroup. You cannot create another voting disk in a separate DG. And the command to add voting disks ist:
          crsctl add votedisk
          However this won't work when on ASM, you can only replace them into another DG:
          crsctl replace votedisk +DG
          The OCR on the other side, since a normal file, allows additional OCRs to be created in different diskgroups.
          Since it is the heart of the cluster (and also containing backups of the voting disks) it does make sense to "logically" mirror it additionally (even if it already is based on the mirroring of the DG). Since if the DG cannot be mounted maybe another one containing actual OCR can. This is a little better than using the OCR backup (which can be up to to 4 hours old).

          Hence in my cluster installation I always put the OCR in every diskgroup I have (up to a number of 5).
          And since my clusters normally have 3 diskgroups (1 OCRVOT, 1 DATA and 1 FRA), I end up with 3.
          Having an uneven number of OCR is a little superior over having an even number (so 3 or 5 vs. 2 or 4). I skip the explanation to why.

          Regards
          Sebastian
          • 2. Re: Only ONE voting disk created on EXTERNAL redundancy
            Jcgo-Oracle
            Thanks Sebastian.

            I do agree with you 100%.

            The Voting disk is place on the disk header of an actual ASM created disk in a diskgroup.
            So I will just have to depend on the redundancy of the hardware itself, RAID array, etc for the voting disks.

            Is there any command to show how many OCR disks I have?
            I have tried ocrcheck and it doesnt show the OCR output..

            thanks
            • 3. Re: Only ONE voting disk created on EXTERNAL redundancy
              Sebastian Solbach -Dba Community-Oracle
              Hi,

              ocrcheck is indeed the right tool.
              It will tell you how many OCR you have, and how many are intact.

              Look at the bottom of the output.

              If you only have one (which will be listed), then it will tell you other as "not configured".

              Regards
              Sebastian
              • 4. Re: Only ONE voting disk created on EXTERNAL redundancy
                phaeus
                Hello,
                here is a short example for sebastions post.
                [oracle11@rac01 ~]$ ocrcheck
                Status of Oracle Cluster Registry is as follows :
                      Version                  :          3
                      Total space (kbytes)     :     262120
                      Used space (kbytes)      :       3144
                      Available space (kbytes) :     258976
                      ID                       : 1435023663
                      Device/File Name         : +DG_CLUSTER
                                                    Device/File integrity check succeeded
                
                                                    Device/File not configured
                
                                                    Device/File not configured
                
                                                    Device/File not configured
                
                                                    Device/File not configured
                
                      Cluster registry integrity check succeeded
                
                      Logical corruption check bypassed due to non-privileged user
                regards
                Peter
                • 5. Re: Only ONE voting disk created on EXTERNAL redundancy
                  Sebastian Solbach -Dba Community-Oracle
                  Hi Peter,

                  thx. Did not have a cluster at hand atm.

                  Regards
                  Sebastian
                  • 6. Re: Only ONE voting disk created on EXTERNAL redundancy
                    Jcgo-Oracle
                    Thanks Peter/Sebastian..

                    Sorry to be sort of hard-headed here...but just curious..

                    1). When configuring the EXT REDUNDANCY option...why does the installer only create 1 Vote disk ?
                    I understand that Oracle expects you to have hardware RAID or similar if you choose EXT redundancy, but why doesnt it create 3 voting disks to begin with?

                    2). I know we can use the "md_backup" tools to backup the voting disks, but then Oracle also goes on to tell you, that you cannot create another voting disk in the same DG...and to top it all off, it tellls you that the Disk group will be offlined if there is a failed write to the disk group....
                    SO why even bother? The DG has Hardware RAID, but yet Oracle will offline the entire DG if it detects a failed write?

                    Im sorry, just being hard headed, but that sort of does not make sense....?

                    thanks
                    • 7. Re: Only ONE voting disk created on EXTERNAL redundancy
                      phaeus
                      Hello,
                      the redundancy level set the maximum voting files for a diskgroup

                      External redundancy: A disk group with external redundancy can store only one voting disk
                      Normal redundancy: A disk group with normal redundancy stores three voting disks
                      High redundancy: A disk group with high redundancy stores five voting disks

                      The point why ? To mirror voting disk make only sense between different failover groups in a diskgroup.

                      What you can do with a single storage (and i think it would be a good pratice) is to create a diskgroup with normal (or maybe high) redundancy even when you have one storage. In that case you whould also get more voting files. You can also change the file template in a diskgroup to get a fine mirror so that every block is mirrord against all partner disks. In that case you can offline one disk without loose your complete diskgroup.

                      Also in my recovery szeanrios i have recreated the voting disks and not restore it from a backup (OCR files where restored from backup).

                      regards
                      Peter
                      • 8. Re: Only ONE voting disk created on EXTERNAL redundancy
                        Jcgo-Oracle
                        Hi Peter

                        When you say recreate the voting disks, what is the impact of this? and why not restore from backup?

                        thanks
                        • 9. Re: Only ONE voting disk created on EXTERNAL redundancy
                          Jcgo-Oracle
                          so what I did was:

                          1). Since its external redudn and 5 LUNS, I removed 3 LUNS from the existing DG

                          2). Used the LUNS to create a new DG with Niormal REDUN...

                          3),. Moved Voting disks to new DG.

                          thanks