6 Replies Latest reply: Dec 29, 2012 12:29 PM by Victor Armbrust RSS

    Wrong ASM Disk Group Configuration?

    CoBy
      Hello together,

      I am working on OEL 6.3 with 11.2 RAC Environment. Recently I noticed that the cluster is not running and started investigating the error messages in alertlog. There I saw many
      CRS-1714:Unable to discover any voting files, so I decided to check the ASM-Configuration. After starting the css with the force option, I could login to ASM:
      There I ran following queries

      SQL> select path, header_status from V$asm_disk;

      PATH HEADER_STATU
      ---------------------------------------- ------------
      ORCL:BCKP PROVISIONED
      ORCL:CREP1 MEMBER
      ORCL:CREP2 MEMBER
      ORCL:CREP3 MEMBER
      ORCL:VDISK3 PROVISIONED
      ORCL:FRA MEMBER
      ORCL:VDISK1 PROVISIONED
      ORCL:VDISK2 PROVISIONED
      ORCL:DATA MEMBER

      9 rows selected.

      HERE I read, that VDISKx are not included in any disk group. Is that correct?

      Then I run:

      SQL> select d.name, d.path, dg.name as dg_name from v$asm_disk d, v$asm_diskgroup dg where d.group_number = dg.group_number;

      NAME PATH DG_NAME
      ------------------------------ ---------------------------------------- ------------------------------
      ORCL:BCKP CRS
      ORCL:DATA DATA
      ORCL:CREP2 CRS
      ORCL:CREP3 CRS
      ORCL:VDISK3 CRS
      ORCL:FRA CRS
      ORCL:VDISK1 CRS
      ORCL:VDISK2 CRS
      ORCL:DATA CRS
      ORCL:BCKP FRA
      ORCL:CREP1 FRA
      ORCL:CREP2 FRA
      ORCL:CREP3 FRA
      ORCL:VDISK3 FRA
      ORCL:FRA FRA
      ORCL:VDISK1 FRA
      ORCL:VDISK2 FRA
      ORCL:DATA FRA
      ORCL:BCKP DATA
      ORCL:CREP1 DATA
      ORCL:CREP2 DATA
      ORCL:CREP3 DATA
      ORCL:VDISK3 DATA
      ORCL:FRA DATA
      ORCL:VDISK1 DATA
      ORCL:VDISK2 DATA
      ORCL:CREP1 CRS

      27 rows selected.

      This means, that each of the ASM-Volumes is in each Disk Group. This is somehow confusing for me. Is this correct? Should I have only one group for each purpose, for example VDISKx are my voting disks, CREPx are repository, FRA is Fast recovery area and so on...?

      Thanks for any advice how to (re)configure this.
        • 1. Re: Wrong ASM Disk Group Configuration?
          Victor Armbrust
          Hi

          How do you create that disks? Are you using ASMLIB? Can you please post "/etc/init.d/oracleasm listdisks"
          • 2. Re: Wrong ASM Disk Group Configuration?
            CoBy
            Yes, I used asmlib. Here is the output.

            [root@srv01 dbs]# oracleasm status
            Checking if ASM is loaded: yes
            Checking if /dev/oracleasm is mounted: yes

            [root@srv01 dbs]# oracleasm listdisks
            BCKP
            CREP1
            CREP2
            CREP3
            DATA
            FRA
            VDISK1
            VDISK2
            VDISK3

            [root@srv01 dbs]# oracleasm querydisk CREP1
            Disk "CREP1" is a valid ASM disk

            [root@srv01 dbs]# oracleasm -V
            oracleasm version 2.1.5
            • 3. Re: Wrong ASM Disk Group Configuration?
              Victor Armbrust
              Yes your should use the diskgroups for separated purposes as you mentioned.

              Is your Cluster runing fine?
              crsctl check css
              What is your Voting disks destination?
              ocrcheck
              • 4. Re: Wrong ASM Disk Group Configuration?
                CoBy
                No, it is not starting properly, the error is

                CRS-1714:Unable to discover any voting files as already mentioned.

                For some reason the voting disks became unavailable for the service and therefore I started investigating.

                I started the grid infrastructure with the -excl and -nocrs as I needed the asm Instance to be up in order to fetch the info above.

                +[root@srv01 bin]# ./ocrcheck+
                Status of Oracle Cluster Registry is as follows :
                Version                  :          3
                Total space (kbytes)     :     262120
                Used space (kbytes)      :       2920
                Available space (kbytes) :     259200
                ID                       : 1433305389
                Device/File Name         :       CRS+
                Device/File integrity check succeeded

                Device/File not configured

                Device/File not configured

                Device/File not configured

                Device/File not configured

                Cluster registry integrity check succeeded

                Logical corruption check succeeded
                • 5. Re: Wrong ASM Disk Group Configuration?
                  krishan
                  Check if file "/opt/oracle/extapi/64/asm/orcl/1/libasm.so" exists.

                  If yes, change the permission for /opt/oracle directory so that "others" on the server can have "executable" permission.

                  chmod o+x /opt/oracle

                  Restart ASM instance.

                  If still issue persists:

                  You need to re-install the ASMLIB
                  • 6. Re: Wrong ASM Disk Group Configuration?
                    Victor Armbrust
                    You can also try to restore a backup of your voting files. Another option is to re-install oracle Clusterware.