1 2 Previous Next 17 Replies Latest reply: Aug 2, 2012 1:29 PM by JDinPA RSS

    Candidate ASM disks not showing up in 11.2 installer (Oracle Linux)

    JDinPA
      I have this same issue of the ASM disk not showing up as the previous thread but the solution listed does not work for me.
      Oracle Linux Server release 5.8.
      Oracle 11.2 with patch number 13888614 and 13920180.

      I change the disk discovery path:/dev/oracleasm/disks/* and I still cannot see the asm disk in the installer.

      Everything looks good at the operating system level from both nodes.

      $ls -l /dev/oracleasm/disks
      total 0
      brw-rw---- 1 grid asmadmin 8, 33 Jul 30 10:57 CRSVOL1
      brw-rw---- 1 grid asmadmin 8, 97 Jul 30 10:57 DATAVOL1
      brw-rw---- 1 grid asmadmin 8, 81 Jul 30 10:57 FRAVOL1

      $ /usr/sbin/oracleasm-discover 'ORCL:*'
      Using ASMLib from /opt/oracle/extapi/64/asm/orcl/1/libasm.so
      [ASM Library - Generic Linux, version 2.0.4 (KABI_V2)]
      Discovered disk: ORCL:CRSVOL1 [4517506 blocks (2312963072 bytes), maxio 512]
      Discovered disk: ORCL:DATAVOL1 [47710176 blocks (24427610112 bytes), maxio 512]
      Discovered disk: ORCL:FRAVOL1 [47710176 blocks (24427610112 bytes), maxio 512]

      $ /etc/init.d/oracleasm listdisks
      CRSVOL1
      DATAVOL1
      FRAVOL1


      What can I check next and how can I get past this error? At this point my project is stuck until I can recognize the ASM disks.
        • 1. Re: Candidate ASM disks not showing up in 11.2 installer.
          Levi Pereira
          Hi,

          What is the Grid user and user groups you are using during Installation of Grid Infrastructure?
          • 2. Re: Candidate ASM disks not showing up in 11.2 installer.
            JDinPA
            OS User      Primary Group      Supplementary Groups
            grid      oinstall      asmadmin, asmdba, asmoper      

            I am installing as the grid user.
            • 3. Re: Candidate ASM disks not showing up in 11.2 installer.
              Levi Pereira
              Hi,

              Run kfod on all nodes and put output here:
              $ kfod asm_diskstring='ORCL:*' disks=all
              $ kfod asm_diskstring='/dev/oracleasm/disks/*' disks=all
              Note:

              Keep in mind that the Grid Infrastructure software does not need to be installed to use kfod, since kfod is located as well on the staging area where you uncompressed the Grid Infrastructure software, as the following example:

              <pre>
              /stage/grid/stage/ext/bin/kfod
              </pre>
              Where: "/stage" is the directory where I uncompressed the source software for the Grid Infrastructure software.
              • 4. Re: Candidate ASM disks not showing up in 11.2 installer.
                Sebastian Solbach -Dba Community-Oracle
                Hi,

                pls do a

                rpm -qa|grep asmlib

                Regards
                Sebastian
                • 5. Re: Candidate ASM disks not showing up in 11.2 installer.
                  JDinPA
                  #node1
                  # rpm -qa|grep asmlib
                  oracleasmlib-2.0.4-1.el5

                  #node2
                  # rpm -qa|grep asmlib
                  oracleasmlib-2.0.4-1.el5
                  • 6. Re: Candidate ASM disks not showing up in 11.2 installer.
                    JDinPA
                    #node1
                    $ /mnt/downloads/oracle/grid/stage/ext/bin/kfod asm_diskstring='ORCL:*' disks=all
                    /mnt/downloads/oracle/grid/stage/ext/bin/kfod: error while loading shared libraries: libcell11.so: cannot open shared object file: No such file or directory

                    #node2
                    $ /mnt/downloads/oracle/grid/stage/ext/bin/kfod asm_diskstring='ORCL:*' disks=all
                    /mnt/downloads/oracle/grid/stage/ext/bin/kfod: error while loading shared libraries: libcell11.so: cannot open shared object file: No such file or directory
                    • 7. Re: Candidate ASM disks not showing up in 11.2 installer.
                      JDinPA
                      Do these files look correct?

                      $ ls -l /dev/dm*
                      brw-rw---- 1 root root 253, 0 Jul 31 13:25 /dev/dm-0
                      brw-rw---- 1 root root 253, 1 Jul 31 13:25 /dev/dm-1

                      $ ls -l /dev/mapper/*
                      crw------- 1 root root 10, 62 Jul 31 13:25 /dev/mapper/control
                      brw-rw---- 1 root disk 253, 0 Jul 31 13:25 /dev/mapper/VolGroup00-LogVol00
                      brw-rw---- 1 root disk 253, 1 Jul 31 13:25 /dev/mapper/VolGroup00-LogVol01
                      brw-rw---- 1 root disk 253, 2 Jul 31 13:25 /dev/mapper/VolGroup01-LogVol00

                      $ dmsetup ls
                      VolGroup01-LogVol00 (253, 2)
                      VolGroup00-LogVol01 (253, 1)
                      VolGroup00-LogVol00 (253, 0)
                      • 8. Re: Candidate ASM disks not showing up in 11.2 installer.
                        JDinPA
                        #Here is an error file:
                        # cat /var/log/oracleasm
                        Creating /dev/oracleasm mount point: /dev/oracleasm
                        Loading module "oracleasm": oracleasm
                        Mounting ASMlib driver filesystem: /dev/oracleasm
                        Reloading disk partitions: done
                        Cleaning any stale ASM disks...
                        Scanning system for ASM disks...
                        oracleasm-read-label: Unable to open device "/dev/dm-2": No such file or directory
                        oracleasm-read-label: Unable to open device "/dev/dm-2": No such file or directory
                        oracleasm-read-label: Unable to open device "/dev/dm-2": No such file or directory
                        oracleasm-read-label: Unable to open device "/dev/dm-2": No such file or directory
                        Disk "CRSVOL1" does not exist or is not instantiated
                        Instantiating disk "CRSVOL1"
                        Disk "FRAVOL1" does not exist or is not instantiated
                        Instantiating disk "FRAVOL1"
                        Disk "DATAVOL1" does not exist or is not instantiated
                        Instantiating disk "DATAVOL1"




                        Shouldn't there be a /dev/dm-2 file?
                        • 9. Re: Candidate ASM disks not showing up in 11.2 installer.
                          JDinPA
                          Is it possible that there are some files left over from a previous failed install? I've read some posts about dd'ing the header on the asm disks.
                          $ ls -l /dev/oracleasm/disks/*
                          brw-rw---- 1 grid asmadmin 8, 33 Jul 31 13:28 /dev/oracleasm/disks/CRSVOL1
                          brw-rw---- 1 grid asmadmin 8, 49 Jul 31 13:28 /dev/oracleasm/disks/DATAVOL1
                          brw-rw---- 1 grid asmadmin 8, 65 Jul 31 13:28 /dev/oracleasm/disks/FRAVOL1

                          Would it make sense to clear out the headers of the asm disks?
                          $ dd if=/dev/zero of=/dev/oracleasm/disks/CRSVOL1 bs=1024 count=100
                          $ dd if=/dev/zero of=/dev/oracleasm/disks/DATAVOL1 bs=1024 count=100
                          $ dd if=/dev/zero of=/dev/oracleasm/disks/FRAVOL1 bs=1024 count=100
                          • 10. Re: Candidate ASM disks not showing up in 11.2 installer.
                            Sebastian Solbach -Dba Community-Oracle
                            Hi,

                            are you using logical volumes for ASM based disks in a cluster?

                            An LVM is not supported, since it is not cluster aware. If you want to use LVM, it must be a CLVM, and the only CLVM supported under Linux is that of Veritas.
                            That might be the issue ASMLIB does not work correctly.

                            Regards
                            Sebastian
                            • 11. Re: Candidate ASM disks not showing up in 11.2 installer.
                              JDinPA
                              I'm using openfiler per this documentation from Oracle:
                              http://www.oracle.com/technetwork/articles/hunter-rac11gr2-iscsi-088677.html#4
                              • 12. Re: Candidate ASM disks not showing up in 11.2 installer.
                                Sebastian Solbach -Dba Community-Oracle
                                Hi,

                                well the OPENFILER is an external filer, so this would be good (though not supported, but good for testing).

                                However the output you gave identifies that you mixed something up.

                                => Either you tried installing ASMLIB on the OPENFILER, which does not make sense, the ASMLIB has to be installed on the RAC Nodes, and they have to access the luns on the openfiler.
                                => Or you ran OPENFILER on one of the RAC Nodes, which again would not make sense, since you cannot use the luns directly there (as pointed out LVM on the RAC Node is not supported).
                                => Or you did not use ISCSI to access the luns on the OPENFILER but used another protocoll to access them - which would be strange since you see LVMs...

                                One way or the other, I believe you misunderstood one thing in the setup.

                                Regards
                                Sebastian
                                • 13. Re: Candidate ASM disks not showing up in 11.2 installer.
                                  JDinPA
                                  Asmlib is installed on the racnodes.
                                  Openfiler is ran on the shared node not the rac nodes.
                                  Iscsi was the protocol used to configure the targets on the openfiler.

                                  They show up fine from the os (per my posts above) but the installer doesn't recognize them.

                                  Shouldn't the installer have a detailed log file where I can debug the issue?
                                  • 14. Re: Candidate ASM disks not showing up in 11.2 installer.
                                    Levi Pereira
                                    Hi,

                                    Put here output of (all nodes);
                                    $ fdisk -l |grep dev
                                    $ asmdsk=`(cd /dev/oracleasm/disks/ && ls *  | awk '{ ORS=" "; print; }')`
                                    $ /etc/init.d/oracleasm querydisk -p $asmdsk |grep LABEL |awk '{print $1,$2}'
                                    P.S: When post output please format text using tag
                                     on begin and end of your output.
                                    
                                    Edited by: Levi Pereira on Aug 2, 2012 11:43 AM                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
                                    1 2 Previous Next