7 Replies Latest reply: Dec 23, 2012 2:56 AM by Billy~Verreynne RSS

    Can I use udev rules file for 11.2.0.3 RAC on Oracle Enterprise Linux 6 ?

    Kavanagh
      Grid Version : 11.2.0.3
      Platfomrm : The latest Oracle Enterprise Linux 6 (Unbreakable Enterprise Kernel)


      Yes the Oracle sales guys have convinced our manager to move to OEL from AIX.
      We are planning to install 11.2.0.3 RAC on Oracle Enterprise Linux 6 (UEK).

      I read this thread about udev rules file and asmlib.
      udev rules list for 10.2 RAC

      As an advantage for ASMLib, Levi periera says (quotes)
      " +A good async IO interface for the database, the entire IO interface is based on an optimal ASYNC model for performance+"

      Oracle hasn't released AsmLib for OEL 6 or RHEL 6. I think they are not going to release asmlib anymore.

      So, we will be using just udev rules file. Without ASMLib,  will we be having an ASYNC IO or other storage related issues?

      Has anyone succesfully installed 11.2.0.3 RAC in OEL 6 with udev rules file ?
        • 1. Re: Can I use udev rules file for 11.2.0.3 RAC on Oracle Enterprise Linux 6 ?
          Billy~Verreynne
          udev is a Linux kernel feature. So it exists, whether you choose to use or ignore it.

          The only reason that comes to my mind to use udev rules is to set device permissions for the grid o/s user to own devices it will use for ASM diskgroups.

          However, one should now be able to set device permissions via +/etc/multipath.conf+ instead - which needs to be configured anyway for white and black listing of devices and assigning logical device names to WWIDs. Assuming this permission setting works, there should be very little reason for touching udev rules files.

          What do you have in mind wrt modifying the rules files?
          • 2. Re: Can I use udev rules file for 11.2.0.3 RAC on Oracle Enterprise Linux 6 ?
            Gennady Sigalaev
            Hi Kavanagh,
            Oracle hasn't released AsmLib for OEL 6 or RHEL 6.
            Please have a look on this link (http://www.oracle.com/technetwork/server-storage/linux/asmlib/ol6-1709075.html).
            "The oracleasm kernel driver is built into the Unbreakable Enterprise Kernel for Oracle Linux 6 and does not need to be installed manually.". It means you don't need to install anything. You need only to set up the yum and update the driver using update-driver command.

            Best regards,
            Gennady
            • 3. Re: Can I use udev rules file for 11.2.0.3 RAC on Oracle Enterprise Linux 6 ?
              Billy~Verreynne
              Kavanagh wrote:

              Without ASMLib,  will we be having an ASYNC IO or other storage related issues?
              Async I/O is a kernel feature. That needs libaio.x86_64 installed.

              Fwiw, we have not used ASMLib on any of our RACs spanning o/s and Oracle versions from RHEL3 to RHEL5/OL5, and 10gr1 to 11gr2.
              • 4. Re: Can I use udev rules file for 11.2.0.3 RAC on Oracle Enterprise Linux 6 ?
                Kavanagh
                Thank you very much Billy, Gennady.

                Hi Gennady,
                Since ASMLib is not mandatory we are not going to use it even though oracleasm is built into OEL 6 (UEK) . Fingers crossed.

                Thank you Billy for letting us know that Async I/O is a kernel feature and the package that needs to be installed for this. The Unix Admin guys here have no prior experience with Linux. Their expertise lies only in AIX and Solaris.

                What do you have in mind wrt modifying the rules files?
                From googling, all we know about udev rules file is that it is used to
                 A. To maintain persistent paths for LUNs across reboots
                 B. To maintain the ownership of LUNs (grid , oinstall )
                So, udev is an alternative to ASMLib.

                Are you suggesting that configuring */etc/multipath.conf* is a better alternative to using udev rules file ? Can i achieve both the above functions (A and B) by using */etc/multipath.conf* ?
                • 5. Re: Can I use udev rules file for 11.2.0.3 RAC on Oracle Enterprise Linux 6 ?
                  Billy~Verreynne
                  I use udev file +/etc/udev/rules.d/40-multipath.rules+ to configure device permissions of multiple I/O path devices for use by o/s user grid.

                  It looks as follows:
                  # modded for setting mpath device permissions
                  
                  SUBSYSTEM!="block", GOTO="end_mpath"
                  
                  KERNEL!="dm-[0-9]*", GOTO="end_mpath"
                  
                  PROGRAM!="/sbin/mpath_wait %M %m", GOTO="end_mpath"
                  
                  ACTION=="add", RUN+="/sbin/dmsetup ls --target multipath --exec '/sbin/kpartx -a -p p' -j %M -m %m"
                  
                  # goto labels added
                  PROGRAM=="/sbin/dmsetup ls --target multipath --exec /bin/basename -j %M -m %m", RESULT=="?*", NAME="%k", SYMLINK="mpath/%c", GOTO="check_cluster_devs"
                  PROGRAM!="/bin/bash -c '/sbin/dmsetup info -c --noheadings -j %M -m %m | /bin/grep -q .*:.*:.*:.*:.*:.*:.*:part[0-9]*-mpath-'", GOTO="check_cluster_devs"
                  PROGRAM=="/sbin/dmsetup ls --target linear --exec /bin/basename -j %M -m %m", NAME="%k", RESULT=="?*", SYMLINK="mpath/%c", GOTO="check_cluster_devs"
                  
                  GOTO="end_mpath"
                  
                  # set device permissions for Grid Infrastructure
                  LABEL="check_cluster_devs"
                  RESULT=="*", GROUP="oinstall", MODE="660"
                  RESULT=="*", OWNER="grid", GROUP="oinstall", MODE="660"
                  OPTIONS="last_rule"
                  
                  LABEL="end_mpath"
                  
                  # eof
                  As I understand this is no longer needed at this can be configured in +/etc/multipath.conf+ as follows:
                  ..
                  multipaths {
                      multipath {
                          wwid  SSCST_BIOscst1_sdb_757d30d0
                          alias scst1_sdb
                          mode 0660
                          uid oinstall
                          gid grid
                        }
                  .. etc.
                  }
                  However, never got this to work properly myself - must be doing something wrong as I've seen a couple of Internet sources showing this to work.

                  AFAIK, it is also with OL6 (as it is with OL5 and prior), not an issue of AsmLib or udev/multipath. Udev is a the device mapper used by the kernel. The decision is whether to add another layer called AsmLib. Personally, I have yet to see any compelling evidence that why I should increase the complexity of the kernel driver stack, by adding AsmLib. We also had some serious incompatibility issues with AsmLib in the past (where introducing it, caused consistent I/O corruption at logical level).

                  If you look at clusters in the www.top500.org list, many of these have Pentabytes of shared storage accessed via udev/multipath and nothing else. Multipath has its roots as a kernel driver for such large clusters. One of the reasons we ceased years ago to use EMC's Powerpath for our RACs that are wired to EMC SANs for shared storage.
                  • 6. Re: Can I use udev rules file for 11.2.0.3 RAC on Oracle Enterprise Linux 6 ?
                    Kavanagh
                    Did a bit of googling. Apparenlty /etc/multipath.conf file is no longer used from RHEL 6/OEL 6 onwards. udev rules file is used instead. Our AIX admin guy (experienced professional ) hasn't even heard of udev rules file. Our PM will be sending him for OEL Training at Oracle University soon.
                    • 7. Re: Can I use udev rules file for 11.2.0.3 RAC on Oracle Enterprise Linux 6 ?
                      Billy~Verreynne
                      AIX is not Linux, and vice versa - despite both having Unix SVR4 roots ito o/s, directory structures and commands.

                      In fact, there are a number of differences too between Debian and RedHat Linux distros - network configuration for example are handled very differently.

                      That said - solid Unix skills are not wasted when encountering Linux. One needs to learn and understand (and accept) the differences.