2 Replies Latest reply: Apr 19, 2007 3:53 PM by 511303 RSS

    ASMLib and Linux block devices

      I'm little confused with the role of ASMLib on Linux (asmlib and kernel asm module) and would
      appreciate if someone from Oracle (maybe Sergio or Wim) clarify it.

      If I got it right, Oracle database storage management has the following choices and combinations
      of them:
      1. Traditional File System (eg. /opt/oracle/data/orcl/file_name.dbf), buffered by OS

      2. Standard UNIX Raw Character Devices (/dev/raw/raw*). One raw device for each
         datafile, control file, and log file must be created

      3. ASM (Automatic Storage Management)

         3.1. ASM with UNIX I/O on Raw Character Devices (/dev/raw/raw*). Devices are
              managed by ASM using standard UNIX I/O

         3.2. ASM on unmounted Block Devices (/dev/hda*, /dev/hdb*, etc.)

              3.2.1. ASM on block devices with ASMLib installed

              3.2.2. ASM on block devices without ASMLib installed
      I experimented with all ASM variants on Unbreakable Linux 4.4:

      - on raw-bound character devices (/dev/raw/raw*)
      - on block devices (/dev/hd**) with ASMLib installed
      - on block devices (/dev/hd**) without ASMLib installed

      and all variants worked fine. I couldn't find out which variant had the best performance
      testing it on single-user uniprocessor system.

      Now, please, forget ASM variant on raw-bound devices and concentrate just on the last two choices:
              3.2.1. ASM on block devices with ASMLib installed
              3.2.2. ASM on block devices without ASMLib installed
      My experience tells me that ASM implementation on unmounted Linux block devices (/dev/hda*,
      /dev/hdb*, etc) doesn't require ASMLib interface to be installed at all, specially not as a
      prerequisite for it. Furthermore, I cannot perceive any performance or stability benefits using
      ASMLib interface and kernel asm module contained in packages
      and discovering, configuring and scanning devices according to it, as opposed to not
      installing it at all and refering to devices directly as /dev/hda*, /dev/hdb* etc.

      Oracle official documentation says that ASMLib interface is an add-on module that simplifies
      the management and discovery of ASM disks and improve identifying and accessing block devices.
      This is not pretty clear to me and I would have two questions:

      1. what is the real role and benefit of ASMLib on Linux block devices?

      2. (indenpendent from 1.) considering only performance (speed and throughput), what is Oracle's
      official recommendation for Unbreakable Linux 4.4: ASM implementation on raw-bound devices
      or ASM implementation on block devices (with or without ASMLib)?

      Thanks in advance.

        • 1. Re: ASMLib and Linux block devices
          asmlib wouldn't necessarily give you much of an io performance benefit, it's mainly for ease of management as it will find/discover the right devices for you, the io effect of asmlib is large the same as doing async io to raw devices.

          3.2.1. ASM on block devices with ASMLib installed
          3.2.2. ASM on block devices without ASMLib installed

          it comes back to another post where youmentioned that /dev/raw is gone and you now do directio to the block device.

          basically, if you give blockdevices to oracle, it will now automatically open with O_DIRECT and by default do async io. so whether you give /dev/raw or /dev/hd (or /dev/sd) to asm or to oracle in general, it will be exactly the same.

          even tho as you stated elsewhere dev/raw is deprecated, reality is that all that /dev/raw was in the end was (open (/dev/blockdevice, O_DIRECT) literally nothing more. so there is no difference there.

          the suggested use definitely is using asmlib and just provide the blockdevice to it. it will then take care of making sure discovery is right, naming is less of an issue and you still get asynchronous IO on pure raw non buffered disk performance.

          so using asm without asmlib on blockdevices dont buffer, oracle will open odirect and do async io. using asmlib will do the same type of io, it's a different actual codepath but in effect does similar stuff.
          • 2. Re: ASMLib and Linux block devices
            Thanks Wim. Thanks a lot. You are the only one who gave me the right answer on what I was asking
            almost three years. In fact, you've just confirmed what I always thought (or at least suspected).
            Namely, after having examined the raw driver source code on Linux kernel source tree as well as
            Stephen Tweedie's `raw' binding utility, I was continuously trying to convince my colleagues that
            Linux raw-bound device interface was nothing else but a fatamorgana, a mirage of something called
            "Sun Solaris native raw character device interface", doing nothing else but opening a block device
            with O_DIRECT. Thanks a lot.

            So, if I got you right, using ASM on Linux implies that setting Oracle startup parameters
            'disk_asynch_io=TRUE' and 'filesystemio_options=SETALL' is obsolete (redundant), except in the case
            that traditional Oracle filesystem is combined with ASM.

            One more question, please, and I'll finish. If ASMLib is just an add-on module with primary function
            to simplify the management and discovery of ASM disks, but doesn't necessarily give us much of an io
            performance benefit, as you wrote, why does it exist only for Linux? There is no ASMLib for other
            unixes (Solaris, AIX, HP-UX etc). Only for Linux. Why would be the management and discovery of ASM
            disks more complicated on Linux than on other unixes, so that we need ASMLib only for Linux?

            Hundred times - thanks a lot.