4 Replies Latest reply on Oct 26, 2016 11:56 AM by Peter_L_

    Label Disk for ASM Filter Driver (AFD) requires restart of afd.service

    Peter_L_

      Hi,

      currently I'm testing out the ASM Filter Driver handling for future use.

       

      I've discovered something and maybe you know if it is expected or an issue:

       

      I want to label a physical disk for ASM DG extension:

       

      [root@repo ~]#asmcmd afd_label DBDATA6 /dev/sdq
      ok
      

       

      I see, that the new disk was added correctly under /dev/oracleafd/disks/ but the permissions are wrong (root:root instead of grid:asmadmin)

      [root@repo ~]# ls -latr /dev/oracleafd/disks/
      insgesamt 28
      drwxrwx---. 3 grid asmadmin  80 24. Okt 12:04 ..
      -rw-r--r--. 1 grid asmadmin  10 24. Okt 12:04 DBFRA
      -rw-r--r--. 1 grid asmadmin   9 24. Okt 12:04 DBDATA5
      -rw-r--r--. 1 grid asmadmin  10 24. Okt 12:04 DBDATA4
      -rw-r--r--. 1 grid asmadmin  10 24. Okt 12:04 DBDATA3
      -rw-r--r--. 1 grid asmadmin  10 24. Okt 12:04 DBDATA2
      -rw-r--r--. 1 grid asmadmin  10 24. Okt 12:04 DBDATA
      -rw-r--r--. 1 root root       9 24. Okt 12:06 DBDATA6
      drwxrwx---. 2 grid asmadmin 180 24. Okt 12:06 .
      

       

      the udev rules seems to be correct:

      [root@repo ~]# cat /etc/udev/rules.d/53-afd.rules
      #
      # AFD devices
      KERNEL=="oracleafd/.*", OWNER="grid", GROUP="asmadmin", MODE="0770"
      KERNEL=="oracleafd/*", OWNER="grid", GROUP="asmadmin", MODE="0770"
      KERNEL=="oracleafd/disks/*", OWNER="grid", GROUP="asmadmin", MODE="0660"
      

       

      but

      udevadm control --reload 
      udevadm trigger
      

      has no effect.

       

      Only

       systemctl restart afd.service
      

      helps and expected udev rules are applied.:

      [root@repo ~]# ls -latr /dev/oracleafd/disks/
      insgesamt 28
      drwxrwx---. 3 grid asmadmin  80 24. Okt 12:08 ..
      -rw-r--r--. 1 grid asmadmin  10 24. Okt 12:08 DBDATA2
      -rw-r--r--. 1 grid asmadmin  10 24. Okt 12:08 DBDATA
      -rw-r--r--. 1 grid asmadmin  10 24. Okt 12:08 DBFRA
      -rw-r--r--. 1 grid asmadmin   9 24. Okt 12:08 DBDATA6
      -rw-r--r--. 1 grid asmadmin   9 24. Okt 12:08 DBDATA5
      -rw-r--r--. 1 grid asmadmin  10 24. Okt 12:08 DBDATA4
      -rw-r--r--. 1 grid asmadmin  10 24. Okt 12:08 DBDATA3
      drwxrwx---. 2 grid asmadmin 180 24. Okt 12:08 .
      

      Could it be a problem to restart the afd.service while ASM running (it works, but I don't know waht happens in heavy load scenarios).

      Is it expected to have to restart afd.service (systemd)?

       

      Thank you

      Peter

        • 1. Re: Label Disk for ASM Filter Driver (AFD) requires restart of afd.service
          Gbenga Ajakaye
          1. udevadmcontrol--reload

          has no effect. is wrong.

           

          You should be using udevadm control --reload-rules  (without reboot)

          Thought your OS matters. The steps is different on CentOS.

           

          Could it be a problem to restart the afd.service while ASM running (it works, but I don't know waht happens in heavy load scenarios).

          Is it expected to have to restart afd.service (systemd)?

           

          Honestly, the only way to know is to stress test your system and make sure you know how it would act under different types of workload before going to prod.

          • 2. Re: Label Disk for ASM Filter Driver (AFD) requires restart of afd.service
            Peter_L_

            Thank you. I use Oracle Linux 7 (RHEL 7). udevadm control --reload-rules is not documented in manpage but I hink it's the same as udevadm control --reload. However both does not apply the udev rules for AFD disks.

            Regards Peter

            • 3. Re: Label Disk for ASM Filter Driver (AFD) requires restart of afd.service
              Marco Mischke

              Hi,

               

              when you initially label the disks, the devices in /dev/oracleafd/disks/ are still owned by root. But you are already able to add them to diskgroups. Even after that "root" is still the owner. Ownership is changed once you reboot the system, but this is not required since you can use the device even with root ownership. I have no clue how they implement that, but it works like this. Just did that to a customers cluster in production

               

              HTH

              Marco

              • 4. Re: Label Disk for ASM Filter Driver (AFD) requires restart of afd.service
                Peter_L_

                Hi Marco,

                I have a little bit  re-engineered the afd scripts and something tested.

                The "asmcmd afd_label" does not set explicit any permission at the resulted disk under "/dev/oracleafd/disks/".

                 

                Rather the permissions will derived from current user which is hitting "asmcmd afd_label"-command:

                For testing this I putted the "grid" user into the "disk" group which owns the device to label. Then I run asmcmd afd_label as grid. The resulting disk under "/dev/oracleafd/disks/" got the owner "grid:oinstall" where oinstall is the main group of "grid".

                 

                In the "/etc/init.d/afd"  you found in start_afd() method the "set permission section"

                start_afd()

                {

                  $LOGMSG "Starting AFD... "

                 

                  works=true

                  if [ ! -r $AFDLOAD ]

                  then

                    works=false

                  fi

                 

                  if [ $works = false ]

                  then

                    $LOGMSG "INFO (Oracle AFD) :: Filesystem containing $AFDLOAD is not available."

                    $LOGMSG "INFO (Oracle AFD) :: Using a fallback option."

                  fi

                 

                # load AFD driver

                afd_loaddriver

                add_subsysfile

                 

                # create /dev/oracleafd/disks & set permissions

                $MKDIR -p $AFD_PORTAL_DEV_DISKS

                $CHMOD 0770 $AFD_PORTAL_DEV_DISKS

                $CHOWN $USER:$GROUP $AFD_PORTAL_DEV_DISKS

                 

                # Discover the afd disks

                afd_scandisk

                 

                # Set permissions on files under AFD_PORTAL_DEV_DISKS (after scan)

                $CHOWN $USER:$GROUP $AFD_PORTAL_DEV_DISKS/*

                 

                # Enable AFD Kernel logging if not done

                # If file system was available, 'afdload' starts logging

                # Check again and attempt to enable kernel logging

                if [ $works = false ]

                then

                   afd_enablelog

                fi

                }

                 

                 

                So this is what I have to do for label disk with right permissions:

                [root@repo ~]# asmcmd afd_label DBDATA6 /dev/sdq --rename

                Connected to an idle instance.

                [root@repo ~]#  ls -latr /dev/oracleafd/disks/DBDATA6

                -rw-r--r--. 1 root root 9 26. Okt 11:31 /dev/oracleafd/disks/DBDATA6

                [root@repo ~]# /etc/init.d/afd start

                AFD-641: Checking for existing AFD installation.

                AFD-643: Validating AFD installation files for operating system.

                AFD-9393: Verifying ASM administrator setup.

                AFD-637: Loading installed AFD drivers.

                AFD-649: Verifying AFD devices.

                AFD-9156: Detecting control device '/dev/oracleafd/admin'.

                AFD-9322: completed

                [root@repo ~]#  ls -latr /dev/oracleafd/disks/DBDATA6

                -rw-r--r--. 1 grid asmadmin 9 26. Okt 11:31 /dev/oracleafd/disks/DBDATA6

                It would be nice, when oracle would the setting of permissions running implicit due "asmcmd afd_label"-command

                 

                Maybe it was com up with this patch:

                Doc ID 1928853.1

                Bug 20011635     UPDATE AFD.SBS SCRIPT : CHANGE PERMISSION TO /DEV/ORACLEAFD/DISKS/*

                 

                Regards Peter