4 Replies Latest reply: Feb 28, 2014 5:09 PM by Dude! RSS

    Partition Scheme for DM Multipath Devices

    user10437903

      I have some questions about Partition Scheme for Device Mapper Multipath Devices.

       

      Current Environment

      Oracle Linux 6.5

       

      I had planned on setting up partitions for each Storage Area and mounting Oracle Software, Oracle Staging, Oracle Backup and OEM Software Library.  The other partitions would serve as Oracle ASM disk devices.  The problem is that I've identified that only 4 primary partitions can exists on one system, so, what partition schema to use and how this should affect the underlining LUN setup and DM Device Mapping.  The Planned Partition Schema below CANNOT work.

       

      Storage AreaBootableFileSystem

      Storage Size

      Requirement

      Disk Devices

      Disk Device

      Size

      Planned

      Partition Scheme

      OS bootyesext4~MB/dev/cciss/c0d0~MB/dev/cciss/c0d0p1
      OSext4~146GB/dev/cciss/c0d0~146GB/dev/cciss/c0d0p2
      Oracle Softwareext4~50GB/dev/mapper/mpatha~100GB/dev/mapper/mpathap1
      Oracle Stagingext4~25GB/dev/mapper/mpatha~100GB/dev/mapper/mpathap2
      Oracle Backupext4~25GB/dev/mapper/mpatha~100GB/dev/mapper/mpathap3
      OEM Software Libraryext4~700GB/dev/mapper/mpathb~700GB/dev/mapper/mpathbp1
      ASM DATA Disk Device 1 (DATADK1)none~200GB/dev/mapper/mpathc~200GB/dev/mapper/mpathcp1
      ASM DATA Disk Device 2 (DATADK2)none~200GB/dev/mapper/mpathd~200GB/dev/mapper/mpathdp1
      ASM DATA Disk Device 3 (DATADK3)none~200GB/dev/mapper/mpathe~200GB/dev/mapper/mpathep1
      ASM DATA Disk Device 4 (DATADK4)none~200GB/dev/mapper/mpathf~200GB/dev/mapper/mpathfp1
      ASM FRA Disk Device 1 (FRADK1)none~200GB/dev/mapper/mpathg~200GB/dev/mapper/mpathgp1
      ASM FRA Disk Device 2 (FRADK2)none~200GB/dev/mapper/mpathh~200GB/dev/mapper/mpathhp1
      ASM FRA Disk Device 3 (FRADK3)none~200GB/dev/mapper/mpathi~200GB/dev/mapper/mpathip1
      ASM FRA Disk Device 4 (FRADK4)none~200GB/dev/mapper/mpathj~200GB/dev/mapper/mpathjp1

       

      So, I am looking for confirmation of my new design.  I constrained the number of Primary Partitions to 4.  The underlining LUNS needed to be combined to produce 4 total disk devices. I used Extended and Logical Partitions to achieve the number of partitions required.  Is there any other way to partition this so I don't need to create the extended and logical partitions.

       

       

      Storage Area

      Storage Size

      Requirement

      Disk Devices

      Disk Device

      Size

      Partition

      Type

      Planned

      Partition Scheme

       

      Partition

      Size

       

      FileSystem

       

      Bootable

      OS boot~MB/dev/cciss/c0d0~MBPrimary/dev/cciss/c0d0p1~MBext4yes
      OS~146GB/dev/cciss/c0d0146GBLVM/dev/cciss/c0d0p2~146MBext4
      Oracle Software~50GB/dev/mapper/mpatha100GBPrimary/dev/mapper/mpathap1~50GBext4
      Oracle Staging~25GB/dev/mapper/mpatha100GB

      Extended

           Logical

      /dev/mapper/mpathap2

           /dev/mapper/mpathap5

      ~25GBext4
      Oracle Backup~25GB/dev/mapper/mpatha100GB

      Extended

           Logical

      /dev/mapper/mpathap2

           /dev/mapper/mpathap6

      ~25GBext4
      OEM Software Library~700GB/dev/mapper/mpathb700GBPrimary/dev/mapper/mpathbp1~700GBext4
      ASM DATA Disk Device 1 (DATADK1)~200GB/dev/mapper/mpathc1600GBPrimary/dev/mapper/mpathcp1~200GBnone
      ASM DATA Disk Device 2 (DATADK2)~200GB/dev/mapper/mpathc1600GB

      Extended

          Logical

      /dev/mapper/mpathcp2

           /dev/mapper/mpathcp5

      ~200GBnone
      ASM DATA Disk Device 3 (DATADK3)~200GB/dev/mapper/mpathc1600GB

      Extended

          Logical

      /dev/mapper/mpathcp2

          /dev/mapper/mpathcp6

      ~200GBnone
      ASM DATA Disk Device 4 (DATADK4)~200GB/dev/mapper/mpathc1600GB

      Extended

          Logical

      /dev/mapper/mpathcp2

          /dev/mapper/mpathcp7

      ~200GBnone
      ASM FRA Disk Device 1 (FRADK1)~200GB/dev/mapper/mpathc1600GB

      Extended

          Logical

      /dev/mapper/mpathcp2

          /dev/mapper/mpathcp8

      ~200GBnone
      ASM FRA Disk Device 2 (FRADK2)~200GB/dev/mapper/mpathc1600GB

      Extended

          Logical

      /dev/mapper/mpathcp2

          /dev/mapper/mpathcp9

      ~200GBnone
      ASM FRA Disk Device 3 (FRADK3)~200GB/dev/mapper/mpathc1600GB

      Extended

          Logical

      /dev/mapper/mpathcp2

          /dev/mapper/mpathcp10

      ~200GBnone
      ASM FRA Disk Device 4 (FRADK4)~200GB/dev/mapper/mpathc1600GB

      Extended

          Logical

      /dev/mapper/mpathcp2

          /dev/mapper/mpathcp11

      ~200GBnone

       

      This doesn't seem to be right since this would seem to constrain the LUN sizes presented since only 4 devices can be referenced and partitioned.  Also, this would seem to complicate the addition of storage for Oracle ASM.  I think my understanding of Linux Partitioning is wrong so please advise.

       

      Lastly,

      I've largely seen two methods to partition multipath devices.

       

      Method 1

       

      The following link uses PARTED to partition DM Multipath Devices.

       

      https://community.oracle.com/message/11244298#11244298

       

      parted /dev/mapper/mpath_drive

      (parted)print

      (parted)mklabel gpt #MAKE SURE THE DISK IS NOT BEING USED OR YOU WILL REMOVE ALL THE DATA

      (parted)print

      (parted)unit Mb

      (parted)print

                Model: Linux device-mapper (multipath) (dm)

                Disk /dev/mapper/asm_data1: 2199023MB

                Sector size (logical/physical): 512B/512B

                Partition Table: gpt

                Number  Start   End     Size   File system  Name     Flags   

       

      (parted)mkpart primary ext2 INIT_MB END_MB  #YOU WILL SET THOSE VALUES BASED ON THE AMOUNT OF SPACE YOU HAVE   

      (parted)print #In this example, I have created four 500gb partitions with a 2tb drive for ASM.

                Model: Linux device-mapper (multipath) (dm)

                Disk /dev/mapper/mpath_drive: 2199023MB

                Sector size (logical/physical): 512B/512B

                Partition Table: gpt

                Number  Start      End        Size      File system  Name     Flags

                 1      1.00MB     549757MB   549756MB               primary

                 2      549758MB   1099515MB  549757MB               primary

                 3      1099516MB  1649272MB  549756MB               primary

                 4      1649273MB  2199023MB  549750MB               primary

      (quit)

       

      Method 2

       

      fdisk one path of the multipath device and NOT the multipath device itself

       

      # fdisk /dev/sda

      # kpartx -a /dev/mapper/mpath0

       

       

      I am uncertain which way is preferred.  I am uncertain about method 2 but it seems much simplier.

       

      Thanks in advance for any help.

        • 1. Re: Partition Scheme for DM Multipath Devices
          Dude!

          The limitation of 4 primary partitions stems from the old MBR partitioning scheme required by the PC BIOS. Perhaps you remember the concept of extended partitions under MSDOS. A GUID type partitioning, which is typically used by EFI or UEFI systems does not have this limitation anymore, or at least no practical limit.

           

          However, device partitions are only a logical constructs. DM multipath is used to provide device path redundancy. If a devices path or LUN fails, you cannot switch between partitions because they will all be gone. The same goes for ASM. If you use partitions for ASM there is no physical data redundancy or performance benefit.

          • 2. Re: Partition Scheme for DM Multipath Devices
            Svetoslav Gyurov

            Hi,

             

            Why not just having these defined on the storage level rather than having OS partitions ? Partition the LUN for the OS and file systems but ask for number of LUNs for your ASM disk group(s). What's more your DATA and FRA disks may have different characteristics, at least that's how I setup mine. Why waste SAS/FC disk space when there is plenty of SATA. When the moment for expanding comes just ask for another LUN with the same size and characteristics, change ownership (create udev rules) and simple add the disk to the ASM disk group.

             

            Regards,

            Sve

            • 3. Re: Partition Scheme for DM Multipath Devices
              user10437903

              Dude:

               

              I was able to use fdisk referencing the /dev/mapper/mpath# and create more than 4 primary partitions.  I am scratching my head at this point.  Is there no warning?  What occurs if you create more than 4 primary partitions?

               

               

              Svetoslav Gyurov


              Yes, your right.  I have LUNs partitioned to the sizes I needed.  I've read that partitions are a way to prevent storage, os or dba from accidentally uses the device when it may already be dedicated for ASM disk.


              Thanks for you help,

              Carlos

              • 4. Re: Partition Scheme for DM Multipath Devices
                Dude!

                user10437903 wrote:

                 

                Dude:

                 

                I was able to use fdisk referencing the /dev/mapper/mpath# and create more than 4 primary partitions.  I am scratching my head at this point.  Is there no warning?  What occurs if you create more than 4 primary partitions?

                  

                 

                Depends on your partitioning scheme, boot loader and architecture. I think GPT can have up to 128 primary partitions and MBR up to 4. If your system relies on a PC BIOS and you have your OS on primary partition no. 5 then system will not be able address and boot from it.

                 

                user10437903 wrote:

                 

                Yes, your right.  I have LUNs partitioned to the sizes I needed.  I've read that partitions are a way to prevent storage, os or dba from accidentally uses the device when it may already be dedicated for ASM disk.


                Thanks for you help,

                Carlos

                 

                Partitions in Unix/Linux are sometimes used to prevent space overrun of one to affect the other. It was also used to work-around address space limitations of 8 bit BIOS and 32-bit OS and file systems. Most of it is no longer really relevant. As far as protecting ASM devices from accidental use, sorry that's not how it works.