This discussion is archived
8 Replies Latest reply: May 9, 2013 8:53 AM by Omega3 RSS

Questions from Linux System Admin on storage

james_p Newbie
Currently Being Moderated
Version Info
+++++++++++
Grid Infrstructure: 11.2.0.3
Platform : Oracle Enterprise Linux 6.3
++++++++++++

We are a big AIX, Solaris shop ! After some meetings with the Oracle Sales guys, our managers have decided to move some of our RAC DBs to Oracle Linux.

Our Linux Admins (RHEL ) never configured machines for RAC before. They have only configured machines for App Servers before.

For the shared LUNs meant for ASM Diskgroups they have asked me how I want to have the LUNs to be ? Physical Volume, Logical Volume .. etc. They spoke about things like pvcreate, vgcreate.. etc.
Any idea how should the storage LUNs have to be presented to Linux RAC Nodes ?
  • 1. Re: Questions from Linux System Admin on storage
    user11977218 Newbie
    Currently Being Moderated
    Each node in a cluster requires external shared disks for storing the Oracle Clusterware (Oracle Cluster
    Registry and voting disk) files, and Oracle Database files.
    Once the LUNs have been presented from the SAN to ALL servers in the cluster, partition the LUNs from
    one node only, run fdisk to create a single whole-disk partition with exactly 1 MB offset on each LUN to be
    used as ASM Disk.
  • 2. Re: Questions from Linux System Admin on storage
    EdStevens Guru
    Currently Being Moderated
    james_p wrote:
    Version Info
    +++++++++++
    Grid Infrstructure: 11.2.0.3
    Platform : Oracle Enterprise Linux 6.3
    ++++++++++++

    We are a big AIX, Solaris shop ! After some meetings with the Oracle Sales guys, our managers have decided to move some of our RAC DBs to Oracle Linux.

    Our Linux Admins (RHEL ) never configured machines for RAC before. They have only configured machines for App Servers before.
    But you have AIX and/or Solaris admins who have configured for RAC?
    Why can't your AIX/Solaris guys help your Linux guys? They already speak the same language, the principles are the same ... I know some purists will say that Linux is not Unix, but from my perspective the difference between Linux and any particular Unix is no greater than the differences between the various particular Unix's. As a dba, I've worked with Solaris, AIX, HP-UX, and Oracle Linux and have seen some of the differences first hand, but they are so trivial that I can't even remember what they are.
    For the shared LUNs meant for ASM Diskgroups they have asked me how I want to have the LUNs to be ? Physical Volume, Logical Volume .. etc. They spoke about things like pvcreate, vgcreate.. etc.
    Any idea how should the storage LUNs have to be presented to Linux RAC Nodes ?
  • 3. Re: Questions from Linux System Admin on storage
    rarain Explorer
    Currently Being Moderated
    Hi James,

    pvcreate Initilizes physical volume for later use by LVM and vgcreate creates a new volume group using the block special device previously configured for LVM with pvcreate. So if you are going to implement LVM then only you need to use pvcreate and vgcreate but since you are implementing ASM so LVM is not required because ASM itself offers LVM features where you can add the disk dynamically if your diskgroup is full and rebalancing is also performed automatically by ASM.

    You just have to Map configured LUN on RAC server which will be represented as block device let's say /dev/sdb and create logical partition under /dev/sdb let's say /dev/sdb1 and further use these devices to create the ASM diskgroup.

    Thanks
  • 4. Re: Questions from Linux System Admin on storage
    Levi-Pereira Guru
    Currently Being Moderated
    james_p wrote:
    Version Info
    +++++++++++
    Grid Infrstructure: 11.2.0.3
    Platform : Oracle Enterprise Linux 6.3
    ++++++++++++

    We are a big AIX, Solaris shop ! After some meetings with the Oracle Sales guys, our managers have decided to move some of our RAC DBs to Oracle Linux.

    Our Linux Admins (RHEL ) never configured machines for RAC before. They have only configured machines for App Servers before.

    For the shared LUNs meant for ASM Diskgroups they have asked me how I want to have the LUNs to be ? Physical Volume, Logical Volume .. etc. They spoke about things like pvcreate, vgcreate.. etc.
    Any idea how should the storage LUNs have to be presented to Linux RAC Nodes ?
    Short answer is : Recommended present to ASM the Physical Disks (Luns)

    Install Docs is pretty clear about theses questions.
    http://docs.oracle.com/cd/E11882_01/install.112/e22489/storage.htm#CDECEBGH


    * All of the devices in an Oracle ASM disk group should be the same size and have the same performance characteristics.

    * Do not specify multiple partitions on a single physical disk as a disk group device. Each disk group device should be on a separate physical disk.

    * Although you can specify a logical volume as a device in an Oracle ASM disk group, Oracle does not recommend their use because it adds a layer of complexity that is unnecessary with Oracle ASM. In addition, Oracle RAC requires a cluster logical volume manager in case you decide to use a logical volume with Oracle ASM and Oracle RAC.

    Oracle recommends that if you choose to use a logical volume manager, then use the logical volume manager to represent a single LUN without striping or mirroring, so that you can minimize the impact of the additional storage layer.

    Using Logical Volume Managers with Oracle Grid Infrastructure and Oracle RAC
    http://docs.oracle.com/cd/E11882_01/install.112/e22489/storage.htm#CHDGEIIB
  • 5. Re: Questions from Linux System Admin on storage
    james_p Newbie
    Currently Being Moderated
    Thank you. I have 2 more questions


    Question1.
    Should it be extended partition or Primary partition ? Should I choose 'e' or 'p' for below ?
    # fdisk /dev/sdb
    .
    .
    .
    .
    Command action
       e   extended
       p   primary partition (1-4)
    Question2.
    What should be the partition type ? What should I choose from the list below
    Command (m for help): t
    Selected partition 1
    Hex code (type L to list codes): l
    
     0  Empty           1e  Hidden W95 FAT1 80  Old Minix       bf  Solaris        
     1  FAT12           24  NEC DOS         81  Minix / old Lin c1  DRDOS/sec (FAT-
     2  XENIX root      39  Plan 9          82  Linux swap / So c4  DRDOS/sec (FAT-
     3  XENIX usr       3c  PartitionMagic  83  Linux           c6  DRDOS/sec (FAT-
     4  FAT16 <32M      40  Venix 80286     84  OS/2 hidden C:  c7  Syrinx        
     5  Extended        41  PPC PReP Boot   85  Linux extended  da  Non-FS data    
     6  FAT16           42  SFS             86  NTFS volume set db  CP/M / CTOS / .
     7  HPFS/NTFS       4d  QNX4.x          87  NTFS volume set de  Dell Utility  
     8  AIX             4e  QNX4.x 2nd part 88  Linux plaintext df  BootIt        
     9  AIX bootable    4f  QNX4.x 3rd part 8e  Linux LVM       e1  DOS access    
     a  OS/2 Boot Manag 50  OnTrack DM      93  Amoeba          e3  DOS R/O        
     b  W95 FAT32       51  OnTrack DM6 Aux 94  Amoeba BBT      e4  SpeedStor      
     c  W95 FAT32 (LBA) 52  CP/M            9f  BSD/OS          eb  BeOS fs        
     e  W95 FAT16 (LBA) 53  OnTrack DM6 Aux a0  IBM Thinkpad hi ee  EFI GPT        
     f  W95 Ext'd (LBA) 54  OnTrackDM6      a5  FreeBSD         ef  EFI (FAT-12/16/
    10  OPUS            55  EZ-Drive        a6  OpenBSD         f0  Linux/PA-RISC b
    11  Hidden FAT12    56  Golden Bow      a7  NeXTSTEP        f1  SpeedStor      
    12  Compaq diagnost 5c  Priam Edisk     a8  Darwin UFS      f4  SpeedStor      
    14  Hidden FAT16 3 61  SpeedStor       a9  NetBSD          f2  DOS secondary  
    16  Hidden FAT16    63  GNU HURD or Sys ab  Darwin boot     fb  VMware VMFS    
    17  Hidden HPFS/NTF 64  Novell Netware  b7  BSDI fs         fc  VMware VMKCORE
    18  AST SmartSleep  65  Novell Netware  b8  BSDI swap       fd  Linux raid auto
    1b  Hidden W95 FAT3 70  DiskSecure Mult bb  Boot Wizard hid fe  LANstep        
    1c  Hidden W95 FAT3 75  PC/IX           be  Solaris boot    ff  BBT            
    Hex code (type L to list codes): 82
    Changed system type of partition 1 to 82 (Linux swap / Solaris)
  • 6. Re: Questions from Linux System Admin on storage
    Levi-Pereira Guru
    Currently Being Moderated
    james_p wrote:
    Thank you. I have 2 more questions


    Question1.
    Should it be extended partition or Primary partition ? Should I choose 'e' or 'p' for below ?
    # fdisk /dev/sdb
    .
    .
    .
    .
    Command action
    e   extended
    p   primary partition (1-4)
    Use primary partition

    >
    Question2.
    What should be the partition type ? What should I choose from the list below
    Command (m for help): t
    Selected partition 1
    Hex code (type L to list codes): l
    
    0  Empty           1e  Hidden W95 FAT1 80  Old Minix       bf  Solaris        
    1  FAT12           24  NEC DOS         81  Minix / old Lin c1  DRDOS/sec (FAT-
    2  XENIX root      39  Plan 9          82  Linux swap / So c4  DRDOS/sec (FAT-
    3  XENIX usr       3c  PartitionMagic  83  Linux           c6  DRDOS/sec (FAT-
    4  FAT16 <32M      40  Venix 80286     84  OS/2 hidden C:  c7  Syrinx        
    5  Extended        41  PPC PReP Boot   85  Linux extended  da  Non-FS data    
    6  FAT16           42  SFS             86  NTFS volume set db  CP/M / CTOS / .
    7  HPFS/NTFS       4d  QNX4.x          87  NTFS volume set de  Dell Utility  
    8  AIX             4e  QNX4.x 2nd part 88  Linux plaintext df  BootIt        
    9  AIX bootable    4f  QNX4.x 3rd part 8e  Linux LVM       e1  DOS access    
    a  OS/2 Boot Manag 50  OnTrack DM      93  Amoeba          e3  DOS R/O        
    b  W95 FAT32       51  OnTrack DM6 Aux 94  Amoeba BBT      e4  SpeedStor      
    c  W95 FAT32 (LBA) 52  CP/M            9f  BSD/OS          eb  BeOS fs        
    e  W95 FAT16 (LBA) 53  OnTrack DM6 Aux a0  IBM Thinkpad hi ee  EFI GPT        
    f  W95 Ext'd (LBA) 54  OnTrackDM6      a5  FreeBSD         ef  EFI (FAT-12/16/
    10  OPUS            55  EZ-Drive        a6  OpenBSD         f0  Linux/PA-RISC b
    11  Hidden FAT12    56  Golden Bow      a7  NeXTSTEP        f1  SpeedStor      
    12  Compaq diagnost 5c  Priam Edisk     a8  Darwin UFS      f4  SpeedStor      
    14  Hidden FAT16 3 61  SpeedStor       a9  NetBSD          f2  DOS secondary  
    16  Hidden FAT16    63  GNU HURD or Sys ab  Darwin boot     fb  VMware VMFS    
    17  Hidden HPFS/NTF 64  Novell Netware  b7  BSDI fs         fc  VMware VMKCORE
    18  AST SmartSleep  65  Novell Netware  b8  BSDI swap       fd  Linux raid auto
    1b  Hidden W95 FAT3 70  DiskSecure Mult bb  Boot Wizard hid fe  LANstep        
    1c  Hidden W95 FAT3 75  PC/IX           be  Solaris boot    ff  BBT            
    Hex code (type L to list codes): 82
    Changed system type of partition 1 to 82 (Linux swap / Solaris)
    Leave to default.


    You will need this:
    http://docs.oracle.com/cd/E11882_01/install.112/e22489/storage.htm#CDEBFDEH
  • 7. Re: Questions from Linux System Admin on storage
    rarain Explorer
    Currently Being Moderated
    Hi James,

    Use primary partition and you don't need to specify partition type because to create ASM diskgroup you need RAW devices. Better to follow docs as suggested.

    http://docs.oracle.com/cd/E11882_01/install.112/e22489/storage.htm#CDEBFDEH

    Thanks

    Edited by: rarain on May 8, 2013 10:52 AM
  • 8. Re: Questions from Linux System Admin on storage
    Omega3 Newbie
    Currently Being Moderated
    Question2.
    What should be the partition type ?


    I think this is all you need
    [oracle@el6 Desktop]$ su -
    Password:
    [root@el6 ~]# fdisk /dev/sdc
    Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
    Building a new DOS disklabel with disk identifier 0x8c2dfecc.
    Changes will remain in memory only, until you decide to write them.
    After that, of course, the previous content won't be recoverable.
    
    Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
    
    WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
             switch off the mode (command 'c') and change display units to
             sectors (command 'u').
    
    Command (m for help): n
    Command action
       e   extended
       p   primary partition (1-4)
    p
    Partition number (1-4): 1
    First cylinder (1-1305, default 1):
    Using default value 1
    Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305):
    Using default value 1305
    
    Command (m for help): w
    The partition table has been altered!
    
    Calling ioctl() to re-read partition table.
    Syncing disks.
    [root@el6 ~]#
    now
    /dev/sdc1
    will be created and ASM will take it from here

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points