This discussion is archived
1 2 Previous Next 23 Replies Latest reply: Oct 24, 2012 10:40 PM by 970402 RSS

OUI cannot list candidate disks for ASM Disk Group during Grid Infra instln

user-1221 Newbie
Currently Being Moderated
Hi,

We are trying to install 2 node Oracle RAC 11gR2 on HP-UX 11.31, all the prerequisites are done, the CVU test is successfull for both the nodes, except the "-x" slewing option for NTP on node 2.

During ASM Disk Group creation, the OUI cannot see the disks;

provided grid:asmadmin ownership on the disks with 660 file permissions, even created the similar names for the shared disks on the both nodes.
_Node 1:_

*/dev/rdisk*
crw-rw----   1 grid       asmadmin    22 0x000010 Jul  9 10:33 ocr1
crw-rw----   1 grid       asmadmin    22 0x000003 Jul  9 10:33 ocr2
crw-rw----   1 grid       asmadmin    22 0x00000d Jul  9 10:33 ocr3
crw-rw----   1 grid       asmadmin    22 0x00000e Jul  9 10:33 vote1
crw-rw----   1 grid       asmadmin    22 0x00000c Jul  9 10:33 vote2
crw-rw----   1 grid       asmadmin    22 0x00000b Jul  9 10:33 vote3
crw-rw----   1 grid       asmadmin    22 0x00000f Jul  9 10:33 data1
crw-rw----   1 grid       asmadmin    22 0x000002 Jul  9 10:33 arch1

*/dev/rdsk*
crw-rw----   1 grid       asmadmin   188 0x030200 Jul  9 09:21 ocr1
crw-rw----   1 grid       asmadmin   188 0x1b0200 Jul  9 09:21 ocr2
crw-rw----   1 grid       asmadmin   188 0x2d0100 Jul  9 09:21 ocr3
crw-rw----   1 grid       asmadmin   188 0x2f0200 Jul  9 09:21 vote1
crw-rw----   1 grid       asmadmin   188 0x110300 Jul  9 09:21 vote2
crw-rw----   1 grid       asmadmin   188 0x0b0300 Jul  9 09:21 vote3
crw-rw----   1 grid       asmadmin   188 0x150100 Jul  9 09:21 data1
crw-rw----   1 grid       asmadmin   188 0x030100 Jul  9 09:21 arch1

_Node 2:_

*/dev/rdisk*
crw-rw----   1 grid       asmadmin    22 0x000014 Jul  9 10:33 ocr2
crw-rw----   1 grid       asmadmin    22 0x000012 Jul  9 10:33 ocr1
crw-rw----   1 grid       asmadmin    22 0x00000e Jul  9 10:33 ocr3
crw-rw----   1 grid       asmadmin    22 0x000010 Jul  9 10:33 vote1
crw-rw----   1 grid       asmadmin    22 0x000011 Jul  9 10:33 vote2
crw-rw----   1 grid       asmadmin    22 0x000013 Jul  9 10:33 vote3
crw-rw----   1 grid       asmadmin    22 0x00000c Jul  9 10:33 arch1
crw-rw----   1 grid       asmadmin    22 0x00000f Jul  9 10:33 data1

*/dev/rdsk/*
crw-rw----   1 grid       asmadmin   188 0x1a0200 Jul  9 09:28 ocr1
crw-rw----   1 grid       asmadmin   188 0x1c0200 Jul  9 09:28 ocr2
crw-rw----   1 grid       asmadmin   188 0x2f0100 Jul  9 09:28 ocr3
crw-rw----   1 grid       asmadmin   188 0x180300 Jul  9 09:28 vote1
crw-rw----   1 grid       asmadmin   188 0x1c0300 Jul  9 09:28 vote2
crw-rw----   1 grid       asmadmin   188 0x2d0200 Jul  9 09:28 vote3
crw-rw----   1 grid       asmadmin   188 0x220100 Jul  9 09:29 arch1
crw-rw----   1 grid       asmadmin   188 0x180100 Jul  9 09:29 data1

Even clean the disk header with dd...
dd if=/dev/zero of=/dev/rdisk/ocr1 bs=8192 count=10240
dd if=/dev/zero of=/dev/rdisk/ocr2 bs=8192 count=10240
dd if=/dev/zero of=/dev/rdisk/ocr3 bs=8192 count=10240
dd if=/dev/zero of=/dev/rdisk/vote1 bs=8192 count=10240
dd if=/dev/zero of=/dev/rdisk/vote2 bs=8192 count=10240
dd if=/dev/zero of=/dev/rdisk/vote3 bs=8192 count=10240
dd if=/dev/zero of=/dev/rdisk/arch1 bs=8192 count=10240
dd if=/dev/zero of=/dev/rdisk/data1 bs=8192 count=10240
Node1_

root@node1:/dev/rdisk# id grid
uid=1100(grid) gid=1000(oinstall) groups=1020(asmadmin),1021(asmdba)
root@node1:/dev/rdisk# id oracle
uid=1101(oracle) gid=1000(oinstall) groups=1021(asmdba),1031(dba),1032(oper)


Node2_

root@node2:/dev/rdsk# id grid
uid=1100(grid) gid=1000(oinstall) groups=1020(asmadmin),1021(asmdba)
root@node2:/dev/rdsk# id oracle
uid=1101(oracle) gid=1000(oinstall) groups=1031(dba),503(oper),1021(asmdba)

tried to list the disks with both the Disk Discovery Paths, "/dev/rdsk/*" and "/dev/rdisk/*" but still the disks are not shown in the OUI Utility.

Please let me know if I am missing something.

Thanks in Advance!
  • 1. Re: OUI cannot list candidate disks for ASM Disk Group during Grid Infra instln
    585179 Expert
    Currently Being Moderated
    HI,


    Are you using LVM disk device? if so can you please post

    #> ls -l /dev/*disk/ocr1


    Cheers
  • 2. Re: OUI cannot list candidate disks for ASM Disk Group during Grid Infra instln
    user-1221 Newbie
    Currently Being Moderated
    Hi,

    Thanks for your reply;

    NODE1:

    root@node1:/dev/rdisk# ls -l /dev/*disk/ocr*
    crw-rw---- 1 grid asmadmin 22 0x00000d Jul 9 10:33 /dev/rdisk/ocr1
    crw-rw---- 1 grid asmadmin 22 0x000010 Jul 9 10:33 /dev/rdisk/ocr2
    crw-rw---- 1 grid asmadmin 22 0x000003 Jul 9 10:33 /dev/rdisk/ocr3

    root@node1:/dev/rdisk# ls -l /dev/*dsk/ocr*
    crw-rw---- 1 grid asmadmin 188 0x2d0100 Jul 9 09:21 /dev/rdsk/ocr1
    crw-rw---- 1 grid asmadmin 188 0x030200 Jul 9 09:21 /dev/rdsk/ocr2
    crw-rw---- 1 grid asmadmin 188 0x1b0200 Jul 9 09:21 /dev/rdsk/ocr3

    NODE2:

    root@node2:/dev/rdisk# ls -l /dev/*disk/*ocr
    crw-rw---- 1 grid asmadmin 22 0x00000e Jul 9 10:33 /dev/rdisk/ocr3
    crw-rw---- 1 grid asmadmin 22 0x000014 Jul 9 10:33 /dev/rdisk/ocr2
    crw-rw---- 1 grid asmadmin 22 0x000012 Jul 9 10:33 /dev/rdisk/ocr1

    root@node2:/dev/rdisk# ls -l /dev/*dsk/*ocr
    crw-rw---- 1 grid asmadmin 188 0x2f0100 Jul 9 09:28 /dev/rdsk/ocr3
    crw-rw---- 1 grid asmadmin 188 0x1c0200 Jul 9 09:28 /dev/rdsk/ocr2
    crw-rw---- 1 grid asmadmin 188 0x1a0200 Jul 9 09:28 /dev/rdsk/ocr1
  • 3. Re: OUI cannot list candidate disks for ASM Disk Group during Grid Infra instln
    585179 Expert
    Currently Being Moderated
    Hi,


    You need give read permission to the corresponding block device as well of each disk you will use

    e.g

    #> chmod 644 [block device location]


    Cheers
  • 4. Re: OUI cannot list candidate disks for ASM Disk Group during Grid Infra instln
    user-1221 Newbie
    Currently Being Moderated
    hi,

    The Installation documentation says the permissions should be

    # chown grid:asmadmin /dev/rdsk/cxtydz
    # chmod 660 /dev/rdsk/cxtydz

    http://download.oracle.com/docs/cd/E11882_01/install.112/e17211/storage.htm#CDEEDHIJ
  • 5. Re: OUI cannot list candidate disks for ASM Disk Group during Grid Infra instln
    585179 Expert
    Currently Being Moderated
    user-1221 wrote:
    hi,

    The Installation documentation says the permissions should be

    # chown grid:asmadmin /dev/rdsk/cxtydz
    # chmod 660 /dev/rdsk/cxtydz

    http://download.oracle.com/docs/cd/E11882_01/install.112/e17211/storage.htm#CDEEDHIJ
    Hi,

    See Unable to discover or use raw devices for ASM in HP-UX Itanium in 11.2.0.2 [ID 1276913.1] for details


    CHeers
  • 6. Re: OUI cannot list candidate disks for ASM Disk Group during Grid Infra instln
    user-1221 Newbie
    Currently Being Moderated
    Hi,

    Thanks for the info.

    I changed the permissions for the corresponding block devices an still the disks does not list in the OUI.


    root@node1:/dev/disk# ll -rt
    total 0
    brw-r--r--   1 bin        sys          3 0x000002 Jul  6 00:07 disk21
    brw-r----- 1 bin sys 3 0x000009 Jul 6 00:07 disk23_p3
    brw-r----- 1 bin sys 3 0x000006 Jul 6 00:07 disk25
    brw-r----- 1 bin sys 3 0x000005 Jul 6 00:07 disk24
    brw-r----- 1 bin sys 3 0x000008 Jul 6 00:07 disk23_p2
    brw-r----- 1 bin sys 3 0x000004 Jul 6 00:07 disk23
    brw-r--r--   1 bin        sys          3 0x000003 Jul  6 00:07 disk22
    brw-r----- 1 bin sys 3 0x000007 Jul 6 16:51 disk23_p1
    brw-r--r--   1 bin        sys          3 0x00000f Jul  6 18:26 disk78
    brw-r--r--   1 bin        sys          3 0x00000e Jul  6 18:26 disk77
    brw-r--r--   1 bin        sys          3 0x00000c Jul  6 18:26 disk75
    brw-r--r--   1 bin        sys          3 0x00000d Jul  6 18:26 disk76
    brw-r--r--   1 bin        sys          3 0x00000b Jul  6 18:26 disk74
    brw-r--r--   1 bin        sys          3 0x000010 Jul  6 18:26 disk79

    root@node1:/dev/disk# ls -l /dev/*disk/disk21
    brw-r--r-- 1 bin sys 3 *0x000002* Jul 6 00:07 /dev/disk/disk21
    crw-r----- 1 bin sys 22 0x000002 Jul 6 00:07 /dev/rdisk/disk21
    root@node1:/dev/disk# ls -l /dev/rdisk/data1
    crw-rw---- 1 grid asmadmin 22 *0x000002* Jul 9 10:33 /dev/rdisk/data1

    I earlier created an alias for /dev/rdisk/disk21 as /dev/rdisk/data1 and gave grid:asmadmin ownership along with 660 permissions.
    the metalink note 1276913.1 has grid:oinstall ownership? do we need to change it to oinstall from asmadmin?
  • 7. Re: OUI cannot list candidate disks for ASM Disk Group during Grid Infra instln
    user-1221 Newbie
    Currently Being Moderated
    Hi,

    I installed the patch PHCO_41479 recommended in ID 1276913.1.
    still I cannot see the devices


    root@node1:/# swlist | grep PHCO_41479
    PHCO_41479 1.0 diskowner(1M) cumulative patch

    root@node2:/# swlist | grep PHCO_41479
    PHCO_41479 1.0 diskowner(1M) cumulative patch
  • 8. Re: OUI cannot list candidate disks for ASM Disk Group during Grid Infra instln
    HungryDBA Newbie
    Currently Being Moderated
    Hi friend,

    For 11gR2 RAC Implementation, you need to create specail Character Device files in Hp-UX 11.31 or older.

    Character Device: The Oracle GI, ASM & Oracle Database Homes binaries are established on the local disk of each of the RAC nodes. The OCR & Vote Disks required by Oracle GI must be on Character Devices.
    • Make sure that the disks are available, enter the following command on every node:

    # ioscan –fnNkCdisk
    # ioscan –C disk -fun

    •If the ioscan command does not display device name information for a device that you want to use, enter the following command to install the special device files for any new devices:
    # insf –e

    •The disk names for the same disk can be different on each node. A disk can be identified as the same one via the WWID. The WWID of a disk can be checked via the following command:
    # scsimgr lun_map -D /dev/rdisk/disk25 | grep WWID

    • At HP-UX, LUNs name could be different for same LUN ID, hence compare both LUNs WWID & map to the one Character Device File.


    • I recommend to create a special Oracle device directory, and to use mknod to create device paths in this special Oracle folder. This has the advantage, that you get same names for OCR and Voting files across all nodes in the cluster. In addition, it ensures that the permissions of these Oracle device files remain untouched of 'insf −e'.

    # mkdir /dev/oracle (on all RAC Nodes)

    # ll /dev/rdisk/disk25
    crw-r----- 1 bin sys *23 0x000019* Jan 16 12:16 /dev/rdisk/disk25

    # mknod /dev/oracle/asmdisk1 c *23 0x000019*

    Later during set-up, for the ASM instance, set the ASM_DISKSTRING parameter to /dev/oracle/*
    Now when 'insf -e' is run, it only touches 'standard' DSF, rather than this special ones.

    • Modify the owner, group, and permissions on the character raw device files (ASM and Voting Disks) on all nodes:

    # chown grid:asmadmin /dev/oracle/*
    # chmod 660 /dev/oracle/*


    That's it. Now you should able to see above LUNs for storing OCR & Voting Disks in ASM.

    I had just completed five 3 Nodes & 2 Nodes 11gR2 RAC Setup on HP-UX 11.31 & didn't find any issues for Shared Storage.


    Regards,
    Manish

    Mail: manish.nashikkar@hotmail.com
    Visit: http://manishn.wordpress.com
  • 9. Re: OUI cannot list candidate disks for ASM Disk Group during Grid Infra instln
    user-1221 Newbie
    Currently Being Moderated
    Hi,

    I ran the CVU and got to knwo that the user grid is missing "dba" membership.
    Check: Membership of user "grid" in group "dba"
      Node Name         User Exists   Group Exists  User in Group  Comment
      ---------------- 
      *node2           yes           yes           no            failed*
      *node1           yes           yes           no            failed*
    *Result: Membership check for user "grid" in group "dba" failed*
    
    
    
    Check default user file creation mask
      Node Name     Available                 Required                  Comment
      ------------  
      *node2       unknown                   0022                      failed*
      node1       022                       0022                      passed
    Result: Default user file creation mask check failed
    
    Starting Clock synchronization checks using Network Time Protocol(NTP)...
    
    NTP Configuration file check started...
    The NTP configuration file "/etc/ntp.conf" is available on all nodes
    NTP Configuration file check passed
    
    Checking daemon liveness...
    
    Check: Liveness for "xntpd"
      Node Name                             Running?
      ------------------- 
      node2                               yes
      node1                               yes
    Result: Liveness check passed for "xntpd"
    
    Checking NTP daemon command line for slewing option "-x"
    Check: NTP daemon command line
      Node Name                             Slewing Option Set?
      ----------------
      *node2                               no*
      node1                               yes
    Result:
    NTP daemon slewing option check failed on some nodes
    PRVF-5436 : The NTP daemon running on one or more nodes lacks the slewing option "-x"
    Result: Clock synchronization check using Network Time Protocol(NTP) failed
    
    Pre-check for cluster services setup was unsuccessful on all the nodes.
    Node1
    root@node1:/dev/rdisk# id grid
    uid=1100(grid) gid=1000(oinstall) groups=1020(asmadmin),1021(asmdba)
    root@node1:/dev/rdisk# id oracle
    uid=1101(oracle) gid=1000(oinstall) groups=1021(asmdba),1031(dba),1032(oper)

    Node2:
    root@node2:/dev/disk# id grid
    uid=1100(grid) gid=1000(oinstall) groups=1020(asmadmin),1021(asmdba)
    root@node2:/dev/disk# id oracle
    uid=1101(oracle) gid=1000(oinstall) groups=1031(dba),503(oper),1021(asmdba)
  • 10. Re: OUI cannot list candidate disks for ASM Disk Group during Grid Infra instln
    user-1221 Newbie
    Currently Being Moderated
    Hi,

    Added grid user to dba group
    Check: Membership of user "grid" in group "dba"
      Node Name         User Exists   Group Exists  User in Group  Comment
      ------------- 
      arcdbs2           yes           yes           yes           passed
      arcdbs1           yes           yes           yes           passed
    Result: Membership check for user "grid" in group "dba" passed
    root@node1:/dev/rdisk# id grid
    uid=1100(grid) gid=1000(oinstall) groups=1020(asmadmin),1021(asmdba),1031(dba)


    Still the disks are not listed in the OUI.
  • 11. Re: OUI cannot list candidate disks for ASM Disk Group during Grid Infra instln
    585179 Expert
    Currently Being Moderated
    Hi,

    Can you please post the output from

    # > ioscan -funN -C disk
    # > /sbin/pvdisplay /dev/disk/disk21


    Cheers
  • 12. Re: OUI cannot list candidate disks for ASM Disk Group during Grid Infra instln
    user-1221 Newbie
    Currently Being Moderated
    Hi,
    root@node1:/#  ioscan -funN -C disk
    Class     I  H/W Path  Driver         S/W State   H/W Type     Description
    ===========================================================================
    disk      0  0/0/2/1.0x0.0x10   UsbScsiAdaptor   CLAIMED     LUN_PATH     USB SCSI Stack Adaptor
    disk     21  64000/0xfa00/0x2   esdisk           CLAIMED     DEVICE       HP      HSV210
                          /dev/disk/disk21     /dev/rdisk/data1
                          /dev/rdisk/disk21
    disk     22  64000/0xfa00/0x3   esdisk           CLAIMED     DEVICE       HP      HSV210
                          /dev/disk/disk22    /dev/rdisk/disk22   /dev/rdisk/ocr1
    disk     23  64000/0xfa00/0x4   esdisk           CLAIMED     DEVICE       HP      IR Volume
                          /dev/disk/disk23      /dev/rdisk/disk23
                          /dev/disk/disk23_p1   /dev/rdisk/disk23_p1
                          /dev/disk/disk23_p2   /dev/rdisk/disk23_p2
                          /dev/disk/disk23_p3   /dev/rdisk/disk23_p3
    disk     24  64000/0xfa00/0x5   esdisk           CLAIMED     DEVICE       HP      DG146ABAB4
                          /dev/disk/disk24   /dev/rdisk/disk24
    disk     25  64000/0xfa00/0x6   esdisk           CLAIMED     DEVICE       HP      DG146ABAB4
                          /dev/disk/disk25   /dev/rdisk/disk25
    disk     74  64000/0xfa00/0xb   esdisk           CLAIMED     DEVICE       HP      HSV210
                          /dev/disk/disk74     /dev/rdisk/vote3
                          /dev/rdisk/disk74
    disk     75  64000/0xfa00/0xc   esdisk           CLAIMED     DEVICE       HP      HSV210
                          /dev/disk/disk75     /dev/rdisk/vote2
                          /dev/rdisk/disk75
    disk     76  64000/0xfa00/0xd   esdisk           CLAIMED     DEVICE       HP      HSV210
                          /dev/disk/disk76    /dev/rdisk/disk76   /dev/rdisk/ocr3
    disk     77  64000/0xfa00/0xe   esdisk           CLAIMED     DEVICE       HP      HSV210
                          /dev/disk/disk77     /dev/rdisk/vote1
                          /dev/rdisk/disk77
    disk     78  64000/0xfa00/0xf   esdisk           CLAIMED     DEVICE       HP      HSV210
                          /dev/disk/disk78     /dev/rdisk/arch1
                          /dev/rdisk/disk78
    disk     79  64000/0xfa00/0x10  esdisk           CLAIMED     DEVICE       HP      HSV210
                          /dev/disk/disk79    /dev/rdisk/disk79   /dev/rdisk/ocr2
    root@node1:/# /sbin/pvdisplay /dev/disk/disk21
    pvdisplay: Couldn't find the volume group to which
    physical volume "/dev/disk/disk21" belongs.
    pvdisplay: Cannot display physical volume "/dev/disk/disk21".
  • 13. Re: OUI cannot list candidate disks for ASM Disk Group during Grid Infra instln
    user-1221 Newbie
    Currently Being Moderated
    Hi Manish,

    Thanks for the detailed explanation,

    I performed everything what you have mentioned in the post, even I performed the action mentioned in the metalink ID 1276913.1, but still the disks are not getting identified by the OUI.

    Please share the Grid Infrastructure installation procedure.

    Please Help me in resolving this issue, the deadline has already passed for this activity.

    Thanks in Advance!
  • 14. Re: OUI cannot list candidate disks for ASM Disk Group during Grid Infra instln
    HungryDBA Newbie
    Currently Being Moderated
    Hi friend,

    You are not seeing LUNs in GUI that means there could be issue with LUNs, which are not persistent across all the Nodes. Find it out correct LUN ID by above mentioned command & map to the Character Special file like /dev/oracle/asmdisk1, so that Oracle would be use the same disk even after reboot. Even check that proper permissions & ownerships you have given to these special files.


    Give me your mail ID, so that I shall be provided you the document for implementing 11gR2 RAC on HP-UX 11.31 (which I followed)


    Regards,
    Manish
1 2 Previous Next

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points