This discussion is archived
1 2 Previous Next 16 Replies Latest reply: Oct 17, 2012 2:18 AM by rukbat RSS

Re: Oracle Server 3.1.1 don't see local HDDs. How to diagnose?

florin cercelaru Newbie
Currently Being Moderated
Hi,

I think I have the same situation as this other post
Oracle Server 3.1.1 don't see local HDDs. How to diagnose?
so I hijacked the discussion away from that original poster. A kind forum moderator has split my part of the thread away to let it stand on its own (just as I should have done from the beginning).

.
I checked the multipathd service..the service is running. Check below:
[root@HPML350-1 ~]# ls -l /dev/mapper
total 0
brw-rw---- 1 root disk 252,   1 Oct 12 12:06 360080e500023b6f0000012954f202c1f
brw-rw---- 1 root disk 252,   2 Oct 12 12:06 360080e500023b6f0000012954f202c1fp1
brw-rw---- 1 root disk 252,   0 Oct 12 12:06 360080e500023b6f0000012974f202c43
brw-rw---- 1 root disk 252,   3 Oct 12 12:06 360080e500023b6f0000012974f202c43p1
crw------- 1 root root  10, 236 Oct 12 12:04 control
brw-rw---- 1 root disk 252,   4 Oct 12 12:07 ovspoolfs
[root@HPML350-1 ~]# cat /proc/partitions
major minor  #blocks  name

 104        0  143338560 cciss/c0d0
 104        1  142279641 cciss/c0d0p1
 104        2    1052257 cciss/c0d0p2
 104       16  430025624 cciss/c0d1
   8       16  524288000 sdb
   8       32   15728640 sdc
   8       33   15728606 sdc1
   8        0   15728640 sda
   8       48  524288000 sdd
   8       49  157286366 sdd1
 252        0  524288000 dm-0
 252        1   15728640 dm-1
 252        2   15728606 dm-2
 252        3  157286366 dm-3
 252        4   10485760 dm-4
[root@HPML350-1 ~]#
  • 1. Re: Oracle Server 3.1.1 don't see local HDDs. How to diagnose?
    Avi Miller Guru
    Currently Being Moderated
    Florin Cercelaru wrote:
    brw-rw---- 1 root disk 252, 1 Oct 12 12:06 360080e500023b6f0000012954f202c1f
    brw-rw---- 1 root disk 252, 2 Oct 12 12:06 360080e500023b6f0000012954f202c1fp1
    brw-rw---- 1 root disk 252, 0 Oct 12 12:06 360080e500023b6f0000012974f202c43
    brw-rw---- 1 root disk 252, 3 Oct 12 12:06 360080e500023b6f0000012974f202c43p1

    You have partitions (p1) on each disk, so they will be suppressed by Oracle VM. The disks must be unpartitioned to be usable by Oracle VM.
  • 2. Re: Oracle Server 3.1.1 don't see local HDDs. How to diagnose?
    florin cercelaru Newbie
    Currently Being Moderated
    Hi,

    my problem is with this disk
    104       16  430025624 cciss/c0d1
  • 3. Re: Oracle Server 3.1.1 don't see local HDDs. How to diagnose?
    Avi Miller Guru
    Currently Being Moderated
    Florin Cercelaru wrote:
    my problem is with this disk
    It's probably being blacklisted: http://docs.oracle.com/cd/E27300_01/E27307/html/vmrns-bugs.html#vmrns-hp-smart-array-error
  • 4. Re: Oracle Server 3.1.1 don't see local HDDs. How to diagnose?
    florin cercelaru Newbie
    Currently Being Moderated
    thanks .. now .. try keep you updated ..
  • 5. Re: Oracle Server 3.1.1 don't see local HDDs. How to diagnose?
    florin cercelaru Newbie
    Currently Being Moderated
    HI,

    I changed the multipath conf file..but.. not work;
    # List of device names to discard as not multipath candidates
    #
    ## IMPORTANT for OVS do not remove the black listed devices.
    blacklist {
            devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st|nbd)[0-9]*"
            devnode "^hd[a-z][0-9]*"
            devnode "^etherd"
            devnode "^cciss!c[0-9]d[0-9]*"  << I add this line
            %include "/etc/blacklisted.wwids"
    }
    Edited by: Florin Cercelaru on Oct 12, 2012 10:09 AM

    Edited by: Florin Cercelaru on Oct 12, 2012 10:09 AM
  • 6. Re: Oracle Server 3.1.1 don't see local HDDs. How to diagnose?
    Avi Miller Guru
    Currently Being Moderated
    Florin Cercelaru wrote:
    devnode "^cciss!c[0-9]d[0-9]*" << I add this line
    No, that will definitely black list the disk. :) You need to make sure it's not being blacklisted.
  • 7. Re: Oracle Server 3.1.1 don't see local HDDs. How to diagnose?
    florin cercelaru Newbie
    Currently Being Moderated
    :)..ok I remove the line..but not work...restart the multipathd service..but nothing..:(
  • 8. Re: Oracle Server 3.1.1 don't see local HDDs. How to diagnose?
    Avi Miller Guru
    Currently Being Moderated
    Florin Cercelaru wrote:
    :)..ok I remove the line..but not work...restart the multipathd service..but nothing..:(
    Check the output from multipath -v3 -- it may be getting blacklisted elsewhere, like being listed in blacklisted.wwids or something.
  • 9. Re: Oracle Server 3.1.1 don't see local HDDs. How to diagnose?
    florin cercelaru Newbie
    Currently Being Moderated
    ..this is the output..
    [root@HPML350-1 ~]# multipath -v3
    Oct 12 13:36:29 | ram0: device node name blacklisted
    Oct 12 13:36:29 | ram1: device node name blacklisted
    Oct 12 13:36:29 | ram2: device node name blacklisted
    Oct 12 13:36:29 | ram3: device node name blacklisted
    Oct 12 13:36:29 | ram4: device node name blacklisted
    Oct 12 13:36:29 | ram5: device node name blacklisted
    Oct 12 13:36:29 | ram6: device node name blacklisted
    Oct 12 13:36:29 | ram7: device node name blacklisted
    Oct 12 13:36:29 | ram8: device node name blacklisted
    Oct 12 13:36:29 | ram9: device node name blacklisted
    Oct 12 13:36:29 | ram10: device node name blacklisted
    Oct 12 13:36:29 | ram11: device node name blacklisted
    Oct 12 13:36:29 | ram12: device node name blacklisted
    Oct 12 13:36:29 | ram13: device node name blacklisted
    Oct 12 13:36:29 | ram14: device node name blacklisted
    Oct 12 13:36:29 | ram15: device node name blacklisted
    Oct 12 13:36:29 | loop0: device node name blacklisted
    Oct 12 13:36:29 | loop1: device node name blacklisted
    Oct 12 13:36:29 | loop2: device node name blacklisted
    Oct 12 13:36:29 | loop3: device node name blacklisted
    Oct 12 13:36:29 | loop4: device node name blacklisted
    Oct 12 13:36:29 | loop5: device node name blacklisted
    Oct 12 13:36:29 | loop6: device node name blacklisted
    Oct 12 13:36:29 | loop7: device node name blacklisted
    Oct 12 13:36:29 | cciss!c0d0: not found in pathvec
    Oct 12 13:36:29 | cciss!c0d0: mask = 0x1f
    Oct 12 13:36:29 | cciss!c0d0: dev_t = 104:0
    Oct 12 13:36:29 | cciss!c0d0: size = 286677120
    Oct 12 13:36:29 | cciss!c0d0: subsystem = cciss
    Oct 12 13:36:29 | cciss!c0d0: h:b:t:l = 0:0:0:0
    Oct 12 13:36:29 | cciss!c0d0: vendor = HP
    Oct 12 13:36:29 | cciss!c0d0: product = LOGICAL VOLUME
    Oct 12 13:36:29 | cciss!c0d0: revision = 1.86
    Oct 12 13:36:29 | cciss!c0d0: get_state
    Oct 12 13:36:29 | loading /lib64/multipath/libchecktur.so checker
    Oct 12 13:36:29 | cciss!c0d0: path checker = tur (controller setting)
    Oct 12 13:36:29 | cciss!c0d0: checker timeout = 300000 ms (internal default)
    Oct 12 13:36:29 | cciss!c0d0: path offline
    Oct 12 13:36:29 | cciss!c0d0: getuid = /sbin/scsi_id -g -u -s /block/%n (controller setting)
    Oct 12 13:36:29 | /sbin/scsi_id exitted with 1
    Oct 12 13:36:29 | error calling out /sbin/scsi_id -g -u -s /block/cciss/c0d0
    Oct 12 13:36:29 | /sbin/scsi_id exitted with 1
    Oct 12 13:36:29 | error calling out /sbin/scsi_id -g -u -s /block/cciss/c0d0
    Oct 12 13:36:29 | cciss!c0d0: prio = const (controller setting)
    Oct 12 13:36:29 | cciss!c0d0: const prio = 1
    Oct 12 13:36:29 | cciss!c0d1: not found in pathvec
    Oct 12 13:36:29 | cciss!c0d1: mask = 0x1f
    Oct 12 13:36:29 | cciss!c0d1: dev_t = 104:16
    Oct 12 13:36:29 | cciss!c0d1: size = 860051248
    Oct 12 13:36:29 | cciss!c0d1: subsystem = cciss
    Oct 12 13:36:29 | cciss!c0d1: h:b:t:l = 0:0:1:0
    Oct 12 13:36:29 | cciss!c0d1: vendor = HP
    Oct 12 13:36:29 | cciss!c0d1: product = LOGICAL VOLUME
    Oct 12 13:36:29 | cciss!c0d1: revision = 1.86
    Oct 12 13:36:29 | cciss!c0d1: get_state
    Oct 12 13:36:29 | cciss!c0d1: path checker = tur (controller setting)
    Oct 12 13:36:29 | cciss!c0d1: checker timeout = 300000 ms (internal default)
    Oct 12 13:36:29 | cciss!c0d1: path offline
    Oct 12 13:36:29 | cciss!c0d1: getuid = /sbin/scsi_id -g -u -s /block/%n (controller setting)
    Oct 12 13:36:29 | /sbin/scsi_id exitted with 1
    Oct 12 13:36:29 | error calling out /sbin/scsi_id -g -u -s /block/cciss/c0d1
    Oct 12 13:36:29 | /sbin/scsi_id exitted with 1
    Oct 12 13:36:29 | error calling out /sbin/scsi_id -g -u -s /block/cciss/c0d1
    Oct 12 13:36:29 | cciss!c0d1: prio = const (controller setting)
    Oct 12 13:36:29 | cciss!c0d1: const prio = 1
    Oct 12 13:36:29 | sr0: device node name blacklisted
    Oct 12 13:36:29 | sda: not found in pathvec
    Oct 12 13:36:29 | sda: mask = 0x1f
    Oct 12 13:36:29 | sda: dev_t = 8:0
    Oct 12 13:36:29 | sda: size = 31457280
    Oct 12 13:36:29 | sda: subsystem = scsi
    Oct 12 13:36:29 | sda: vendor = IBM
    Oct 12 13:36:29 | sda: product = 1746      FAStT
    Oct 12 13:36:29 | sda: rev = 1070
    Oct 12 13:36:29 | sda: h:b:t:l = 0:0:0:0
    Oct 12 13:36:29 | sda: serial = SV11618036
    Oct 12 13:36:29 | sda: get_state
    Oct 12 13:36:29 | loading /lib64/multipath/libcheckrdac.so checker
    Oct 12 13:36:29 | sda: path checker = rdac (controller setting)
    Oct 12 13:36:29 | sda: checker timeout = 60000 ms (sysfs setting)
    Oct 12 13:36:29 | sda: state = running
    Oct 12 13:36:29 | sda: state = 5
    Oct 12 13:36:29 | sda: getuid = /sbin/scsi_id -g -u -s /block/%n (controller setting)
    Oct 12 13:36:29 | sda: uid = 360080e500023b6f0000012954f202c1f (callout)
    Oct 12 13:36:29 | loading /lib64/multipath/libpriordac.so prioritizer
    Oct 12 13:36:29 | sda: prio = rdac (controller setting)
    Oct 12 13:36:29 | sda: rdac prio = 1
    Oct 12 13:36:29 | sdb: not found in pathvec
    Oct 12 13:36:29 | sdb: mask = 0x1f
    Oct 12 13:36:29 | sdb: dev_t = 8:16
    Oct 12 13:36:29 | sdb: size = 1048576000
    Oct 12 13:36:29 | sdb: subsystem = scsi
    Oct 12 13:36:29 | sdb: vendor = IBM
    Oct 12 13:36:29 | sdb: product = 1746      FAStT
    Oct 12 13:36:29 | sdb: rev = 1070
    Oct 12 13:36:29 | sdb: h:b:t:l = 0:0:0:1
    Oct 12 13:36:29 | sdb: serial = SV11618036
    Oct 12 13:36:29 | sdb: get_state
    Oct 12 13:36:29 | sdb: path checker = rdac (controller setting)
    Oct 12 13:36:29 | sdb: checker timeout = 60000 ms (sysfs setting)
    Oct 12 13:36:29 | sdb: state = running
    Oct 12 13:36:29 | sdb: state = 5
    Oct 12 13:36:29 | sdb: getuid = /sbin/scsi_id -g -u -s /block/%n (controller setting)
    Oct 12 13:36:29 | sdb: uid = 360080e500023b6f0000012974f202c43 (callout)
    Oct 12 13:36:29 | sdb: prio = rdac (controller setting)
    Oct 12 13:36:29 | sdb: rdac prio = 1
    Oct 12 13:36:29 | sdc: not found in pathvec
    Oct 12 13:36:29 | sdc: mask = 0x1f
    Oct 12 13:36:29 | sdc: dev_t = 8:32
    Oct 12 13:36:29 | sdc: size = 31457280
    Oct 12 13:36:29 | sdc: subsystem = scsi
    Oct 12 13:36:29 | sdc: vendor = IBM
    Oct 12 13:36:29 | sdc: product = 1746      FAStT
    Oct 12 13:36:29 | sdc: rev = 1070
    Oct 12 13:36:29 | sdc: h:b:t:l = 0:0:1:0
    Oct 12 13:36:29 | sdc: serial = SV11807225
    Oct 12 13:36:29 | sdc: get_state
    Oct 12 13:36:29 | sdc: path checker = rdac (controller setting)
    Oct 12 13:36:29 | sdc: checker timeout = 60000 ms (sysfs setting)
    Oct 12 13:36:29 | sdc: state = running
    Oct 12 13:36:29 | sdc: state = 3
    Oct 12 13:36:29 | sdc: getuid = /sbin/scsi_id -g -u -s /block/%n (controller setting)
    Oct 12 13:36:29 | sdc: uid = 360080e500023b6f0000012954f202c1f (callout)
    Oct 12 13:36:29 | sdc: prio = rdac (controller setting)
    Oct 12 13:36:29 | sdc: rdac prio = 6
    Oct 12 13:36:29 | sdd: not found in pathvec
    Oct 12 13:36:29 | sdd: mask = 0x1f
    Oct 12 13:36:29 | sdd: dev_t = 8:48
    Oct 12 13:36:29 | sdd: size = 1048576000
    Oct 12 13:36:29 | sdd: subsystem = scsi
    Oct 12 13:36:29 | sdd: vendor = IBM
    Oct 12 13:36:29 | sdd: product = 1746      FAStT
    Oct 12 13:36:29 | sdd: rev = 1070
    Oct 12 13:36:29 | sdd: h:b:t:l = 0:0:1:1
    Oct 12 13:36:29 | sdd: serial = SV11807225
    Oct 12 13:36:29 | sdd: get_state
    Oct 12 13:36:29 | sdd: path checker = rdac (controller setting)
    Oct 12 13:36:29 | sdd: checker timeout = 60000 ms (sysfs setting)
    Oct 12 13:36:29 | sdd: state = running
    Oct 12 13:36:29 | sdd: state = 3
    Oct 12 13:36:29 | sdd: getuid = /sbin/scsi_id -g -u -s /block/%n (controller setting)
    Oct 12 13:36:29 | sdd: uid = 360080e500023b6f0000012974f202c43 (callout)
    Oct 12 13:36:29 | sdd: prio = rdac (controller setting)
    Oct 12 13:36:29 | sdd: rdac prio = 6
    Oct 12 13:36:29 | md0: device node name blacklisted
    Oct 12 13:36:29 | dm-0: device node name blacklisted
    Oct 12 13:36:29 | dm-1: device node name blacklisted
    Oct 12 13:36:29 | dm-2: device node name blacklisted
    Oct 12 13:36:29 | dm-3: device node name blacklisted
    Oct 12 13:36:29 | dm-4: device node name blacklisted
    ===== paths list =====
    uuid                              hcil    dev        dev_t  pri dm_st chk_st v
                                      0:0:0:0 cciss!c0d0 104:0  1   undef faulty H
                                      0:0:1:0 cciss!c0d1 104:16 1   undef faulty H
    360080e500023b6f0000012954f202c1f 0:0:0:0 sda        8:0    1   undef ghost  I
    360080e500023b6f0000012974f202c43 0:0:0:1 sdb        8:16   1   undef ghost  I
    360080e500023b6f0000012954f202c1f 0:0:1:0 sdc        8:32   6   undef ready  I
    360080e500023b6f0000012974f202c43 0:0:1:1 sdd        8:48   6   undef ready  I
    Oct 12 13:36:29 | params = 3 queue_if_no_path pg_init_retries 50 1 rdac 2 1 round-robin 0 1 1 8:48 1000 round-robin 0 1 1 8:16 1000
    Oct 12 13:36:29 | status = 2 0 0 0 2 1 E 0 1 0 8:48 A 0 E 0 1 0 8:16 A 0
    Oct 12 13:36:29 | params = 3 queue_if_no_path pg_init_retries 50 1 rdac 2 1 round-robin 0 1 1 8:32 1000 round-robin 0 1 1 8:0 1000
    Oct 12 13:36:29 | status = 2 0 0 0 2 1 E 0 1 0 8:32 A 0 E 0 1 0 8:0 A 0
    [root@HPML350-1 ~]#
  • 10. Re: Oracle Server 3.1.1 don't see local HDDs. How to diagnose?
    Avi Miller Guru
    Currently Being Moderated
    Florin Cercelaru wrote:
    ..this is the output..
    Oct 12 13:36:29 | cciss!c0d0: getuid = /sbin/scsi_id -g -u -s /block/%n (controller setting)
    Oct 12 13:36:29 | /sbin/scsi_id exitted with 1
    Oct 12 13:36:29 | error calling out /sbin/scsi_id -g -u -s /block/cciss/c0d0
    Oct 12 13:36:29 | /sbin/scsi_id exitted with 1
    Oct 12 13:36:29 | error calling out /sbin/scsi_id -g -u -s /block/cciss/c0d0
    The call to get the uuid of the disk is failing. I don't have any cciss disks handy, so I have no idea why or what the correct configuration is supposed to be. I recommend opening an SR with Oracle Support.
  • 11. Re: Oracle Server 3.1.1 don't see local HDDs. How to diagnose?
    florin cercelaru Newbie
    Currently Being Moderated
    Hi Avi,

    thanks for help .. good to know for the future ..
    I studied the problem and it seems that version 3.1.1 does not work ..only version, 3.0.3 ...
    for the moment I exported local disks as nfs..

    a good day
  • 12. Re: Oracle Server 3.1.1 don't see local HDDs. How to diagnose?
    Avi Miller Guru
    Currently Being Moderated
    Florin Cercelaru wrote:
    for the moment I exported local disks as nfs..
    Please don't do this -- Dom0 has very, very limited resources and NFS writes are just going to consume lots and lots of memory, leading to potential disk corruption or node fencing. If this is for personal testing with very low usage, you can probably get away with it, but definitely not for anything resembling production workloads.

    If you have Oracle Support, please log an SR.
  • 13. Re: Oracle Server 3.1.1 don't see local HDDs. How to diagnose?
    rukbat Guru Moderator
    Currently Being Moderated
    Moderator Action:

    @ user Florin,
    Your post and its replies have been split away from that other thread.
    You hijacked that thread and then asked your question.
    That's questionable forum etiquette.

    How would you feel if you had a forum thread and someone ripped ownership away from you?

    The discussion of your issue has value to the general knowledgebase. That's why it wasn't just summarily deleted.
  • 14. Re: Oracle Server 3.1.1 don't see local HDDs. How to diagnose?
    florin cercelaru Newbie
    Currently Being Moderated
    Hi Avi,

    thanks for the guidance..will give up the idea..local disc export as nfs.
1 2 Previous Next