1 2 Previous Next 16 Replies Latest reply: Oct 17, 2012 4:18 AM by rukbat RSS

    Re: Oracle Server 3.1.1 don't see local HDDs. How to diagnose?

    florin cercelaru
      Hi,

      I think I have the same situation as this other post
      Oracle Server 3.1.1 don't see local HDDs. How to diagnose?
      so I hijacked the discussion away from that original poster. A kind forum moderator has split my part of the thread away to let it stand on its own (just as I should have done from the beginning).

      .
      I checked the multipathd service..the service is running. Check below:
      [root@HPML350-1 ~]# ls -l /dev/mapper
      total 0
      brw-rw---- 1 root disk 252,   1 Oct 12 12:06 360080e500023b6f0000012954f202c1f
      brw-rw---- 1 root disk 252,   2 Oct 12 12:06 360080e500023b6f0000012954f202c1fp1
      brw-rw---- 1 root disk 252,   0 Oct 12 12:06 360080e500023b6f0000012974f202c43
      brw-rw---- 1 root disk 252,   3 Oct 12 12:06 360080e500023b6f0000012974f202c43p1
      crw------- 1 root root  10, 236 Oct 12 12:04 control
      brw-rw---- 1 root disk 252,   4 Oct 12 12:07 ovspoolfs
      [root@HPML350-1 ~]# cat /proc/partitions
      major minor  #blocks  name
      
       104        0  143338560 cciss/c0d0
       104        1  142279641 cciss/c0d0p1
       104        2    1052257 cciss/c0d0p2
       104       16  430025624 cciss/c0d1
         8       16  524288000 sdb
         8       32   15728640 sdc
         8       33   15728606 sdc1
         8        0   15728640 sda
         8       48  524288000 sdd
         8       49  157286366 sdd1
       252        0  524288000 dm-0
       252        1   15728640 dm-1
       252        2   15728606 dm-2
       252        3  157286366 dm-3
       252        4   10485760 dm-4
      [root@HPML350-1 ~]#
        • 1. Re: Oracle Server 3.1.1 don't see local HDDs. How to diagnose?
          Avi Miller-Oracle
          Florin Cercelaru wrote:
          brw-rw---- 1 root disk 252, 1 Oct 12 12:06 360080e500023b6f0000012954f202c1f
          brw-rw---- 1 root disk 252, 2 Oct 12 12:06 360080e500023b6f0000012954f202c1fp1
          brw-rw---- 1 root disk 252, 0 Oct 12 12:06 360080e500023b6f0000012974f202c43
          brw-rw---- 1 root disk 252, 3 Oct 12 12:06 360080e500023b6f0000012974f202c43p1

          You have partitions (p1) on each disk, so they will be suppressed by Oracle VM. The disks must be unpartitioned to be usable by Oracle VM.
          • 2. Re: Oracle Server 3.1.1 don't see local HDDs. How to diagnose?
            florin cercelaru
            Hi,

            my problem is with this disk
            104       16  430025624 cciss/c0d1
            • 3. Re: Oracle Server 3.1.1 don't see local HDDs. How to diagnose?
              Avi Miller-Oracle
              Florin Cercelaru wrote:
              my problem is with this disk
              It's probably being blacklisted: http://docs.oracle.com/cd/E27300_01/E27307/html/vmrns-bugs.html#vmrns-hp-smart-array-error
              • 4. Re: Oracle Server 3.1.1 don't see local HDDs. How to diagnose?
                florin cercelaru
                thanks .. now .. try keep you updated ..
                • 5. Re: Oracle Server 3.1.1 don't see local HDDs. How to diagnose?
                  florin cercelaru
                  HI,

                  I changed the multipath conf file..but.. not work;
                  # List of device names to discard as not multipath candidates
                  #
                  ## IMPORTANT for OVS do not remove the black listed devices.
                  blacklist {
                          devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st|nbd)[0-9]*"
                          devnode "^hd[a-z][0-9]*"
                          devnode "^etherd"
                          devnode "^cciss!c[0-9]d[0-9]*"  << I add this line
                          %include "/etc/blacklisted.wwids"
                  }
                  Edited by: Florin Cercelaru on Oct 12, 2012 10:09 AM

                  Edited by: Florin Cercelaru on Oct 12, 2012 10:09 AM
                  • 6. Re: Oracle Server 3.1.1 don't see local HDDs. How to diagnose?
                    Avi Miller-Oracle
                    Florin Cercelaru wrote:
                    devnode "^cciss!c[0-9]d[0-9]*" << I add this line
                    No, that will definitely black list the disk. :) You need to make sure it's not being blacklisted.
                    • 7. Re: Oracle Server 3.1.1 don't see local HDDs. How to diagnose?
                      florin cercelaru
                      :)..ok I remove the line..but not work...restart the multipathd service..but nothing..:(
                      • 8. Re: Oracle Server 3.1.1 don't see local HDDs. How to diagnose?
                        Avi Miller-Oracle
                        Florin Cercelaru wrote:
                        :)..ok I remove the line..but not work...restart the multipathd service..but nothing..:(
                        Check the output from multipath -v3 -- it may be getting blacklisted elsewhere, like being listed in blacklisted.wwids or something.
                        • 9. Re: Oracle Server 3.1.1 don't see local HDDs. How to diagnose?
                          florin cercelaru
                          ..this is the output..
                          [root@HPML350-1 ~]# multipath -v3
                          Oct 12 13:36:29 | ram0: device node name blacklisted
                          Oct 12 13:36:29 | ram1: device node name blacklisted
                          Oct 12 13:36:29 | ram2: device node name blacklisted
                          Oct 12 13:36:29 | ram3: device node name blacklisted
                          Oct 12 13:36:29 | ram4: device node name blacklisted
                          Oct 12 13:36:29 | ram5: device node name blacklisted
                          Oct 12 13:36:29 | ram6: device node name blacklisted
                          Oct 12 13:36:29 | ram7: device node name blacklisted
                          Oct 12 13:36:29 | ram8: device node name blacklisted
                          Oct 12 13:36:29 | ram9: device node name blacklisted
                          Oct 12 13:36:29 | ram10: device node name blacklisted
                          Oct 12 13:36:29 | ram11: device node name blacklisted
                          Oct 12 13:36:29 | ram12: device node name blacklisted
                          Oct 12 13:36:29 | ram13: device node name blacklisted
                          Oct 12 13:36:29 | ram14: device node name blacklisted
                          Oct 12 13:36:29 | ram15: device node name blacklisted
                          Oct 12 13:36:29 | loop0: device node name blacklisted
                          Oct 12 13:36:29 | loop1: device node name blacklisted
                          Oct 12 13:36:29 | loop2: device node name blacklisted
                          Oct 12 13:36:29 | loop3: device node name blacklisted
                          Oct 12 13:36:29 | loop4: device node name blacklisted
                          Oct 12 13:36:29 | loop5: device node name blacklisted
                          Oct 12 13:36:29 | loop6: device node name blacklisted
                          Oct 12 13:36:29 | loop7: device node name blacklisted
                          Oct 12 13:36:29 | cciss!c0d0: not found in pathvec
                          Oct 12 13:36:29 | cciss!c0d0: mask = 0x1f
                          Oct 12 13:36:29 | cciss!c0d0: dev_t = 104:0
                          Oct 12 13:36:29 | cciss!c0d0: size = 286677120
                          Oct 12 13:36:29 | cciss!c0d0: subsystem = cciss
                          Oct 12 13:36:29 | cciss!c0d0: h:b:t:l = 0:0:0:0
                          Oct 12 13:36:29 | cciss!c0d0: vendor = HP
                          Oct 12 13:36:29 | cciss!c0d0: product = LOGICAL VOLUME
                          Oct 12 13:36:29 | cciss!c0d0: revision = 1.86
                          Oct 12 13:36:29 | cciss!c0d0: get_state
                          Oct 12 13:36:29 | loading /lib64/multipath/libchecktur.so checker
                          Oct 12 13:36:29 | cciss!c0d0: path checker = tur (controller setting)
                          Oct 12 13:36:29 | cciss!c0d0: checker timeout = 300000 ms (internal default)
                          Oct 12 13:36:29 | cciss!c0d0: path offline
                          Oct 12 13:36:29 | cciss!c0d0: getuid = /sbin/scsi_id -g -u -s /block/%n (controller setting)
                          Oct 12 13:36:29 | /sbin/scsi_id exitted with 1
                          Oct 12 13:36:29 | error calling out /sbin/scsi_id -g -u -s /block/cciss/c0d0
                          Oct 12 13:36:29 | /sbin/scsi_id exitted with 1
                          Oct 12 13:36:29 | error calling out /sbin/scsi_id -g -u -s /block/cciss/c0d0
                          Oct 12 13:36:29 | cciss!c0d0: prio = const (controller setting)
                          Oct 12 13:36:29 | cciss!c0d0: const prio = 1
                          Oct 12 13:36:29 | cciss!c0d1: not found in pathvec
                          Oct 12 13:36:29 | cciss!c0d1: mask = 0x1f
                          Oct 12 13:36:29 | cciss!c0d1: dev_t = 104:16
                          Oct 12 13:36:29 | cciss!c0d1: size = 860051248
                          Oct 12 13:36:29 | cciss!c0d1: subsystem = cciss
                          Oct 12 13:36:29 | cciss!c0d1: h:b:t:l = 0:0:1:0
                          Oct 12 13:36:29 | cciss!c0d1: vendor = HP
                          Oct 12 13:36:29 | cciss!c0d1: product = LOGICAL VOLUME
                          Oct 12 13:36:29 | cciss!c0d1: revision = 1.86
                          Oct 12 13:36:29 | cciss!c0d1: get_state
                          Oct 12 13:36:29 | cciss!c0d1: path checker = tur (controller setting)
                          Oct 12 13:36:29 | cciss!c0d1: checker timeout = 300000 ms (internal default)
                          Oct 12 13:36:29 | cciss!c0d1: path offline
                          Oct 12 13:36:29 | cciss!c0d1: getuid = /sbin/scsi_id -g -u -s /block/%n (controller setting)
                          Oct 12 13:36:29 | /sbin/scsi_id exitted with 1
                          Oct 12 13:36:29 | error calling out /sbin/scsi_id -g -u -s /block/cciss/c0d1
                          Oct 12 13:36:29 | /sbin/scsi_id exitted with 1
                          Oct 12 13:36:29 | error calling out /sbin/scsi_id -g -u -s /block/cciss/c0d1
                          Oct 12 13:36:29 | cciss!c0d1: prio = const (controller setting)
                          Oct 12 13:36:29 | cciss!c0d1: const prio = 1
                          Oct 12 13:36:29 | sr0: device node name blacklisted
                          Oct 12 13:36:29 | sda: not found in pathvec
                          Oct 12 13:36:29 | sda: mask = 0x1f
                          Oct 12 13:36:29 | sda: dev_t = 8:0
                          Oct 12 13:36:29 | sda: size = 31457280
                          Oct 12 13:36:29 | sda: subsystem = scsi
                          Oct 12 13:36:29 | sda: vendor = IBM
                          Oct 12 13:36:29 | sda: product = 1746      FAStT
                          Oct 12 13:36:29 | sda: rev = 1070
                          Oct 12 13:36:29 | sda: h:b:t:l = 0:0:0:0
                          Oct 12 13:36:29 | sda: serial = SV11618036
                          Oct 12 13:36:29 | sda: get_state
                          Oct 12 13:36:29 | loading /lib64/multipath/libcheckrdac.so checker
                          Oct 12 13:36:29 | sda: path checker = rdac (controller setting)
                          Oct 12 13:36:29 | sda: checker timeout = 60000 ms (sysfs setting)
                          Oct 12 13:36:29 | sda: state = running
                          Oct 12 13:36:29 | sda: state = 5
                          Oct 12 13:36:29 | sda: getuid = /sbin/scsi_id -g -u -s /block/%n (controller setting)
                          Oct 12 13:36:29 | sda: uid = 360080e500023b6f0000012954f202c1f (callout)
                          Oct 12 13:36:29 | loading /lib64/multipath/libpriordac.so prioritizer
                          Oct 12 13:36:29 | sda: prio = rdac (controller setting)
                          Oct 12 13:36:29 | sda: rdac prio = 1
                          Oct 12 13:36:29 | sdb: not found in pathvec
                          Oct 12 13:36:29 | sdb: mask = 0x1f
                          Oct 12 13:36:29 | sdb: dev_t = 8:16
                          Oct 12 13:36:29 | sdb: size = 1048576000
                          Oct 12 13:36:29 | sdb: subsystem = scsi
                          Oct 12 13:36:29 | sdb: vendor = IBM
                          Oct 12 13:36:29 | sdb: product = 1746      FAStT
                          Oct 12 13:36:29 | sdb: rev = 1070
                          Oct 12 13:36:29 | sdb: h:b:t:l = 0:0:0:1
                          Oct 12 13:36:29 | sdb: serial = SV11618036
                          Oct 12 13:36:29 | sdb: get_state
                          Oct 12 13:36:29 | sdb: path checker = rdac (controller setting)
                          Oct 12 13:36:29 | sdb: checker timeout = 60000 ms (sysfs setting)
                          Oct 12 13:36:29 | sdb: state = running
                          Oct 12 13:36:29 | sdb: state = 5
                          Oct 12 13:36:29 | sdb: getuid = /sbin/scsi_id -g -u -s /block/%n (controller setting)
                          Oct 12 13:36:29 | sdb: uid = 360080e500023b6f0000012974f202c43 (callout)
                          Oct 12 13:36:29 | sdb: prio = rdac (controller setting)
                          Oct 12 13:36:29 | sdb: rdac prio = 1
                          Oct 12 13:36:29 | sdc: not found in pathvec
                          Oct 12 13:36:29 | sdc: mask = 0x1f
                          Oct 12 13:36:29 | sdc: dev_t = 8:32
                          Oct 12 13:36:29 | sdc: size = 31457280
                          Oct 12 13:36:29 | sdc: subsystem = scsi
                          Oct 12 13:36:29 | sdc: vendor = IBM
                          Oct 12 13:36:29 | sdc: product = 1746      FAStT
                          Oct 12 13:36:29 | sdc: rev = 1070
                          Oct 12 13:36:29 | sdc: h:b:t:l = 0:0:1:0
                          Oct 12 13:36:29 | sdc: serial = SV11807225
                          Oct 12 13:36:29 | sdc: get_state
                          Oct 12 13:36:29 | sdc: path checker = rdac (controller setting)
                          Oct 12 13:36:29 | sdc: checker timeout = 60000 ms (sysfs setting)
                          Oct 12 13:36:29 | sdc: state = running
                          Oct 12 13:36:29 | sdc: state = 3
                          Oct 12 13:36:29 | sdc: getuid = /sbin/scsi_id -g -u -s /block/%n (controller setting)
                          Oct 12 13:36:29 | sdc: uid = 360080e500023b6f0000012954f202c1f (callout)
                          Oct 12 13:36:29 | sdc: prio = rdac (controller setting)
                          Oct 12 13:36:29 | sdc: rdac prio = 6
                          Oct 12 13:36:29 | sdd: not found in pathvec
                          Oct 12 13:36:29 | sdd: mask = 0x1f
                          Oct 12 13:36:29 | sdd: dev_t = 8:48
                          Oct 12 13:36:29 | sdd: size = 1048576000
                          Oct 12 13:36:29 | sdd: subsystem = scsi
                          Oct 12 13:36:29 | sdd: vendor = IBM
                          Oct 12 13:36:29 | sdd: product = 1746      FAStT
                          Oct 12 13:36:29 | sdd: rev = 1070
                          Oct 12 13:36:29 | sdd: h:b:t:l = 0:0:1:1
                          Oct 12 13:36:29 | sdd: serial = SV11807225
                          Oct 12 13:36:29 | sdd: get_state
                          Oct 12 13:36:29 | sdd: path checker = rdac (controller setting)
                          Oct 12 13:36:29 | sdd: checker timeout = 60000 ms (sysfs setting)
                          Oct 12 13:36:29 | sdd: state = running
                          Oct 12 13:36:29 | sdd: state = 3
                          Oct 12 13:36:29 | sdd: getuid = /sbin/scsi_id -g -u -s /block/%n (controller setting)
                          Oct 12 13:36:29 | sdd: uid = 360080e500023b6f0000012974f202c43 (callout)
                          Oct 12 13:36:29 | sdd: prio = rdac (controller setting)
                          Oct 12 13:36:29 | sdd: rdac prio = 6
                          Oct 12 13:36:29 | md0: device node name blacklisted
                          Oct 12 13:36:29 | dm-0: device node name blacklisted
                          Oct 12 13:36:29 | dm-1: device node name blacklisted
                          Oct 12 13:36:29 | dm-2: device node name blacklisted
                          Oct 12 13:36:29 | dm-3: device node name blacklisted
                          Oct 12 13:36:29 | dm-4: device node name blacklisted
                          ===== paths list =====
                          uuid                              hcil    dev        dev_t  pri dm_st chk_st v
                                                            0:0:0:0 cciss!c0d0 104:0  1   undef faulty H
                                                            0:0:1:0 cciss!c0d1 104:16 1   undef faulty H
                          360080e500023b6f0000012954f202c1f 0:0:0:0 sda        8:0    1   undef ghost  I
                          360080e500023b6f0000012974f202c43 0:0:0:1 sdb        8:16   1   undef ghost  I
                          360080e500023b6f0000012954f202c1f 0:0:1:0 sdc        8:32   6   undef ready  I
                          360080e500023b6f0000012974f202c43 0:0:1:1 sdd        8:48   6   undef ready  I
                          Oct 12 13:36:29 | params = 3 queue_if_no_path pg_init_retries 50 1 rdac 2 1 round-robin 0 1 1 8:48 1000 round-robin 0 1 1 8:16 1000
                          Oct 12 13:36:29 | status = 2 0 0 0 2 1 E 0 1 0 8:48 A 0 E 0 1 0 8:16 A 0
                          Oct 12 13:36:29 | params = 3 queue_if_no_path pg_init_retries 50 1 rdac 2 1 round-robin 0 1 1 8:32 1000 round-robin 0 1 1 8:0 1000
                          Oct 12 13:36:29 | status = 2 0 0 0 2 1 E 0 1 0 8:32 A 0 E 0 1 0 8:0 A 0
                          [root@HPML350-1 ~]#
                          • 10. Re: Oracle Server 3.1.1 don't see local HDDs. How to diagnose?
                            Avi Miller-Oracle
                            Florin Cercelaru wrote:
                            ..this is the output..
                            Oct 12 13:36:29 | cciss!c0d0: getuid = /sbin/scsi_id -g -u -s /block/%n (controller setting)
                            Oct 12 13:36:29 | /sbin/scsi_id exitted with 1
                            Oct 12 13:36:29 | error calling out /sbin/scsi_id -g -u -s /block/cciss/c0d0
                            Oct 12 13:36:29 | /sbin/scsi_id exitted with 1
                            Oct 12 13:36:29 | error calling out /sbin/scsi_id -g -u -s /block/cciss/c0d0
                            The call to get the uuid of the disk is failing. I don't have any cciss disks handy, so I have no idea why or what the correct configuration is supposed to be. I recommend opening an SR with Oracle Support.
                            • 11. Re: Oracle Server 3.1.1 don't see local HDDs. How to diagnose?
                              florin cercelaru
                              Hi Avi,

                              thanks for help .. good to know for the future ..
                              I studied the problem and it seems that version 3.1.1 does not work ..only version, 3.0.3 ...
                              for the moment I exported local disks as nfs..

                              a good day
                              • 12. Re: Oracle Server 3.1.1 don't see local HDDs. How to diagnose?
                                Avi Miller-Oracle
                                Florin Cercelaru wrote:
                                for the moment I exported local disks as nfs..
                                Please don't do this -- Dom0 has very, very limited resources and NFS writes are just going to consume lots and lots of memory, leading to potential disk corruption or node fencing. If this is for personal testing with very low usage, you can probably get away with it, but definitely not for anything resembling production workloads.

                                If you have Oracle Support, please log an SR.
                                • 13. Re: Oracle Server 3.1.1 don't see local HDDs. How to diagnose?
                                  Moderator Action:

                                  @ user Florin,
                                  Your post and its replies have been split away from that other thread.
                                  You hijacked that thread and then asked your question.
                                  That's questionable forum etiquette.

                                  How would you feel if you had a forum thread and someone ripped ownership away from you?

                                  The discussion of your issue has value to the general knowledgebase. That's why it wasn't just summarily deleted.
                                  • 14. Re: Oracle Server 3.1.1 don't see local HDDs. How to diagnose?
                                    florin cercelaru
                                    Hi Avi,

                                    thanks for the guidance..will give up the idea..local disc export as nfs.
                                    1 2 Previous Next