Infrastructure Software

Weird. I'm using the same blades with the same HBAs against EMC FC SANs (CLARiiON and SYMMETRIX) and I'm not having any issues. If you have Oracle VM support, I urge you to raise an SR on this so that device-mapper-multipath in Oracle VM can be investigated and perhaps upgraded.

Hello,

I closed this issue thanks to an SR I opened, but moving the root partition to local disks. Explanation:

- to install 5u3 multipath rpms was not enough to resolve the problem.
- support noticed OVM2.1.2 do not support booting from external multipath devices. OVM2.1 is based on a OEL 5 base (in turn, based on RH5 base), and, although it is not explicitly indicated in the documentation, this does not work.
- Technically, in oracle 5 base that uses a multipath root partition, some static information is stored in the initrd init scripts. This behavior changed in OEL 5u2, for example.

Regards,

Daniel Caballero

Hi dcaba,

can you (or anybody else with experience on multipath SAN setup) please detail a bit more the steps you performed to TRY to implement SAN boot + multipath?

I'm trying to do the same right now. My system is as follows:
- Oracle VM Server 2.1.5
- C3000 Bladesystem enclosure
- BL460C Blades
- Qlogic QMH2462 HBA's
- Brocade FC Switches
- HP 2012FC SAN Storage

I have results similar to yours...

SAME AS YOU:
I get the following messages at system boot:
device-mapper: table: 253:26: multipath: error getting device
device-mapper: ioctl: error adding target to table

DIFFERENT FROM YOU:
my system boots correctly.

My problem is that e.g. if I shutdown the primary controller on my SAN disk array, I see that Paths are failing over. The system is still usable.

BUT AFTER FAILOVER, "/" FILE SYSTEM GETS TO BE "READ ONLY FILESYSTEM" (that's exactly the error message that I get if I try, for instance, to touch or edit whatever text file under "/root".

I'd like to know what I've "left behind". Something must definitely be incomplete, as if I type "mount", I get the following: So it looks like JUST "/OVS" is mounted under "/dev/mapper". I would expect also "/", "/boot" etc. to be under "/dev/mapper/something", but they are under "/dev/sdaXXX"...

I'd like to, at least, get to the end of it correctly, before giving up. Here is what I do:

---------------------------------------------------------------------------
cd /etc
vi multipath.conf
### ADD THE FOLLOWING BLACKLIST AFTER"defaults user_friendly_names yes" SECTION
## Blacklist non-SAN devices
devnode_blacklist {
devnode "sd[a]$"
devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
devnode "^hd[a-z]"
devnode "^cciss!c[0-9]d[0-9]*"
}

# add the following under "devices" (UNCOMMENT THE SECTION FIRST!!!)
# section:
# For MSA2012fc/MSA2212fc/MSA2012i
device
{
vendor "HP"
product "MSA2[02]12fc|MSA2012i"
getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
hardware_handler "0"
path_selector "round-robin 0"
path_grouping_policy multibus
failback immediate
rr_weight uniform
no_path_retry 18
rr_min_io 100
path_checker tur
}

# Configuring QLogic HBA parameters
# To configure the QLogic HBA parameters for QLogic 2xxx family of HBAs, complete the following
# steps:
cd /etc
vi modprobe.conf

# ADD THE FOLLOWING LINE
options qla2xxx ql2xmaxqdepth=16 qlport_down_retry=10 ql2xloginretrycount=30

####
# Rebuild the initrd by executing the following commands:
####
# Backup the existing initrd image
mv /boot/initrd-2.6.18-8.1.15.3.1.el5xen.img /boot/initrd-2.6.18-8.1.15.3.1.el5xen.img.old

# then create the new one:
mkinitrd /boot/initrd-2.6.18-8.1.15.3.1.el5xen.img `uname -r`

# CHECK THAT the value for default parameter in /boot/grub/menu.lst file to boot with the
# new initrd image --> OK, SO I LEAVE THE FILE UNCHANGED

### CONFIGURES MULTIPATHD DAEMON
chkconfig --add multipathd
chkconfig multipathd on
/etc/init.d/multipathd start

### REBOOT THE HOST
------------------------------------------------------------------------------------------

What are the steps that I'm missing to see EVERYTHING under /dev/mapper? Or maybe it's OK to see / mounted under /dev/sdaXXX, and the error is somewhere else?

Thank you for eventually pointing me to a "check" procedure, or to a troubleshooting guide, or for any hints and ideas...

Edited by: rlomba on Aug 13, 2009 3:33 AM

Edited by: rlomba on Aug 13, 2009 3:36 AM

UPDATE:

After changing a configuration on the HP 2012FC storage, reinstalling OracleVM Server, and performing again the above listed post-install actions (SPECIFIC FOR MY HARDWARE), I have now everything apparently working....

- It boots from SAN
- If I now type "mount", everything is under /dev/mapper (probably, a mix of issues prevented the previous installation to work)

STILL I HAVE A PROBLEM. After shutting down a Disk Array Controller, to test the path's failover, I get a "journal aborted" message on the system console, followed by a "remounting filesystem read-only".

Therefore, this is not exactly perfect. It's not usable.

Anyone that knows about specific settings to be used with the following hardware combination:
- Qlogic QMH2462 HBA's
- Brocade 4/12 SAN switches
- HP 2012FC SAN storage?

I guess right now, it's all a matter of vendor-specific settings... But the settings I'm already using (listed in the previous message) have been found on HP sources....

Any ideas?

All, sorry for the "noise". It works.

I realized now that:
is not enough for correctly starting up the multipathd daemon. It just started the PID, but not the actual service/daemon (at least according to the output of "service multipathd status").

So I had to simply:
So, hopefully it's going to be useful to somebody, sooner or later... The above listed "VMServer Version+HW configuration+parameters" looks like a working configuration for both SAN boot and multipathing.

I still haven't tested under load, but at least the system now is not complaining after a path failure, it's already a good result :-)

Thank you.