Infrastructure Software

Well, according to this http://www.sun.com/software/cluster/osp/ibm_interop.xml
its supported by mpxio. (the multipath options listed include the standard san driver in solaris 10 ie mpxio).

That document is actually about sun cluster support. But if its supported by sun cluster, then standard filesystems without sun cluster arent going to be a problem :-).

But you should probably check with some IBM documentation.

Thanks robert.cohen.

It looks like IBM disk array DS4300 should support MPxIO:
http://www-07.ibm.com/storage/includes/content/product/system_x/pdf/interop-matrix.pdf

Unfortunatelly using the Solaris documentation and other knowledge sources I wasn't able to configure MPxIO.

I have changed the following:
1. /kernel/drv/fp.conf
mpxio-disable="no";
2. /kernel/drv/scsi_vhci.conf
load-balance="round-robin";
device-type-scsi-options-list="IBM 1722","symetric-option";
symmetric-option = 0x1000000;
3. stmsboot -e

After reboot there is only one device listed (this is internal disk)

# stmsboot -L
non-STMS device name                    STMS device name
------------------------------------------------------------------
/dev/rdsk/c1t0d0        /dev/rdsk/c4t500000E01276E730d0

Maybe somebody had such issue?

Thanks,
Daniel

I think your misunderstanding the issue and the relevance of mpxio.
Mpxio is a multipath driver. So if you have multiple paths to disk, mpxio handles the loadbalancing or the failover in the case of non symmetric arrays.
So for example, if you only have one path to disk, you can use a SAN filesystem without using mpxio at all.


Your issue isnt with handling multiple paths properly. Your problem is that arent seeing any paths to disk at all.
Thats not an mpxio issue, thats a SAN connectivity issue.

See what cfgadm -al tells you.
And see if you can get it working with a single path first before you start bringing in mpxio.

Dynamic Multipathing on Solaris SPARC 10 with disk array DS4300 works fine using Veritas DMP.
I can see one disk using vxdis list command.
When one path is disabled I can still access the disk because second path is connected.

I would like to have the same functionality using MPxIO without Veritas DMP.

Without DMP when I connect to disk using one path and this path will fail I will lose access to disk.
With multipathing I can have still access through second path.

Output of cfgadm:

# cfgadm -al
Ap_Id                          Type         Receptacle   Occupant     Condition
c0                             scsi-bus     connected    configured   unknown
c0::dsk/c0t0d0                 CD-ROM       connected    configured   unknown
c1                             scsi-bus     connected    unconfigured unknown
c2                             fc-fabric    connected    configured   unknown
c2::200800a0b8163a93           disk         connected    configured   unknown
c2::200900a0b8163a93           disk         connected    configured   unknown
c2::210000e08b871314           unknown      connected    unconfigured unknown
c2::210000e08b8f6aee           unknown      connected    unconfigured unknown
c2::210000e08b8fa3ef           unknown      connected    unconfigured unknown
c2::210000e08b8fbaed           unknown      connected    unconfigured unknown
c3                             fc           connected    unconfigured unknown
...

So how to check if MPxIO works correctly and is enabled?

Thanks,
Daniel

First, the early versions of scsi_vhci aren't going to be able to be configured to handle an assymetric array like the DS4300. If you do get it configured so the LUNs are visible, the failover model will be completely wrong, although if you have AVT disabled on the DS4300 it will appear to work OK, but you won't have any failover capability that way. So with those versions of MPxIO Solaris pretty much has to handle the array as installed.

Can you see the LUNs at all with an unmodified scsi_vhci.conf? You have to get that working first.

Once you can see the LUNs, what does this command return (sh/bash):

for port in `fcinfo hba-port | grep 'HBA Port WWN' | awk '{ print $4 }'`;
    do fcinfo remote-port -ls -p $port;
done

That should show you the LUNs from the array, whether MPXIO handles them properly or no. You should see something like this (IBM DS4100 in this case):

Remote Port WWN: 200700a0b8166aee
        Active FC4 Types: 
        SCSI Target: yes
        Port Symbolic Name: 
        Node WWN: 200600a0b8166aec
        Link Error Statistics:
                Link Failure Count: 0
                Loss of Sync Count: 0
                Loss of Signal Count: 1
                Primitive Seq Protocol Error Count: 0
                Invalid Tx Word Count: 0
                Invalid CRC Count: 0
        LUN: 0
          Vendor: IBM     
          Product: 1742-900        
          OS Device Name: /dev/rdsk/c0t600A0B8000166B0D000000CB4A7D9796d0s2
        LUN: 1
          Vendor: IBM     
          Product: 1742-900        
          OS Device Name: /dev/rdsk/c0t600A0B8000166AEC000000A14A7D94A7d0s2
        LUN: 2
          Vendor: IBM     
          Product: 1742-900        
          OS Device Name: /dev/rdsk/c0t600A0B8000166B0D000000CE4A7D97F8d0s2
        LUN: 3
          Vendor: IBM     
          Product: 1742-900        
          OS Device Name: /dev/rdsk/c0t600A0B8000166AEC000000A34A7D9509d0s2
        LUN: 31
          Vendor: IBM     
          Product: Universal Xport 
          OS Device Name: /dev/rdsk/c5t3d31s2

Now, what does a SCSI inquiry show?

-bash-3.2# format -e
Searching for disks...done


AVAILABLE DISK SELECTIONS:
       0. c0t600A0B8000166AEC000000A14A7D94A7d0 <IBM-1742-900-0520-1.36TB>
          /scsi_vhci/disk@g600a0b8000166aec000000a14a7d94a7
       1. c0t600A0B8000166AEC000000A34A7D9509d0 <IBM-1742-900-0520-1.36TB>
          /scsi_vhci/disk@g600a0b8000166aec000000a34a7d9509
       2. c0t600A0B8000166B0D000000CB4A7D9796d0 <IBM-1742-900-0520-1.36TB>
          /scsi_vhci/disk@g600a0b8000166b0d000000cb4a7d9796
       3. c0t600A0B8000166B0D000000CE4A7D97F8d0 <IBM-1742-900-0520-1.36TB>
          /scsi_vhci/disk@g600a0b8000166b0d000000ce4a7d97f8
       4. c1t0d31 <DEFAULT cyl 18 alt 2 hd 64 sec 32>
          /pci@4,0/pci1077,100@1/fp@0,0/disk@w200600a0b8166aed,1f
       5. c1t1d31 <DEFAULT cyl 18 alt 2 hd 64 sec 32>
          /pci@4,0/pci1077,100@1/fp@0,0/disk@w200700a0b8166aed,1f
       6. c2t0d0 <DEFAULT cyl 4406 alt 2 hd 255 sec 63>
          /pci@1,0/pci1014,2f2@2/sd@0,0
       7. c2t1d0 <IBM-ServeRAID 8i-V1.0-465.62GB>
          /pci@1,0/pci1014,2f2@2/sd@1,0
       8. c2t3d0 <IBM-ServeRAID 8i-V1.0-68.26GB>
          /pci@1,0/pci1014,2f2@2/sd@3,0
       9. c2t4d0 <IBM-ServeRAID 8i-V1.0-68.26GB>
          /pci@1,0/pci1014,2f2@2/sd@4,0
      10. c5t2d31 <DEFAULT cyl 18 alt 2 hd 64 sec 32>
          /pci@2,0/pci1077,100@1/fp@0,0/disk@w200600a0b8166aee,1f
      11. c5t3d31 <DEFAULT cyl 18 alt 2 hd 64 sec 32>
          /pci@2,0/pci1077,100@1/fp@0,0/disk@w200700a0b8166aee,1f
Specify disk (enter its number): 0
selecting c0t600A0B8000166AEC000000A14A7D94A7d0
[disk formatted]
Note: capacity in disk label is smaller than the real disk capacity.
Select <partition> <expand> to adjust the label capacity. 


FORMAT MENU:
        disk       - select a disk
        type       - select (define) a disk type
        partition  - select (define) a partition table
        current    - describe the current disk
        format     - format and analyze the disk
        fdisk      - run the fdisk program
        repair     - repair a defective sector
        label      - write label to the disk
        analyze    - surface analysis
        defect     - defect list management
        backup     - search for backup labels
        verify     - read and display labels
        inquiry    - show vendor, product and revision
        scsi       - independent SCSI mode selects
        cache      - enable, disable or query SCSI disk cache
        volname    - set 8-character volume name
        !<cmd>     - execute <cmd>, then return
        quit
format> i
Vendor:   IBM     
Product:  1742-900        
Revision: 0520
format> 

Does that product string appear in your LSI-specific scsi_vhci module?

-bash-3.2# strings -a /kernel/misc/scsi_vhci/scsi_vhci_f_asym_lsi | grep 1742 
IBM     1742-900        
IBM     1742            
-bash-3.2# strings -a /kernel/misc/scsi_vhci/scsi_vhci_f_asym_lsi | grep 1722 
IBM     1722-600        
-bash-3.2# 

If you don't have that file, you have an earlier version of MPxIO. The examples I posted are from the SNV 96 version of OpenSolaris. I'm not sure when the updated MPxIO made it's way into Solaris 10, or if it ever has, although an install of Solaris 10u6 that I have does show this:

-bash-3.00#  strings -a /kernel/drv/scsi_vhci | grep 1722 
1722-600        
-bash-3.00# 

One thing that's important: spaces. The strings are space-padded. IIRC the vendor string is 8 characters, and the product string is 16 characters. You MUST have the correct space padding on your scsi_vhci.conf file.

Thank you all for help in configuring MPxIO. I have sucessfully configures Solaris multipathing on Solaris SPARC U8.
I think the problem was related to space padding.

I've have found interesting information from:
# man scsi_vhci.7d

...
In previous releases,  Solaris  I/O  multipathing  was  also
     known as MPxIO and Sun StorEdge Traffic  Manager (STMS).
...

I can see all luns under format command.
All disks size is listed via command iostat -En

fcinfo command gives the following output:

# for port in `fcinfo hba-port | grep 'HBA Port WWN' | awk '{ print $4 }'`
do

fcinfo remote-port -ls -p $port

done
Remote Port WWN: 210000e08b8fbaed
        Active FC4 Types: SCSI,IP
        SCSI Target: no
        Node WWN: 200000e08b8fbaed
        Link Error Statistics:
                Link Failure Count: 1
                Loss of Sync Count: 0
                Loss of Signal Count: 0
                Primitive Seq Protocol Error Count: 0
                Invalid Tx Word Count: 0
                Invalid CRC Count: 0
Remote Port WWN: 200800a0b8163a93
        Active FC4 Types: SCSI
        SCSI Target: yes
        Node WWN: 200800a0b8163a92
        Link Error Statistics:
                Link Failure Count: 12
                Loss of Sync Count: 391
                Loss of Signal Count: 552
                Primitive Seq Protocol Error Count: 0
                Invalid Tx Word Count: 0
                Invalid CRC Count: 0
        LUN: 0
          Vendor: IBM
          Product: 1722-600
          OS Device Name: /dev/rdsk/c2t200800A0B8163A93d0s2
        LUN: 1
          Vendor: IBM
          Product: 1722-600
          OS Device Name: /dev/rdsk/c2t200800A0B8163A93d1s2
        LUN: 2
          Vendor: IBM
          Product: 1722-600
          OS Device Name: /dev/rdsk/c2t200800A0B8163A93d2s2
        LUN: 3
          Vendor: IBM
          Product: 1722-600
          OS Device Name: /dev/rdsk/c2t200800A0B8163A93d3s2
        LUN: 4
          Vendor: IBM
          Product: 1722-600
          OS Device Name: /dev/rdsk/c2t200800A0B8163A93d4s2
...

SCSI inquiry shows:

format> i
Vendor:   IBM
Product:  1722-600
Revision: 0914
format>

Also it looks like DS4300 disk array is supported by MPxIO:

# strings -a /kernel/drv/sparcv9/scsi_vhci |grep 1722-600
1722-600

My configuration files:

# cat /kernel/drv/fp.conf

#ident  "@(#)fp.conf    1.4     06/06/22 SMI"
#

name="fp" class="fibre-channel" port=0;
name="fp" class="fibre-channel" port=1;

#
...
#
scsi-binding-set="fcp";

#
# List of ULP modules for loading during port driver attach time
#
load-ulp-list="1","fcp";

#
# Force attach driver to support hotplug activity (do not remove the property)
#
ddi-forceattach=1;

#
...
# NOTE: If you just want to enable or disable MPxIO on all fp ports, it is
# better to use stmsboot(1M) as it also updates /etc/vfstab.
#
mpxio-disable="no";
#
...

# cat /kernel/drv/scsi_vhci.conf

#pragma ident   "@(#)scsi_vhci.conf     1.9     04/08/26 SMI"
#
name="scsi_vhci" class="root";
#
# Load balancing global configuration: setting load-balance="none" will cause
# all I/O to a given device (which supports multipath I/O) to occur via one
# path.  Setting load-balance="round-robin" will cause each path to the device
# to be used in turn.
#
load-balance="round-robin";
#
# Automatic failback configuration
# possible values are auto-failback="enable" or auto-failback="disable"
auto-failback="disable";
#
# For enabling MPxIO support for 3rd party symmetric device need an
# entry similar to following in this file. Just replace the "SUN     SENA"
# part with the Vendor ID/Product ID for the device, exactly as reported by
# Inquiry cmd.
#
 device-type-scsi-options-list =
 "IBM     1722-600", "symmetric-option";
 symmetric-option = 0x1000000;
# "SUN     SENA", "symmetric-option";
#
# symmetric-option = 0x1000000;

Now stmsboot command shows all disks:

# stmsboot -L
non-STMS device name                    STMS device name
------------------------------------------------------------------
/dev/rdsk/c1t0d0        /dev/rdsk/c4t500000E01276E730d0
/dev/rdsk/c2t200900A0B8163A93d22        /dev/rdsk/c4t600A0B8000163A9200006B324A12DBFCd0
/dev/rdsk/c2t200900A0B8163A93d21        /dev/rdsk/c4t600A0B8000163A9200006B314A12DBE6d0
/dev/rdsk/c2t200900A0B8163A93d20        /dev/rdsk/c4t600A0B8000163A9200006B304A12DBD0d0
/dev/rdsk/c2t200900A0B8163A93d19        /dev/rdsk/c4t600A0B8000163A9200006B2F4A12DBBAd0
/dev/rdsk/c2t200900A0B8163A93d18        /dev/rdsk/c4t600A0B8000163A9200006B2D4A12DB62d0
/dev/rdsk/c2t200900A0B8163A93d17        /dev/rdsk/c4t600A0B8000163A9200006B2C4A12DB4Cd0
/dev/rdsk/c2t200900A0B8163A93d16        /dev/rdsk/c4t600A0B8000163A9200006B2B4A12DB34d0
/dev/rdsk/c2t200900A0B8163A93d15        /dev/rdsk/c4t600A0B8000163A9200006B2A4A12DB1Ed0
/dev/rdsk/c2t200900A0B8163A93d14        /dev/rdsk/c4t600A0B8000163A9200006B294A12DB08d0
/dev/rdsk/c2t200900A0B8163A93d13        /dev/rdsk/c4t600A0B8000163A9200006B284A12DAF0d0
/dev/rdsk/c2t200900A0B8163A93d12        /dev/rdsk/c4t600A0B8000163A9200006B244A12DAAEd0
/dev/rdsk/c2t200900A0B8163A93d11        /dev/rdsk/c4t600A0B8000163A9200006B234A12DA98d0
/dev/rdsk/c2t200900A0B8163A93d10        /dev/rdsk/c4t600A0B8000163A9200006B224A12DA82d0
/dev/rdsk/c2t200900A0B8163A93d9 /dev/rdsk/c4t600A0B8000163A9200006B214A12DA6Ed0
/dev/rdsk/c2t200900A0B8163A93d8 /dev/rdsk/c4t600A0B8000163A9200006B204A12DA56d0
/dev/rdsk/c2t200900A0B8163A93d7 /dev/rdsk/c4t600A0B8000163A9200006B1F4A12DA40d0
/dev/rdsk/c2t200900A0B8163A93d6 /dev/rdsk/c4t600A0B8000163A9200006B1E4A12DA2Ad0
/dev/rdsk/c2t200900A0B8163A93d5 /dev/rdsk/c4t600A0B8000163A9200006B1D4A12DA12d0
/dev/rdsk/c2t200900A0B8163A93d4 /dev/rdsk/c4t600A0B8000163A9200006B1C4A12D9FAd0
/dev/rdsk/c2t200900A0B8163A93d3 /dev/rdsk/c4t600A0B8000163A9200006B1B4A12D9E0d0
/dev/rdsk/c2t200900A0B8163A93d2 /dev/rdsk/c4t600A0B8000163A9200006B1A4A12D9BAd0
/dev/rdsk/c2t200900A0B8163A93d1 /dev/rdsk/c4t600A0B8000163A9200006B194A12D99Ed0
/dev/rdsk/c2t200900A0B8163A93d0 /dev/rdsk/c4t600A0B8000163A9200006B164A12D95Cd0
/dev/rdsk/c2t200800A0B8163A93d22        /dev/rdsk/c4t600A0B8000163A9200006B324A12DBFCd0
/dev/rdsk/c2t200800A0B8163A93d21        /dev/rdsk/c4t600A0B8000163A9200006B314A12DBE6d0
/dev/rdsk/c2t200800A0B8163A93d20        /dev/rdsk/c4t600A0B8000163A9200006B304A12DBD0d0
...

Thanks,
Daniel