5 Replies Latest reply on Jul 18, 2012 11:56 AM by Nik

    2048 LUNs on a single host

    950211
      Hi all,

      We are currently in the middle of attaching a large scale storage to a single Solaris 11 x86 server.
      We are experiencing a lot of issues when trying to write to the drives.

      Any know best practices to configure the OS to support this amount (2048) of LUNs?
      This does not work out of the box.

      Thank you for your assistance.
        • 1. Re: 2048 LUNs on a single host
          Nik
          Can you more describe current connections ( FC or SAS ; zonning etc) and OS settings ( load balancing ; how many path to every LUN; total nu,ber of path... )

          What kind of problem you have.


          Regards.
          • 2. Re: 2048 LUNs on a single host
            950211
            sure.

            This is an FC connection to a Symmetrix storage using FC disks. There are no Solaris zones on the os (well, global oviously). We are running PowerPath 5.5 and we have 2 paths per LUN (we had more but removed them in order to check if this is the issue, we usually have 4 paths per LUN).

            The problem we are having is that we get the following error message when running cfgadm -c configure c5(example):
            cfgadm: Library error: report LUNs failed: 50000972c00865d9
            failed to configure ANY device on FCA port

            moreover, when running the powermt display I get:
            root@a65t2457:~# powermt display
            Symmetrix logical device count=2000
            ==============================================================================
            ----- Host Bus Adapters --------- ------ I/O Paths ----- ------ Stats ------
            ### HW Path Summary Total Dead IO/Sec Q-IOs Errors
            ==============================================================================
            3072 pci@0,0/pci8086,d138@3/pci1077,15d@0/fp@0,0 optimal 2000 0 - 0 0
            3074 pci@0,0/pci8086,d138@3/pci1077,15d@0,1/fp@0,0 optimal 1582 0 - 0 0

            Please note that there is a difference netween the amount of paths from each adapter where they should be identical. We actually see it per LUN.

            After all that when we try actually writing to the devices themselves we get an I/O error (both powerpath devices and regular devices).

            thank you again.
            • 3. Re: 2048 LUNs on a single host
              Nik
              Hi.

              Try check
              luxadm -e port - for see path to controllers
              luxadm -e dump_map <for on of controllers>


              cfgadm -al -o show_FCP_dev



              Regards.
              • 4. Re: 2048 LUNs on a single host
                950211
                Here are the results:

                #luxadm -e port
                /devices/pci@0,0/pci8086,d138@3/pci1077,15d@0/fp@0,0:devctl CONNECTED
                /devices/pci@0,0/pci8086,d138@3/pci1077,15d@0,1/fp@0,0:devctl CONNECTED

                # luxadm -e dump_map /devices/pci@0,0/pci8086,d138@3/pci1077,15d@0,1/fp@0,0:devctl
                Pos Port_ID Hard_Addr Port WWN Node WWN Type
                0 10a00 0 50000972c0086518 50000972c0086400 0x0 (Disk device)
                1 11300 0 50000972c00865d9 50000972c0086400 0x0 (Disk device)
                2 11900 0 21000024ff34c925 20000024ff34c925 0x1f (Unknown Type,Host Bus Adapter)

                # luxadm -e dump_map /devices/pci@0,0/pci8086,d138@3/pci1077,15d@0/fp@0,0:devctl
                Pos Port_ID Hard_Addr Port WWN Node WWN Type
                0 10a00 0 50000972c0086518 50000972c0086400 0x0 (Disk device)
                1 11300 0 50000972c00865d9 50000972c0086400 0x0 (Disk device)
                2 11a00 0 21000024ff34c924 20000024ff34c924 0x1f (Unknown Type,Host Bus Adapter)

                # cfgadm -al -o show_FCP_dev
                Ap_Id Type Receptacle Occupant Condition
                c5 fc-fabric connected configured unknown
                c5::50000972c0086518 disk connected configured unknown
                c5::50000972c00865d9 disk connected configured unknown
                c6 fc-fabric connected configured unknown
                c6::50000972c0086518 disk connected configured unknown
                c6::50000972c00865d9 disk connected configured unknown
                • 5. Re: 2048 LUNs on a single host
                  Nik
                  Hi.

                  According output of
                  ======
                  # cfgadm -al -o show_FCP_dev
                  Ap_Id Type Receptacle Occupant Condition
                  c5 fc-fabric connected configured unknown
                  c5::50000972c0086518 disk connected configured unknown
                  c5::50000972c00865d9 disk connected configured unknown
                  c6 fc-fabric connected configured unknown
                  c6::50000972c0086518 disk connected configured unknown
                  c6::50000972c00865d9 disk connected configured unknown
                  =================

                  It't look like host can't see LUNS on array.

                  In case you don't reboot server after creating LUNS, you can try reinitialize FC port.

                  luxadm -e forcelip /devices/pci@0,0/pci8086,d138@3/pci1077,15d@0/fp@0,0:devctl
                  luxadm -e forcelip /devices/pci@0,0/pci8086,d138@3/pci1077,15d@0,1/fp@0,0:devctl

                  Do you configure any LUN masking, port permissons on array ?

                  For test reason,may be first live just some LUN's.


                  Regards.