3 Replies Latest reply on May 4, 2013 10:36 PM by 1002848

    cfgadm unable to configure disk shelf/disks

      Hello all,
      I am attempting to add a JBOD shelf of disks to my existing Solaris environment - and I am unable to "initialize" or "configure" the disks so that I can use them in a ZFS volume.
      The server has a quad port HBA, of which only one port goes to a NetApp DS14MK2 shelf with FC-ATX modules. Inside this shelf are SATA drives.

      When I run cfgadm, Solaris can see the disks but it cannot use them as they are set to an Occupant type of "Unconfigured". If I attempt to run the cfgadm -c configure <ap_id>, the command completes without error but the devices (controller and disks) remained "Unconfigured". I've tried "cfgadm -c configure" for both controller and disk ap_ids, with no luck.

      Now, if I use a different disk shelf, the NetApp DS14MK4 (ESH4 modules) with FC disks, I am able to utilize the disks immediately upon booting into Solaris, and I can add them to a ZFS volume without manually initializing/configuring them. In this case, the cfgadm -al command shows the controller and disks as "Connected" and "Configured". This leads me to believe that my issue may reside in the fact that the non-working disk shelf is "FC-ATX" style, whereas the working shelf is FC-AL (I think that statement is complete!?!). However, if I install Windows on this same server, Windows can see and initialize and use the FC-ATX shelf disks just fine, and I can create dynamic or standard volumes in Windows this way.
      So, I'm speculating that the Solaris driver for my HBA card (Qlogic 2464) is unable to negotiate with a FC-ATX shelf... would you say my assumption is correct?

      If you can please advise on how I can get these disks initialized. I would be glad to offer additional information and command outputs in pursuit of an answer!

      Here are some to get our brains clicking:

      root@mNetFile:/volumes# uname -a
      SunOS mNetFile 5.11 NexentaOS_134f i86pc i386 i86pc Solaris

      root@mNetFile:/volumes# fcinfo hba-port
      HBA Port WWN: 2100001b32061720
      Port Mode: Initiator
      Port ID: 1
      OS Device Name: /dev/cfg/c4
      Manufacturer: QLogic Corp.
      Model: QLE2464
      Firmware Version: 05.03.01
      FCode/BIOS Version: BIOS: 3.20; fcode: 3.19; EFI: 2.25;
      Serial Number: GFC0742E01431
      Driver Name: qlc
      Driver Version: 20100408-3.01
      Type: L-port
      State: online
      Supported Speeds: 1Gb 2Gb 4Gb
      Current Speed: 2Gb
      Node WWN: 2000001b32061720
      Max NPIV Ports: 127
      NPIV port list:
      HBA Port WWN: 2101001b32261720

      root@mNetFile:/volumes# cfgadm -al
      Ap_Id Type Receptacle Occupant Condition
      c0 scsi-bus connected configured unknown
      c0::dsk/c0t0d0 disk connected configured unknown
      c0::dsk/c0t1d0 disk connected configured unknown
      c0::scsi/array_ctrl/c0t7d0 array-ctrl connected configured unknown
      c2 fc connected unconfigured unknown
      c3 fc connected unconfigured unknown
      c4 fc-private connected unconfigured unknown
      c4::2209000a338d4da0 disk connected unconfigured unknown
      c4::2209000a338d4db0 disk connected unconfigured unknown
      c4::2209000a338d4dbd disk connected unconfigured unknown
      c4::2209000a338d4dc9 disk connected unconfigured unknown
      c4::2209000a338d4dd3 disk connected unconfigured unknown
      c4::2209000a338d4ddb disk connected unconfigured unknown
      c4::2209000a338d4de5 disk connected unconfigured unknown
      c4::2209000a338d4def disk connected unconfigured unknown
      c4::2209000a338d4dfb disk connected unconfigured unknown
      c4::2209000a338d4e37 disk connected unconfigured unknown
      c4::2209000a338d4e39 disk connected unconfigured unknown
      c4::2209000a338d4e3b disk connected unconfigured unknown
      c4::2209000a338d4e3e disk connected unconfigured unknown
      c4::2209000a338d4e41 disk connected unconfigured unknown
      c4::220d000a33330a46 ESI connected unconfigured unknown
      c4::220d000a33330a47 ESI connected unconfigured unknown
      c5 fc connected unconfigured unknown
      usb0/1 unknown empty unconfigured ok
      usb0/2 unknown empty unconfigured ok

      Edited by: 999845 on Apr 12, 2013 5:40 PM
        • 1. Re: cfgadm unable to configure disk shelf/disks
          You get JBOD for NetAPP. It can correct work only with NetAAPP.

          1. NetApp format disks with another sector size. (520 byte. Sure about FC disk, not sure about SATA)
          2. FATA disk ( ATA disk with FC converter - is not FC disk. Native Solaris can't work with this disks).
          3. Prolbem with trubleshutings this configuration, problem with identify bad disk etc.

          Connect this JBOD to original NetAPP and enjoy.

          • 2. Re: cfgadm unable to configure disk shelf/disks
            Thank you for your reply. It seems as though I need to clarify about the disk shelves and my experiences with them..

            When I first received the drives, they wouldn't be recognized by Linux or Windows. I researched and figured out they were formatted with the 520 byte sector size so I used Linux-based utility "parted" to re-format the sector size to 512 bytes. After I did that, I could use the FC drives in Windows and Linux, using the ESH4(4gb FC) module shelf. This shelf and these drives I am not having problems with.

            Then, I got my hands on another shelf - the shelf that has FC-ATX(2gb FC) modules and SATA drvies (with SATA->SCA-2 convertor in-caddy; Is this the "FATA" interposer?; the shelf's backplane is still SCA-2). Oddly enough, these drives were already formatted in 512-byte sector size (I think because NetApp has recently switched back to 512-byte for all drives...?). Anyway, I could immediately use these drives in Windows and Linux just fine. This shelf and these drives I cannot use in Solaris to make a ZFS volume.

            These "FATA" drives in the FC-ATX shelf - they work natively in Windows 2008R2 and Ubuntu 10.04 LTS. Is there anyway to get them to work with Solaris / Nexenta?
            The incompatibility of FATA drives in Solaris would explain why the drives are "Unconfigured" in cfgadm, however, why can't we configure the controller (and ESI) devices? P.S. does the listing of the ESI devices in cfgadm -al mean anything? Notice, they do not come up when I do cfgadm -al with ESH4 shelf plugged in...

            Does this help clear things up? Thank you for any help you can offer.
            • 3. Re: cfgadm unable to configure disk shelf/disks
              I would like to share some more experiences with these FATA disks.
              After some discussion with a colleague at work, I have decided to give FreeBSD's port of ZFS a try. Lo and Behold, the FATA drives show up as soon as FreeBSD is booted. I am able to configure them into a ZFS pool and all is happy.

              However, I do like Solaris's implementation of ZFS better, and it would be peachy if I could figure out how to get the FATA drives supported in Solaris. I find it counter-progressive that Solaris, the birthplace of ZFS, doesn't support a storage device that multiple other OSs do. Does anyone have any other suggestions RE this issue?

              Edited by: 999845 on May 4, 2013 3:31 PM