I am attempting to add a JBOD shelf of disks to my existing Solaris environment - and I am unable to "initialize" or "configure" the disks so that I can use them in a ZFS volume.
The server has a quad port HBA, of which only one port goes to a NetApp DS14MK2 shelf with FC-ATX modules. Inside this shelf are SATA drives.
When I run cfgadm, Solaris can see the disks but it cannot use them as they are set to an Occupant type of "Unconfigured". If I attempt to run the cfgadm -c configure <ap_id>, the command completes without error but the devices (controller and disks) remained "Unconfigured". I've tried "cfgadm -c configure" for both controller and disk ap_ids, with no luck.
Now, if I use a different disk shelf, the NetApp DS14MK4 (ESH4 modules) with FC disks, I am able to utilize the disks immediately upon booting into Solaris, and I can add them to a ZFS volume without manually initializing/configuring them. In this case, the cfgadm -al command shows the controller and disks as "Connected" and "Configured". This leads me to believe that my issue may reside in the fact that the non-working disk shelf is "FC-ATX" style, whereas the working shelf is FC-AL (I think that statement is complete!?!). However, if I install Windows on this same server, Windows can see and initialize and use the FC-ATX shelf disks just fine, and I can create dynamic or standard volumes in Windows this way.
So, I'm speculating that the Solaris driver for my HBA card (Qlogic 2464) is unable to negotiate with a FC-ATX shelf... would you say my assumption is correct?
If you can please advise on how I can get these disks initialized. I would be glad to offer additional information and command outputs in pursuit of an answer!
Here are some to get our brains clicking:
root@mNetFile:/volumes# uname -a
SunOS mNetFile 5.11 NexentaOS_134f i86pc i386 i86pc Solaris
root@mNetFile:/volumes# fcinfo hba-port
HBA Port WWN: 2100001b32061720
Port Mode: Initiator
Port ID: 1
OS Device Name: /dev/cfg/c4
Manufacturer: QLogic Corp.
Firmware Version: 05.03.01
FCode/BIOS Version: BIOS: 3.20; fcode: 3.19; EFI: 2.25;
Serial Number: GFC0742E01431
Driver Name: qlc
Driver Version: 20100408-3.01
Supported Speeds: 1Gb 2Gb 4Gb
Current Speed: 2Gb
Node WWN: 2000001b32061720
Max NPIV Ports: 127
NPIV port list:
HBA Port WWN: 2101001b32261720
root@mNetFile:/volumes# cfgadm -al
Ap_Id Type Receptacle Occupant Condition
c0 scsi-bus connected configured unknown
c0::dsk/c0t0d0 disk connected configured unknown
c0::dsk/c0t1d0 disk connected configured unknown
c0::scsi/array_ctrl/c0t7d0 array-ctrl connected configured unknown
c2 fc connected unconfigured unknown
c3 fc connected unconfigured unknown
c4 fc-private connected unconfigured unknown
c4::2209000a338d4da0 disk connected unconfigured unknown
c4::2209000a338d4db0 disk connected unconfigured unknown
c4::2209000a338d4dbd disk connected unconfigured unknown
c4::2209000a338d4dc9 disk connected unconfigured unknown
c4::2209000a338d4dd3 disk connected unconfigured unknown
c4::2209000a338d4ddb disk connected unconfigured unknown
c4::2209000a338d4de5 disk connected unconfigured unknown
c4::2209000a338d4def disk connected unconfigured unknown
c4::2209000a338d4dfb disk connected unconfigured unknown
c4::2209000a338d4e37 disk connected unconfigured unknown
c4::2209000a338d4e39 disk connected unconfigured unknown
c4::2209000a338d4e3b disk connected unconfigured unknown
c4::2209000a338d4e3e disk connected unconfigured unknown
c4::2209000a338d4e41 disk connected unconfigured unknown
c4::220d000a33330a46 ESI connected unconfigured unknown
c4::220d000a33330a47 ESI connected unconfigured unknown
c5 fc connected unconfigured unknown
usb0/1 unknown empty unconfigured ok
usb0/2 unknown empty unconfigured ok
You get JBOD for NetAPP. It can correct work only with NetAAPP.
1. NetApp format disks with another sector size. (520 byte. Sure about FC disk, not sure about SATA)
2. FATA disk ( ATA disk with FC converter - is not FC disk. Native Solaris can't work with this disks).
3. Prolbem with trubleshutings this configuration, problem with identify bad disk etc.
Thank you for your reply. It seems as though I need to clarify about the disk shelves and my experiences with them..
When I first received the drives, they wouldn't be recognized by Linux or Windows. I researched and figured out they were formatted with the 520 byte sector size so I used Linux-based utility "parted" to re-format the sector size to 512 bytes. After I did that, I could use the FC drives in Windows and Linux, using the ESH4(4gb FC) module shelf. This shelf and these drives I am not having problems with.
Then, I got my hands on another shelf - the shelf that has FC-ATX(2gb FC) modules and SATA drvies (with SATA->SCA-2 convertor in-caddy; Is this the "FATA" interposer?; the shelf's backplane is still SCA-2). Oddly enough, these drives were already formatted in 512-byte sector size (I think because NetApp has recently switched back to 512-byte for all drives...?). Anyway, I could immediately use these drives in Windows and Linux just fine. This shelf and these drives I cannot use in Solaris to make a ZFS volume.
These "FATA" drives in the FC-ATX shelf - they work natively in Windows 2008R2 and Ubuntu 10.04 LTS. Is there anyway to get them to work with Solaris / Nexenta?
The incompatibility of FATA drives in Solaris would explain why the drives are "Unconfigured" in cfgadm, however, why can't we configure the controller (and ESI) devices? P.S. does the listing of the ESI devices in cfgadm -al mean anything? Notice, they do not come up when I do cfgadm -al with ESH4 shelf plugged in...
Does this help clear things up? Thank you for any help you can offer.
I would like to share some more experiences with these FATA disks.
After some discussion with a colleague at work, I have decided to give FreeBSD's port of ZFS a try. Lo and Behold, the FATA drives show up as soon as FreeBSD is booted. I am able to configure them into a ZFS pool and all is happy.
However, I do like Solaris's implementation of ZFS better, and it would be peachy if I could figure out how to get the FATA drives supported in Solaris. I find it counter-progressive that Solaris, the birthplace of ZFS, doesn't support a storage device that multiple other OSs do. Does anyone have any other suggestions RE this issue?