Looking for some insight on the Logzilla drives. Sorry - long post, lots of questions.
We have a 7310 cluster with a single J4400 cage with 22 spindles and 2 ZIL's. They're 18GB drives.
Last year I wanted to add another drive cage to my cluster, and was told that I was basically out of luck and had to get an entirely new SAN because they couldn't sell me a J4400 and the 4410's weren't compatible due to the SAS & SAS2 differences. Oracle DID do us well and gave me a GREAT deal on a new 7320 cluster with 2 fully populated 4410's for an absolutely great price. Was for sure more than adding a drive shelf, but not anywhere close to what it would cost on the street, to make up for the mistake on their end. Kudo's to Oracle.
So now I have a 7320 & 2 4410's.
Problem is - the 4410's didn't include Logzilla drives, and the controllers came with a mere 24GB memory.
I do intend to upgrade the ARC, but I'm assuming my I/O is suffering quite a bit due to the lack of ZILs. However I am struggling to find a metric in DTRACE to properly assess the latency, by LUN, the lack of ZIL drives is imposing on the system. How can I quantify this bottleneck that must surely exist? Workload is mainly 10 VMware 2TB LUNs attached to 6 hosts, approximately 200 VM's. Pool contains 18 spindles across both chassis, mirror config. I have another pool of another 18 disks (and I have some unused currently) that is mostly NFS and iSCSI traffic, light use - nonissue, I'd say currently.
Also, I keep hearing from support that I cannot use the ZIL drives from the J4400 on the J4410 chassis. I understand fully that there's a difference between SAS2 and SAS, but isn't the SAS standard in general backward compatible? Further, these aren't hard drives w/spindles, they're flash operating on a SAS bus. Why is it so unthinkable to want to use the ZIL from my 4400 in a 4410? Why not just swap the caddy out and insert? What am I missing?
From what I understand of ZFS in general, I could even use a traditional spindle HDD as a log drive, technically. Why can't I use a slightly lower-performing drive than current generation top of the line write-optimized device if ZFS itself will support a single spindle for caching writes?
I am not certain how much, if at all, the VMware workload would benefit from ZIL, there aren't any heavy I/O demanding guests inside the environment currently. We do intend to move our production data warehouse from the 7310 (iSCSI connected 1Gb, all 22 spindles mirror set pretty much dedicated to warehouse, 96GB memory) to the 7320, which essentailly eliminates the 7310 from production entirely, and I can steal and pillage the device for testing or other uses. The warehouse workload would be much more sequential in nature and so I'm assuming would have more benefit from ZIL, but I can't quantify this assumption, currently. If I do identify this need, why couldn't I just use the ZIL's from the 7310? Why do I need to buy more?
Sorry for the long post. I can't seem to get anything other than what feels like a surface, almost sales-driven response, from Oracle support and I don't have a local VAR who can help.