The question is about usable space in a ZFS 7320. System has two shelves - total (44) 900GB and (4) write-accelerators, so around 40TB raw. The summary per ZFS software is as follows:
Usable Data - 28.3TB
Parity - 5.62TB
Reserved - 480GB
Spare - 1.64TB
Data & Parity - 42 Disks
Spare - 2 Disks
Log - 4 Disks
Cache - 4 Disks
Snapshot can be seen here - http://systemsmarketing.com/ZFS_Snapshot.png
After spares, parity and reserved there is 28.3TB available. User wants to add another shelf but wants to be sure it will give him what he needs.
Adding a shelf with (24) 900GB is 21.6TB raw. The goal with that shelf is to be well over 40TB usable, hopefully at least 45TB. What we are trying to determine is, after adding the shelf, wiping everything and restarting with one pool, how much usable will be added? We assume it will still be two spares so the 1.64TB won’t change. The big question is the parity and whether that allocation is linear. We're assuming not as there is probably some initial overhead in that figure, but don’t know.
Our experience with Sun formatting on earlier drives is that firmware can account for as much as 15% but don’t know if that applies to these ZFS HDs. If so that would take off 6TB of the initial 40 which could explain most of the differential (expected had been around 35TB). If that 15% came off the 21.6TB shelf we would be starting at 18TB and any significant parity loss could be problematic.
Input on this question would be appreciated. Thank you.