I am setting up an enviornment with a ZFS 7120 for iSCSI storage, connected to ESXi 5.1 host via a dedicated Dell PowerConnect 6224 switch.
The 7120 has 2 ports dedicated to iSCSI, igb2 and igb1. These are in an LACP Aggregation with MTU of 9000 bytes. The PowerConnect docs recommend MTU on the ports of 9216 bytes. There is an EQuallogic SAN also connected to this switch (switch ports also at 9216 MTU, and jumbo frames enabled on equallogic side ... it all is working well).
The ESXi 5.1 host has a vSwitch with MTU 9000, which also has 2 VMKernel ports with MTU 9000, and each has 1 dedicated vmnic for uplink to the Dell switch. In this configuration, I confirm that jumbo frames is OK by using vmkping -d -s 8972 <ipofsanarray> . I get response from Equallogic, indicating tha end-to-end jumbo frames is working. I get no response from the IP of the 7120, which means that there is some jumbo frame issue. I do a regular ping of the ZFS just fine, so the devices are connected and able to talk to each other. vmkping to regular servers connected (with Jumbo enabled) to the SAN works.
It seems the issue is on the 7120 side of things (I think), but not sure where to start troubleshooting? Maybe I should break the LACP config?
What's the output from "ifconfig -a" show on the 7120?
Can you snoop the interface(s) from the 7120 and see if the packets are getting there? And if they are, are responses getting sent?
Are all the other connections identical between the various storage devices? Same switch? Same switch port configs? If everything else is the same, yeah, I think I'd try simplifying things by dropping LACP and seeing if I could get it to work in a simpler configuration first.