I run a 7320 2-peer cluster, latest OS. Hardware includes onboard 4 ports 1Gb, a 2-port 8Gb FC card, clusteron card and the SAS card. Means I can't really add any ethernet ports.
I am trying to get the most bandwidth possible, without sacrificing availability or manageability during takeover.
After a long time of whiteboarding, I've come up with the following:
igb0, igb1 make LACP1
igb2, igb3 make LACP2
Create locked interface INT1 on Node1 attached to LACP1, IP 10.105.100.2 (lock for guarantee of IP/node relationship)
Create locked interface INT2 on Node2 attached to LACP2, IP 10.105.100.3 (lock for guarantee of IP/node relationship)
Create IPMP1 on Node1, unlocked attached to INT1 and INT2, both Active, IP 10.105.100.4
Create IPMP2 on Node2, unlocked attached to INT2 and INT1, both Active, IP 10.105.100.5
My thinking is if both interfaces are marked active, in normal operation, IPMP1 living on node1 will see INT1 is alive and working, INT2 is offline. During takeover to node2, IPMP1 slides to node2, INT1 then shows down but INT2 is alive. Vice-versa for other IPMP.
I don't want a configuration that prevents me from hosting up LUN from either storage node/storage pool. Creating a single LACP/INT attached to a single node prevents me from presenting LUNs through controller that doesn't own the interface. That seems obvious, so I need each node to each own an interface capable of having LUNs presented through it, and set the iSCSI target to listen on both physical (locked) interfaces. Make sense?
Follow up questions:
Currently, we have a single "storage switch" that both nodes are plugged into. Highly redundant nexxus switch, not worried about single point of failure here.
Should my LACP channels include physical port igb0, igb1 of BOTH nodes, or should I create 2 independent channels, 1 per node per group?
For example, lets say the following:
IGB0 Node1 - MAC Address = A
IGB1 Node1 - MAC Address = B
IGB2 Node1 - MAC Address = C
IGB3 Node1 - MAC Address = D
IGB0 Node2 - MAC Address = E
IGB1 Node2 - MAC Address = F
IGB2 Node2 - MAC Address = G
IGB3 Node2 - MAC Address = H
On the switch, do I configure LACP Channel1 to include "A, B" or "A, B, E, F"?
Edited by: John Berisford on Oct 11, 2012 10:41 AM
My reason for creating the IPMP is because it can be assigned an IP address which is a clusterable resource that I can serve up iSCSI from.
To be a good working configuration, I need an interface/IP that does not move during takeover/failback operations. But on top of that, I create an IPMP interface that can slide - therefore follow cluster operations as needed.
I'm mid-way through configuring this, so right now igb2,3 are an AGG, with a locked interface on top of it, and an IPMP on top of that. IGB0, 1 are currently the admin ports on regular VLAN switch, but they do serve up SMB as a temporary need popped up. That temporary need is being removed this weekend, after which I'll kill those datalinks and create LACP for those sets, set a locked interface on top of that, then a movable IPMP on top of that new interface. Should give me 2 LACPs of 2 ports each, 1 locked interface per LACP and 1 moving interface per LACP.