Sun datacenter infiniband switch 36 with zfssa 7420 clustered installation
We are trying to configure 2 Sun Datacenter InfiniBand switches which we plan to use between a zfssa 7420 (clustered 2 controller configuration) and 3 nodes of RAC servers. We need the zfssa to expose nfs shares to the servers in an fully redundant manner without any single point of failure. Each zfssa controller head have 4 infiniband ports on 2 pci slots which are connected to the 2 IB switches (i.e. 1 port from each pci slot goes to a different switch), the same thing holds for each of the RAC servers (4 ports/2 Slots/each slot is connected to 2 switches).
We are having problems with configurations.Trying to configure zfssa controller IB IPMPs seem to work but two controllers cannot cannot ping each other. Same thing goes on with the servers, one IB link from a RAC server can ping zfssa IPs but other IB links do not ping etc.