We have about 20 solaris 10u7 installations built from identical templates that run oracle databases with windows 2k3 2k8 front end on dell poweredge servers with Broadcom teamed nics. The problem affects only some connections. dell1 can ping 100% solaris1 but drops packets against solaris2 where dell2 can ping 100% solaris1 but drops packets against solaris2 and dell3 and solaris3 can ping everyone 100%. All the servers are on the same subnet. When the dell box cannot ping the solaris box I run an arp -a on the solaris box and it returns the wrong mac address for the dell box. The mac address can be a duplicate address from another server / gateway. When the mac address is correct the pings return. I have now changed the ip address on one of the solaris boxes and everyones happy for now...... I have tried setting static mac addresses with arp -s but these are overwritten. We are starting to see arp entries for computers no longer on our network.
We have tried packet sniffing. I am not sure how you can get the source of the arp from snoop I am trying to find the information for that. We cleared most of the issues by upgrading all the dell broadcom drivers and firmware on the same subnet. There is an issue between the latest Solaris OS 10u4 upwards and broadcom NICs. Solaris adheres closer to rfc86 the older broadcom drivers upto version 4.1 do not run to the same standard. We are still trying to find where the odd 2 bous arp entries come from. We think it is Broadcom based issues and not Solaris there seems to be a lot of entries on the internet re this issue.
It seems the issue was caused by drivers on our dell windows servers, the broadcom drivers and teaming software were broadcasting false mac addresses unfortunately one of the mac addresses broadcast was our default gateway. Once we updated all our broadcom host drivers and firmware the issue went away.
We have tried to packet sniffing. I do not know how you can get the source from the prying eyes of the ARP I tried to find the information. Just like the firewire device, you need to make sure that your [sata dock station|http://www.espow.com/wholesale-sata-hdd-docking-station-for-mac-support-1394b-1394a-firewire-port.html] is ok, and then we raise all in the same subnet Dell Broadcom hard drivers and firmware of most issues. There is a new Solaris operating system 10u4 hdd dock and Broadcom network card problem. Adhere to the old drivers close rfc86 Solaris version 4.1 does not run upto Broadcom to the same level. We are still looking into more than 2 Boos from the ARP table entry. We think this is Broadcom's problem, not the existence of Solaris seems to be a re-entries on the Internet a lot of the problem.