I have two nic cards on node B (oracle linux 6.3). one card with default gateway is alright, but I can not ping the other card(200 segement) from A node(just pick any one node). on the same A node, i can ping all other nodes in the same 200 segment. the following is the "route" table and ifconfig eth1 on node B:
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default 10.100.216.1 0.0.0.0 UG 0 0 0 eth2
10.100.200.0 * 255.255.255.0 U 0 0 0 eth1
10.100.216.0 * 255.255.255.0 U 0 0 0 eth2
192.168.122.0 * 255.255.255.0 U 0 0 0 virbr0
Please be more specific in your description. I'm not sure if I can follow, but glancing your addresses and network mask I'd say you cannot ping between the networks because of your netmask being 255.255.255.0, which prevents broadcasting between the 10.100.0.0 and 10.200.0.0 networks. If you change your netmask to 255.0.0.0 or 255.255.0.0 it should work.
Sure. On node B, there are 2 nic cards, and IP's are 10.100.200.81(eth1), and 10.100.216.81(eth2), respectively. After the configurations all being done, nobody can ping 10.100.200.81. For instance, on node A, I could not ping 10.100.200.81, however, I could ping all nodes in 200 segment from the node A.
I can ping 10.100.216.81 witout a problem. 10.100.216.1 is the default gateway.
Based on ifconfig eth1 on node A, this nic card seems UP, and configured correctly.
Please check out the output of ifconfig eth1 and route in my previous message.
Hopefully, i got this clearer a little bit more this time.
As far as I can tell from your description, you cannot communicate between the 10.100.216 and 10.100.200 network without a 255.255.0.0 netmask or having a separate gateway in each of the networks that bridges the two networks together. The gateway has be be within the same subnet as the source IP in order to be seen. With your current netmask, your 10.100.200.0 network cannot see the 10.100.216.1 gateway.
got your idea, however, I have a basic question on changing the netmask.
I chnaged NETMASK=255.255.0.0 in /etc/sysconfig/network-scripts/ifcfg-eht1. and then "service network restart". How come the netmask 255.255.0.0 could not be shown in "route" command, it is still showing 255.255.255.0. I even reboot the server, same thing.
I suggest to check ifcfg-eth1 to see what value you have defined for PREFIX. Technically it should be 8 for a 255.0.0.0 netmask and a 10.0.0.0 network. If you want to use the value for NETMASK=255.255.0.0, remove PREFIX or change it to 16 and restart the network using "service network restart".