Let me try to answer some:
>>1.Lets say a connection request comes to scan listener , scan listener find that least loaded node , say node 1 ,what scan listener will do if it find that node listener is down on the node1?
If scan listener finds a node listener as down, it will hand over the connection to another node.
>>2.If any scan_listener can hand-off to any node_listener in the cluster , then why there are multiple scan_listener/scan_vips ? In that case only one scan_listener is enough right ,as i see there is no "fail over" concept for scan-vip/scan_listener,they will be running some where in the cluster , with no worries
If you have your scan name registered in DNS with only 1 ip, then only 1 scan_listener will be there. Multiple scan listeners are for load balancing.
If you have only 1 scan-vip then there will be only one scan_listener and if there is some config issue at the grid side there will be a complete downtime. This is another reason to suggest multiple scan-vips.
>>3.And how do scan listener will identify the correct node_listener , if a node is having multiple node_listeners ?
First PMON picks the local listener details from the init parameter LOCAL_LISTENER and tell it "hey local listener, i also run on the same server as you and i run the following services. So if you get any connection requests for these services hand them over".
Secondly PMON picks the the parameter remote_listener points(should point) to the scan_name and the PMON will talk to the scan_listeners and updates them about the services and local listener running on that node. If you have 3 scan_listeners , all of them will be updated with the details by respective PMONs.
SYS:AINTPRD1 > show parameters remote NAME TYPE VALUE ------------------------------------ ----------- ------------------------------ remote_listener string tprod-scan:1521
Now, when the Scan listener gets a connection request from a client it forwards it to the least loaded local_listener which in turns forks a foreground process.
>>4. The scan-vip and node-vips will be up from the similar sub interfaces (eth0:1& eth0:2) of public interface (eth0). So why scan-vip can hold scan_listener on any node , why not node-vip can hold node-listener when failed over to other node?
The scan-vip and node-vips are for different functionality but serve similar purposes. In a 10 node RAC, you need 10 node-vips but only 3 scan-vips is required.
Earlier in 10gR2 in TNS entry we use a series of 10 hostname-vips(for a 10 node RAC) so that the connections failover to otehr nodes of the 1st node is not available.
In 11gR2 in TNS entry we use scan-name so that you don't have to mention all the 10 hostname-vips.
>>5.Via which IP ? VIP or Pub-IP ? The scan_listener communicates with node_listener?
The scan_listener will communicate to which ever IP the local_listener is running on.
>>6. A node_listener in RAC will run on two end points by default (VIP/TCP/1521 & PUBIP/TCP/1521), if for any reason the VIP is failed over to other node , then why not the listener continue to run the existing pub-ip?
A hostname-vip failover to other node when that host is not available. If the host is not not avaialble then why would the scan_listener continue on that.
Remember that scan-vips are not associated to any specific interface on any node. They hop within the grid based on the availability.
J have already answered this question. Some addition.
VIPs were introduced in 10g. When VIP fails over to other node, client that connects to this VIP will get "TNS:no listener" error and can quickly connect to other node listed in tnsnames. In case of connecting to public IP, client will wait for TCP timeout.
11g with SCAN Listeners simplifies this mechanism, eliminating requirement of maintance client's tnsnames.