This discussion is archived
8 Replies Latest reply: May 15, 2013 8:23 AM by 1008097 RSS

RAC11gR2 - unable to connect via SCAN - can connect directly on nodes

1008097 Newbie
Currently Being Moderated
Hello,

I recently installed a two node RAC 11gR2 database on Oracle Linux. I'm trying to connect from an external windows client, using SQL Developer, but I keep getting the error "IO Error: The Network adapter could not establish the connection".

If I attempt to connect directly to one of the nodes (instead of putting the scan hostname I just use the node's hostname) it works without a problem. I'm thinking this might have something to do with a bad configuration on the scan listeners or the VIP.

After installing clusterware and the database I never manually configured any listener. I assume this was done automatically. Let me know what kind of information I should provide to make this easier to troubleshoot. I'm not using any hosts file entries (except the private interconnect hostnames) since everything has been configured on my DNS. This is the network schema I had in mind when creating the cluster (and it's how my DNS is configured):

#10.16.1.93 rac01-dev.elqrd.local rac01-dev -- public interfaces
#10.16.1.94 rac02-dev.elqrd.local rac02-dev

#10.16.1.95 rac01-vip.elqrd.local rac01-vip -- vip interfaces
#10.16.1.96 rac02-vip.elqrd.local rac02-vip

172.16.125.1 rac01-priv.elqrd.local rac01-priv -- private interconnect
172.16.125.2 rac02-priv.elqrd.local rac02-priv

#10.16.1.97 rac01-scan -- scan ip's
#10.16.1.98
#10.16.1.99

Here's the scan status of my cluster:

[grid@rac01-dev ~]$ srvctl config scan
SCAN name: rac01-scan.elqrd.local, Network: 1/10.16.1.0/255.255.255.0/eth0
SCAN VIP name: scan1, IP: /rac01-scan.elqrd.local/10.16.1.97
SCAN VIP name: scan2, IP: /rac01-scan.elqrd.local/10.16.1.98
SCAN VIP name: scan3, IP: /rac01-scan.elqrd.local/10.16.1.99


And here's the vip configuration:
[grid@rac01-dev ~]$ srvctl config vip -n rac01-dev
VIP exists.:rac01-dev
VIP exists.: /rac01-vip/10.16.1.95/255.255.255.0/eth0
[grid@rac01-dev ~]$ srvctl config vip -n rac02-dev
VIP exists.:rac02-dev
VIP exists.: /rac02-vip/10.16.1.96/255.255.255.0/eth0
[grid@rac01-dev ~]$


Here is the listener status:
grid@rac01-dev ~]$ lsnrctl service

LSNRCTL for Linux: Version 11.2.0.1.0 - Production on 09-MAY-2013 17:01:01

Copyright (c) 1991, 2009, Oracle. All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER)))
Services Summary...
Service "+ASM" has 1 instance(s).
Instance "+ASM1", status READY, has 1 handler(s) for this service...
Handler(s):
"DEDICATED" established:70 refused:0 state:ready
LOCAL SERVER
Service "ORCL" has 1 instance(s).
Instance "ORCL1", status READY, has 1 handler(s) for this service...
Handler(s):
"DEDICATED" established:83 refused:0 state:ready
LOCAL SERVER
Service "ORCLXDB" has 1 instance(s).
Instance "ORCL1", status READY, has 1 handler(s) for this service...
Handler(s):
"D000" established:0 refused:0 current:0 max:1022 state:ready
DISPATCHER <machine: rac01-dev.elqrd.local, pid: 9331>
(ADDRESS=(PROTOCOL=tcp)(HOST=rac01-dev.elqrd.local)(PORT=64963))
The command completed successfully

It's the same on the second node, with the difference that it refers to the other instances (+ASM2, ORCL2)



A similar thread suggested I post this information also:
select inst_id, name, value from gv$parameter where name like '%_listener%'

1     local_listener     (DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=rac01-vip)(PORT=1521))))
1     remote_listener     rac01-scan.elqrd.local:1521
2     local_listener     (DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=rac02-vip)(PORT=1521))))
2     remote_listener     rac01-scan.elqrd.local:1521

Edited by: 1005094 on May 9, 2013 2:01 PM
Added listener status information

Edited by: 1005094 on May 9, 2013 2:06 PM

Edited by: 1005094 on May 9, 2013 2:17 PM
  • 1. Re: RAC11gR2 - unable to connect via SCAN - can connect directly on nodes
    damorgan Oracle ACE Director
    Currently Being Moderated
    The scan listener is installed, configured, and managed as part of clusterware.

    Can you ping it from the client?

    Wen you do ps -ef can you see one running?
  • 2. Re: RAC11gR2 - unable to connect via SCAN - can connect directly on nodes
    Levi-Pereira Guru
    Currently Being Moderated
    Hi,
    Try use this checklist:
    http://levipereira.wordpress.com/2011/05/03/configuring-client-to-use-scan-11-2-0/
  • 3. Re: RAC11gR2 - unable to connect via SCAN - can connect directly on nodes
    1008097 Newbie
    Currently Being Moderated
    [grid@rac01-dev ~]$ ps -ef | grep SCAN
    grid 8317 1 0 May09 ? 00:00:03 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER_SCAN2 -inherit
    grid 8326 1 0 May09 ? 00:00:02 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER_SCAN3 -inherit

    [oracle@rac02-dev ~]$ ps -ef | grep SCAN
    grid 5828 1 0 May09 ? 00:00:02 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER_SCAN1 -inherit

    So it appears the scan listeners are working on both nodes.


    From the client I can telnet and ping both nodes and their vip as well as the scan dnsname & ip.

    Now the odd thing is this:
    When I do lsnrctl service listener_scan1 (| 2 | 3) I notice that the 3 scan listeners are running on either node 01 or node 02. This is intended. However take a look at the output:

    [grid@rac01-dev ~]$ lsnrctl service listener_scan3 (it's more or less the same for the other 2 scan listeners)

    LSNRCTL for Linux: Version 11.2.0.1.0 - Production on 10-MAY-2013 09:13:43

    Copyright (c) 1991, 2009, Oracle. All rights reserved.

    Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN3)))
    Services Summary...
    Service "ORCL" has 2 instance(s).
    Instance "ORCL1", status READY, has 1 handler(s) for this service...
    Handler(s):
    "DEDICATED" established:0 refused:0 state:ready
    REMOTE SERVER
    (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=rac01-vip)(PORT=1521)))
    Instance "ORCL2", status READY, has 1 handler(s) for this service...
    Handler(s):
    "DEDICATED" established:2 refused:0 state:ready
    REMOTE SERVER
    (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=rac02-vip)(PORT=1521)))
    Service "ORCLXDB" has 2 instance(s).
    Instance "ORCL1", status READY, has 1 handler(s) for this service...
    Handler(s):
    "D000" established:0 refused:0 current:0 max:1022 state:ready
    DISPATCHER <machine: rac01-dev.elqrd.local, pid: 9331>
    (ADDRESS=(PROTOCOL=tcp)(HOST=rac01-dev.elqrd.local)(PORT=64963))
    Instance "ORCL2", status READY, has 1 handler(s) for this service...
    Handler(s):
    "D000" established:0 refused:0 current:0 max:1022 state:ready
    DISPATCHER <machine: rac02-dev.elqrd.local, pid: 6174>
    (ADDRESS=(PROTOCOL=tcp)(HOST=rac02-dev.elqrd.local)(PORT=63648))
    The command completed successfully


    What exactly is the ORCLXDB service? When I installed the database the provided SID was ORCL (With the instances ORCL1 & ORCL2 running on each of the nodes). I am able to connect on ORCLXDB (via SCAN) from my client but not on ORCL. I can still connect normally to the database if I specify the node hostname instead of the scan. But this is not desired as it kind of negates the point of having SCAN (and removes the load balancing functionality).

    P.S I'm trying to connect to the scan via SQL Developer and without providing any tnsnames.ora file. It fails for scan (for ORCL service - but works if it is ORCLXDB) but works for the nodes directly. The same thing happens from a second client when I try via sqlplus ("The network adapter could not establish the connection")

    Edited by: 1005094 on May 10, 2013 6:26 AM

    Edited by: 1005094 on May 10, 2013 6:31 AM
  • 4. Re: RAC11gR2 - unable to connect via SCAN - can connect directly on nodes
    1008097 Newbie
    Currently Being Moderated
    Also I noticed this:

    [grid@rac02-dev ~]$ cat /u01/app/11.2.0/grid/network/admin/endpoints_listener.ora
    LISTENER_RAC02-DEV=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=rac02-vip)(PORT=1521))(ADDRESS=(PROTOCOL=TCP)(HOST=10.16.1.94)(PORT=1521)(IP=FIRST))))

    [grid@rac01-dev admin]$ cat endpoints_listener.ora
    LISTENER_RAC01-DEV=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=rac01-vip)(PORT=1521))(ADDRESS=(PROTOCOL=TCP)(HOST=10.16.1.93)(PORT=1521)(IP=FIRST))))          # line added by Agent

    According to my initial design, 10.16.1.93 and 94 are the public interfaces for the two nodes. The VIPs are defined on my dns as .95 (rac01-vip) and .96 (rac02-vip):

    rac01-dev = .93
    rac01-vip = .95

    rac02-dev = .94
    rac02-vip = .96

    Why is the endpoint listener containing both the vip and the public ip? Is this as intended?
  • 5. Re: RAC11gR2 - unable to connect via SCAN - can connect directly on nodes
    onedbguru Pro
    Currently Being Moderated
    https://www.lmgtfy.com/?q=oracle+scan+pdf

    make sure your local and remote_listener parameters are properly configured on all nodes.
  • 6. Re: RAC11gR2 - unable to connect via SCAN - can connect directly on nodes
    986120 Explorer
    Currently Being Moderated
    Hi,

    Is your scan IP resolvable from windows client.
    i.e: ping scan hostname as well ip.
    example: ping rac01-scan

    Try to add *"elqrd.local"* to your windows DNS suffix

    --
    Bala:)
  • 7. Re: RAC11gR2 - unable to connect via SCAN - can connect directly on nodes
    1008097 Newbie
    Currently Being Moderated
    I've tried this from a windows, a linux and my macbook host. I can ping and telnet the SCAN ip & hostanme without a problem from anywhere.

    From linux it seems I can tnsping the service but not able to login via sqlplus:

    [oracle@queue02-sbox admin]$ tnsping ORCL

    TNS Ping Utility for Linux: Version 11.2.0.1.0 - Production on 14-MAY-2013 10:21:18

    Copyright (c) 1997, 2009, Oracle. All rights reserved.

    Used parameter files:
    /u01/app/oracle/product/11.2.0/dbhome_1/network/admin/sqlnet.ora


    Used TNSNAMES adapter to resolve the alias
    Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = rac01-scan.elqrd.local)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = ORCL)))
    OK (0 msec)
    [oracle@queue02-sbox admin]$ sqlplus sys@ORCL as sysdba

    SQL*Plus: Release 11.2.0.1.0 Production on Tue May 14 10:21:30 2013

    Copyright (c) 1982, 2009, Oracle. All rights reserved.

    Enter password:
    ERROR:
    ORA-12537: TNS:connection closed
  • 8. Re: RAC11gR2 - unable to connect via SCAN - can connect directly on nodes
    1008097 Newbie
    Currently Being Moderated
    Apparently this was a silly problem that was missed by me due to making attempts on various OS's. I just had to add a dns suffix on windows (and update /etc/resolv.conf on my linux client). For some reason when trying to make connections to the SCAN hostname without having a DNS search domain configured it fails. Even if I explicitly set the hostname as fully qualified. I was still able to ping the fully qualified name but sqlplus (and sql developer) failed to login. Thanks for the feedback.

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points