13 Replies Latest reply on Jan 23, 2012 10:32 AM by Erni Papakroni

    Oracle lsnrctl stops and hangs

    Erni Papakroni
      Hi ,
      I installed Oracle 11g in OEL Linux . OEL Linux is inside Oracle VM . I installed in another machine (OEL Linux inside Oracle VM) . In the other machine installed Weblogic 10.3.2 , SOA Suite and RCU . I've noticed connection problems and hangs since installation of RCU .Database listener gets down or hangs every time i try to connect with it . Installation took a lot of time (this was unusual ) but in the end was successful . When i start soa_server i see errors regarding ConnectionDeadSQLException and DataSource related , when i try lsnrctl status it hangs a lot and sometimes it stops by itself .I'm 100% sure that it's listeners problem (I've tested connection from other machines too). I'm attaching listener.ora file . Please help me cause this is really urgent for me



      # listener.ora Network Configuration File: /u01/app/oracle/product/11.2.0/dbhome_1/network/admin/listener.ora
      # Generated by Oracle configuration tools.

      LISTENER =
      (DESCRIPTION_LIST =
      (DESCRIPTION =
      (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC1))
      (ADDRESS = (PROTOCOL = TCP)(HOST = localhost)(PORT = 1521))
      (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.1.51)(PORT = 1521))
      )
      )

      SUBSCRIBE_FOR_NODE_DOWN_EVENT_LISTENER=OFF

      SID_LIST_LISTENER =
      (SID_LIST =
      (SID_DESC =
      (SID_NAME = PLSExtProc)
      (ORACLE_HOME = /u01/app/oracle/product/11.2.0/dbhome_1)
      (PROGRAM = extproc)
      )
      (SID_DESC =
      (GLOBAL_DBNAME = orcl)
      (ORACLE_HOME =/u01/app/oracle/product/11.2.0/dbhome_1)
      (SID_NAME = orcl)
      )
      )



      ADR_BASE_LISTENER = /u01/app/oracle
        • 1. Re: Oracle lsnrctl stops and hangs
          sb92075
          no listener.ora file is require for V10+

          as a test, rename the listener.ora file out of the way to a new/different name

          issue command below

          lsnrctl stop
          lsnrctl start

          COPY commands & results then PASTE all back here
          • 2. Re: Oracle lsnrctl stops and hangs
            Billy~Verreynne
            The Listener is not a very complex piece of s/w (unlike the database instance). It very seldom will simply crash or simply hang.

            It opens a tcp end point on the network, listens for incoming connections and services them. It also can get system requests (e.g. registration events from a db instance) to handle.

            And in a nutshell, that is it.

            Listener problems are typically network related - i.e. issues in the network layer itself. Such as reverse IP lookup issues. Network link running in half duplex. Etc.

            To see just what the Listener does, one can enable tracing (with different verbosity levels). Such a trace make it usually pretty easy to spot where the error occurs and diagnose it.

            Enable tracing. Also, have a look at the kernel log (using dmesg or <i>/var/log/messages</i>) to see if there are network related errors. ethtool can be used to look at the interface counters to see if any error counters are ticking over.
            • 3. Re: Oracle lsnrctl stops and hangs
              Erni Papakroni
              Hi ,
              I renamed listener.ora
              Stopped the listener, started the listener.

              lsnrctl status shows



              Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))
              STATUS of the LISTENER
              ------------------------
              Alias LISTENER
              Version TNSLSNR for Linux: Version 11.2.0.1.0 - Production
              Start Date 20-JAN-2012 11:20:38
              Uptime 0 days 0 hr. 9 min. 10 sec
              Trace Level off
              Security ON: Local OS Authentication
              SNMP OFF
              Listener Log File /u01/app/oracle/diag/tnslsnr/ora/listener/alert/log.xml
              Listening Endpoints Summary...
              (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=ora.facilization.net)(PORT=1521)))
              Services Summary...
              Service "orcl.facilization.net" has 1 instance(s).
              Instance "orcl", status READY, has 1 handler(s) for this service...
              Service "orclXDB.facilization.net" has 1 instance(s).
              Instance "orcl", status READY, has 1 handler(s) for this service...
              The command completed successfully
              ----------------------------------------------------


              Now when i start SOA , it throws exceptions

              The DBMS driver exception was: Listener refused the connection with the following error:
              ORA-12514, TNS:listener does not currently know of service requested in connect descriptor

              When i try to connect with another app to db it throws me the same error
              ORA-12514, TNS:listener does not currently know of service requested in connect descriptor

              So i think listener.ora is needed , just an opinion.
              Before renaming an listener.ora i was able to connect now not.


              Regards Erni
              • 4. Re: Oracle lsnrctl stops and hangs
                Billy~Verreynne
                Erni Papakroni wrote:

                So i think listener.ora is needed , just an opinion.
                Not really. The error is the client requesting a service that does not exist (is not registered with the listener). I.e.
                ORA-12514, TNS:listener does not currently know of service requested in connect descriptor

                A listener.ora would be needed for defining non-database services - like a heterogeneous service or external procedure service. A database instance registers its services with the local and remote configured listeners. There is no need to do this manually via the listener.ora file - registration is automated (instance talks directly with the listener).

                Your listener has the following services:
                - service: orcl.facilization.net
                - sid: orcl
                - service: orclXDB.facilization.net

                What does the client's TNS connection string request? A service or a SID? What is this value?
                • 5. Re: Oracle lsnrctl stops and hangs
                  sb92075
                  Erni Papakroni wrote:
                  Hi ,
                  I renamed listener.ora
                  Stopped the listener, started the listener.

                  lsnrctl status shows



                  Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))
                  STATUS of the LISTENER
                  ------------------------
                  Alias LISTENER
                  Version TNSLSNR for Linux: Version 11.2.0.1.0 - Production
                  Start Date 20-JAN-2012 11:20:38
                  Uptime 0 days 0 hr. 9 min. 10 sec
                  Trace Level off
                  Security ON: Local OS Authentication
                  SNMP OFF
                  Listener Log File /u01/app/oracle/diag/tnslsnr/ora/listener/alert/log.xml
                  Listening Endpoints Summary...
                  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=ora.facilization.net)(PORT=1521)))
                  Services Summary...
                  Service "orcl.facilization.net" has 1 instance(s).
                  Instance "orcl", status READY, has 1 handler(s) for this service...
                  Service "orclXDB.facilization.net" has 1 instance(s).
                  Instance "orcl", status READY, has 1 handler(s) for this service...
                  The command completed successfully
                  ----------------------------------------------------


                  Now when i start SOA , it throws exceptions

                  The DBMS driver exception was: Listener refused the connection with the following error:
                  ORA-12514, TNS:listener does not currently know of service requested in connect descriptor

                  When i try to connect with another app to db it throws me the same error
                  ORA-12514, TNS:listener does not currently know of service requested in connect descriptor

                  So i think listener.ora is needed , just an opinion.
                  Before renaming an listener.ora i was able to connect now not.


                  Regards Erni
                  within the file "/u01/app/oracle/diag/tnslsnr/ora/listener/alert/log.xml" & near the end of the file will be "12514"

                  post excerpt from file starting about 10 - 15 lines above "12514" line to 20 or so lines below
                  • 6. Re: Oracle lsnrctl stops and hangs
                    Erni Papakroni
                    These are some lines of log.xml file


                    host_addr='192.168.1.51'>
                    <txt>20-JAN-2012 11:49:22 * service_update * orcl * 0
                    </txt>
                    </msg>
                    <msg time='2012-01-20T11:49:40.723-05:00' org_id='oracle' comp_id='tnslsnr'
                    type='UNKNOWN' level='16' host_id='ora.facilization.net'
                    host_addr='192.168.1.51'>
                    <txt>20-JAN-2012 11:49:40 * (CONNECT_DATA=(CID=(PROGRAM=)(HOST=__jdbc__)(USER=oracle))(SERVICE_NAME=orcl)(CID=(PROGRAM=)(HOST=__jdbc__)(USER=oracle))) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.1.53)(PORT=38489)) * establish * orcl * 12514
                    </txt>
                    </msg>
                    <msg time='2012-01-20T11:49:40.724-05:00' org_id='oracle' comp_id='tnslsnr'
                    type='UNKNOWN' level='16' host_id='ora.facilization.net'
                    host_addr='192.168.1.51'>
                    <txt>TNS-12514: TNS:listener does not currently know of service requested in connect descriptor
                    </txt>
                    </msg>
                    <msg time='2012-01-20T11:49:41.768-05:00' org_id='oracle' comp_id='tnslsnr'
                    type='UNKNOWN' level='16' host_id='ora.facilization.net'
                    host_addr='192.168.1.51'>
                    <txt>20-JAN-2012 11:49:41 * (CONNECT_DATA=(CID=(PROGRAM=)(HOST=__jdbc__)(USER=oracle))(SERVICE_NAME=orcl)(CID=(PROGRAM=)(HOST=__jdbc__)(USER=oracle))) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.1.53)(PORT=38490)) * establish * orcl * 12514
                    </txt>
                    </msg>
                    <msg time='2012-01-20T11:49:41.768-05:00' org_id='oracle' comp_id='tnslsnr'
                    type='UNKNOWN' level='16' host_id='ora.facilization.net'
                    host_addr='192.168.1.51'>
                    <txt>TNS-12514: TNS:listener does not currently know of service requested in connect descriptor
                    </txt>
                    </msg>
                    <msg time='2012-01-20T11:49:42.782-05:00' org_id='oracle' comp_id='tnslsnr'
                    type='UNKNOWN' level='16' host_id='ora.facilization.net'
                    host_addr='192.168.1.51'>
                    <txt>20-JAN-2012 11:49:42 * (CONNECT_DATA=(CID=(PROGRAM=)(HOST=__jdbc__)(USER=oracle))(SERVICE_NAME=orcl)(CID=(PROGRAM=)(HOST=__jdbc__)(USER=oracle))) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.1.53)(PORT=38491)) * establish * orcl * 12514
                    </txt>
                    </msg>
                    <msg time='2012-01-20T11:49:42.782-05:00' org_id='oracle' comp_id='tnslsnr'
                    type='UNKNOWN' level='16' host_id='ora.facilization.net'
                    host_addr='192.168.1.51'>
                    <txt>TNS-12514: TNS:listener does not currently know of service requested in connect descriptor
                    </txt>
                    </msg>
                    <msg time='2012-01-20T11:49:52.272-05:00' org_id='oracle' comp_id='tnslsnr'
                    type='UNKNOWN' level='16' host_id='ora.facilization.net'
                    host_addr='192.168.1.51'>
                    <txt>20-JAN-2012 11:49:52 * service_update * orcl * 0
                    </txt>
                    </msg>
                    • 7. Re: Oracle lsnrctl stops and hangs
                      sb92075
                      Erni Papakroni wrote:
                      These are some lines of log.xml file


                      host_addr='192.168.1.51'>
                      <txt>20-JAN-2012 11:49:22 * service_update * orcl * 0
                      </txt>
                      </msg>
                      <msg time='2012-01-20T11:49:40.723-05:00' org_id='oracle' comp_id='tnslsnr'
                      type='UNKNOWN' level='16' host_id='ora.facilization.net'
                      host_addr='192.168.1.51'>
                      <txt>20-JAN-2012 11:49:40 * (CONNECT_DATA=(CID=(PROGRAM=)(HOST=__jdbc__)(USER=oracle))(SERVICE_NAME=orcl)(CID=(PROGRAM=)(HOST=__jdbc__)(USER=oracle))) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.1.53)(PORT=38489)) * establish * orcl * 12514
                      </txt>
                      Above shows request for "SERVICE_NAME=orcl" but
                      Below shows that listener knows about Service "orcl.facilization.net"
                      Service "orcl.facilization.net" has 1 instance(s).
                      So modify tnsnames.ora file to request
                      SERVICE=orcl.facilization.net
                      • 8. Re: Oracle lsnrctl stops and hangs
                        Erni Papakroni
                        Ok ,
                        i changed it , now i can connect with ora.facilization.net but still seems slow
                        For example when i try to connect with rcu for creatin repositories the response is delayed with
                        10secs or more even for a simple connection , connection from navicat takes a lot and in the meantime
                        i exexute lsnrctl status from terminal it hangs a lot .
                        This is log.xml again

                        <msg time='2012-01-20T12:37:02.677-05:00' org_id='oracle' comp_id='tnslsnr'
                        type='UNKNOWN' level='16' host_id='ora.facilization.net'
                        host_addr='192.168.1.51'>
                        <txt>20-JAN-2012 12:37:02 * (CONNECT_DATA=(CID=(PROGRAM=)(HOST=__jdbc__)(USER=oracle))(service_name=ORCL.FACILIZATION.NET)(server=dedicated)) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.1.53)(PORT=39979)) * establish * ORCL.FACILIZATION.NET * 0
                        </txt>
                        </msg>
                        <msg time='2012-01-20T12:37:02.677-05:00' org_id='oracle' comp_id='tnslsnr'
                        type='UNKNOWN' level='16' host_id='ora.facilization.net'
                        host_addr='192.168.1.51'>
                        <txt>20-JAN-2012 12:37:02 * service_update * orcl * 0
                        </txt>
                        </msg>
                        <msg time='2012-01-20T12:37:02.678-05:00' org_id='oracle' comp_id='tnslsnr'
                        type='UNKNOWN' level='16' host_id='ora.facilization.net'
                        host_addr='192.168.1.51'>
                        <txt>20-JAN-2012 12:37:02 * service_update * orcl * 0
                        </txt>
                        </msg>
                        <msg time='2012-01-20T12:37:02.679-05:00' org_id='oracle' comp_id='tnslsnr'
                        type='UNKNOWN' level='16' host_id='ora.facilization.net'
                        host_addr='192.168.1.51'>
                        <txt>WARNING: Subscription for node down event still pending
                        </txt>
                        </msg>
                        <msg time='2012-01-20T12:37:02.679-05:00' org_id='oracle' comp_id='tnslsnr'
                        type='UNKNOWN' level='16' host_id='ora.facilization.net'
                        host_addr='192.168.1.51'>
                        <txt>20-JAN-2012 12:37:02 * (CONNECT_DATA=(CID=(PROGRAM=)(HOST=ora.facilization.net)(USER=oracle))(COMMAND=status)(ARGUMENTS=64)(SERVICE=LISTENER)(VERSION=186646784)) * status * 0
                        </txt>
                        </msg>
                        <msg time='2012-01-20T12:37:05.681-05:00' org_id='oracle' comp_id='tnslsnr'
                        type='UNKNOWN' level='16' host_id='ora.facilization.net'
                        host_addr='192.168.1.51'>
                        <txt>20-JAN-2012 12:37:05 * service_update * orcl * 0
                        </txt>
                        </msg>
                        <msg time='2012-01-20T12:37:11.689-05:00' org_id='oracle' comp_id='tnslsnr'
                        type='UNKNOWN' level='16' host_id='ora.facilization.net'
                        host_addr='192.168.1.51'>
                        <txt>20-JAN-2012 12:37:11 * service_update * orcl * 0
                        </txt>
                        </msg>
                        <msg time='2012-01-20T12:37:32.715-05:00' org_id='oracle' comp_id='tnslsnr'
                        type='UNKNOWN' level='16' host_id='ora.facilization.net'
                        host_addr='192.168.1.51'>
                        <txt>20-JAN-2012 12:37:32 * service_update * orcl * 0
                        </txt>
                        </msg>
                        • 9. Re: Oracle lsnrctl stops and hangs
                          sb92075
                          Erni Papakroni wrote:
                          Ok ,
                          i changed it , now i can connect with ora.facilization.net but still seems slow
                          I conclude there are problem at the OS/Networking level
                          & Oracle is the victim; not the culprit.

                          You are free to blame Oracle but realize that many, Many, MANY other sites do not suffer what you report.

                          Best Wishes at solving your mystery!
                          • 10. Re: Oracle lsnrctl stops and hangs
                            Erni Papakroni
                            I'm not blaming Oracle ,
                            i just need suggestions .
                            • 11. Re: Oracle lsnrctl stops and hangs
                              sb92075
                              Erni Papakroni wrote:
                              I'm not blaming Oracle ,
                              Could have fooled me!
                              I'm 100% sure that it's listeners
                              consider packet sniffing
                              • 12. Re: Oracle lsnrctl stops and hangs
                                Billy~Verreynne
                                Erni Papakroni wrote:

                                i changed it , now i can connect with ora.facilization.net but still seems slow
                                For example when i try to connect with rcu for creatin repositories the response is delayed with
                                10secs or more even for a simple connection , connection from navicat takes a lot and in the meantime
                                i exexute lsnrctl status from terminal it hangs a lot .
                                Sounds like networking issues - before blaming the Listener, check the health of the network. The symptoms you are describing sounds like network issues.

                                The Listener will usually only be slow when it takes time to hand-off a connection to a dispatcher or dedicated server process - and that would not be a Listener issue specifically, but an o/s or instance related one.

                                But from the sounds of it.. and seeing that you're virtualising the platforms and network... it does not sound like a Listener specific issue at all. And no evidence has been posted so far to convince me that it could be.
                                • 13. Re: Oracle lsnrctl stops and hangs
                                  Erni Papakroni
                                  Thank you guys
                                  Now it's working evereything good
                                  Schemas are created in less than 2 minutes from RCU


                                  Regards Erni