5 Replies Latest reply: Jul 10, 2013 5:11 PM by chris_here RSS

    ORA-29003 when checking server SSL certificate

    chris_here

      Hi there,

       

      I'm bumping into an annoying issue when trying to configure SSL authentication on an 11gR2 database on RedHat Linux 6.3 64bit. The database is a physical standby instance in an Active Dataguard configuration, in case that's relevant to the case. Everything goes ok up to a point:

      bash-4.1$ sqlplus /@smsgadm1
      SQL*Plus: Release 11.2.0.3.0 Production on Sun Jun 30 21:51:45 2013
      Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
      SQL>

       

      until I switch to SSL_SERVER_DN_MATCH = TRUE in the client's sqlnet.ora; then I get:

      bash-4.1$ sqlplus /@smsgadm1
      SQL*Plus: Release 11.2.0.3.0 Production on Sun Jun 30 22:07:23 2013
      ERROR: ORA-29003: SSL transport detected mismatched server certificate.
      Enter user-name:
      

       

      The relevant part of the client's tnsnames.ora is as follows; the DN string is a pure copy/paste from the output of "orapki wallet display".

      SMSGADM1 =
        (DESCRIPTION =
          (ADDRESS = (PROTOCOL = TCPS)(HOST = admnode)(PORT = 2484))
          (CONNECT_DATA = (SERVER = DEDICATED)(SERVICE_NAME = smsgadm1))
          (SECURITY = (SSL_SERVER_CERT_DN="CN=smsgw,OU=etc.."))
        )
      

       

      The documentation  mentions that the CN field should contain the global database name ("Server DN matching prevents the database server from faking its identity to the client during connections by matching the server's global database name against the DN from the server certificate."), which in my case is "smsgw". However I have also tried using the database SID "smsgadm1" to no avail, i.e., I get the exact same result. I've also tried to follow this recent post step by step without more success.

      SQL> show parameter name
      NAME                                 TYPE        VALUE
      ------------------------------------ ----------- ------------------------------
      db_name                              string      SMSGW
      db_unique_name                       string      SMSGADM1
      instance_name                        string      smsgadm1
      
      SQL> show parameter domain
      NAME                                 TYPE        VALUE
      ------------------------------------ ----------- ------------------------------
      db_domain                            string
      
      SQL> select * from global_name;
      GLOBAL_NAME
      --------------------------------------------------------------------------------
      SMSGW
      

       

      The most puzzling thing is that this ORA-29003 error returns extremely few links on Google, so this seems to be a very uncommon error. I'm totally at a loss here about what may be going on. Has anybody got an idea? Or knows of a way to get more information from the client about what it's trying to match the DN against?

       

      Thanks for your help,

      Chris

        • 1. Re: ORA-29003 when checking server SSL certificate
          DK2010

          Hi,

           

          Have you checked the MOS doc

          Oracle Advanced Security SSL Troubleshooting Guide [ID 166492.1]

           

          HTH

          • 2. Re: ORA-29003 when checking server SSL certificate
            chris_here

            Hi DK,

             

            thanks for the answer. In fact if I look at the configuration everything seems to be as expected. To clarify the setup a bit, I'm using for these tests a Grid home as server and the corresponding Database home on the same host as a client.

             

            On the server side, the listener configuration is as follows:

            > srvctl config listener
            Name: LISTENER
            Home: /opt/grid
            End points: TCP:1521/TCPS:2484
            

             

            In /opt/grid/network/admin/listener.ora:

            WALLET_LOCATION =
            (SOURCE =
              (METHOD = FILE)
              (METHOD_DATA = (DIRECTORY = /opt/oracle/admin/wallet/smsgw)))
            

             

            > orapki wallet display -wallet /opt/oracle/admin/wallet/smsgw

            Oracle PKI Tool : Version 11.2.0.3.0 - Production
            User Certificates:
            Subject:        CN=smsgw,OU=blah,O=blah,C=blah,C=blah
            

             

            On the client side, in /opt/oracle/db11gr2/admin/network/sqlnet.ora:

            SSL_VERSION = 3.0
            SSL_CLIENT_AUTHENTICATION = TRUE
            SSL_SERVER_DN_MATCH = TRUE
            
            WALLET_LOCATION =
              (SOURCE =
                (METHOD = FILE)
                (METHOD_DATA =
                  (DIRECTORY = /opt/oracle/admin/wallet/client)))
            

             

            In tnsnames.ora:

            SMSGW =
              (DESCRIPTION =
                (ADDRESS = (PROTOCOL = TCPS)(HOST = admnode)(PORT = 2484))
                (CONNECT_DATA = (SERVER = DEDICATED)(SERVICE_NAME = smsgadm1))
                (SECURITY = (SSL_SERVER_CERT_DN="CN=smsgw,OU=blah,O=blah,L=blah,C=blah"))
              )
            

             

            The DN string was copied using copy/paste (right-click in Putty) and contains no special characters, only pure 7-bit ASCII. I even updated the "rdbms_server_dn" parameter in the database, although it doesn't apply here from my understanding (it's only relevant to the Enterprise Security deployment):

            SQLPLUS> show parameter dn
            NAME                                 TYPE        VALUE
            ------------------------------------ ----------- ------------------------------
            rdbms_server_dn                      string      CN=smsgw,OU=blah,O=blah,L=blah,C=blah
            

             

            I have no idea what the issue might be. I see that in several examples the DB string contains the entry "CN=OracleContext", is this setting mandatory? Does it mean anything special? (Although I have also tested with it without success either.

             

            Thanks for any ideas!

            Chris

            • 3. Re: ORA-29003 when checking server SSL certificate
              DK2010

              Hi,

              do you have server side sqlnet.ora, what about that Entry. make it simmiler with client

              can you try to set SSL_SERVER_CERT_DN as in doc.: Configuring Secure Sockets Layer Authentication

              • 4. Re: ORA-29003 when checking server SSL certificate
                chris_here

                Ok the plot thickens... I was about to post yesterday that I had found the solution, but as it appears it's not complete.

                 

                First thing first, here is the server-side sqlnet.ora that I did not include in my previous post:

                ADR_BASE = /opt/oracle 
                NAMES.DIRECTORY_PATH = (TNSNAMES, EZCONNECT)
                
                SQLNET.ENCRYPTION_SERVER = REJECTED 
                SQLNET.CRYPTO_CHECKSUM_SERVER = REJECTED 
                
                SSL_VERSION = 3.0 
                SSL_CLIENT_AUTHENTICATION = TRUE 
                SSL_CIPHER_SUITES= (SSL_RSA_WITH_AES_256_CBC_SHA) 
                
                WALLET_LOCATION = 
                (SOURCE = 
                  (METHOD = FILE) 
                  (METHOD_DATA = (DIRECTORY = /opt/oracle/admin/wallet/smsgw))) 
                

                 

                I was trying to use SSL3.0 because I had noted earlier that asking for TLS 1.0 by setting SSL_VERSION=3.1 in both server and client sqlnet.ora would fail with "ORA-12560: TNS:protocol adapter error". Then reading the documentation a little harder as suggested, I ended up on this annex which states:

                "Note that the cipher suites that use Advanced Encryption Standard (AES) work with Transport Layer Security (TLS 1.0) only."

                 

                So this was obviously at least part of my problem. Googling a little more I found this post which explains what happens and the solution. Bug 9682150 : SSL_VERSION=3.1 IS CAUSING ORA-12560 IN SSL AUTHENTICATION prevents from explicitly setting SSL_VERSION to 3.1 in sqlnet.ora. Simply commenting out the setting should solve the issue since TLS 1.0 is going to end up being used anyway as it's the only protocol that's compatible with the required cipher. So did I:

                 #SSL_VERSION = 3.1
                

                 

                and indeed the connection now works:

                -bash-4.1$ sqlplus /@smsgw
                SQL*Plus: Release 11.2.0.3.0 Production on Sat Jul 6 12:49:36 2013
                Connected to:
                Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
                With the Partitioning option
                >
                

                 

                However the crazy thing is that it only works for a minute or so. After that amount of time, without any other intervention on my part, I get the same error again:

                -bash-4.1$ sqlplus /@smsgw
                SQL*Plus: Release 11.2.0.3.0 Production on Sat Jul 6 13:03:52 2013
                ERROR:
                ORA-29003: SSL transport detected mismatched server certificate.
                Enter user-name:
                

                 

                and the sqlnet trace shows this:

                2013-07-06 13:03:52.453207 : ntzgpcd:entry
                2013-07-06 13:03:52.453232 : ntzgpcd:found value in connect data for "SSL_SERVER_CERT_DN": "CN=smsgw,OU=blah,O=blah,L=blah,C=blah"
                2013-07-06 13:03:52.453250 : ntzgpcd:exit
                2013-07-06 13:03:52.453292 : ntzCheckServerDN:returning NZ error 29003 in result structure
                2013-07-06 13:03:52.453332 : ntzCheckServerDN:failed with error 540
                2013-07-06 13:03:52.453353 : ntzCheckServerDN:exit
                2013-07-06 13:03:52.453372 : ntzcontrol:failed with error 540
                2013-07-06 13:03:52.453390 : ntzcontrol:exit
                2013-07-06 13:03:52.453408 : nszntcontrol:operation not supported
                2013-07-06 13:03:52.453425 : nszntcontrol:exit
                

                 

                Each time I restart the listener

                -bash-4.1$ srvctl stop listener
                -bash-4.1$ srvctl start listener
                

                 

                the connection works for a couple minutes then fails again.

                Help!

                • 5. Re: ORA-29003 when checking server SSL certificate
                  chris_here

                  Some more information about the problem: I came to realize that the SSL authentication works when using the statically registered services, and fails when using a service that registered dynamically with the listener. The cause of the behavior I described above is that after the listener is restarted, it takes about 20s for pmon to register the services, after which service smsgadm1 gets an additional dynamic entry, which makes SSL authentication fail.

                   

                  -bash-4.1$  lsnrctl status
                  LSNRCTL for Linux: Version 11.2.0.3.0 - Production on 10-JUL-2013 21:14:22
                  [...]
                  Service "smsgadm1" has 2 instance(s).
                    Instance "smsgadm1", status UNKNOWN, has 1 handler(s) for this service...
                    Instance "smsgadm1", status READY, has 1 handler(s) for this service... <- this dynamic entry breaks SSL
                  

                   

                  So the question now is: does anybody has an idea about why would SSL server authentication work with static services and not dynamic ones?

                   

                  Any help is greatly appreciated!

                  Chris