Skip to Main Content
Forums

Infrastructure Software

Announcement

For appeals, questions and feedback about Oracle Forums, please email oracle-forums-moderators_us@oracle.com. Technical questions should be asked in the appropriate category. Thank you!

Failed to connect to Web Service API

TakeTheLongPathAug 19 2015 — edited Aug 21 2015

I have switched my OVMM (3.3.3) weblogic server to using a non-self-signed SSL certificate.

I am able to get into Weblogic console. But the OVM console and CLI have errors.

Log entry:

==> AdminServer.out <==

<2015-08-19T10:08:13.812-0500> <Error> <com.oracle.ovm.appfw.coreinterface.ConnectionManager> <BEA-000000> <AppFw session 1: Failed to connect to Web Service API.

com.sun.jersey.api.client.ClientHandlerException: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

  at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151)

  at com.sun.jersey.api.client.Client.handle(Client.java:648)

  at com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)

  at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)

  at com.sun.jersey.api.client.WebResource$Builder.post(WebResource.java:558)

  Truncated. see log file for complete stacktrace

Caused By: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

  at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)

  at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1904)

  at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:279)

  at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:273)

  at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1446)

  Truncated. see log file for complete stacktrace

Caused By: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

  at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385)

  at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292)

  at sun.security.validator.Validator.validate(Validator.java:260)

  at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326)

  at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:231)

  Truncated. see log file for complete stacktrace

Caused By: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

  at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196)

  at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268)

  at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380)

  at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292)

  at sun.security.validator.Validator.validate(Validator.java:260)

  Truncated. see log file for complete stacktrace

>

Steps to get here:

1. I have a private key and certificate generated using OpenSSL because I need PEM version to use with Apache as well.

2. I import the private key and certificate into a brand new keystore using an ImportKey.class file (Import private key and certificate into Java Key Store (JKS))

3. Import the CA for the certificate above into jks and mark as trusted

4. copy the new jks to   /u01/app/oracle/ovm-manager-3/domains/ovm_domain/security/

5. Set the ENV variables:

export JAVA_HOME=/u01/app/oracle/java/
export WL_HOME=/u01/app/oracle/Middleware/wlserver_10.3/
export MW_HOME=/u01/app/oracle/Middleware

6. run script provided to "setsslkey" to the one in the jks

"  /u01/app/oracle/ovm-manager-3/ovm_upgrade/bin/ovmkeytool.sh setsslkey "

Was successful.

Restarted OVMM service and connected to weblogic and OVM console and the Certificate is green! Success!

Able to log in to weblogic console.

Not able to log in to the OVMM console.

Google showed me a blog and script to run to configure the client certs:

/u01/app/oracle/ovm-manager-3/bin/configure_client_cert_login.sh from Harri's Oracle Technology Blog: OracleVM 3.3.1 and External Authentication

This completes -- this solved an error before when I had to restore from a backup after a failed update.

Still get the errors above.

I have imported my host certificate and my CA's certificate into the ovmtrust JKS file in /u01/app/oracle/ovm-manager-3/domains/ovm_domain/security/

(because I know the passwords to them, having previously run the re-create-all script...)
This post has been answered by TakeTheLongPath on Aug 21 2015
Jump to Answer

Comments

843790
Hello,

No answer at all so far... I'm wondering, perhaps am I asking for something stupid?

In the meantime, I've implemented a working workaround to manually resolve all names against the right server at the right time, but I have the feeling I shouldn't have to resort to such hacks. Any comments?

Thanks in advance.
jtahlborn
http://java.sun.com/j2se/1.5.0/docs/guide/net/properties.html
843790
Apart from the 3 JNDI DNS service provider settings properties which I didn't know about and are interesting, the informations in this link do not seem relevant to the problem described above: why does Java names resolution break if the system's DNS server to use change while the application is running?
jtahlborn
Oh, i didn't notice that your code was already setting the ttl properties. did you test your code setting those properties on the command line? some properties do not have any affect if set at runtime.
843790
The problem described here is not about DNS results caching. It's about some kind of broken "DNS server to query" caching.

Please re-read the whole stuff. I know I probably made it too long, but I wanted to be as precise and complete as possible.

Thanks.
jtahlborn
I understand what you are saying. what i'm saying, is that your test code does not prove that it is a "dns server caching" problem instead of a simple "dns lookup caching" problem. see the referenced bug report for many complaints that java does not honor these settings when set prgrammatically, as well as comments about using sun specific properties instead of the "generic" ones:

http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6247501

if you can prove that the dns lookup is still happening incorrectly after the server change, then it really is a "dns server caching problem". then you might want to try futzing with some of the "*.nameservice.*" properties.
843790
Ok, I understand your point.

This very same test case can also be used in the "real" situation (which I obviously did), thus proving that it is not a "dns lookup caching" problem.

1) Connect the system to the LAN (and only to the LAN, no ppp link at this point). Check that network access and DNS are OK from command line. Start the test case: names are resolved as expected
2) Bring up the the ppp link, which will become the default route, and also update /etc/resolv.conf. Do not disconnect the LAN: names are still resolved so far.
3) Now unplug the LAN cable. Check from command line that system-wide network access and DNS is still fine (it should be). Check the test-case output: name resolution is now failing. As we get a failure, and we were getting a result earlier, it is safe to tell that positive DNS lookups are not cached.
4) Reconnect the LAN cable (not touching the ppp link). Names will be successfully resolved again. As we now get results and were previously getting failure, it is also safe to tell that negative DNS lookups are not cached either

My conclusions:
a) There is no dns lookup caching taking place in this case, as expected
b) For the whole run duration, java tries to resolve names against the LAN's DNS server (the one in use at the beginning), despite the system configuration changing mid-run. This is an obvious deduction as we see that name resolution fails at step 3), but works again at step 4), just by restoring physical network access to this DNS server.

Do you better see the point?
jtahlborn
Yep.

Have you looked at any of the nameservice properties. i think with one you can specify a list of dns servers. you could try specifying the lan and ppp dns servers in that property and see if that works (obviously not ideal solution).

i looked into the java code which instantiates the nameserver (at least in jdk 1.6), and it looks like it caches the data from the resolv.conf for 300 seconds (and there is no way to change this config). looks like the only way to get a faster reaction would be to find an alternate NameServer implementation (you can specify your own impl with sun.net.spi.nameservice.provider).
843790
Well, the nameservice property allowing to specify my DNS server of choice is interesting (I didn't know about those), but not very usable, as the DNS server to use is not known in advance (it may be whatever the ISP tells me to use at the time where the ppp link gets established). So no real solution there.

The fact that it caches /etc/resolv.conf content for 300s is very interesting, but having no possibility to impact on this duration really is a pity. There should be some kind of property to fix this behaviour. So as you say, a custom provider may be the only solution.

So far, the hack I use to get this working is based on code similar to this one (it is presented here in a similar form than my test case above). Obviously, reading the /etc/resolv.conf for each dns resolution is not an option in a real environment, but you get the idea.
package com.test.dnsresolver;

import java.io.BufferedReader;

public class DnsResolver {
	private static final String urlString = "www.google.com";
	private static final String resolvConf = "/etc/resolv.conf";

	public static void main(String[] args) {

		int loopCounter = 0;
		while (true) {
			loopCounter++;
			try {
				Thread.sleep(1000);
			} catch (InterruptedException e) {}

			// Parse the current DNS server to be used in the config
			String nameserver = null;
			try {
				BufferedReader input =  new BufferedReader(new FileReader(new File(resolvConf)));
				String currentLine = null;
				while (( currentLine = input.readLine()) != null){
					// Take care of potential comments
					currentLine = currentLine.substring(0, currentLine.indexOf("#") == -1
							? currentLine.length() : currentLine.indexOf("#") );

					if (currentLine.contains("nameserver")) {
						// It is the line we are looking for
						nameserver = currentLine;
						break;
					}
				}
			} catch (FileNotFoundException e) {
				System.out.println("Loop " + loopCounter + ": FileNotFoundException");
			} catch (IOException e) {
				System.out.println("Loop " + loopCounter + ": IOException");
			}

			if (nameserver == null) {
				// No "nameserver" line found
				System.out.println("Loop " + loopCounter + ": No nameserver found in configration file!");
				continue;
			}
			
			// Trim it to just contain the IP address
			nameserver = (nameserver.replace("nameserver", "")).trim();
			System.out.println("Loop " + loopCounter + ": Going to use DNS server " + nameserver);

			// At this point, we know which server to use, now perform the resolution
			Hashtable<String, String> env = new Hashtable<String, String>();
			env.put("java.naming.factory.initial", "com.sun.jndi.dns.DnsContextFactory");
			env.put("java.naming.provider.url",    "dns://" + nameserver);

			DirContext ictx;
			try {
				ictx = new InitialDirContext(env);
				Attributes attrs1 = ictx.getAttributes(urlString, new String[] {"A"});
				System.out.println("Loop " + loopCounter + ": Manual resolution: +" + attrs1.get("a").get() + "+");

			} catch (NamingException e) {
				System.out.println("Loop " + loopCounter + ": NamingException");
			}
		}
	}
}
So maybe I should adapt and package this into a proper provider and specify it for sun.net.spi.nameservice.provider. Any link, info or example about how a proper provider should look like?

Thanks for your advices!
jtahlborn
Well, i looked at the implementation sun provides in openjdk 1.6. if you (or your company) are ok with open source licenses (GPL), you should be able to just copy that one and modify it to suit your needs. my thought was changing the implementation to check the timestamp on the resolv.conf file, and reload it whenever the timestamp changes (File.lastModified()).
1 - 10
Locked Post
New comments cannot be posted to this locked post.

Post Details

Locked on Sep 18 2015
Added on Aug 19 2015
#oracle-virtualization, #oracle-vm-server-for-x86, #ovm, #ssl
2 comments
3,619 views