I'm attempting to do work necessary for an application to be FIPS 140-2 compliant in preparation for testing submission. I'm currently having a problem when the application attempts to access a remote server using an HttpsURLConnection.
I've setup the JRE to use the SunPKCS11 JSSE provider with Mozilla NSS as the security token. I've already setup the NSS crypto db to be running in a FIPS mode. I had to disable some code in the HttpsURLConnection client that uses a custom TrustManager implementation to accept self-signed certs. Even after doing this, the point where the app attempts to connect to the remote server using an HttpsURLConnection, a java.security.KeyManagementException is through.
I've looked through the code in com.sun.net.ssl.internal.ssl.SSLContextImpl.chooseTrustManager, and it throws that error in FIPS mode if the SSLContext's trust manager is not an instanceof X509TrustManagerImpl. I've tried to explicitly initialize this class in an attempt to circumvent the mechanism by which JSSE chooses a trust manager, but that is not allowed by the API.
At this point, what I'm looking for is a definitive way to ensure that my JRE's java.security configuration or any property based configuration will result in JSSE selecting the com.sun.net.ssl.internal.ssl.X509TrustManagerImpl trust manager that is checked for by com.sun.net.ssl.internal.ssl.SSLContextImpl#chooseTrustManager when running in FIPS mode.