Oracle Analytics and AI

OBIEE 12c will not start after configuring SSL on WebLogic

Hello All,

I recently configured the SSL in our OBIEE 12c environment as per the Oracle documentation - OBIEE 12c: How To Configure SSL Including Examples (Doc ID 2188982.1) using custom certificates. After configuration when i tried to restart the BI services using ./start.sh command, I get the following error.

Starting domain; Using domainHome: /opt/oracle/obiee_domains/fibbi ...

Initializing WebLogic Scripting Tool (WLST) ...

Welcome to WebLogic Server Administration Scripting Shell

Type help() for help on available commands

<Mar 4, 2020 6:17:42 PM MST> <Info> <Security> <BEA-090906> <Changing the default Random Number Generator in RSA CryptoJ from ECDRBG128 to HMACDRBG. To disable this change, specify -Dweblog

<Mar 4, 2020 6:17:42 PM MST> <Info> <Security> <BEA-090909> <Using the configured custom SSL Hostname Verifier implementation: weblogic.security.utils.SSLWLSHostnameVerifier$NullHostnameVer

Node manager not running. Starting it...

NMProcess: NODEMGR_HOME is already set to /opt/oracle/obiee_domains/fibbi/nodemanager

NMProcess: CLASSPATH=/opt/java/jdk1.8.0_221/lib/tools.jar:/opt/oracle/middleware/wlserver/server/lib/weblogic.jar:/opt/oracle/middleware/wlserver/../oracle_common/modules/thirdparty/ant-conmodules/features/oracle.wls.common.nodemanager.jar::/opt/java/jdk1.8.0_221/lib/tools.jar:/opt/oracle/middleware/wlserver/modules/features/wlst.wls.classpath.jar:/opt/oracle/middleware/wlserfeatures/oracle.wls.common.grizzly.jar

NMProcess: + /opt/java/jdk1.8.0_221/bin/java -server -Xms32m -Xmx200m -Djdk.tls.ephemeralDHKeySize=2048 -Dcoherence.home=/opt/oracle/middleware/wlserver/../coherence -Dbea.home=/opt/oracle/racle/obiee_domains/fibbi/nodemanager -DLogToStderr=false -DQuitEnabled=true -Dweblogic.RootDirectory=/opt/oracle/obiee_domains/fibbi -Doracle.security.jps.config=/opt/oracle/obiee_domains/mon.components.home=/opt/oracle/middleware/oracle_common -Dopss.version=12.2.1.3 -Dweblogic.RootDirectory=/opt/oracle/obiee_domains/fibbi -Doracle.bi.home.dir=/opt/oracle/middleware/bi -Dorbbi/config/fmwconfig/biconfig -Doracle.bi.environment.dir=/opt/oracle/obiee_domains/fibbi/config/fmwconfig/bienv -Doracle.bi.12c=true -Ddomain.home=/opt/oracle/obiee_domains/fibbi -Dfile.encle.classloader.weblogic.LaunchClassLoader -Djava.security.policy=/opt/oracle/middleware/wlserver/server/lib/weblogic.policy -Dweblogic.nodemanager.JavaHome=/opt/java/jdk1.8.0_221 weblogic.

NMProcess: Mar 04, 2020 6:17:46 PM oracle.security.opss.internal.runtime.ServiceContextManagerImpl getContext

NMProcess: WARNING: Bootstrap services are used by OPSS internally and clients should never need to directly read/write bootstrap credentials. If required, use Wlst or configuration managem

NodeManager started

Reading domain...

/Servers/AdminServer/ListenPort=9500

Accessing admin server using URL t3s://<myserver>:9501

Starting AdminServer ...

nmStart(AdminServer) succeeded

Start Admin Server connect Exception caught Error occurred while performing connect : Cannot connect via t3s or https. If using demo certs, verify that the -Dweblogic.security.TrustKeyStoreinitialize JNDI context, tried 2 time or times totally, the interval of each time is 0ms.

t3s://vmtnofsaarpt01.fib.system.root:9501: Destination <myserverip>, 9501 unreachable.; nested exception is:

java.net.ConnectException: Connection refused (Connection refused); No available router to destination.; nested exception is:

java.rmi.ConnectException: No available router to destination.

Use dumpStack() to view the full stacktrace :

Reading domain...

/Servers/AdminServer/ListenPort=9500

Accessing admin server using URL t3s://<myserver>:9501

Requesting credentials ...

Enter Weblogic login details at prompt

Weblogic Username:

The nodemanager log shows the following:

weblogic.nodemanager.common.ConfigException: Key store identity alias does not contain a certificate chain: weblogic

at weblogic.nodemanager.server.SSLConfig.loadKeyStoreConfig(SSLConfig.java:239)

at weblogic.nodemanager.server.SSLConfig.access$000(SSLConfig.java:33)

at weblogic.nodemanager.server.SSLConfig$1.run(SSLConfig.java:118)

at java.security.AccessController.doPrivileged(Native Method)

at weblogic.nodemanager.server.SSLConfig.<init>(SSLConfig.java:115)

at weblogic.nodemanager.server.NMServer.<init>(NMServer.java:169)

at weblogic.nodemanager.server.NMServer.getInstance(NMServer.java:134)

at weblogic.nodemanager.server.NMServer.main(NMServer.java:589)

at weblogic.NodeManager.main(NodeManager.java:31)

The certificate imported on the keystore is this one:

$ keytool -list -v -keystore keystore.jks

Enter keystore password:

Keystore type: PKCS12

Keystore provider: SUN

Your keystore contains 1 entry

Alias name: weblogic

Creation date: Mar 4, 2020

Entry type: PrivateKeyEntry

Certificate chain length: 1

Certificate[1]:

Owner: CN=<myserver>, OU=Unknown, O=<myorg>, L=<mylocation>, ST=<mystate>, C=US

Issuer: CN=<myserver>, OU=Unknown, O=<myorg>, L=<mylocation>, ST=<mystate>, C=US

Serial number: 1e28f3e

Valid from: Wed Oct 09 15:19:27 MDT 2019 until: Thu Oct 08 15:19:27 MDT 2020

Certificate fingerprints:

.....

Signature algorithm name: SHA256withRSA

Subject Public Key Algorithm: 2048-bit RSA key

Version: 3

I am reading something about Certificate Root, Intermediary, etc. Not sure if that's the cause of the problem, but, I don't really understand that concept of root, intermediary certificates.

Please advise, I am stuck on this and can't find any specific documentation that has anything additional to do.

Thanks!

Find more posts tagged with

OBIEE

Accepted answers

All comments

[Deleted User]

Eduardo,

Which precise OBI? Which precise Operating System? Have you first tested it on ONLY the Managed Server before moving on? that's normally the first test you do beore moving on to the Admin Server and potentially destroy your system.

SSL is pretty tricky so without an exact specification of what you have done it will be nigh impossible to help you.

Eduardo Ferrari

OBIEE 12.2.1.x and RHEL 7

Not sure how does that make a difference though, as the Oracle document ID I provided you was specifically to my system.

You make a good point about the Node Manager, but the Oracle instructions obviously don’t state that.

I have a feeling that might be my certificate that it does not “go all the way up to the root”. I am currently checking that with the team which generates the certificates.

[Deleted User]

Eduardo Ferrari wrote:OBIEE 12.2.1.x and RHEL 7Not sure how does that make a difference though

Your so called ".x" omits just about any useful information you could have possibly given because that's the only part with which the - so far - 30 releases of the 12c family distinguish themselves and are ALL called "12.2.1...".

And I hate to say it, but: software changes over the span of five years and there where literally thousands of bug fixes. It's how IT works.

Eduardo Ferrari wrote:as the Oracle document ID I provided you was specifically to my system.

Again, sorry, but it isn't and working with such an assumption is dangerous. Believe me, many cases posted in here have people claiming they "have done everything" and "were precise" just to end up with "oh right, this was wrong". Precision helps solving issues. Being vague wastes everybody's time and makes threads unusable for other users.

Eduardo Ferrari wrote:You make a good point about the Node Manager, but the Oracle instructions obviously don’t state that.

Of course it doesn't because it's written for ONE specific case that worked. The other 99'000 flavours of the use case which can differ ever so slightly aren't covered.

Eduardo Ferrari wrote:I have a feeling that might be my certificate that it does not “go all the way up to the root”. I am currently checking that with the team which generates the certificates.

Yes. The certificate are always the absolute crux. I had huge issues when following 2188982.1 step by step or trying to help clients who followed it and maneuvered themselves into a dead-end where they had to set up the machines again. 2188982.1 simply doesn't ever tell you WHY you do WHAT and how you should do it differently in different cases and how it can impact the solution.

Is there one certificate chain? Chain with key? Singular, distinct certificates?

Did you import it all into the original keystore?

etc etc

As I said it's not a straightforward thing and hard to ask the exact question that leads to the solution, especially if you're (me in this case) isn't sitting in front of the system.

SonPat99

Generally, when you get a certificate, you extract the Root, Intermediate and Personal Certificate, install them and then import them into your identity.

You should see 3 entries in your identity: 1 for Root, 1 for Intermediate and 1 for Personal.

Once all the steps have been covered, you should start the NODEMANAGER Standalone to see if it is pulling all the configurations and is not complaining about any of the changes.

Regarding, the doc, I would suggest you should try and refer a few more docs as this doc is not 100%. Its at very high level.

[Deleted User]

SonPat99 wrote:Generally, when you get a certificate, you extract the Root, Intermediate and Personal Certificate, install them and then import them into your identity.

Not necessarily. It depends on what you are being provided with by your security department.

SonPat99 wrote:Once all the steps have been covered, you should start the NODEMANAGER Standalone to see if it is pulling all the configurations and is not complaining about any of the changes.

No. Doing "all steps" is definitely an error. Only ever configure one managed server to use SSL and test it. Do not touch any other component until you have completely proven that it works.

Eduardo Ferrari

After I got the correct certificate, then it worked.

User_WUVLD

Eduardo Ferrari Please help me

I create new keystore to configure SSL . I do step by step

keytool -genkeypair
keytool -certreq
java utils.CertGen
java utils.ImportPrivateKey
cteare Trust.jks from standard java Trust
keytool -import -v -trustcacert

everything is ok

But finally I check chainhing of certificate it is incomplete

java utils.ValidateCertChain

Certificate chain is incomplete, can't confirm the entire is valid

How to You got the correct certificate ?

Thanks so much

Eduardo Ferrari

Are you using a self-signed cert? It seems like a problem on the cert you have. You must have (in most cases) 3 certs if you are not using self-signed.