Oracle Business Intelligence

Products Banner

OBIEE 12.2.1.4 doesnt start

Received Response
167
Views
14
Comments

We have two node test environment. It has been working for over 4 years

But yesterday after server restart it didnt started.

Main error

"Unable to connect to NodeManager on host: ANALYTICHYP, due to Error occurred whi

le performing nmConnect : Cannot connect to Node Manager"

Starting stack

C:\Users\admin>C:\Oracle\Middleware\Oracle_Home\bi\modules\oracle.bi.sysman\scri

pts\start.cmd

BI_PRODUCT_HOME set as C:\Oracle\Middleware\Oracle_Home\bi\

ORACLE_HOME set as C:\Oracle\Middleware\Oracle_Home

Starting domain; Using domainHome: C:\Oracle\Middleware\Oracle_Home\bi\bin\..\..

\user_projects\domains\bi_domain ...


Initializing WebLogic Scripting Tool (WLST) ...


Welcome to WebLogic Server Administration Scripting Shell


Type help() for help on available commands


<26.02.2024 20:30:33> <Info> <Security> <BEA-090905> <Disabling the CryptoJ

JCE Provider self-integrity check for better startup performance. To enable this

 check, specify -Dweblogic.security.allowCryptoJDefaultJCEVerification=true.>

<26.02.2024 20:30:33> <Info> <Security> <BEA-090906> <Changing the default R

andom Number Generator in RSA CryptoJ from ECDRBG128 to HMACDRBG. To disable thi

s change, specify -Dweblogic.security.allowCryptoJDefaultPRNG=true.>

<26.02.2024 20:30:33> <Info> <Security> <BEA-090909> <Using the configured c

ustom SSL Hostname Verifier implementation: weblogic.security.utils.SSLWLSHostna

meVerifier$NullHostnameVerifier.>

Node manager not running. Starting it...

NMProcess: NODEMGR_HOME is already set to C:\Oracle\Middleware\Oracle_Home\user_

projects\domains\bi_domain\nodemanager

NMProcess: CLASSPATH=.;C:\Java\jdk1.8.0_202\lib\tools.jar;C:\Oracle\Middleware\O

racle_Home\wlserver\server\lib\weblogic.jar;C:\Oracle\Middleware\Oracle_Home\ora

cle_common\modules\thirdparty\ant-contrib-1.0b3.jar;C:\Oracle\Middleware\Oracle_

Home\wlserver\modules\features\oracle.wls.common.nodemanager.jar;C:\Java\jdk1.8.

0_202\lib\tools.jar;C:\Oracle\Middleware\Oracle_Home\wlserver\modules\features\w

lst.wls.classpath.jar;C:\Oracle\Middleware\Oracle_Home\wlserver\modules\features

\oracle.wls.common.grizzly.jar;C:\Oracle\Middleware\Oracle_Home\oracle_common\mo

dules\oracle.jps\jps-manifest.jar

NMProcess:

NMProcess: C:\Oracle\Middleware\Oracle_Home\user_projects\domains\bi_domain\node

manager>"C:\Java\jdk1.8.0_202\bin\java.exe" -server -Xms1024m -Xmx2048m -Djdk.t

ls.ephemeralDHKeySize=2048 -Dcoherence.home=C:\Oracle\Middleware\Oracle_Home\coh

erence -Dbea.home=C:\Oracle\Middleware\Oracle_Home -DNodeManagerHome=C:\Oracle\M

iddleware\Oracle_Home\user_projects\domains\bi_domain\nodemanager -DLogToStderr=

false -DQuitEnabled=true -Dweblogic.RootDirectory=C:\Oracle\Middleware\Oracle_Ho

me\user_projects\domains\bi_domain -Doracle.security.jps.config=C:\Oracle\Middl

eware\Oracle_Home\bi\bin\..\..\user_projects\domains\bi_domain\config\fmwconfig\

jps-config-jse.xml -Dcommon.components.home=C:\Oracle\Middleware\Oracle_Home\ora

cle_common -Dopss.version=12.2.1.3 -Dweblogic.RootDirectory=C:\Oracle\Middleware

\Oracle_Home\user_projects\domains\bi_domain  -Doracle.bi.home.dir=C:\Oracle\Mi

ddleware\Oracle_Home\bi -Doracle.bi.config.dir=C:\Oracle\Middleware\Oracle_Home\

bi\bin\..\..\user_projects\domains\bi_domain\config\fmwconfig\biconfig -Doracle.

bi.environment.dir=C:\Oracle\Middleware\Oracle_Home\bi\bin\..\..\user_projects\d

omains\bi_domain\config\fmwconfig\bienv -Doracle.bi.12c=true -Ddomain.home=C:\Or

acle\Middleware\Oracle_Home\user_projects\domains\bi_domain -Dfile.encoding=utf-

8 -Djava.system.class.loader=com.oracle.classloader.weblogic.LaunchClassLoader "

-Djava.security.policy=C:\Oracle\Middleware\Oracle_Home\wlserver\server\lib\webl

ogic.policy" "-Dweblogic.nodemanager.JavaHome=C:\Java\jdk1.8.0_202" weblogic.Nod

eManager -v

NMProcess: С?Р?Р? 26, 2024 8:30:38 PM oracle.security.opss.internal.runtime.Serv

iceContextManagerImpl getContext

NMProcess: WARNING: Bootstrap services are used by OPSS internally and clients s

hould never need to directly read/write bootstrap credentials. If required, use

Wlst or configuration management interfaces.

Node manager started, but cannot connect due to: Error occurred while performing

 nmConnect : Cannot connect to Node Manager. : General SSLEngine problem

Use dumpStack() to view the full stacktrace :

Unable to connect to NodeManager on host: ANALYTICHYP, due to Error occurred whi

le performing nmConnect : Cannot connect to Node Manager. : General SSLEngine pr

oblem

Use dumpStack() to view the full stacktrace :


Fatal: Failed to start one or more Servers (return 3)

Answers

  • Federico Venturin
    Federico Venturin ✭✭✭✭✭

    Hi @DENISKA80 ,

    Has SSL been enabled? I believe that your SSL certificate expired, and you have to renew it.

  • I cant remember we have ever enabled it.

    How can we check it?

  • RanaAshutosh-Oracle
    RanaAshutosh-Oracle ✭✭✭✭✭
    edited February 27

    @DENISKA80 Please check if this helps : OBIEE 12c: Start Script Fails to Connect to the Node Manager with Error "General SSLEngine Problem" (Doc ID 2330254.1)

  • Federico Venturin
    Federico Venturin ✭✭✭✭✭

    Hi @DENISKA80 ,

    Are you able to access the WebLogic Administration Console as weblogic (or any other administrator)?

    If so, in the Domain Structure panel expand Environment > Servers > AdminServer, and check whether the SSL checkbox is ticked. Then do the same for bi_server1 to understand if SSL has been enabled or not.

    Then, if you expand Environment > Machines, you can monitor the NodeManager status. When everything is working as expected it should display Reachable as status.

  • "Are you able to access the WebLogic Administration Console as weblogic (or any other administrator)?"

    Yes I can start weblogic manually

    "If so, in the Domain Structure panel expand Environment > Servers > AdminServer, and check whether the SSL checkbox is ticked. Then do the same for bi_server1 to understand if SSL has been enabled or not."

    Not enabled

    "Then, if you expand Environment > Machines, you can monitor the NodeManager status. When everything is working as expected it should display Reachable as status."

    Status Inactive

    Listen port 9506

    Type SSL

    I can even start bi_server1 with "startManagedWebLogic bi_server1" command and see that it has Running state in EM but all of the services in biinctance are down. When I trying to start it(obis1 f.e.) i get the same error from the 1st post

  • Federico Venturin
    Federico Venturin ✭✭✭✭✭

    Hi @DENISKA80 ,

    NodeManager status Inactive: this is the issue that you must fix!

    I recommend to stop/kill all your OBIEE processes. Then start the NodeManager manually using the startNodeManager script (do not close the command prompt otherwise NodeManager will stop). Do you get any error message?

    If it started properly, then start the Weblogic server manually using the startWeblogic command. If it completes successfully, then login to the Administration Console, confirm that the NodeManager is reachable, and start the managed server (bi_server1) from the console.

  • Restarted the whole machine.

    Opened cmd with admin rights and started nodemanager

    c:\Oracle\Middleware\Oracle_Home\user_projects\domains\bi_domain\bin\startNodeManager.cmd

    (Secure scoket listener started on port 9506)

    Opened cmd with admin rights and started weblogic (during starting entered credentials)

    c:\Oracle\Middleware\Oracle_Home\user_projects\domains\bi_domain\bin\startWebLogic.cmd

    In weblogic console status of the NodeManager - Inactive

  • @DENISKA80 Can you please upload the screenshot of the weblogic console where you see the status of the nodemanager.

    Also I advice you to raise an SR with support for this issue. So, we can thoroughly study the logs, debug the issue and fix your connectivity issue between admin server and nodemanager.


    Thanks.

  • Hi @DENISKA80

    Nodemanager uses demo certificates. Your OBIEE demo certificate would have got expired.

    So, your weblogic admin server is not able to trust the nodemanager.

    Here we have two options -

    Recreate demo certificate

    or

    Turn off secure listener in your nodemanager properties and then make the connection type to Plain in Admin Console.

  • I've noticed one warning in admin server log

    <BEA-090154> <Identity certificate has expired...

    But we didnt set any certificates. But after some searching i found that there is default certificate that expires after 5 year of using the System)).

    Great article

    https://chronicler.tech/regenerating-the-demoidentity-certificate-in-weblogic-12c/

    Ive did all described there but faced one issue

    We have 2 nodes

    I chose bi domain name with prefix (DemoCertifFor_bi_domain) for setting "Common Name" like it was before.

    Then after configuring and executing start.bat no one of my 2 bi servers started (admin server started).

    And I have a lot of such messages

    <BEA-090504> <Certificate chain received from MALAKHITHYP - 192.168.10.78 failed hostname verification check. Certificate contained DemoCertifFor_bi_domain but check expected MALAKHITHYP> 

     <Unable to start the server bi_server1...

    <BEA-090504> <Certificate chain received from MALAKHIT02 - 192.168.10.77 failed hostname verification check. Certificate contained DemoCertifFor_bi_domain but check expected MALAKHIT02> 

     <Unable to start the server bi_server2...

    Then i recreated cert with Common name=MALAKHITHYP and a half part of my instance started to work!

    BUT I dont know how to start the second one(MALAKHIT02) because I can specify only one in "Common name"

  • Hi @DENISKA80

    Nodemanager uses demo certificates. You OBIEE demo certificate would have got expired.

    So, your weblogic admin server is not able to trust the nodemanager.

    Here we have two options -

    Recreate democertificate

    or

    Turn off secure listener in your nodemanager properties and then make the connection type to Plain in Admin Console.

  • DENISKA80
    DENISKA80 ✭✭✭

    Thanks

    About 2 options

    1. We did it but faced interesting issue. When you recreate certificate you should specify Common name. If you have 2 machines with diffrent names(not like domainX.machine but f.e. machine1 and supermachine2) you cant specify Common name for both or they wont start . To make it work you should add following start parameter Dweblogic.security.SSL.ignoreHostnameVerification=true
    2. Nope. You can do this but some of the bi services wont start since they work only with ssl