Oracle Analytics Cloud and Server

Welcome to the Oracle Analytics Community: Please complete your User Profile and upload your Profile Picture

OBIEE 12c start up error

Received Response
531
Views
11
Comments
Raj453
Raj453 Rank 3 - Community Apprentice

Hi Gurus,

we have an issue when trying to start/stop the OBIEE instance. Below is the message upon attempting to start.  Please share inputs. Thanks.

====================================================================================================================

[oracle@****** bin]$ ./stop.sh

Stopping domain; Using domainHome: /u01/app/oracle/product/12.2.1/Middleware_Home/user_projects/domains/bi ...

Initializing WebLogic Scripting Tool (WLST) ...

Welcome to WebLogic Server Administration Scripting Shell

Type help() for help on available commands

<Nov 28, 2018 12:26:17 PM EST> <Info> <Security> <BEA-090905> <Disabling the CryptoJ JCE Provider self-integrity check for better startup performance. To enable this check, specify -Dweblogic.security.allowCryptoJDefaultJCEVerification=true.>

<Nov 28, 2018 12:26:17 PM EST> <Info> <Security> <BEA-090906> <Changing the default Random Number Generator in RSA CryptoJ from ********* to **********. To disable this change, specify -Dweblogic.security.allowCryptoJDefaultPRNG=true.>

<Nov 28, 2018 12:26:17 PM EST> <Info> <Security> <BEA-090909> <Using the configured custom SSL Hostname Verifier implementation: weblogic.security.utils.SSLWLSHostnameVerifier$NullHostnameVerifier.>

Reading domain...

/Servers/AdminServer/ListenPort=9500

Accessing admin server using URL t3://********:9500

Start Admin Server connect Exception caught Error occurred while performing connect : Error getting the initial context. There is no server running at t3://*********:9500 : Failed to initialize JNDI context, tried 2 time or times totally, the interval of each time is 0ms.

t3://*********:9500: Destination ************, 9500 unreachable.; nested exception is:

        java.net.ConnectException: Connection refused (Connection refused); No available router to destination.; nested exception is:

        java.rmi.ConnectException: No available router to destination.

Use dumpStack() to view the full stacktrace :

Reading domain...

/Servers/AdminServer/ListenPort=9500

Accessing admin server using URL t3://*********:9500

Requesting credentials ...

Enter Weblogic login details at prompt

Weblogic Username: weblogic

Weblogic Password:

Starting AdminServer ...

Unable to connect to AdminServer to shutdown all servers

Problem invoking WLST - Traceback (innermost last):

  File "/u01/app/oracle/product/12.2.1/Middleware_Home/bi/modules/oracle.bi.sysman/scripts/stop_servers.py", line 188, in ?

  File "/u01/app/oracle/product/12.2.1/Middleware_Home/bi/modules/oracle.bi.sysman/scripts/stop_servers.py", line 80, in stopManagedServersAndSystemComponents

AttributeError: 'NoneType' object has no attribute 'getAdminServerName'

Stopping Derby with pid: 14962; killing (SIGTERM) ...

[oracle@************* bin]$

«1

Answers

  • Raj453
    Raj453 Rank 3 - Community Apprentice

    Hi.  any suggestions please ..

  • Charles M
    Charles M Rank 6 - Analytics Lead

    Could you check the admin server log(s)?

    Also, see if this note helps you: weblogic.Admin PING fails with java.net.SocketException: Connection reset; No available router to destination (Doc ID 1398139.1)

    Regards,

    Charles

  • Charles M
    Charles M Rank 6 - Analytics Lead

    Also ... where is that output coming from? Is that standard out, or from a log? I'm a bit confused because you show a stop command, but I can see entries trying to start the admin server? Can you confirm that is the exact output you get, once you execute the stop command.

    Also, also, explain more about your configuration. SSL-enabled?, etc.

    Better explanations/better answers

  • Raj453
    Raj453 Rank 3 - Community Apprentice

    Hi Charles,

    the URL you shared prompting for support identifier, i am not using corporate account,  can you please share the PDF or Word version of that link. Thanks.

  • Charles M
    Charles M Rank 6 - Analytics Lead

    Sorry, can't do that. That is a My Oracle Support note. This is a public forum.

    My recommendation is that you provide some additional details, so others can help out. Give some context ... when did this last work, any changes made, OS your using, etc. Is this a fresh install (you get the idea)?

  • Raj453
    Raj453 Rank 3 - Community Apprentice

    Hi Charles,

    this is on Oracle linux  (on a VM),  not a new install, previously working fine,

    tried some suggestions like  kill all the oracle user processes,  deleted Admin server  "lok"  files (only after confirming admin server not running using netstat ),  replaced system generated weblogic username and password in "boot.properties" file with the actual values.  Still no luck, 

  • Raj453
    Raj453 Rank 3 - Community Apprentice

    Hi Charles,

    From Admin server log file, copying some of the error messages , please take a look.

    <Nov 28, 2018 4:55:54,825 PM EST> <Error> <Diagnostics> <Server or Host name> <AdminServer> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <8f6393a9-8666-4129-8edf-ba3b13f230f4-00000001> <1543442154825> <[severity-value: 8] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-320185> <Failed to open diagnostic store file /u01/app/oracle/product/12.2.1/Middleware_Home/user_projects/domains/bi/servers/AdminServer/data/store/diagnostics/WLS_DIAGNOSTICS.DAT after 10 attempts. Some diagnostics features will be unavailable.

    weblogic.store.PersistentStoreFatalException: [Store:280105]The persistent file store "WLS_DIAGNOSTICS" cannot open file WLS_DIAGNOSTICS000000.DAT.

    <Nov 28, 2018 4:56:57,278 PM EST> <Warning> <JMX> <Server or Host name> <AdminServer> <[ACTIVE] ExecuteThread: '7' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <8f6393a9-8666-4129-8edf-ba3b13f230f4-00000006> <1543442217278> <[severity-value: 16] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-149513> <JMX Connector Server stopped at service:jmx:iiop://172.29.16.181:9500/jndi/weblogic.management.mbeanservers.runtime.>

    ####<Nov 28, 2018 4:56:57,295 PM EST> <Critical> <WebLogicServer> <Server or Host name> <AdminServer> <main> <<WLS Kernel>> <> <8f6393a9-8666-4129-8edf-ba3b13f230f4-00000007> <1543442217295> <[severity-value: 4] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-000386> <Server subsystem failed. Reason: A MultiException has 22 exceptions.  They are:

    1. java.lang.NullPointerException

    <Nov 28, 2018 4:56:57,312 PM EST> <Error> <WebLogicServer> <Server or Host name> <AdminServer> <main> <<WLS Kernel>> <> <8f6393a9-8666-4129-8edf-ba3b13f230f4-00000007> <1543442217312> <[severity-value: 8] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-000383> <A critical service failed. The server will shut itself down.>

    <Nov 28, 2018 4:58:28,254 PM EST> <Error> <Diagnostics> <Server or Host name> <AdminServer> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <a070f2d1-faf8-4eab-a331-cfdc9a2d4181-00000001> <1543442308254> <[severity-value: 8] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-320185> <Failed to open diagnostic store file /u01/app/oracle/product/12.2.1/Middleware_Home/user_projects/domains/bi/servers/AdminServer/data/store/diagnostics/WLS_DIAGNOSTICS.DAT after 10 attempts. Some diagnostics features will be unavailable.

    weblogic.store.PersistentStoreFatalException: [Store:280105]The persistent file store "WLS_DIAGNOSTICS" cannot open file WLS_DIAGNOSTICS000000.DAT

    <Nov 28, 2018 4:59:54,532 PM EST> <Warning> <JMX> <Server or Host name> <AdminServer> <[ACTIVE] ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <a070f2d1-faf8-4eab-a331-cfdc9a2d4181-00000006> <1543442394532> <[severity-value: 16] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-149513> <JMX Connector Server stopped at service:jmx:iiop://172.29.16.181:9500/jndi/weblogic.management.mbeanservers.runtime.>

    ####<Nov 28, 2018 4:59:54,553 PM EST> <Critical> <WebLogicServer> <Server or Host name> <AdminServer> <main> <<WLS Kernel>> <> <a070f2d1-faf8-4eab-a331-cfdc9a2d4181-00000007> <1543442394553> <[severity-value: 4] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-000386> <Server subsystem failed. Reason: A MultiException has 22 exceptions.  They are:

    <Nov 28, 2018 5:01:34,207 PM EST> <Warning> <JMX> <Server or Host name> <AdminServer> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <2a692f87-4fd5-42d5-9b96-f8a098db5a8f-00000005> <1543442494207> <[severity-value: 16] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-149513> <JMX Connector Server stopped at service:jmx:iiop://172.29.16.181:9500/jndi/weblogic.management.mbeanservers.runtime.>

    ####<Nov 28, 2018 5:01:34,244 PM EST> <Critical> <WebLogicServer> <Server or Host name> <AdminServer> <main> <<WLS Kernel>> <> <2a692f87-4fd5-42d5-9b96-f8a098db5a8f-00000006> <1543442494244> <[severity-value: 4] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-000386> <Server subsystem failed. Reason: A MultiException has 6 exceptions.  They are:

    <Nov 28, 2018 5:03:34,428 PM EST> <Error> <WebLogicServer> <Server or Host name> <AdminServer> <main> <<WLS Kernel>> <> <2515cf00-51f2-4e19-bc67-e5e3ffcfa333-00000007> <1543442614428> <[severity-value: 8] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-000383> <A critical service failed. The server will shut itself down.>

    ####<Nov 28, 2018 5:03:34,431 PM EST> <Notice> <WebLogicServer> <Server or Host name> <AdminServer> <main> <<WLS Kernel>> <> <2515cf00-51f2-4e19-bc67-e5e3ffcfa333-00000007> <1543442614431> <[severity-value: 32] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-000365> <Server state changed to FORCE_SHUTTING_DOWN.>

    ####<Nov 28, 2018 5:03:34,431 PM EST> <Error> <Kernel> <Server or Host name> <AdminServer> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <2515cf00-51f2-4e19-bc67-e5e3ffcfa333-00000008> <1543442614431> <[severity-value: 8] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-000802> <ExecuteRequest failed

    A MultiException has 1 exceptions

  • Hi,

    Raj453 wrote:Hi Gurus,we have an issue when trying to start/stop the OBIEE instance. Below is the message upon attempting to start. Please share inputs. Thanks.====================================================================================================================[oracle@****** bin]$ ./stop.shStopping domain; Using domainHome: /u01/app/oracle/product/12.2.1/Middleware_Home/user_projects/domains/bi ...

    I just had a quick read through the rest of the thread as I blocked on that: you call stop.sh to attempt to start OBIEE? Very little chances it will work and of course the output is lot of issues because nothing is running ...

    So: what is the output when you call start.sh ?

  • Raj453
    Raj453 Rank 3 - Community Apprentice

    Hi Gianni,

    My mistake, i mean for stopping i ran stop.sh.

    When i attempt to start,  please see below

    [oracle@Server or Host name bin]$ ./start.sh

    Starting domain; Using domainHome: /u01/app/oracle/product/12.2.1/Middleware_Home/user_projects/domains/bi ...

    Initializing WebLogic Scripting Tool (WLST) ...

    Welcome to WebLogic Server Administration Scripting Shell

    Type help() for help on available commands

    <Nov 29, 2018 8:40:53 AM EST> <Info> <Security> <BEA-090905> <Disabling the CryptoJ JCE Provider self-integrity check for better startup performance. To enable this check                                                                    , specify -Dweblogic.security.allowCryptoJDefaultJCEVerification=true.>

    <Nov 29, 2018 8:40:53 AM EST> <Info> <Security> <BEA-090906> <Changing the default Random Number Generator in RSA CryptoJ from ECDRBG128 to HMACDRBG. To disable this chan                                                                    ge, specify -Dweblogic.security.allowCryptoJDefaultPRNG=true.>

    <Nov 29, 2018 8:40:53 AM EST> <Info> <Security> <BEA-090909> <Using the configured custom SSL Hostname Verifier implementation: weblogic.security.utils.SSLWLSHostnameVeri                                                                    fier$NullHostnameVerifier.>

    NodeManager already running

    Reading domain...

    /Servers/AdminServer/ListenPort=9500

    Accessing admin server using URL t3://Server or Host name:9500

    Starting AdminServer ...

    ^[[B^[[BAdmin Server Start Exception: Error occurred while performing nmStart : Error Starting server AdminServer : Received error message from Node Manager Server: [Server start command for WebLogic server 'AdminServer' failed due to: [Server failed to start up but Node Manager was not aware of the reason]. Please check Node Manager log and/or server 'AdminServer' log for detailed information.]. Please check Node Manager log for details.

    Use dumpStack() to view the full stacktrace :

    Reading domain...

    /Servers/AdminServer/ListenPort=9500

    Accessing admin server using URL t3://Server or Host name:9500

    Requesting credentials ...

    Enter Weblogic login details at prompt

    Weblogic Username: weblogic

    Weblogic Password:

    Starting AdminServer ...

    Unable to connect to AdminServer on host: Server or Host name

    Failed to start one or more Servers

    /Servers/AdminServer/ListenPort=9500

    Accessing admin server using URL t3://Server or Host name:9500

    Start Admin Server connect Exception caught Error occurred while performing connect : Error getting the initial context. There is no server running at t3://Server or Host name:9500 : Failed to initialize JNDI context, tried 2 time or times totally, the interval of each time is 0ms.

    t3://Server or Host name:9500: Destination 172.29.16.181, 9500 unreachable.; nested exception is:

            java.net.ConnectException: Connection refused (Connection refused); No available router to destination.; nested exception is:

            java.rmi.ConnectException: No available router to destination.

    Use dumpStack() to view the full stacktrace :

    Reading domain...

    Error: runCmd() failed. Do dumpStack() to see details.

    Failed to get Status of Servers and System Components

    [oracle@Server or Host name bin]$

  • Charles M
    Charles M Rank 6 - Analytics Lead

    Hi,

    Can you take a look at the node manager log, and paste any errors?

    Nodemanager: [DOMAIN_HOME]/nodemanager/nodemanager.log

    Regards,

    Charles