OBIEE 12c 12.2.1.2 - config.sh Configuration error on BI Startup - bi_server1 failed — Oracle Analytics

Oracle Analytics Cloud and Server

Welcome to the Oracle Analytics Community: Please complete your User Profile and upload your Profile Picture

OBIEE 12c 12.2.1.2 - config.sh Configuration error on BI Startup - bi_server1 failed

Received Response
258
Views
4
Comments
chillychin
chillychin Rank 6 - Analytics Lead

I was having some issues with the final step of my OBIEE 12c 12.2.1.2 installation.

I am at the point where I am running the /bi/bin/config.sh and initially was hitting errors on the Essbase step (resolved this in a previous thread, it had to do with the java /dev/random - /dev/urandom java issue) and was able to get my configuration to successfully get to the BI Startup step

Once the tool hit this step it basically seemed to sit there for nearly an hour.

I could tell that there was "something" happening as I saw that files were being created, and some of the log files were updating.

It finally failed though and I cant seem to find much info on the actual error that caused it to fail?

I checked the [startallservers.log] and I see the following

Starting domain; Using domainHome: /itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi ...Initializing WebLogic Scripting Tool (WLST) ...Welcome to WebLogic Server Administration Scripting ShellType help() for help on available commandsRequesting credentials ...Enter Weblogic login details at promptWeblogic Username: Weblogic Password: <Mar 9, 2017 2:14:00 PM EST> <Info> <Security> <BEA-090905> <Disabling the CryptoJ JCE Provider self-integrity check for better startup performance. To enable this check, specify -Dweblogic.security.allowCryptoJDefaultJCEVerification=true.> <Mar 9, 2017 2:14:00 PM EST> <Info> <Security> <BEA-090906> <Changing the default Random Number Generator in RSA CryptoJ from ECDRBG128 to HMACDRBG. To disable this change, specify -Dweblogic.security.allowCryptoJDefaultPRNG=true.> <Mar 9, 2017 2:14:00 PM EST> <Info> <Security> <BEA-090909> <Using the configured custom SSL Hostname Verifier implementation: weblogic.security.utils.SSLWLSHostnameVerifier$NullHostnameVerifier.> Node manager not running. Starting it...NMProcess: NODEMGR_HOME is already set to /itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi/nodemanagerNMProcess: + /usr/java/jdk1.8.0_111/bin/java -server -Xms32m -Xmx200m -Dcoherence.home=/itadm/u01/obiee/Middleware/obiee12c_home/wlserver/../coherence -Dbea.home=/itadm/u01/obiee/Middleware/obiee12c_home/wlserver/.. -DNodeManagerHome=/itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi/nodemanager -DLogToStderr=false -DQuitEnabled=true -Dweblogic.RootDirectory=/itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi -Doracle.security.jps.config=/itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi/config/fmwconfig/jps-config-jse.xml -Dcommon.components.home=/itadm/u01/obiee/Middleware/obiee12c_home/oracle_common -Dopss.version=12.2.1.2 -Dweblogic.RootDirectory=/itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi -Doracle.bi.home.dir=/itadm/u01/obiee/Middleware/obiee12c_home/bi -Doracle.bi.config.dir=/itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi/config/fmwconfig/biconfig -Doracle.bi.environment.dir=/itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi/config/fmwconfig/bienv -Doracle.bi.12c=true -Ddomain.home=/itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi -Dfile.encoding=utf-8 -Djava.system.class.loader=com.oracle.classloader.weblogic.LaunchClassLoader -Djava.security.policy=/itadm/u01/obiee/Middleware/obiee12c_home/wlserver/server/lib/weblogic.policy -Dweblogic.nodemanager.JavaHome=/usr/java/jdk1.8.0_111 weblogic.NodeManager -vNMProcess: CLASSPATH=/usr/java/jdk1.8.0_111/lib/tools.jar:/itadm/u01/obiee/Middleware/obiee12c_home/wlserver/server/lib/weblogic.jar:/itadm/u01/obiee/Middleware/obiee12c_home/wlserver/../oracle_common/modules/net.sf.antcontrib_1.1.0.0_1-0b3/lib/ant-contrib.jar:/itadm/u01/obiee/Middleware/obiee12c_home/wlserver/modules/features/oracle.wls.common.nodemanager.jar::/usr/java/jdk1.8.0_111/lib/tools.jar:/itadm/u01/obiee/Middleware/obiee12c_home/wlserver/modules/features/wlst.wls.classpath.jar:/itadm/u01/obiee/Middleware/obiee12c_home/wlserver/..NMProcess: Mar 09, 2017 2:18:39 PM oracle.security.opss.internal.runtime.ServiceContextManagerImpl getContextNMProcess: WARNING: Bootstrap services are used by OPSS internally and clients should never need to directly read/write bootstrap credentials. If required, use Wlst or configuration management interfaces.Storing encrypted NodeManager credentials: /itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi/nodemanager/nmUserConfig.propertiesCreating the key file can reduce the security of your system if it is not kept in a secured location after it is created. Creating new key...NodeManager startedReading domain.../Servers/AdminServer/ListenPort=9500Accessing admin server using URL t3://edwsnd.uit:9500Reading domain.../Servers/AdminServer/ListenPort=9500Accessing admin server using URL t3://edwsnd.uit:9500Starting AdminServer ...Started AdminServerStoring encrypted AdminServer credentials: /itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi/servers/AdminServer/security/adminServerUserConfig.propertiesCreating the key file can reduce the security of your system if it is not kept in a secured location after it is created. Creating new key...Starting all servers ...Starting bi_server1 (Original State:SHUTDOWN) ...Received error message from Node Manager Server: [Server start command for WebLogic server 'bi_server1' failed due to: [Server failed to start up but Node Manager was not aware of the reason]. Please check Node Manager log and/or server 'bi_server1' log for detailed information.]. Please check Node Manager log for details.Starting obiccs1 (Original State:SHUTDOWN) ...Received error message from Node Manager Server: [Server start command for OBICCS server 'obiccs1' failed due to: [Error creating directory "/itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi/system_components/OBICCS/obiccs1/logs"]. Please check Node Manager log and/or server 'obiccs1' log for detailed information.]. Please check Node Manager log for details.Starting obis1 (Original State:SHUTDOWN) ...Received error message from Node Manager Server: [Server start command for OBIS server 'obis1' failed due to: [Error creating directory "/itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi/system_components/OBIS/obis1/logs"]. Please check Node Manager log and/or server 'obis1' log for detailed information.]. Please check Node Manager log for details.Starting obips1 (Original State:SHUTDOWN) ...Received error message from Node Manager Server: [Server start command for OBIPS server 'obips1' failed due to: [Error creating directory "/itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi/system_components/OBIPS/obips1/logs"]. Please check Node Manager log and/or server 'obips1' log for detailed information.]. Please check Node Manager log for details.Starting obijh1 (Original State:SHUTDOWN) ...Received error message from Node Manager Server: [Server start command for OBIJH server 'obijh1' failed due to: [Error creating directory "/itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi/system_components/OBIJH/obijh1/logs"]. Please check Node Manager log and/or server 'obijh1' log for detailed information.]. Please check Node Manager log for details.Starting obisch1 (Original State:SHUTDOWN) ...Received error message from Node Manager Server: [Server start command for OBISCH server 'obisch1' failed due to: [Error creating directory "/itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi/system_components/OBISCH/obisch1/logs"]. Please check Node Manager log and/or server 'obisch1' log for detailed information.]. Please check Node Manager log for details.Failure to start one or more ManagedServers and or ComponentsFailed to start one or more ServersStatus of Domain: /itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/biNodeManager (edwsnd.uit:9506): RUNNINGName            Type            Machine                   Status         ----            ----            -------                   ------         AdminServer     Server          edwsnd.uit       RUNNING        bi_server1      Server          edwsnd.uit       FAILED_NOT_RESTARTABLEobips1          OBIPS           edwsnd.uit       SHUTDOWN       obijh1          OBIJH           edwsnd.uit       SHUTDOWN       obiccs1         OBICCS          edwsnd.uit       SHUTDOWN       obisch1         OBISCH          edwsnd.uit       SHUTDOWN       obis1           OBIS            edwsnd.uit       SHUTDOWN       Domain status; Using domainHome: /itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi ...Initializing WebLogic Scripting Tool (WLST) ...Welcome to WebLogic Server Administration Scripting ShellType help() for help on available commands<Mar 9, 2017 3:08:27 PM EST> <Info> <Security> <BEA-090905> <Disabling the CryptoJ JCE Provider self-integrity check for better startup performance. To enable this check, specify -Dweblogic.security.allowCryptoJDefaultJCEVerification=true.> <Mar 9, 2017 3:08:27 PM EST> <Info> <Security> <BEA-090906> <Changing the default Random Number Generator in RSA CryptoJ from ECDRBG128 to HMACDRBG. To disable this change, specify -Dweblogic.security.allowCryptoJDefaultPRNG=true.> <Mar 9, 2017 3:08:27 PM EST> <Info> <Security> <BEA-090909> <Using the configured custom SSL Hostname Verifier implementation: weblogic.security.utils.SSLWLSHostnameVerifier$NullHostnameVerifier.> /Servers/AdminServer/ListenPort=9500Accessing admin server using URL t3://edwsnd.uit:9500Status of Domain: /itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/biNodeManager (edwsnd.uit:9506): RUNNINGName            Type            Machine                   Status         ----            ----            -------                   ------         AdminServer     Server          edwsnd.uit       RUNNING        bi_server1      Server          edwsnd.uit       FAILED_NOT_RESTARTABLEobips1          OBIPS           edwsnd.uit       SHUTDOWN       obijh1          OBIJH           edwsnd.uit       SHUTDOWN       obiccs1         OBICCS          edwsnd.uit       SHUTDOWN       obisch1         OBISCH          edwsnd.uit       SHUTDOWN       obis1           OBIS            edwsnd.uit       SHUTDOWN       

So only Adminserver was able to start

On the step [Starting bi_server1] it just simply states "Server failed to start up but Node Manager was not aware of the reason"

The remainder of the error messages Im not sure why they would pop up, but I can create the folders easily enough. But the bi_server1 error really stumps me.

Has anyone else come across this issue?

Are there some other logs I can check?

This is a brand new install of OBIEE 12c, so I cant see what may be the issue?

Any help much appreciated!

Answers

  • Hi,

    I had something similar again once (still in my docker container), but as I was in a hurry I didn't spend time on it: killed the container, removed it, cleaned up the RCU schemas and started a new one

    You can check the logs of bi_server1 in user_projects/domains/bi/servers/bi_server1/logs/

    There you must find a lot more info about the issue.

    PS: post OBIEE questions directly in as they will anyway move it there

  • chillychin
    chillychin Rank 6 - Analytics Lead

    Hi Gianni

    What do you mean by "docker container"?

    Are you saying you did a complete uninstall/removal of JUST the RCU schemas?

    Or do you mean that you totally uninstalled EVERYTHING and re-instaled everything all over again?

    Ah my mistake I must have totally missed that forum, I will keep that in mind to post in the other forum, thanks for the tip

  • For my tests I often need a "new OBIEE", so instead of installing on my host or on VM I do it in docker as it takes less resources than a VM. That's why I didn't have to uninstall and all I did was calling the RCU doing a "dropRepository" as I didn't want to use a new RCU prefix. But then it's like a VM, so I just deleted it and started a new one directly running the config and the second one worked (so 10 minutes later I had my new OBIEE up and running for my tests).

  • chillychin
    chillychin Rank 6 - Analytics Lead

    Just wanted to close the loop on this thread

    Very stupid rookie mistake on my part, but apparently my installation had JUST enough space to install everything with no issues, but when it came to start up time it could not as it actually ran out of physical space

    I pretty much installed most of the examples, so the total installation was about 11GB~

    I had to dig through a lot of logs, as a lot of logs just said "unknown error" until I finally found one error that said "no space"