OBIEE 12c 12.2.1.2 - config.sh Configuration error on BI Startup - bi_server1 failed

Question

I was having some issues with the final step of my OBIEE 12c 12.2.1.2 installation. I am at the point where I am running the /bi/bin/config.sh and initially was hitting errors on the Essbase step (resolved this in a previous thread, it had to do with the java /dev/random - /dev/urandom java issue) and was able to get my configuration to successfully get to the BI Startup step Once the tool hit this step it basically seemed to sit there for nearly an hour. I could tell that there was "something" happening as I saw that files were being created, and some of the log files were updating. It finally failed though and I cant seem to find much info on the actual error that caused it to fail? I checked the [startallservers.log] and I see the following Starting domain; Using domainHome: /itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi ...Initializing WebLogic Scripting Tool (WLST) ...Welcome to WebLogic Server Administration Scripting ShellType help() for help on available commandsRequesting credentials ...Enter Weblogic login details at promptWeblogic Username: Weblogic Password:

Node manager not running. Starting it...NMProcess: NODEMGR_HOME is already set to /itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi/nodemanagerNMProcess: + /usr/java/jdk1.8.0_111/bin/java -server -Xms32m -Xmx200m -Dcoherence.home=/itadm/u01/obiee/Middleware/obiee12c_home/wlserver/../coherence -Dbea.home=/itadm/u01/obiee/Middleware/obiee12c_home/wlserver/.. -DNodeManagerHome=/itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi/nodemanager -DLogToStderr=false -DQuitEnabled=true -Dweblogic.RootDirectory=/itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi -Doracle.security.jps.config=/itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi/config/fmwconfig/jps-config-jse.xml -Dcommon.components.home=/itadm/u01/obiee/Middleware/obiee12c_home/oracle_common -Dopss.version=12.2.1.2 -Dweblogic.RootDirectory=/itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi -Doracle.bi.home.dir=/itadm/u01/obiee/Middleware/obiee12c_home/bi -Doracle.bi.config.dir=/itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi/config/fmwconfig/biconfig -Doracle.bi.environment.dir=/itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi/config/fmwconfig/bienv -Doracle.bi.12c=true -Ddomain.home=/itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi -Dfile.encoding=utf-8 -Djava.system.class.loader=com.oracle.classloader.weblogic.LaunchClassLoader -Djava.security.policy=/itadm/u01/obiee/Middleware/obiee12c_home/wlserver/server/lib/weblogic.policy -Dweblogic.nodemanager.JavaHome=/usr/java/jdk1.8.0_111 weblogic.NodeManager -vNMProcess: CLASSPATH=/usr/java/jdk1.8.0_111/lib/tools.jar:/itadm/u01/obiee/Middleware/obiee12c_home/wlserver/server/lib/weblogic.jar:/itadm/u01/obiee/Middleware/obiee12c_home/wlserver/../oracle_common/modules/net.sf.antcontrib_1.1.0.0_1-0b3/lib/ant-contrib.jar:/itadm/u01/obiee/Middleware/obiee12c_home/wlserver/modules/features/oracle.wls.common.nodemanager.jar::/usr/java/jdk1.8.0_111/lib/tools.jar:/itadm/u01/obiee/Middleware/obiee12c_home/wlserver/modules/features/wlst.wls.classpath.jar:/itadm/u01/obiee/Middleware/obiee12c_home/wlserver/..NMProcess: Mar 09, 2017 2:18:39 PM oracle.security.opss.internal.runtime.ServiceContextManagerImpl getContextNMProcess: WARNING: Bootstrap services are used by OPSS internally and clients should never need to directly read/write bootstrap credentials. If required, use Wlst or configuration management interfaces.Storing encrypted NodeManager credentials: /itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi/nodemanager/nmUserConfig.propertiesCreating the key file can reduce the security of your system if it is not kept in a secured location after it is created. Creating new key...NodeManager startedReading domain.../Servers/AdminServer/ListenPort=9500Accessing admin server using URL t3://edwsnd.uit:9500Reading domain.../Servers/AdminServer/ListenPort=9500Accessing admin server using URL t3://edwsnd.uit:9500Starting AdminServer ...Started AdminServerStoring encrypted AdminServer credentials: /itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi/servers/AdminServer/security/adminServerUserConfig.propertiesCreating the key file can reduce the security of your system if it is not kept in a secured location after it is created. Creating new key...Starting all servers ...Starting bi_server1 (Original State:SHUTDOWN) ...Received error message from Node Manager Server: [Server start command for WebLogic server 'bi_server1' failed due to: [Server failed to start up but Node Manager was not aware of the reason]. Please check Node Manager log and/or server 'bi_server1' log for detailed information.]. Please check Node Manager log for details.Starting obiccs1 (Original State:SHUTDOWN) ...Received error message from Node Manager Server: [Server start command for OBICCS server 'obiccs1' failed due to: [Error creating directory "/itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi/system_components/OBICCS/obiccs1/logs"]. Please check Node Manager log and/or server 'obiccs1' log for detailed information.]. Please check Node Manager log for details.Starting obis1 (Original State:SHUTDOWN) ...Received error message from Node Manager Server: [Server start command for OBIS server 'obis1' failed due to: [Error creating directory "/itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi/system_components/OBIS/obis1/logs"]. Please check Node Manager log and/or server 'obis1' log for detailed information.]. Please check Node Manager log for details.Starting obips1 (Original State:SHUTDOWN) ...Received error message from Node Manager Server: [Server start command for OBIPS server 'obips1' failed due to: [Error creating directory "/itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi/system_components/OBIPS/obips1/logs"]. Please check Node Manager log and/or server 'obips1' log for detailed information.]. Please check Node Manager log for details.Starting obijh1 (Original State:SHUTDOWN) ...Received error message from Node Manager Server: [Server start command for OBIJH server 'obijh1' failed due to: [Error creating directory "/itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi/system_components/OBIJH/obijh1/logs"]. Please check Node Manager log and/or server 'obijh1' log for detailed information.]. Please check Node Manager log for details.Starting obisch1 (Original State:SHUTDOWN) ...Received error message from Node Manager Server: [Server start command for OBISCH server 'obisch1' failed due to: [Error creating directory "/itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi/system_components/OBISCH/obisch1/logs"]. Please check Node Manager log and/or server 'obisch1' log for detailed information.]. Please check Node Manager log for details.Failure to start one or more ManagedServers and or ComponentsFailed to start one or more ServersStatus of Domain: /itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/biNodeManager (edwsnd.uit:9506): RUNNINGName Type Machine Status ---- ---- ------- ------ AdminServer Server edwsnd.uit RUNNING bi_server1 Server edwsnd.uit FAILED_NOT_RESTARTABLEobips1 OBIPS edwsnd.uit SHUTDOWN obijh1 OBIJH edwsnd.uit SHUTDOWN obiccs1 OBICCS edwsnd.uit SHUTDOWN obisch1 OBISCH edwsnd.uit SHUTDOWN obis1 OBIS edwsnd.uit SHUTDOWN Domain status; Using domainHome: /itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/bi ...Initializing WebLogic Scripting Tool (WLST) ...Welcome to WebLogic Server Administration Scripting ShellType help() for help on available commands

/Servers/AdminServer/ListenPort=9500Accessing admin server using URL t3://edwsnd.uit:9500Status of Domain: /itadm/u01/obiee/Middleware/obiee12c_domain_home/user_projects/domains/biNodeManager (edwsnd.uit:9506): RUNNINGName Type Machine Status ---- ---- ------- ------ AdminServer Server edwsnd.uit RUNNING bi_server1 Server edwsnd.uit FAILED_NOT_RESTARTABLEobips1 OBIPS edwsnd.uit SHUTDOWN obijh1 OBIJH edwsnd.uit SHUTDOWN obiccs1 OBICCS edwsnd.uit SHUTDOWN obisch1 OBISCH edwsnd.uit SHUTDOWN obis1 OBIS edwsnd.uit SHUTDOWN So only Adminserver was able to start On the step [Starting bi_server1] it just simply states "Server failed to start up but Node Manager was not aware of the reason" The remainder of the error messages Im not sure why they would pop up, but I can create the folders easily enough. But the bi_server1 error really stumps me. Has anyone else come across this issue? Are there some other logs I can check? This is a brand new install of OBIEE 12c, so I cant see what may be the issue? Any help much appreciated!

chillychin · Answer

Just wanted to close the loop on this thread

Very stupid rookie mistake on my part, but apparently my installation had JUST enough space to install everything with no issues, but when it came to start up time it could not as it actually ran out of physical space

I pretty much installed most of the examples, so the total installation was about 11GB~

I had to dig through a lot of logs, as a lot of logs just said "unknown error" until I finally found one error that said "no space"

Gianni Ceresa · Answer

For my tests I often need a "new OBIEE", so instead of installing on my host or on VM I do it in docker as it takes less resources than a VM. That's why I didn't have to uninstall and all I did was calling the RCU doing a "dropRepository" as I didn't want to use a new RCU prefix. But then it's like a VM, so I just deleted it and started a new one directly running the config and the second one worked (so 10 minutes later I had my new OBIEE up and running for my tests).

chillychin · Answer

Hi Gianni

What do you mean by "docker container"?

Are you saying you did a complete uninstall/removal of JUST the RCU schemas?

Or do you mean that you totally uninstalled EVERYTHING and re-instaled everything all over again?

Ah my mistake I must have totally missed that forum, I will keep that in mind to post in the other forum, thanks for the tip