2 Replies Latest reply on Mar 24, 2016 3:15 AM by Alex Sharkov

    Horizontally scaling BI 12c     bi_server2 did not startup

    Alex Sharkov

      Hi

      Im using doc for horizont scaling

      https://docs.oracle.com/middleware/1221/biee/BIESG/cluster.htm#BIESG9280

       

      1. I have 2 servers

      one server - installed and configured domain with active directory auth. It's work- all ok.

      other server - installed infrastructure and BI-it is my new node for cluster.

      2. I have shared disk by NFS between server and i set Singleton Data Directory (SDD)

      and Global Cache on shared disk

      3. by command line i create machine  by clone_bi_machine.sh

      4. move file on new node and do unpack


      Logs successfully copied to /home/oracle/oraInventory/logs.

      [oracle@appbitest2 Linux x86_64]$ unpack.sh -domain=/home/oracle/Oracle/Middleware/Oracle_Home/user_projects/domains/bi -template=/home/oracle/tmp/machine_appbitest2.jar -nodemanager_type=PerDomainNodeManager

      << read template from "/home/oracle/tmp/machine_appbitest2.jar"

      >>  succeed: read template from "/home/oracle/tmp/machine_appbitest2.jar"

      << set config option NodeManagerType to "PerDomainNodeManager"

      >>  succeed: set config option NodeManagerType to "PerDomainNodeManager"

      << set config option DomainName to "bi"

      >>  succeed: set config option DomainName to "bi"

      << write Domain to "/home/oracle/Oracle/Middleware/Oracle_Home/user_projects/domains/bi"

      >>  warning:write Domain to "/home/oracle/Oracle/Middleware/Oracle_Home/user_projects/domains/bi"

      >>  40326: Invalid coherence server address.

      40326: At least one server in a Coherence cluster should have non localhost address.

      40326: Enter a non localhost listen address for at least one Coherence server.

      ..........................................................................................

      >>  succeed: write Domain to "/home/oracle/Oracle/Middleware/Oracle_Home/user_projects/domains/bi"

      << close template

      >>  succeed: close template

       

       

      I did   sync_midtier_db  -that 's ok

      5. I do startNodemanager on new node and start all on primary node.

      6. I see on new node starting bi server but have error

      <Feb 11, 2016 1:14:27 PM ALMT> <INFO> <bi> <bi_server2> <Server output log file is '/home/oracle/Oracle/Middleware/Oracle_Home/user_projects/domains/bi/servers/bi_server2/logs/bi_server2.out'>

      <Feb 11, 2016 1:19:49 PM ALMT> <INFO> <bi> <bi_server2> <The server 'bi_server2' with process id 13105 is no longer alive; waiting for the process to die.>

      <Feb 11, 2016 1:19:49 PM ALMT> <INFO> <bi> <bi_server2> <Server failed during startup. It may be retried according to the auto restart configuration.>

      <Feb 11, 2016 1:19:49 PM ALMT> <INFO> <bi> <bi_server2> <Server failed but will not be restarted because the maximum number of restart attempts has been exceeded>

      <Feb 11, 2016 1:19:49 PM ALMT> <WARNING> <Server start command for WebLogic server 'bi_server2' failed due to: [Server failed to start up but Node Manager was not aware of the reason]. Please check Node Manager log and/or server 'bi_server2' log for detailed information.>

       

       

      BI Logs on new node

      <Feb 11, 2016 1:06:58 PM ALMT> <Notice> <Security> <BEA-090946> <Security pre-initializing using security realm: myrealm>

      <Feb 11, 2016 1:06:58 PM ALMT> <Notice> <Security> <BEA-090947> <Security post-initializing using security realm: myrealm>

      <Feb 11, 2016 1:06:59 PM ALMT> <Notice> <Security> <BEA-090082> <Security initialized using administrative security realm: myrealm>

      <Feb 11, 2016 1:06:59 PM ALMT> <Critical> <Security> <BEA-090402> <Authentication denied: Boot identity not valid. The user name or password or both from the boot identity file (boot.properties) is not valid. The boot identity may have been changed since the boot identity file was created. Please edit and update the boot identity file with the proper values of username and password. The first time the updated boot identity file is used to start the server, these new values are encrypted.>

      <Feb 11, 2016 1:09:27 PM ALMT> <Critical> <WebLogicServer> <BEA-000386> <Server subsystem failed. Reason: A MultiException has 6 exceptions.  They are:

      1. weblogic.security.SecurityInitializationException: Authentication denied: Boot identity not valid. The user name or password or both from the boot identity file (boot.properties) is not valid. The boot identity may have been changed since the boot identity file was created. Please edit and update the boot identity file with the proper values of username and password. The first time the updated boot identity file is used to start the server, these new values are encrypted.

      2. java.lang.IllegalStateException: Unable to perform operation: post construct on weblogic.security.SecurityService

      3. java.lang.IllegalArgumentException: While attempting to resolve the dependencies of weblogic.jndi.internal.RemoteNamingService errors were found

      4. java.lang.IllegalStateException: Unable to perform operation: resolve on weblogic.jndi.internal.RemoteNamingService

      5. java.lang.IllegalArgumentException: While attempting to resolve the dependencies of weblogic.deployment.DeploymentRegistrationService errors were found

      6. java.lang.IllegalStateException: Unable to perform operation: resolve on weblogic.deployment.DeploymentRegistrationService

       

       

      A MultiException has 6 exceptions.  They are:

      1. weblogic.security.SecurityInitializationException: Authentication denied: Boot identity not valid. The user name or password or both from the boot identity file (boot.properties) is not valid. The boot identity may have been changed since the boot identity file was created. Please edit and update the boot identity file with the proper values of username and password. The first time the updated boot identity file is used to start the server, these new values are encrypted.

      2. java.lang.IllegalStateException: Unable to perform operation: post construct on weblogic.security.SecurityService

      3. java.lang.IllegalArgumentException: While attempting to resolve the dependencies of weblogic.jndi.internal.RemoteNamingService errors were found

      4. java.lang.IllegalStateException: Unable to perform operation: resolve on weblogic.jndi.internal.RemoteNamingService

      5. java.lang.IllegalArgumentException: While attempting to resolve the dependencies of weblogic.deployment.DeploymentRegistrationService errors were found

      6. java.lang.IllegalStateException: Unable to perform operation: resolve on weblogic.deployment.DeploymentRegistrationService

       

       

              at org.jvnet.hk2.internal.Collector.throwIfErrors(Collector.java:89)

              at org.jvnet.hk2.internal.ClazzCreator.resolveAllDependencies(ClazzCreator.java:249)

              at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:357)

              at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:471)

              at org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext.findOrCreate(AsyncRunLevelContext.java:228)

              Truncated. see log file for complete stacktrace

      Caused By: weblogic.security.SecurityInitializationException: Authentication denied: Boot identity not valid. The user name or password or both from the boot identity file (boot.properties) is not valid. The boot identity may have been changed since the boot identity file was created. Please edit and update the boot identity file with the proper values of username and password. The first time the updated boot identity file is used to start the server, these new values are encrypted.

              at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.doBootAuthorization(CommonSecurityServiceManagerDelegateImpl.java:1097)

              at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.postInitialize(CommonSecurityServiceManagerDelegateImpl.java:1216)

              at weblogic.security.service.SecurityServiceManager.postInitialize(SecurityServiceManager.java:578)

              at weblogic.security.SecurityService.start(SecurityService.java:134)

              at weblogic.server.AbstractServerService.postConstruct(AbstractServerService.java:76)

              Truncated. see log file for complete stacktrace

      Caused By: javax.security.auth.login.FailedLoginException: [Security:090938]Authentication failure: The specified user failed to log in. javax.security.auth.login.FailedLoginException: [Security:090302]Authentication Failed: User specified user denied

              at com.bea.common.security.utils.ExceptionHandler.throwFailedLoginException(ExceptionHandler.java:62)

              at weblogic.security.providers.authentication.LDAPAtnLoginModuleImpl.login(LDAPAtnLoginModuleImpl.java:378)

              at com.bea.common.security.internal.service.LoginModuleWrapper$1.run(LoginModuleWrapper.java:117)

              at java.security.AccessController.doPrivileged(Native Method)

              at com.bea.common.security.internal.service.LoginModuleWrapper.login(LoginModuleWrapper.java:114)

              Truncated. see log file for complete stacktrace

      >

      <Feb 11, 2016 1:09:27 PM ALMT> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED.>

      <Feb 11, 2016 1:09:27 PM ALMT> <Error> <WebLogicServer> <BEA-000383> <A critical service failed. The server will shut itself down.>

      <Feb 11, 2016 1:09:27 PM ALMT> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FORCE_SHUTTING_DOWN.>

      Stopping Derby server...

      Derby server stopped.

      <Feb 11, 2016 1:09:28 PM ALMT> <INFO> <NodeManager> <The server 'bi_server2' with process id 12492 is no longer alive; waiting for the process to die.>

      <Feb 11, 2016 1:09:28 PM ALMT> <FINEST> <NodeManager> <Process died.>

      <Feb 11, 2016 1:09:28 PM ALMT> <INFO> <NodeManager> <Server failed during startup. It may be retried according to the auto restart configuration.>

      <Feb 11, 2016 1:09:28 PM ALMT> <FINEST> <NodeManager> <get latest startup configuration before deciding/trying to restart the server>

      <Feb 11, 2016 1:09:28 PM ALMT> <INFO> <NodeManager> <Server failed so attempting to restart (restart count = 1)>

      So all PATH, DOMAIN_HOME identical on my nodes

      I try correct boot.properties with login("weblogic") and correct password for my domain but unsuccess.

      After restaring bi_server2 error message again Authentication denied: Boot identity not valid


      There is no documentation about this problem.

       

      What i have to do? Please help!