This discussion is archived
0 Replies Latest reply: Mar 7, 2013 9:54 PM by Krishna1 RSS

Not able to start the managed servers throughing "Failed-not restartable "

Krishna1 Newbie
Currently Being Moderated
Hello All...

We have installed New weblogic portal 10.3.5 on solaris sparc server
We have used Jrockit 1.6.37 version.
We have installed Derby default database for portal.


We are able to browse the admin console and able to see two managed nodes.

Now we are not able to start the managed nodes.
We received error “Failed-not restartable” in Admin console page for managed node.

We have created nodemanager in multicast.

Please find below the managed node error logs for your reference.

<Error> <NodeManager> <BEA-300048> <Unable to start the server wl-qmtctx-qmt-1 : Exception while starting server 'wl-qmtctx-qmt-1'>
<Error> <NodeManager> <BEA-300048> <Unable to start the server wl-qmtctx-qmt-1 : Exception while starting server 'wl-qmtctx-qmt-1'>
Feb 28, 2013 10:46:18 AM MET> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED>
<Feb 28, 2013 10:46:18 AM MET> <Error> <WebLogicServer> <BEA-000383> <A critical service failed. The server will shut itself down>
<Feb 28, 2013 10:46:18 AM MET> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FORCE_SHUTTING_DOWN>
<Feb 28, 2013 10:46:19 AM> <FINEST> <NodeManager> <Waiting for the process to die: 29186>
<Feb 28, 2013 10:46:19 AM> <INFO> <NodeManager> <Server failed during startup so will not be restarted>
<Feb 28, 2013 10:46:19 AM> <FINEST> <NodeManager> <runMonitor returned, setting finished=true and notifying waiters>
<Feb 28, 2013 10:46:17 AM MET> <Critical> <WebLogicServer> <BEA-000362> <Server failed. Reason:

There are 1 nested errors:

weblogic.security.service.SecurityServiceRuntimeException: [Security:090399]Security Services Unavailable
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
<Feb 28, 2013 10:46:17 AM MET> <Error> <Security> <BEA-090870> <The realm "myrealm" failed to be loaded: weblogic.security.service.SecurityServiceException:
com.bea.common.engine.ServiceInitializationException: <openjpa-1.1.1-SNAPSHOT-r422266:965591 fatal internal error> org.apache.openjpa.util.InternalException:
There was an error when invoking the static getInstance method on the named factory class "kodo.jdbc.kernel.KodoJDBCBrokerFactory". See the nested exceptio
n for details..
weblogic.security.service.SecurityServiceException: com.bea.common.engine.ServiceInitializationException: <openjpa-1.1.1-SNAPSHOT-r422266:965591 fatal intern
al error> org.apache.openjpa.util.InternalException: There was an error when invoking the static getInstance method on the named factory class "kodo.jdbc.ker
nel.KodoJDBCBrokerFactory". See the nested exception for details.
at weblogic.security.service.CSSWLSDelegateImpl.initializeServiceEngine(CSSWLSDelegateImpl.java:341)
at weblogic.security.service.CSSWLSDelegateImpl.initialize(CSSWLSDelegateImpl.java:220)
at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.InitializeServiceEngine(CommonSecurityServiceManagerDelegateImpl.java:1785)
at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.initializeRealm(CommonSecurityServiceManagerDelegateImpl.java:442)
at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.loadRealm(CommonSecurityServiceManagerDelegateImpl.java:840)
Truncated. see log file for complete stacktrace
Caused By: com.bea.common.engine.ServiceInitializationException: <openjpa-1.1.1-SNAPSHOT-r422266:965591 fatal internal error> org.apache.openjpa.util.Interna
lException: There was an error when invoking the static getInstance method on the named factory class "kodo.jdbc.kernel.KodoJDBCBrokerFactory". See the nest
ed exception for details.
at com.bea.common.engine.internal.ServiceEngineImpl.findOrStartService(ServiceEngineImpl.java:365)
at com.bea.common.engine.internal.ServiceEngineImpl.findOrStartService(ServiceEngineImpl.java:315)
at com.bea.common.engine.internal.ServiceEngineImpl.lookupService(ServiceEngineImpl.java:257)
at com.bea.common.engine.internal.ServicesImpl.getService(ServicesImpl.java:72)
:$
>
<Feb 28, 2013 10:46:17 AM MET> <Notice> <Security> <BEA-090082> <Security initializing using security realm myrealm.>
<Feb 28, 2013 10:46:17 AM MET> <Critical> <WebLogicServer> <BEA-000362> <Server failed. Reason:

There are 1 nested errors:

weblogic.security.service.SecurityServiceRuntimeException: [Security:090399]Security Services Unavailable
at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.doBootAuthorization(CommonSecurityServiceManagerDelegateImpl.java:916)
at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.initialize(CommonSecurityServiceManagerDelegateImpl.java:1050)
at weblogic.security.service.SecurityServiceManager.initialize(SecurityServiceManager.java:873)
at weblogic.security.SecurityService.start(SecurityService.java:141)
at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:178)

>
<Feb 28, 2013 10:46:18 AM MET> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED>
<Feb 28, 2013 10:46:18 AM MET> <Error> <WebLogicServer> <BEA-000383> <A critical service failed. The server will shut itself down>
<Feb 28, 2013 10:46:18 AM MET> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FORCE_SHUTTING_DOWN>
<Feb 28, 2013 10:46:19 AM> <FINEST> <NodeManager> <Waiting for the process to die: 29186>
<Feb 28, 2013 10:46:19 AM> <INFO> <NodeManager> <Server failed during startup so will not be restarted>
<Feb 28, 2013 10:46:19 AM> <FINEST> <NodeManager> <runMonitor returned, setting finished=true and notifying waiters>
Nodemanager logs as follows:


<Feb 28, 2013 10:36:07 AM> <WARNING> <Node manager configuration properties file '/app/Middleware/wlserver_10.3/common/nodemanager/nodemanager.properties' no
t found. Using default settings.>
<Feb 28, 2013 10:39:10 AM> <WARNING> <Exception while starting server 'wl-qmtctx-qmt-1'>
java.io.IOException: Server failed to start up. See server output log for more details.
at weblogic.nodemanager.server.AbstractServerManager.start(AbstractServerManager.java:200)
at weblogic.nodemanager.server.ServerManager.start(ServerManager.java:23)
at weblogic.nodemanager.server.Handler.handleStart(Handler.java:604)
Admin Server logs:

####<Feb 28, 2013 10:34:22 AM MET> <Error> <Security> <esvmzn007> <wl-qmt-admin> <[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tunin
g)'> <<WLS Kernel>> <> <59424a3c17cd2a6b:382c951d:13d20260c63:-8000-0000000000000003> <1362044062548> <BEA-000000> <Commit events are not broadcast because n
o JMS Transport is available. Remote L2 caches will be out of sync.>
####<Feb 28, 2013 10:34:22 AM MET> <Error> <Security> <esvmzn007> <wl-qmt-admin> <[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tunin
g)'> <<WLS Kernel>> <> <59424a3c17cd2a6b:382c951d:13d20260c63:-8000-0000000000000003> <1362044062713> <BEA-000000> <Commit events are not broadcast because n
o JMS Transport is available. Remote L2 caches will be out of sync.>

=============================================================
we have troubleshoot like below...still not able to start the managed server....please can any one help me

1) FAILED NOT RESTARTABLE :

FAILED NOT RESTARTABLE is a common error in the Node Manager environment and This could be because of two reasons
1) WebLogic Server which you are trying to start is already running.
2) WebLogic Server which you are trying to start did not stop cleanly.

When Web Logic server stops, it removes these two files. If you not stop server properly these lock files remains on that location and creating a problem when you attempt to start a server again and after more than one attempt your server goes in FAILED_NOT_RESTARTABLE state.
i) Domain home/servers/<server_name>/tmp/<servermame>.lok
ii) Domain home /servers/<server_name>/data/ldap/ldapfiles/EmbeddedLDAP.lok

and also we have stop the all servers and we stop the servers..same error we are getting...


Regards..
Krishna

Edited by: 988913 on 7 Mar, 2013 9:51 PM

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points