3 Replies Latest reply: Jan 2, 2013 3:26 AM by Fabian RSS

    Managed Server started but reflected as Shutdown in Admin Console

    982232
      Hi,
      Our managed server is running as service (but the error log shows it is unable to contact the Admin server, thus running in independence mode). On our admin console, it shows the managed server as shutdown. How can we resolve this issue?

      We have tried to clear all the folders under \servers\<managedserver> except the security folder containing boot.properties. However it kept prompting that invalid username/password has been entered. If we restore the original folders, the managed server was able to startup, although in independence mode. Is it because of some ldap ldift file which we find in the managed server's subfolder? How can we resolve this??

      Please help, as this is a production environment and this issue just suddenly happened. It was running fine earlier
        • 1. Re: Managed Server started but reflected as Shutdown in Admin Console
          982280
          Hi,

          1. What's the version of Weblogic you are using?
          2. Please check the following logs for more details
          managed server log
          Admin Server log
          Domain log
          3. Is both servers (Admin and managed servers) are running on the same host?
          • 2. Re: Managed Server started but reflected as Shutdown in Admin Console
            982232
            Hi

            1) 10gR3
            2) The log for managed server shows it is started and running. However, it has the following lines of error:
            ####<Dec 28, 2012 6:43:42 PM SGT> <Emergency> <Management> <KIMPRDDMAPP> <> <main> <> <> <> <1356691422536> <BEA-141151> <The admin server could not be reached at https://kimprddmapp:9002.>
            ####<Dec 28, 2012 6:43:42 PM SGT> <Info> <Configuration Management> <KIMPRDDMAPP> <> <main> <> <> <> <1356691422552> <BEA-150018> <This server is being started in managed server independence mode in the absence of the admin server.>

            It is also unable to deploy our application:
            ####<Dec 28, 2012 6:43:51 PM SGT> <Error> <Deployer> <KIMPRDDMAPP> <KIMServer> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1356691431398> <BEA-149205> <Failed to initialize the application 'da' due to error weblogic.management.DeploymentException: Exception occured while downloading files.
            weblogic.management.DeploymentException: Exception occured while downloading files
                 at weblogic.deploy.internal.targetserver.datamanagement.AppDataUpdate.doDownload(AppDataUpdate.java:43)
                 at weblogic.deploy.internal.targetserver.datamanagement.DataUpdate.download(DataUpdate.java:56)
                 at weblogic.deploy.internal.targetserver.datamanagement.Data.prepareDataUpdate(Data.java:97)
                 at weblogic.deploy.internal.targetserver.BasicDeployment.prepareDataUpdate(BasicDeployment.java:682)
                 at weblogic.deploy.internal.targetserver.BasicDeployment.stageFilesForStatic(BasicDeployment.java:725)
                 at weblogic.deploy.internal.targetserver.AppDeployment.prepare(AppDeployment.java:104)
                 at weblogic.management.deploy.internal.DeploymentAdapter$1.doPrepare(DeploymentAdapter.java:39)
                 at weblogic.management.deploy.internal.DeploymentAdapter.prepare(DeploymentAdapter.java:187)
                 at weblogic.management.deploy.internal.AppTransition$1.transitionApp(AppTransition.java:21)
                 at weblogic.management.deploy.internal.ConfiguredDeployments.transitionApps(ConfiguredDeployments.java:233)
                 at weblogic.management.deploy.internal.ConfiguredDeployments.prepare(ConfiguredDeployments.java:165)
                 at weblogic.management.deploy.internal.ConfiguredDeployments.deploy(ConfiguredDeployments.java:122)
                 at weblogic.management.deploy.internal.DeploymentServerService.resume(DeploymentServerService.java:173)
                 at weblogic.management.deploy.internal.DeploymentServerService.start(DeploymentServerService.java:89)
                 at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
                 at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
                 at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
            java.net.SocketException: Network is unreachable: connect
                 at java.net.PlainSocketImpl.socketConnect(Native Method)
                 at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
                 at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
                 at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
                 at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
                 at java.net.Socket.connect(Socket.java:525)
                 at java.net.Socket.connect(Socket.java:475)
                 at java.net.Socket.<init>(Socket.java:372)
                 at java.net.Socket.<init>(Socket.java:273)
                 at javax.net.ssl.SSLSocket.<init>(SSLSocket.java:195)
                 at javax.net.ssl.impl.SSLSocketImpl.<init>(Unknown Source)
                 at javax.net.ssl.impl.SSLSocketFactoryImpl.createSocket(Unknown Source)
                 at weblogic.socket.ChannelSSLSocketFactory.createSocket(ChannelSSLSocketFactory.java:78)
                 at weblogic.net.http.HttpsClient.openServer(HttpsClient.java:274)
                 at weblogic.net.http.HttpsClient.openServer(HttpsClient.java:502)
                 at weblogic.net.http.HttpsClient.New(HttpsClient.java:566)
                 at weblogic.net.http.HttpsURLConnection.connect(HttpsURLConnection.java:242)
                 at weblogic.deploy.service.datatransferhandlers.HttpDataTransferHandler.getDataAsStream(HttpDataTransferHandler.java:75)
                 at weblogic.deploy.service.datatransferhandlers.DataHandlerManager$RemoteDataTransferHandler.getDataAsStream(DataHandlerManager.java:153)
                 at weblogic.deploy.internal.targetserver.datamanagement.AppDataUpdate.doDownload(AppDataUpdate.java:39)
                 at weblogic.deploy.internal.targetserver.datamanagement.DataUpdate.download(DataUpdate.java:56)
                 at weblogic.deploy.internal.targetserver.datamanagement.Data.prepareDataUpdate(Data.java:97)
                 at weblogic.deploy.internal.targetserver.BasicDeployment.prepareDataUpdate(BasicDeployment.java:682)
                 at weblogic.deploy.internal.targetserver.BasicDeployment.stageFilesForStatic(BasicDeployment.java:725)
                 at weblogic.deploy.internal.targetserver.AppDeployment.prepare(AppDeployment.java:104)
                 at weblogic.management.deploy.internal.DeploymentAdapter$1.doPrepare(DeploymentAdapter.java:39)
                 at weblogic.management.deploy.internal.DeploymentAdapter.prepare(DeploymentAdapter.java:187)
                 at weblogic.management.deploy.internal.AppTransition$1.transitionApp(AppTransition.java:21)
                 at weblogic.management.deploy.internal.ConfiguredDeployments.transitionApps(ConfiguredDeployments.java:233)
                 at weblogic.management.deploy.internal.ConfiguredDeployments.prepare(ConfiguredDeployments.java:165)
                 at weblogic.management.deploy.internal.ConfiguredDeployments.deploy(ConfiguredDeployments.java:122)
                 at weblogic.management.deploy.internal.DeploymentServerService.resume(DeploymentServerService.java:173)
                 at weblogic.management.deploy.internal.DeploymentServerService.start(DeploymentServerService.java:89)
                 at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
                 at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
                 at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
            >
            ####<Dec 28, 2012 6:43:51 PM SGT> <Info> <WebService> <KIMPRDDMAPP> <KIMServer> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<anonymous>> <> <> <1356691431991> <BEA-220103> <Async web service support is not fully configured. The async response web service /AsyncResponseServiceHttps for this server was not fully deployed because the JMS reliability queue was not defined/deployed: weblogic.wsee.DefaultQueue. The server will periodically retry completing the deploy for the service. This message can usually be ignored unless there are async web service applications. To completely disable async web service support, thus avoiding this message, set -Dweblogic.wsee.skip.async.response=true.>
            ####<Dec 28, 2012 6:43:52 PM SGT> <Info> <WebService> <KIMPRDDMAPP> <KIMServer> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<anonymous>> <> <> <1356691432085> <BEA-220103> <Async web service support is not fully configured. The async response web service /AsyncResponseServiceJms for this server was not fully deployed because the JMS reliability queue was not defined/deployed: weblogic.wsee.DefaultQueue. The server will periodically retry completing the deploy for the service. This message can usually be ignored unless there are async web service applications. To completely disable async web service support, thus avoiding this message, set -Dweblogic.wsee.skip.async.response=true.>

            Noticed that on my Admin Server, it has the following error message, could this be the cause? How can I fix it?

            ####<Dec 28, 2012 6:35:45 PM SGT> <Warning> <Security> <KIMPRDDMAPP> <AdminServer> <[ACTIVE] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1356690945499> <BEA-090482> <BAD_CERTIFICATE alert was received from KIMPRDDMAPP.pubnet.sg - 10.206.12.30. Check the peer to determine why it rejected the certificate chain (trusted CA configuration, hostname verification). SSL debug tracing may be required to determine the exact reason the certificate was rejected.>
            ####<Dec 28, 2012 6:35:45 PM SGT> <Error> <Console> <KIMPRDDMAPP> <AdminServer> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <kimdmweblogic> <> <> <1356690945499> <BEA-240003> <Console encountered the following error javax.help.HelpSetException: Could not parse
            Got an IOException ([Security:090477]Certificate chain received from docs.kim.pubnet.sg - 10.206.12.30 was not trusted causing SSL handshake failure.)
            Parsing failed for null
                 at javax.help.HelpSet.<init>(HelpSet.java:146)
                 at com.bea.help.utils.HelpSetHelper.getExtensionHelpSetMap(Unknown Source)
                 at com.bea.help.utils.HelpSetHelper.mergeExtensionHelpSet(Unknown Source)
                 at com.bea.help.utils.HelpSetHelper.createHelpSet(Unknown Source)
                 at com.bea.help.utils.HelpSetHelper.getHelpSet(Unknown Source)
                 at com.bea.help.utils.HelpSetHelper.ensureHelpSet(Unknown Source)
                 at jsp_servlet._jsp.__tocview._jspService(__tocview.java:110)
                 at weblogic.servlet.jsp.JspBase.service(JspBase.java:34)
                 at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)
                 at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)
                 at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:292)
                 at weblogic.servlet.internal.ServletStubImpl.onAddToMapException(ServletStubImpl.java:408)
                 at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:318)
                 at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:175)
                 at weblogic.servlet.internal.RequestDispatcherImpl.invokeServlet(RequestDispatcherImpl.java:505)
                 at weblogic.servlet.internal.RequestDispatcherImpl.include(RequestDispatcherImpl.java:432)
                 at com.bea.netuix.servlets.controls.content.JspContent.beginRender(JspContent.java:552)
                 at com.bea.netuix.nf.ControlLifecycle$7.visit(ControlLifecycle.java:485)
                 at com.bea.netuix.nf.ControlTreeWalker.walkRecursiveRender(ControlTreeWalker.java:518)
                 at com.bea.netuix.nf.ControlTreeWalker.walkRecursiveRender(ControlTreeWalker.java:529)
                 at com.bea.netuix.nf.ControlTreeWalker.walkRecursiveRender(ControlTreeWalker.java:529)
                 at com.bea.netuix.nf.ControlTreeWalker.walkRecursiveRender(ControlTreeWalker.java:529)
                 at com.bea.netuix.nf.ControlTreeWalker.walk(ControlTreeWalker.java:220)
                 at com.bea.netuix.nf.Lifecycle.processLifecycles(Lifecycle.java:395)
                 at com.bea.netuix.nf.Lifecycle.processLifecycles(Lifecycle.java:361)
                 at com.bea.netuix.nf.Lifecycle.processLifecycles(Lifecycle.java:352)
                 at com.bea.netuix.nf.Lifecycle.run(Lifecycle.java:326)
                 at com.bea.netuix.nf.UIControl.render(UIControl.java:582)
                 at com.bea.netuix.servlets.controls.PresentationContext.render(PresentationContext.java:486)
                 at com.bea.netuix.servlets.util.RenderToolkit.renderChild(RenderToolkit.java:146)
                 at com.bea.netuix.servlets.jsp.taglib.skeleton.Child.doTag(Child.java:63)
                 at jsp_servlet._framework._skeletons._console.__nolayout._jspService(__nolayout.java:119)
                 at weblogic.servlet.jsp.JspBase.service(JspBase.java:34)
                 at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)
                 at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)
                 at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:292)
                 at weblogic.servlet.internal.ServletStubImpl.onAddToMapException(ServletStubImpl.java:408)
                 at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:318)
                 at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:175)
                 at weblogic.servlet.internal.RequestDispatcherImpl.invokeServlet(RequestDispatcherImpl.java:505)
                 at weblogic.servlet.internal.RequestDispatcherImpl.include(RequestDispatcherImpl.java:432)
                 at com.bea.netuix.servlets.controls.application.laf.JspTools.renderJsp(JspTools.java:130)
                 at com.bea.netuix.servlets.controls.application.laf.JspControlRenderer.beginRender(JspControlRenderer.java:72)
                 at com.bea.netuix.servlets.controls.application.laf.PresentationControlRenderer.beginRender(PresentationControlRenderer.java:65)
                 at com.bea.netuix.nf.ControlLifecycle$7.visit(ControlLifecycle.java:481)
                 at com.bea.netuix.nf.ControlTreeWalker.walkRecursiveRender(ControlTreeWalker.java:518)
                 at com.bea.netuix.nf.ControlTreeWalker.walkRecursiveRender(ControlTreeWalker.java:529)
                 at com.bea.netuix.nf.ControlTreeWalker.walkRecursiveRender(ControlTreeWalker.java:529)
                 at com.bea.netuix.nf.ControlTreeWalker.walkRecursiveRender(ControlTreeWalker.java:529)
                 at com.bea.netuix.nf.ControlTreeWalker.walkRecursiveRender(ControlTreeWalker.java:529)
                 at com.bea.netuix.nf.ControlTreeWalker.walkRecursiveRender(ControlTreeWalker.java:529)
                 at com.bea.netuix.nf.ControlTreeWalker.walk(ControlTreeWalker.java:220)
                 at com.bea.netuix.nf.Lifecycle.processLifecycles(Lifecycle.java:395)
                 at com.bea.netuix.nf.Lifecycle.processLifecycles(Lifecycle.java:361)
                 at com.bea.netuix.nf.Lifecycle.processLifecycles(Lifecycle.java:352)
                 at com.bea.netuix.nf.Lifecycle.run(Lifecycle.java:326)
                 at com.bea.netuix.nf.UIControl.render(UIControl.java:582)
                 at com.bea.netuix.servlets.controls.PresentationContext.render(PresentationContext.java:486)
                 at com.bea.netuix.servlets.util.RenderToolkit.renderChild(RenderToolkit.java:146)
                 at com.bea.netuix.servlets.jsp.taglib.skeleton.Child.doTag(Child.java:63)
                 at jsp_servlet._framework._skeletons._console.__twocollayout._jspService(__twocollayout.java:246)
                 at weblogic.servlet.jsp.JspBase.service(JspBase.java:34)
                 at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)
                 at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)
                 at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:292)
                 at weblogic.servlet.internal.ServletStubImpl.onAddToMapException(ServletStubImpl.java:408)
                 at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:318)
                 at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:175)
                 at weblogic.servlet.internal.RequestDispatcherImpl.invokeServlet(RequestDispatcherImpl.java:505)
                 at weblogic.servlet.internal.RequestDispatcherImpl.include(RequestDispatcherImpl.java:432)
                 at com.bea.netuix.servlets.controls.application.laf.JspTools.renderJsp(JspTools.java:130)
                 at com.bea.netuix.servlets.controls.application.laf.JspControlRenderer.beginRender(JspControlRenderer.java:72)
                 at com.bea.netuix.servlets.controls.application.laf.PresentationControlRenderer.beginRender(PresentationControlRenderer.java:65)
                 at com.bea.netuix.nf.ControlLifecycle$7.visit(ControlLifecycle.java:481)
                 at com.bea.netuix.nf.ControlTreeWalker.walkRecursiveRender(ControlTreeWalker.java:518)
                 at com.bea.netuix.nf.ControlTreeWalker.walkRecursiveRender(ControlTreeWalker.java:529)
                 at com.bea.netuix.nf.ControlTreeWalker.walkRecursiveRender(ControlTreeWalker.java:529)
                 at com.bea.netuix.nf.ControlTreeWalker.walkRecursiveRender(ControlTreeWalker.java:529)
                 at com.bea.netuix.nf.ControlTreeWalker.walkRecursiveRender(ControlTreeWalker.java:529)
                 at com.bea.netuix.nf.ControlTreeWalker.walkRecursiveRender(ControlTreeWalker.java:529)
                 at com.bea.netuix.nf.ControlTreeWalker.walkRecursiveRender(ControlTreeWalker.java:529)
                 at com.bea.netuix.nf.ControlTreeWalker.walkRecursiveRender(ControlTreeWalker.java:529)
                 at com.bea.netuix.nf.ControlTreeWalker.walkRecursiveRender(ControlTreeWalker.java:529)
                 at com.bea.netuix.nf.ControlTreeWalker.walk(ControlTreeWalker.java:220)
                 at com.bea.netuix.nf.Lifecycle.processLifecycles(Lifecycle.java:395)
                 at com.bea.netuix.nf.Lifecycle.processLifecycles(Lifecycle.java:361)
                 at com.bea.netuix.nf.Lifecycle.runOutbound(Lifecycle.java:208)
                 at com.bea.netuix.nf.Lifecycle.run(Lifecycle.java:162)
                 at com.bea.netuix.servlets.manager.UIServlet.runLifecycle(UIServlet.java:388)
                 at com.bea.netuix.servlets.manager.UIServlet.doPost(UIServlet.java:258)
                 at com.bea.netuix.servlets.manager.UIServlet.doGet(UIServlet.java:211)
                 at com.bea.netuix.servlets.manager.UIServlet.service(UIServlet.java:196)
                 at com.bea.netuix.servlets.manager.SingleFileServlet.service(SingleFileServlet.java:251)
                 at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
                 at weblogic.servlet.AsyncInitServlet.service(AsyncInitServlet.java:130)
                 at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)
                 at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)
                 at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:292)
                 at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:175)
                 at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3498)
                 at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
                 at weblogic.security.service.SecurityManager.runAs(Unknown Source)
                 at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2180)
                 at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2086)
                 at weblogic.servlet.internal.ServletRequestImpl.run(Unknown Source)
                 at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
                 at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)

            3) Both Admin and managed servers are running on same host
            • 3. Re: Managed Server started but reflected as Shutdown in Admin Console
              Fabian
              Hi,
              ####<Dec 28, 2012 6:43:42 PM SGT> <Emergency> <Management> <KIMPRDDMAPP> <main> <1356691422536> <BEA-141151> <The admin server >could not be reached at https://kimprddmapp:9002.>
              This Indicates your Admin Server is down,and when managed server is starting up,it cant find the Admin server due to which its starting on managed server independence mode(MSI mode).
              <kimdmweblogic> <1356690945499> <BEA-240003> <Console encountered the following error javax.help.HelpSetException: Could not parse
              Got an IOException ([Security:090477]Certificate chain received from docs.kim.pubnet.sg - 10.206.12.30 was not trusted causing SSL handshake failure.)
              Warning: Certificate chain received from peer was not trusted causing SSL handshake failure.
              Description: Peer certificate chain was not trusted by the server.
              Cause: A peer presented a certificate chain to the server that the server does not trust
              Action:Check the certificate chain to determine if it should be trusted or not. If it should be trusted, then update the server trusted CA configuration to trust the CA certificate that signed the peer certificate chain.

              Also Provide us the specific Weblogic Version ie 10.3.x.x

              Regards
              Fabian