5 Replies Latest reply on Sep 7, 2012 3:55 AM by 392 Guest

    Cannot stop domain

    slominskir
      I've had Glassfish 3.1.1 running for several months now with no issue and now when I try to shut it down with asadmin stop-domain the response is that it isn't running, even though it is (I can request pages just fine).  If I navigate to the web console on port 4848 I get a blank page.   The server.log shows a bunch of errors:
      [#|2012-08-30T14:52:57.375-0400|WARNING|glassfish3.1.1|javax.enterprise.system.container.web.com.sun.enterprise.web|_ThreadID=31;_ThreadName=Thread-2;|ApplicationDispatcher[] PWC1231: Servlet.service() for servlet FacesServlet threw exception
      java.lang.RuntimeException: java.lang.reflect.InvocationTargetException while attempting to process a 'beforeCreate' event for 'event1'.
              at com.sun.jsftemplating.layout.descriptors.LayoutElementBase.dispatchHandlers(LayoutElementBase.java:422)
              at com.sun.jsftemplating.layout.descriptors.LayoutElementBase.dispatchHandlers(LayoutElementBase.java:394)
              at com.sun.jsftemplating.layout.descriptors.LayoutComponent.beforeCreate(LayoutComponent.java:348)
              at com.sun.jsftemplating.layout.descriptors.LayoutComponent.getChild(LayoutComponent.java:288)
              at com.sun.jsftemplating.layout.LayoutViewHandler.buildUIComponentTree(LayoutViewHandler.java:556)
              at com.sun.jsftemplating.layout.LayoutViewHandler.createView(LayoutViewHandler.java:255)
              at com.sun.faces.lifecycle.RestoreViewPhase.execute(RestoreViewPhase.java:247)
              at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101)
              at com.sun.faces.lifecycle.RestoreViewPhase.doPhase(RestoreViewPhase.java:116)
              at com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:118)
              at javax.faces.webapp.FacesServlet.service(FacesServlet.java:593)
              at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1539)
              at org.apache.catalina.core.ApplicationDispatcher.doInvoke(ApplicationDispatcher.java:787)
              at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:649)
              at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:483)
              at org.apache.catalina.core.ApplicationDispatcher.doDispatch(ApplicationDispatcher.java:454)
              at org.apache.catalina.core.ApplicationDispatcher.dispatch(ApplicationDispatcher.java:350)
              at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:300)
              at org.glassfish.admingui.common.security.AdminConsoleAuthModule.validateRequest(AdminConsoleAuthModule.java:231)
              at com.sun.enterprise.security.jmac.config.GFServerConfigProvider$GFServerAuthContext.validateRequest(GFServerConfigProvider.java:1171)
              at com.sun.web.security.RealmAdapter.validate(RealmAdapter.java:1445)
              at com.sun.web.security.RealmAdapter.invokeAuthenticateDelegate(RealmAdapter.java:1323)
              at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:551)
              at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:623)
              at org.apache.catalina.core.StandardPipeline.doChainInvoke(StandardPipeline.java:600)
              at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:96)
              at com.sun.enterprise.web.PESessionLockingStandardPipeline.invoke(PESessionLockingStandardPipeline.java:91)
              at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:162)
              at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)
              at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)
              at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:327)
              at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)
              at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:232)
              at com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:828)
              at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:725)
              at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1019)
              at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:225)
              at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)
              at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)
              at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)
              at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)
              at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)
              at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)
              at com.sun.grizzly.ContextTask.run(ContextTask.java:71)
              at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)
              at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)
              at java.lang.Thread.run(Thread.java:662)
      Caused by: java.lang.reflect.InvocationTargetException
              at sun.reflect.GeneratedMethodAccessor659.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
              at java.lang.reflect.Method.invoke(Method.java:597)
              at com.sun.jsftemplating.layout.descriptors.handler.Handler.invoke(Handler.java:442)
              at com.sun.jsftemplating.layout.descriptors.LayoutElementBase.dispatchHandlers(LayoutElementBase.java:420)
              ... 46 more
      Caused by: com.sun.jersey.api.client.ClientHandlerException: java.net.SocketException: Unexpected end of file from server
              at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
              at com.sun.jersey.api.client.Client.handle(Client.java:648)
              at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
              at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
              at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:503)
              at org.glassfish.admingui.common.util.RestUtil.get(RestUtil.java:711)
              at org.glassfish.admingui.common.util.RestUtil.restRequest(RestUtil.java:190)
              at org.glassfish.admingui.common.handlers.RestApiHandlers.restRequest(RestApiHandlers.java:216)
              ... 51 more
      Caused by: java.net.SocketException: Unexpected end of file from server
              at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:777)
              at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:640)
              at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:774)
              at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:640)
              at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1195)
              at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:379)
              at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:240)
              at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147)
              ... 58 more
      |#]
      
      [#|2012-08-30T14:53:34.542-0400|SEVERE|glassfish3.1.1|com.sun.grizzly.config.GrizzlyServiceListener|_ThreadID=29;_ThreadName=Thread-2;|GRIZZLY0039: Request URI is too large.
      java.nio.BufferOverflowException
              at com.sun.grizzly.tcp.http11.InternalInputBuffer.fill(InternalInputBuffer.java:765)
              at com.sun.grizzly.tcp.http11.InternalInputBuffer.parseRequestLine(InternalInputBuffer.java:402)
              at com.sun.grizzly.http.ProcessorTask.parseRequest(ProcessorTask.java:861)
              at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:692)
              at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1019)
              at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:225)
              at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)
              at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)
              at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)
              at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)
              at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)
              at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)
              at com.sun.grizzly.ContextTask.run(ContextTask.java:71)
              at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)
              at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)
              at java.lang.Thread.run(Thread.java:662)
      |#]
      
      [#|2012-08-30T14:53:34.542-0400|SEVERE|glassfish3.1.1|com.sun.grizzly.config.GrizzlyServiceListener|_ThreadID=29;_ThreadName=Thread-2;|GRIZZLY0051: ProcessorTask exception.
      java.lang.NullPointerException
              at java.nio.CharBuffer.put(CharBuffer.java:896)
              at com.sun.enterprise.web.accesslog.CommonAccessLogFormatterImpl.appendRequestInfo(CommonAccessLogFormatterImpl.java:203)
              at com.sun.enterprise.web.accesslog.CommonAccessLogFormatterImpl.appendLogEntry(CommonAccessLogFormatterImpl.java:133)
              at com.sun.enterprise.web.accesslog.CombinedAccessLogFormatterImpl.appendLogEntry(CombinedAccessLogFormatterImpl.java:68)
              at com.sun.enterprise.web.PEAccessLogValve.postInvoke(PEAccessLogValve.java:592)
              at com.sun.enterprise.web.VirtualServer$2.onParsingError(VirtualServer.java:1698)
              at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:709)
              at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1019)
              at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:225)
              at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)
              at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)
              at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)
              at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)
              at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)
              at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)
              at com.sun.grizzly.ContextTask.run(ContextTask.java:71)
              at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)
              at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)
              at java.lang.Thread.run(Thread.java:662)
      |#]
      
        • 1. Re: Cannot stop domain
          rdarbh
          Try to do a kill -9 on the PID (ps -ef | grep glassfish), if you are on Linux, or just end process on Windows, and then try start-domain.
          • 2. Re: Cannot stop domain
            slominskir
            I was really looking for an explanation for why this is happening;  This seems like a fairly serious bug.  I don't want to abruptly kill my server!  To abruptly kill the server I can use the --kill=true flag to asadmin.
            • 4. Re: Cannot stop domain
              slominskir
              OK, executed stop domain and this time I get the following response:
              ./asadmin stop-domain
              java.net.SocketException: Unexpected end of file from server
              
              and the log file shows:
              [#|2012-09-06T15:50:47.381-0400|SEVERE|glassfish3.1.1|com.sun.grizzly.config.GrizzlyServiceListener|_ThreadID=21;_ThreadName=Thread-2;|GRIZZLY0039: Request URI is too large.
              java.nio.BufferOverflowException
                      at com.sun.grizzly.tcp.http11.InternalInputBuffer.fill(InternalInputBuffer.java:765)
                      at com.sun.grizzly.tcp.http11.InternalInputBuffer.parseRequestLine(InternalInputBuffer.java:402)
                      at com.sun.grizzly.http.ProcessorTask.parseRequest(ProcessorTask.java:861)
                      at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:692)
                      at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1019)
                      at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:225)
                      at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)
                      at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)
                      at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)
                      at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)
                      at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)
                      at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)
                      at com.sun.grizzly.ContextTask.run(ContextTask.java:71)
                      at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)
                      at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)
                      at java.lang.Thread.run(Thread.java:662)
              |#]
              
              [#|2012-09-06T15:50:47.381-0400|SEVERE|glassfish3.1.1|com.sun.grizzly.config.GrizzlyServiceListener|_ThreadID=21;_ThreadName=Thread-2;|GRIZZLY0051: ProcessorTask exception.
              java.lang.NullPointerException
                      at java.nio.CharBuffer.put(CharBuffer.java:896)
                      at com.sun.enterprise.web.accesslog.CommonAccessLogFormatterImpl.appendRequestInfo(CommonAccessLogFormatterImpl.java:203)
                      at com.sun.enterprise.web.accesslog.CommonAccessLogFormatterImpl.appendLogEntry(CommonAccessLogFormatterImpl.java:133)
                      at com.sun.enterprise.web.accesslog.CombinedAccessLogFormatterImpl.appendLogEntry(CombinedAccessLogFormatterImpl.java:68)
                      at com.sun.enterprise.web.PEAccessLogValve.postInvoke(PEAccessLogValve.java:592)
                      at com.sun.enterprise.web.VirtualServer$2.onParsingError(VirtualServer.java:1698)
                      at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:709)
                      at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1019)
                      at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:225)
                      at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)
                      at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)
                      at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)
                      at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)
                      at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)
                      at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)
                      at com.sun.grizzly.ContextTask.run(ContextTask.java:71)
                      at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)
                      at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)
                      at java.lang.Thread.run(Thread.java:662)
              |#]
              
              The server isn't really hung though because it continues to service other requests just fine.  I ran jstack, but there are too many threads to include here.  I don't see anything all that interesting in the stack trace.  Is there any particular thread that I should be examining?
              • 5. Re: Cannot stop domain
                392 Guest
                OK, executed stop domain and this time I get the following response: ./asadmin stop-domain java.net.SocketException: Unexpected end of file from server and the log file shows: [#|2012-09-06T15:50:47.381-0400|SEVERE|glassfish3.1.1|com.sun.grizzly.config.GrizzlyServiceListener|_ThreadID=21;_ThreadName=Thread-2;|GRIZZLY0039: Request URI is too large. java.nio.BufferOverflowException at com.sun.grizzly.tcp.http11.InternalInputBuffer.fill(InternalInputBuffer.java:765) at com.sun.grizzly.tcp.http11.InternalInputBuffer.parseRequestLine(InternalInputBuffer.java:402) at com.sun.grizzly.http.ProcessorTask.parseRequest(ProcessorTask.java:861) at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:692) at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1019) at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:225) at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137) at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104) at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90) at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79) at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54) at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59) at com.sun.grizzly.ContextTask.run(ContextTask.java:71) at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532) at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513) at java.lang.Thread.run(Thread.java:662) |#] [#|2012-09-06T15:50:47.381-0400|SEVERE|glassfish3.1.1|com.sun.grizzly.config.GrizzlyServiceListener|_ThreadID=21;_ThreadName=Thread-2;|GRIZZLY0051: ProcessorTask exception. java.lang.NullPointerException at java.nio.CharBuffer.put(CharBuffer.java:896) at com.sun.enterprise.web.accesslog.CommonAccessLogFormatterImpl.appendRequestInfo(CommonAccessLogFormatterImpl.java:203) at com.sun.enterprise.web.accesslog.CommonAccessLogFormatterImpl.appendLogEntry(CommonAccessLogFormatterImpl.java:133) at com.sun.enterprise.web.accesslog.CombinedAccessLogFormatterImpl.appendLogEntry(CombinedAccessLogFormatterImpl.java:68) at com.sun.enterprise.web.PEAccessLogValve.postInvoke(PEAccessLogValve.java:592) at com.sun.enterprise.web.VirtualServer$2.onParsingError(VirtualServer.java:1698) at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:709) at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1019) at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:225) at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137) at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104) at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90) at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79) at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54) at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59) at com.sun.grizzly.ContextTask.run(ContextTask.java:71) at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532) at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513) at java.lang.Thread.run(Thread.java:662) |#] The server isn't really hung though because it continues to service other requests just fine. I ran jstack, but there are too many threads to include here. I don't see anything all that interesting in the stack trace. Is there any particular thread that I should be examining? -- [Message sent by forum member 'slominskir'] View Post: http://forums.java.net/node/889712