1 Reply Latest reply: Jun 7, 2010 11:55 PM by 605669 RSS

    Stuck Thread - JMS Business proxy Service (ALSB 2.5)

    687626
      Hi All,

      We are having a jms proxy service which uses a JMS Business Service to send messages to a jms queue on weblogic 8.1 sp6. The alsb domain runs on weblogic 9.2.The jms proxy service is made single threaded by configuring a suitable work manager.

      Now and then, we get a stuck thread issue in production and the proxy service stops consuming the messaging off the jms queue in wls 9.2. These are the errors in the logs:

      <[ACTIVE] ExecuteThread: '18' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1235437298516> <BEA-000337> <[STUCK] ExecuteThread: '8' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "695" seconds working on the request "weblogic.work.ServerWorkManagerImpl$WorkAdapterImpl@5321caf", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
      Thread-70 "[STUCK] ExecuteThread: '8' for queue: 'weblogic.kernel.Default (self-tuning)'" <alive, in native, waiting, priority=1, DAEMON> {
      -- Waiting for notification on: weblogic.rjvm.ResponseImpl@51961a5[fat lock]
      jrockit.vm.Threads.waitForSignal(Native Method)
      java.lang.Object.wait(Native Method)
      java.lang.Object.wait(Native Method)
      weblogic.rjvm.ResponseImpl.waitForData(ResponseImpl.java:73)
      ^-- Lock released while waiting: weblogic.rjvm.ResponseImpl@51961a5[fat lock]
      weblogic.rjvm.ResponseImpl.getTxContext(ResponseImpl.java:100)
      weblogic.rjvm.BasicOutboundRequest.sendReceive(BasicOutboundRequest.java:109)
      weblogic.rmi.internal.BasicRemoteRef.invoke(BasicRemoteRef.java:223)
      weblogic.jms.dispatcher.DispatcherImpl_816_WLStub.dispatchSyncTranFuture(Unknown Source)
      weblogic.jms.dispatcher.DispatcherInteropAdapter.dispatchSyncTranFuture(DispatcherInteropAdapter.java:80)
      weblogic.messaging.dispatcher.DispatcherWrapperState.dispatchSyncTran(DispatcherWrapperState.java:230)
      weblogic.jms.dispatcher.DispatcherAdapter.dispatchSyncTran(DispatcherAdapter.java:51)
      weblogic.jms.client.JMSProducer.toFEProducer(JMSProducer.java:1047)
      weblogic.jms.client.JMSProducer.deliveryInternal(JMSProducer.java:670)
      weblogic.jms.client.JMSProducer.sendInternal(JMSProducer.java:455)
      weblogic.jms.client.JMSProducer.send(JMSProducer.java:352)
      weblogic.jms.client.WLProducerImpl.send(WLProducerImpl.java:1009)
      weblogic.deployment.jms.WrappedMessageProducer.send(WrappedMessageProducer.java:157)
      com.bea.wli.sb.transports.jms.JmsOutboundMessageContext$SendAction.run(JmsOutboundMessageContext.java:795)
      com.bea.wli.sb.transports.jms.JmsOutboundMessageContext.send(JmsOutboundMessageContext.java:500)
      com.bea.wli.sb.transports.jms.JmsTransportProvider.sendMessageAsync(JmsTransportProvider.java:612)
      jrockit.reflect.CompiledMethodInvoker.invoke0(Unknown Source)
      jrockit.reflect.CompiledMethodInvoker.invoke(Unknown Source)
      java.lang.reflect.Method.invoke(Unknown Source)
      com.bea.wli.sb.transports.Util$1.invoke(Util.java:241)
      $Proxy39.sendMessageAsync(Unknown Source)
      com.bea.wli.sb.transports.LoadBalanceFailoverListener.sendMessageAsync(LoadBalanceFailoverListener.java:137)
      com.bea.wli.sb.transports.TransportManagerImpl.sendMessageToService(TransportManagerImpl.java:437)
      com.bea.wli.sb.transports.TransportManagerImpl.sendMessageAsync(TransportManagerImpl.java:328)
      com.bea.wli.sb.pipeline.PipelineContextImpl.doDispatch(PipelineContextImpl.java:503)
      com.bea.wli.sb.pipeline.PipelineContextImpl.dispatch(PipelineContextImpl.java:415)
      stages.publish.runtime.PublishRuntimeStep.processMessage(PublishRuntimeStep.java:106)
      com.bea.wli.sb.stages.StageMetadataImpl$WrapperRuntimeStep.processMessage(StageMetadataImpl.java:327)
      com.bea.wli.sb.stages.impl.SequenceRuntimeStep.processMessage(SequenceRuntimeStep.java:33)
      com.bea.wli.sb.pipeline.PipelineStage.processMessage(PipelineStage.java:100)
      com.bea.wli.sb.pipeline.PipelineContextImpl.execute(PipelineContextImpl.java:771)
      com.bea.wli.sb.pipeline.Pipeline.processMessage(Pipeline.java:73)
      com.bea.wli.sb.pipeline.PipelineContextImpl.execute(PipelineContextImpl.java:771)
      com.bea.wli.sb.pipeline.PipelineNode.doRequest(PipelineNode.java:47)
      com.bea.wli.sb.pipeline.Node.processMessage(Node.java:57)
      com.bea.wli.sb.pipeline.PipelineContextImpl.execute(PipelineContextImpl.java:771)
      com.bea.wli.sb.pipeline.Router.processMessage(Router.java:141)
      com.bea.wli.sb.pipeline.MessageProcessor.processRequest(MessageProcessor.java:75)
      com.bea.wli.sb.pipeline.RouterManager$1.run(RouterManager.java:646)
      com.bea.wli.sb.pipeline.RouterManager$1.run(RouterManager.java:645)
      weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
      weblogic.security.service.SecurityManager.runAs(SecurityManager.java:147)
      com.bea.wli.sb.pipeline.RouterManager.processMessage(RouterManager.java:627)
      com.bea.wli.sb.transports.TransportManagerImpl.receiveMessage(TransportManagerImpl.java:264)
      com.bea.wli.sb.transports.jms.JmsInboundMDB.onMessage(JmsInboundMDB.java:108)
      weblogic.ejb.container.internal.MDListener.execute(MDListener.java:429)
      weblogic.ejb.container.internal.MDListener.transactionalOnMessage(MDListener.java:335)
      weblogic.ejb.container.internal.MDListener.onMessage(MDListener.java:292)
      weblogic.jms.client.JMSSession.onMessage(JMSSession.java:4060)
      weblogic.jms.client.JMSSession.execute(JMSSession.java:4012)
      weblogic.jms.client.JMSSession$UseForRunnable.run(JMSSession.java:4467)
      weblogic.work.ServerWorkManagerImpl$WorkAdapterImpl.run(ServerWorkManagerImpl.java:518)
      weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)
      weblogic.work.ExecuteThread.run(ExecuteThread.java:181)
      jrockit.vm.RNI.c2java(Native Method)
      }

      Any one know the root cause of this issue or ways to prevent this?
      Also Is there a way to make the proxy service to reconsume messages off the queue without a server restart?

      Thanks,
      Atheek