3 Replies Latest reply: Oct 7, 2013 7:57 AM by User393914 RSS

    weblogic thread is getting stuck

    901560
      Hi,

      We have java application deployed on weblogic server 10.3.5 (clustered with 2 managed server instances in active-passive mode).

      After some time application is getting slower and status of managed server comes to WARNING state. From admin console when we go to domain - monitoring, for both the managed server it is showing "ThreadPool has stuck threads".

      I tried taking 2-3 thread dump from Admin console and tried to analyze but I am not able to find out reason for same.

      Here is the snippet of thread dump for your reference.

      "[STUCK] ExecuteThread: '4' for queue: 'weblogic.kernel.Default (self-tuning)'" waiting for lock weblogic.ejb.container.timer.TimerImpl@4b80cf8f BLOCKED
      weblogic.ejb.container.timer.TimerImpl.handleTimeoutSuccess(TimerImpl.java:477)
      weblogic.ejb.container.timer.EJBTimerManager$TimerExpirationOperation.onCommit(EJBTimerManager.java:1069)
      weblogic.store.gxa.internal.GXAOperationWrapperImpl.onCommit(GXAOperationWrapperImpl.java:168)
      weblogic.store.gxa.internal.GXAAbstractTransaction.loopOperationCallbacks(GXAAbstractTransaction.java:250)
      weblogic.store.gxa.internal.GXATransactionImpl.doOperationCallbacks(GXATransactionImpl.java:183)
      weblogic.store.gxa.internal.GXAResourceImpl.commit(GXAResourceImpl.java:1461)
      weblogic.transaction.internal.XAServerResourceInfo.commit(XAServerResourceInfo.java:1418)
      weblogic.transaction.internal.XAServerResourceInfo.commit(XAServerResourceInfo.java:608)
      weblogic.transaction.internal.ServerSCInfo$2.run(ServerSCInfo.java:499)
      weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)
      weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)
      weblogic.work.ExecuteThread.run(ExecuteThread.java:178)
      "DynamicListenThread[Default]" RUNNABLE native
      java.net.PlainSocketImpl.socketAccept(Native Method)
      java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)
      java.net.ServerSocket.implAccept(ServerSocket.java:462)
      java.net.ServerSocket.accept(ServerSocket.java:430)
      weblogic.socket.WeblogicServerSocket.accept(WeblogicServerSocket.java:38)
      weblogic.server.channels.DynamicListenThread$SocketAccepter.accept(DynamicListenThread.java:535)
      weblogic.server.channels.DynamicListenThread$SocketAccepter.access$200(DynamicListenThread.java:417)
      weblogic.server.channels.DynamicListenThread.run(DynamicListenThread.java:173)
      java.lang.Thread.run(Thread.java:662)
      "weblogic.GCMonitor" waiting for lock java.lang.ref.ReferenceQueue$Lock@7da50940 TIMED_WAITING
      java.lang.Object.wait(Native Method)
      java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
      weblogic.platform.GCMonitorThread.waitForNotification(GCMonitorThread.java:88)
      weblogic.platform.GCMonitorThread.run(GCMonitorThread.java:64)


      "[STUCK] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'" waiting for lock weblogic.transaction.internal.ServerCoordinatorDescriptor$Barrier@5d359fc0 WAITING
      java.lang.Object.wait(Native Method)
      weblogic.transaction.internal.ServerCoordinatorDescriptor$Barrier.await(ServerCoordinatorDescriptor.java:409)
      weblogic.transaction.internal.ServerSCInfo.startCommit(ServerSCInfo.java:531)
      weblogic.transaction.internal.ServerTransactionImpl.localCommit(ServerTransactionImpl.java:2024)
      weblogic.transaction.internal.ServerTransactionImpl.globalRetryCommit(ServerTransactionImpl.java:2751)
      weblogic.transaction.internal.ServerTransactionImpl.globalCommit(ServerTransactionImpl.java:2661)
      weblogic.transaction.internal.ServerTransactionImpl.internalCommit(ServerTransactionImpl.java:291)
      weblogic.transaction.internal.ServerTransactionImpl.commit(ServerTransactionImpl.java:239)
      weblogic.ejb.container.internal.BaseLocalObject.postInvoke1(BaseLocalObject.java:622)
      weblogic.ejb.container.internal.BaseLocalObject.postInvoke(BaseLocalObject.java:766)
      weblogic.ejb.container.internal.TimerDrivenLocalObject.postInvoke(TimerDrivenLocalObject.java:79)
      weblogic.ejb.container.timer.TimerImpl.timerExpired(TimerImpl.java:403)
      weblogic.timers.internal.TimerImpl.run(TimerImpl.java:273)
      weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)
      weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)
      weblogic.work.ExecuteThread.run(ExecuteThread.java:178)
      "NM Reaper" waiting for lock java.util.TaskQueue@6b8b260e TIMED_WAITING
      java.lang.Object.wait(Native Method)
      java.util.TimerThread.mainLoop(Timer.java:509)
      java.util.TimerThread.run(Timer.java:462)
      "weblogic.timers.TimerThread" waiting for lock weblogic.timers.internal.TimerThread@6fd61086 TIMED_WAITING
      java.lang.Object.wait(Native Method)
      weblogic.timers.internal.TimerThread$Thread.run(TimerThread.java:262)
      "weblogic.time.TimeEventGenerator" waiting for lock weblogic.time.common.internal.TimeTable@15a92901 TIMED_WAITING
      java.lang.Object.wait(Native Method)
      weblogic.time.common.internal.TimeTable.snooze(TimeTable.java:286)
      weblogic.time.common.internal.TimeEventGenerator.run(TimeEventGenerator.java:117)
      java.lang.Thread.run(Thread.java:662)


      Any pointer will reallly help.

      -Bhavin