0 Replies Latest reply: Oct 18, 2012 4:01 PM by 801013 RSS

    Please help me in analysing JVM crash

    801013
      Guys,

      Can some one help me in analyzing this hotspot err.log :

      #
      # An unexpected error has been detected by Java Runtime Environment:
      #
      # SIGSEGV (0xb) at pc=0xfec2af88, pid=3772, tid=110
      #
      # Java VM: Java HotSpot(TM) Server VM (11.0-b16 mixed mode solaris-sparc)
      # Problematic frame:
      # V [libjvm.so+0x42af88]
      #
      # If you would like to submit a bug report, please visit:
      # http://java.sun.com/webapps/bugreport/crash.jsp
      #

      --------------- T H R E A D ---------------

      Current thread (0x017de400): JavaThread "[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon [_thread_in_vm, id=110, stack(0xcc480000,0xcc500000)]

      siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR), si_addr=0x00000010

      Registers:
      O0=0xcc4f6b08 O1=0xcc4f6b20 O2=0xcc4f6b20 O3=0x00000000
      O4=0xfc000224 O5=0xcc0f8158 O6=0xcc4f6aa8 O7=0xfef73384
      G1=0xfc00021c G2=0x00000006 G3=0xfc400000 G4=0x00000000
      G5=0xfc000000 G6=0x00000000 G7=0xfe066200 Y=0x00000000
      PC=0xfec2af88 nPC=0xfec2af8c


      Top of Stack: (sp=0xcc4f6aa8)
      0xcc4f6aa8: fc00022c ff139a80 0003ba80 0003b800
      0xcc4f6ab8: fc005eb8 fffffffc ff13c524 027141c8
      0xcc4f6ac8: 027141a8 ff139a80 cc4f6b24 0003ba80
      0xcc4f6ad8: 027141ac ff0fe000 cc4f6d68 fef733c0
      0xcc4f6ae8: 00000000 ff1491f4 00000000 0000000e
      0xcc4f6af8: 00000000 00000001 00000000 c58bdd66
      0xcc4f6b08: cc0f8158 fc000224 fc000108 00000000
      0xcc4f6b18: cc0f80d0 00000000 ff13c524 cc4febd0

      Instructions: (pc=0xfec2af88)
      0xfec2af78: 81 c7 e0 08 81 e8 20 00 da 02 20 00 d6 03 60 20
      0xfec2af88: d4 02 e0 10 92 20 00 0a 8b 32 70 3f 81 c3 e0 08

      Stack: [0xcc480000,0xcc500000], sp=0xcc4f6aa8, free space=474k
      Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
      V [libjvm.so+0x42af88]
      V [libjvm.so+0x7733c8]
      V [libjvm.so+0x2d60a8]
      V [libjvm.so+0x2d6410]
      V [libjvm.so+0x2d72f8]
      V [libjvm.so+0x70aa6c]
      V [libjvm.so+0x11a308]
      j java.util.Collections$SynchronizedMap.get(Ljava/lang/Object;)Ljava/lang/Object;+7
      j weblogic.management.jmx.ObjectNameManagerBase.lookupRegisteredObjectName(Ljava/lang/Object;)Ljavax/management/ObjectName;+5
      j weblogic.management.jmx.modelmbean.WLSModelMBeanContext.lookupOrRegister(Ljava/lang/Object;)Ljavax/management/ObjectName;+5
      j weblogic.management.jmx.modelmbean.WLSModelMBeanContext.mapToJMX(Ljava/lang/Class;Ljava/lang/Object;Ljavax/management/openmbean/OpenType;)Ljava/lang/Object;+290
      j weblogic.management.jmx.modelmbean.WLSModelMBean.getAttribute(Ljava/lang/String;)Ljava/lang/Object;+182
      j com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+107
      j com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+10
      j weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase$12.run()Ljava/lang/Object;+15
      v ~StubRoutines::call_stub
      V [libjvm.so+0x134704]
      V [libjvm.so+0x192b74]
      C [libjava.so+0xefd0] Java_java_security_AccessController_doPrivileged__Ljava_security_PrivilegedExceptionAction_2+0x1c
      j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+32
      j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+0
      j weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+16
      j weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase$12.run()Ljava/lang/Object;+15
      v ~StubRoutines::call_stub
      V [libjvm.so+0x134704]
      V [libjvm.so+0x192b74]
      C [libjava.so+0xefd0] Java_java_security_AccessController_doPrivileged__Ljava_security_PrivilegedExceptionAction_2+0x1c
      j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+32
      j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+0
      j weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+16
      j weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase$12.run()Ljava/lang/Object;+15
      v ~StubRoutines::call_stub
      V [libjvm.so+0x134704]
      V [libjvm.so+0x192b74]
      C [libjava.so+0xefd0] Java_java_security_AccessController_doPrivileged__Ljava_security_PrivilegedExceptionAction_2+0x1c
      j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+32
      j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+0
      j weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+16
      j weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase$12.run()Ljava/lang/Object;+15
      v ~StubRoutines::call_stub
      V [libjvm.so+0x134704]
      V [libjvm.so+0x192b74]
      C [libjava.so+0xefd0] Java_java_security_AccessController_doPrivileged__Ljava_security_PrivilegedExceptionAction_2+0x1c
      j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+32
      j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+0
      j weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+16
      j weblogic.management.mbeanservers.internal.SecurityInterceptor.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+9
      j weblogic.management.mbeanservers.internal.AuthenticatedSubjectInterceptor$5$1.run()Ljava/lang/Object;+18
      v ~StubRoutines::call_stub
      V [libjvm.so+0x134704]
      V [libjvm.so+0x192b74]
      C [libjava.so+0xefd0] Java_java_security_AccessController_doPrivileged__Ljava_security_PrivilegedExceptionAction_2+0x1c
      j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+32
      j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+0
      j weblogic.management.mbeanservers.internal.AuthenticatedSubjectInterceptor$5.run()Ljava/lang/Object;+20
      j weblogic.security.acl.internal.AuthenticatedSubject.doAs(Lweblogic/security/subject/AbstractSubject;Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+34
      j weblogic.management.mbeanservers.internal.AuthenticatedSubjectInterceptor.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+22
      j weblogic.management.jmx.mbeanserver.WLSMBeanServer.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+10
      j javax.management.remote.rmi.RMIConnectionImpl.doOperation(I[Ljava/lang/Object;)Ljava/lang/Object;+268
      j javax.management.remote.rmi.RMIConnectionImpl.access$200(Ljavax/management/remote/rmi/RMIConnectionImpl;I[Ljava/lang/Object;)Ljava/lang/Object;+3
      j javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run()Ljava/lang/Object;+12
      v ~StubRoutines::call_stub
      V [libjvm.so+0x134704]
      V [libjvm.so+0x192b74]
      C [libjava.so+0xf5d8] Java_java_security_AccessController_doPrivileged__Ljava_security_PrivilegedExceptionAction_2Ljava_security_AccessControlContext_2+0x1c
      j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;Ljava/security/AccessControlContext;)Ljava/lang/Object;+32
      j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;Ljava/security/AccessControlContext;)Ljava/lang/Object;+0
      j javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(I[Ljava/lang/Object;Ljavax/security/auth/Subject;)Ljava/lang/Object;+121
      j javax.management.remote.rmi.RMIConnectionImpl.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;Ljavax/security/auth/Subject;)Ljava/lang/Object;+77
      j javax.management.remote.rmi.RMIConnectionImpl_WLSkel.invoke(I[Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;+411
      j weblogic.rmi.internal.ServerRequest.sendReceive()Lweblogic/rmi/spi/InboundResponse;+216
      j weblogic.rmi.internal.BasicRemoteRef.invoke(Ljava/rmi/Remote;Lweblogic/rmi/extensions/server/RuntimeMethodDescriptor;[Ljava/lang/Object;Ljava/lang/reflect/Method;)Ljava/lang/Object;+129
      j javax.management.remote.rmi.RMIConnectionImpl_1030_WLStub.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;Ljavax/security/auth/Subject;)Ljava/lang/Object;+50
      j javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+66
      j javax.management.MBeanServerInvocationHandler.invoke(Ljava/lang/Object;Ljava/lang/reflect/Method;[Ljava/lang/Object;)Ljava/lang/Object;+162
      j weblogic.management.jmx.MBeanServerInvocationHandler.doInvoke(Ljava/lang/Object;Ljava/lang/reflect/Method;[Ljava/lang/Object;)Ljava/lang/Object;+219
      j weblogic.management.jmx.MBeanServerInvocationHandler.invoke(Ljava/lang/Object;Ljava/lang/reflect/Method;[Ljava/lang/Object;)Ljava/lang/Object;+495
      j $Proxy60.getConfigurationManager()Lweblogic/management/mbeanservers/edit/ConfigurationManagerMBean;+9
      j com.bea.console.utils.MBeanChangeManager.getCurrentLockOwner()Ljava/lang/String;+7
      j com.bea.console.utils.MBeanChangeManager.isCurrentLockOwner(Ljavax/servlet/http/HttpServletRequest;)Z+0
      j jsp_servlet._jsp._contentheader.__contentmenu._jspService(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+443
      j weblogic.servlet.jsp.JspBase.service(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+9
      j weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run()Ljava/lang/Object;+43
      j weblogic.servlet.internal.StubSecurityHelper.invokeServlet(Ljavax/servlet/ServletRequest;Ljavax/servlet/http/HttpServletRequest;Lweblogic/servlet/internal/ServletRequestImpl;Ljavax/servlet/ServletResponse;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/Servlet;)Ljava/lang/Throwable;+110
      j weblogic.servlet.internal.ServletStubImpl.execute(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Lweblogic/servlet/internal/FilterChainImpl;)V+356
      j weblogic.servlet.internal.ServletStubImpl.onAddToMapException(Ljava/lang/Throwable;Lweblogic/servlet/internal/ServletRequestImpl;Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Lweblogic/servlet/internal/FilterChainImpl;)V+261
      j weblogic.servlet.internal.ServletStubImpl.execute(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Lweblogic/servlet/internal/FilterChainImpl;)V+382
      j weblogic.servlet.internal.ServletStubImpl.execute(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+4
      j weblogic.servlet.internal.RequestDispatcherImpl.invokeServlet(ZLjavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Lweblogic/servlet/internal/ServletResponseImpl;I)V+105
      j weblogic.servlet.internal.RequestDispatcherImpl.include(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+495
      j com.bea.netuix.servlets.controls.content.JspContent.beginRender(Ljava/io/Writer;)Z+854
      j com.bea.netuix.nf.ControlLifecycle$7.visit(Lcom/bea/netuix/nf/UIControl;Ljava/lang/Object;)Z+94
      j com.bea.netuix.nf.ControlTreeWalker.walkRecursiveRender(Lcom/bea/netuix/nf/VisitorType;Lcom/bea/netuix/nf/UIControl;Ljava/lang/Object;)V+568
      j com.bea.netuix.nf.ControlTreeWalker.walk(Lcom/bea/netuix/nf/VisitorType;Lcom/bea/netuix/nf/UIControl;Lcom/bea/netuix/nf/concurrency/IControlTreeWalkerPool;)V+535
      j com.bea.netuix.nf.Lifecycle.processLifecycles(II[Lcom/bea/netuix/nf/VisitorType;Lcom/bea/netuix/nf/UIControl;Lcom/bea/netuix/nf/UIContext;Lcom/bea/netuix/nf/concurrency/IControlTreeWalkerPool;Z)I+147
      j com.bea.netuix.nf.Lifecycle.processLifecycles(I[Lcom/bea/netuix/nf/VisitorType;Lcom/bea/netuix/nf/UIControl;Lcom/bea/netuix/nf/UIContext;Lcom/bea/netuix/nf/concurrency/IControlTreeWalkerPool;)I+11
      j com.bea.netuix.nf.Lifecycle.processLifecycles(I[Lcom/bea/netuix/nf/VisitorType;Lcom/bea/netuix/nf/UIControl;Lcom/bea/netuix/nf/concurrency/IControlTreeWalkerPool;)I+12
      ...<more frames>...

      Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
      j java.util.Collections$SynchronizedMap.get(Ljava/lang/Object;)Ljava/lang/Object;+7
      j weblogic.management.jmx.ObjectNameManagerBase.lookupRegisteredObjectName(Ljava/lang/Object;)Ljavax/management/ObjectName;+5
      j weblogic.management.jmx.modelmbean.WLSModelMBeanContext.lookupOrRegister(Ljava/lang/Object;)Ljavax/management/ObjectName;+5
      j weblogic.management.jmx.modelmbean.WLSModelMBeanContext.mapToJMX(Ljava/lang/Class;Ljava/lang/Object;Ljavax/management/openmbean/OpenType;)Ljava/lang/Object;+290
      j weblogic.management.jmx.modelmbean.WLSModelMBean.getAttribute(Ljava/lang/String;)Ljava/lang/Object;+182
      j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+0
      j weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+16
      j weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase$12.run()Ljava/lang/Object;+15
      v ~StubRoutines::call_stub
      j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+0
      j weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+16
      j weblogic.management.mbeanservers.internal.SecurityInterceptor.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+9
      j weblogic.management.mbeanservers.internal.AuthenticatedSubjectInterceptor$5$1.run()Ljava/lang/Object;+18
      v ~StubRoutines::call_stub
      j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+0
      j weblogic.management.mbeanservers.internal.AuthenticatedSubjectInterceptor$5.run()Ljava/lang/Object;+20
      j weblogic.security.acl.internal.AuthenticatedSubject.doAs(Lweblogic/security/subject/AbstractSubject;Ljava/security/PrivilegedExceptionAction;)Ljava/lang/Object;+34
      j weblogic.management.mbeanservers.internal.AuthenticatedSubjectInterceptor.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+22
      j weblogic.management.jmx.mbeanserver.WLSMBeanServer.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+10
      j javax.management.remote.rmi.RMIConnectionImpl.doOperation(I[Ljava/lang/Object;)Ljava/lang/Object;+268
      j javax.management.remote.rmi.RMIConnectionImpl.access$200(Ljavax/management/remote/rmi/RMIConnectionImpl;I[Ljava/lang/Object;)Ljava/lang/Object;+3
      j javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run()Ljava/lang/Object;+12
      v ~StubRoutines::call_stub
      j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;Ljava/security/AccessControlContext;)Ljava/lang/Object;+0
      j javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(I[Ljava/lang/Object;Ljavax/security/auth/Subject;)Ljava/lang/Object;+121
      j javax.management.remote.rmi.RMIConnectionImpl.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;Ljavax/security/auth/Subject;)Ljava/lang/Object;+77
      j javax.management.remote.rmi.RMIConnectionImpl_WLSkel.invoke(I[Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;+411
      j weblogic.rmi.internal.ServerRequest.sendReceive()Lweblogic/rmi/spi/InboundResponse;+216
      j weblogic.rmi.internal.BasicRemoteRef.invoke(Ljava/rmi/Remote;Lweblogic/rmi/extensions/server/RuntimeMethodDescriptor;[Ljava/lang/Object;Ljava/lang/reflect/Method;)Ljava/lang/Object;+129
      j javax.management.remote.rmi.RMIConnectionImpl_1030_WLStub.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;Ljavax/security/auth/Subject;)Ljava/lang/Object;+50
      j javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.getAttribute(Ljavax/management/ObjectName;Ljava/lang/String;)Ljava/lang/Object;+66
      j javax.management.MBeanServerInvocationHandler.invoke(Ljava/lang/Object;Ljava/lang/reflect/Method;[Ljava/lang/Object;)Ljava/lang/Object;+162
      j weblogic.management.jmx.MBeanServerInvocationHandler.doInvoke(Ljava/lang/Object;Ljava/lang/reflect/Method;[Ljava/lang/Object;)Ljava/lang/Object;+219
      j weblogic.management.jmx.MBeanServerInvocationHandler.invoke(Ljava/lang/Object;Ljava/lang/reflect/Method;[Ljava/lang/Object;)Ljava/lang/Object;+495
      j $Proxy60.getConfigurationManager()Lweblogic/management/mbeanservers/edit/ConfigurationManagerMBean;+9
      j com.bea.console.utils.MBeanChangeManager.getCurrentLockOwner()Ljava/lang/String;+7
      j com.bea.console.utils.MBeanChangeManager.isCurrentLockOwner(Ljavax/servlet/http/HttpServletRequest;)Z+0
      j jsp_servlet._jsp._contentheader.__contentmenu._jspService(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+443
      j weblogic.servlet.jsp.JspBase.service(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+9
      j weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run()Ljava/lang/Object;+43
      j weblogic.servlet.internal.StubSecurityHelper.invokeServlet(Ljavax/servlet/ServletRequest;Ljavax/servlet/http/HttpServletRequest;Lweblogic/servlet/internal/ServletRequestImpl;Ljavax/servlet/ServletResponse;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/Servlet;)Ljava/lang/Throwable;+110


      --------------- P R O C E S S ---------------

      Java Threads: ( => current thread )
      0x021e1000 JavaThread "Thread-12" daemon [_thread_in_native, id=223, stack(0xc5280000,0xc5300000)]
      0x031f4800 JavaThread "[STANDBY] ExecuteThread: '5' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon [_thread_blocked, id=222, stack(0xc5380000,0xc5400000)]
      0x0323c400 JavaThread "[ACTIVE] ExecuteThread: '4' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon [_thread_blocked, id=221, stack(0xc5480000,0xc5500000)]
      0x0323d000 JavaThread "[STANDBY] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon [_thread_blocked, id=220, stack(0xc5580000,0xc5600000)]
      0x01e39000 JavaThread "[STANDBY] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon [_thread_blocked, id=219, stack(0xc5680000,0xc5700000)]
      0x01e39c00 JavaThread "DynamicListenThread[Default]" daemon [_thread_in_native, id=218, stack(0xc5780000,0xc5800000)]
      0x01b7a400 JavaThread "weblogic.GCMonitor" daemon [_thread_blocked, id=217, stack(0xc5880000,0xc5900000)]
      0x031ff000 JavaThread "Thread-11" daemon [_thread_blocked, id=216, stack(0xc5c80000,0xc5d00000)]
      0x02996400 JavaThread "DoSManager" daemon [_thread_blocked, id=215, stack(0xc5b80000,0xc5c00000)]
      0x02997000 JavaThread "VDE Replication Thread" daemon [_thread_blocked, id=213, stack(0xc5d80000,0xc5e00000)]
      0x02997400 JavaThread "VDE Transaction Processor Thread" daemon [_thread_blocked, id=212, stack(0xc5e80000,0xc5f00000)]
      0x02997c00 JavaThread "ExecuteThread: '96' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=211, stack(0xc5f80000,0xc6000000)]
      0x0298a400 JavaThread "ExecuteThread: '95' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=210, stack(0xc6080000,0xc6100000)]
      0x0298a800 JavaThread "ExecuteThread: '94' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=209, stack(0xc6180000,0xc6200000)]
      0x0298b000 JavaThread "ExecuteThread: '93' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=208, stack(0xc6280000,0xc6300000)]
      0x0298b400 JavaThread "ExecuteThread: '92' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=207, stack(0xc6380000,0xc6400000)]
      0x0298bc00 JavaThread "ExecuteThread: '91' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=206, stack(0xc6480000,0xc6500000)]
      0x02980400 JavaThread "ExecuteThread: '90' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=205, stack(0xc6580000,0xc6600000)]
      0x02980800 JavaThread "ExecuteThread: '89' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=204, stack(0xc6680000,0xc6700000)]
      0x02981000 JavaThread "ExecuteThread: '88' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=203, stack(0xc6780000,0xc6800000)]
      0x02981400 JavaThread "ExecuteThread: '87' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=202, stack(0xc6880000,0xc6900000)]
      0x02981c00 JavaThread "ExecuteThread: '86' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=201, stack(0xc6980000,0xc6a00000)]
      0x02974400 JavaThread "ExecuteThread: '85' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=200, stack(0xc6a80000,0xc6b00000)]
      0x02974800 JavaThread "ExecuteThread: '84' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=199, stack(0xc6b80000,0xc6c00000)]
      0x02975000 JavaThread "ExecuteThread: '83' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=198, stack(0xc6c80000,0xc6d00000)]
      0x02975400 JavaThread "ExecuteThread: '82' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=197, stack(0xc6d80000,0xc6e00000)]
      0x02975c00 JavaThread "ExecuteThread: '81' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=196, stack(0xc6e80000,0xc6f00000)]
      0x02956400 JavaThread "ExecuteThread: '80' for queue: 'weblogic.socket.Muxer'" daemon [_thread_blocked, id=195, stack(0xc6f80000,0xc7000000)]