0 Replies Latest reply on Jul 27, 2009 1:40 PM by 843810

    Calling GetStackTrace cause JVM crash

      Hi all,
      I have written a monitor agent ( using JVMTI ) to sample the stack trace for all threads in the process every 100 micro-seconds. On each sampling point, the agent will get all threads information and for each of the thread it uses "GetStackTrace" to dump the stack traces. But this GetStackTrace function sometimes causes JVM crash. On the other hand, using "GetAllStackTraces", it seems everything is OK. I have attached the error logs in the following part, could any one give some suggestion about it ? Is there any potential issue when calling the "GetStackTrace" function frequently? Thanks for your time and help.

      # A fatal error has been detected by the Java Runtime Environment:
      # SIGSEGV (0xb) at pc=0xf7734e5a, pid=18986, tid=1867705232
      # JRE version: 6.0_14-b08
      # Java VM: Java HotSpot(TM) Server VM (14.0-b16 mixed mode linux-x86 )
      # Problematic frame:
      # V [libjvm.so+0x2ebe5a]
      # If you would like to submit a bug report, please visit:
      # http://java.sun.com/webapps/bugreport/crash.jsp

      T H R E A D
      Current thread (0x0900f400): VMThread [stack: 0x6f4ae000,0x6f52f000] [id=18997]

      siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR), si_addr=0x00001124

      EAX=0x00001124, EBX=0xf7b5e5a0, ECX=0x00000002, EDX=0xf439f640
      ESP=0x6f52dd54, EBP=0x6f52dd78, ESI=0x6f52de00, EDI=0x0900f674
      EIP=0xf7734e5a, CR2=0x00001124, EFLAGS=0x00010212

      Top of Stack: (sp=0x6f52dd54)
      0x6f52dd54: 6f52dde0 6f52ddb8 f7734db0 f443b7c8
      0x6f52dd64: 00000000 00000000 f7b5e5a0 6f52de00
      0x6f52dd74: 6f52de00 6f52ddd8 f7731b16 6f52de00
      0x6f52dd84: 0900f674 6f52de20 f7734edd 6f52de40
      0x6f52dd94: 0907523c 6f52de60 6f52dda0 08f2de58
      0x6f52dda4: 00000001 00000001 f7b5e5a0 6f52de40
      0x6f52ddb4: 6f52de40 6f52de18 f7731b7b f439f608
      0x6f52ddc4: 0907523c 6f52de60 f7b5e5a0 00000000

      Instructions: (pc=0xf7734e5a)
      0xf7734e4a: f4 89 f0 5b 5e 5f 5d c2 04 00 8b 47 10 83 ec 0c
      0xf7734e5a: 8b 38 8d 48 08 8b 40 04 89 7e 10 89 0e 89 4e 14

      Stack: [0x6f4ae000,0x6f52f000], sp=0x6f52dd54, free space=511k
      Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
      V [libjvm.so+0x2ebe5a]
      V [libjvm.so+0x2e8b16]
      V [libjvm.so+0x64779c]
      V [libjvm.so+0x613c3a]
      V [libjvm.so+0x4694f7]
      V [libjvm.so+0x467793]
      V [libjvm.so+0x65e3d6]
      V [libjvm.so+0x65d883]
      V [libjvm.so+0x65daff]
      V [libjvm.so+0x65d5e0]
      V [libjvm.so+0x531ade]
      C [libpthread.so.0+0x545b]

      VM_Operation (0x6f1b95e0): GetStackTrace, mode: safepoint, requested by thread 0x6e3a2000

      P R O C E S S
      Java Threads: ( => current thread )
      0x6e3a2000 JavaThread "threaddump_thread" daemon [_thread_blocked, id=19001, stack(0x6e7ba000,0x6f1bb000)]
      0x6e326c00 JavaThread "LeaseChecker@DFSClient[clientName=DFSClient_attempt_200907271527_0001_m_000885_0, ugi=Grace,DP]: java.lang.Throwable: for testing
      at org.apache.hadoop.hdfs.DFSClient$LeaseChecker.toString(DFSClient.java:992)
      at org.apache.hadoop.util.Daemon.<init>(Daemon.java:38)
      at org.apache.hadoop.hdfs.DFSClient$LeaseChecker.put(DFSClient.java:919)
      at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:457)
      at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:190)
      at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:487)
      at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:387)
      at org.apache.hadoop.examples.terasort.TeraOutputFormat.getRecordWriter(TeraOutputFormat.java:85)
      at org.apache.hadoop.mapred.MapTask$DirectMapOutputCollector.<init>(MapTask.java:377)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
      at org.apache.hadoop.mapred.Child.main(Child.java:158)
      " daemon [_thread_blocked, id=19093, stack(0x6ce48000,0x6ce99000)]
      0x6e325400 JavaThread "Comm thread for attempt_200907271527_0001_m_000885_0" daemon [_thread_blocked, id=19089, stack(0x6ceea000,0x6cf3b000)]
      0x6e2e1c00 JavaThread "IPC Client (47) connection to / from Grace" daemon [_thread_blocked, id=19088, stack(0x6cf3b000,0x6cf8c000)]
      0x6e2df000 JavaThread "Timer thread for monitoring jvm" daemon [_thread_blocked, id=19085, stack(0x6cf8c000,0x6cfdd000)]
      0x6e498000 JavaThread "Thread for syncLogs" daemon [_thread_blocked, id=19079, stack(0x6cfdd000,0x6d02e000)]
      0x6e280c00 JavaThread "IPC Client (47) connection to / from an unknown user" daemon [_thread_blocked, id=19078, stack(0x6d02e000,0x6d07f000)]
      0x0902f000 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=19004, stack(0x6e667000,0x6e6b8000)]
      0x0902d400 JavaThread "CompilerThread1" daemon [_thread_blocked, id=19003, stack(0x6e6b8000,0x6e739000)]
      0x09028c00 JavaThread "CompilerThread0" daemon [_thread_blocked, id=19002, stack(0x6e739000,0x6e7ba000)]
      0x09027400 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=19000, stack(0x6f1bb000,0x6f20c000)]
      0x09014800 JavaThread "Finalizer" daemon [_thread_blocked, id=18999, stack(0x6f40c000,0x6f45d000)]
      0x09013400 JavaThread "Reference Handler" daemon [_thread_blocked, id=18998, stack(0x6f45d000,0x6f4ae000)]
      0x08f2a400 JavaThread "main" [_thread_blocked, id=18988, stack(0xf73f8000,0xf7449000)]

      Other Threads:
      =>0x0900f400 VMThread [stack: 0x6f4ae000,0x6f52f000] [id=18997]
      0x09031000 WatcherThread [stack: 0x6e5e6000,0x6e667000] [id=19005]

      VM state:at safepoint (normal execution)

      VM Mutex/Monitor currently owned by a thread: ([mutex/lock_event])
      [0x08f28a00] Threads_lock - owner thread: 0x0900f400

      PSYoungGen total 12800K, used 6914K [0xe5f50000, 0xe78f0000, 0xf42d0000)
      eden space 12288K, 52% used [0xe5f50000,0xe6592b00,0xe6b50000)
      from space 512K, 98% used [0xe6b50000,0xe6bce0e0,0xe6bd0000)
      to space 832K, 0% used [0xe7820000,0xe7820000,0xe78f0000)
      PSOldGen total 4096K, used 195K [0x742d0000, 0x746d0000, 0xe5f50000)
      object space 4096K, 4% used [0x742d0000,0x74300f20,0x746d0000)
      PSPermGen total 16384K, used 5996K [0x702d0000, 0x712d0000, 0x742d0000)
      object space 16384K, 36% used [0x702d0000,0x708ab1d8,0x712d0000)