This discussion is archived
4 Replies Latest reply: Jul 1, 2009 4:45 PM by 843829 RSS

JVM crashes in CompilerThread CompileTask

843829 Newbie
Currently Being Moderated
We got quite constant JVM crashes running some automated tests since we upgraded to 1.5.0_17. Note, that it seems like the JIT compiler is active even though the -Djava.compile=NONE flag is set. The crashes are always in the compile task when compiling the method org.apache.derby.impl.store.raw.data.StoredPage.logRow from the Derby project (recent stable version).

The crashes seem to occur a little less, when the debug-options are not set.

This has been tested with the sun-java5 package distributed with Debian Lenny and the same version (JDK) directly installed from the .bin-archive provided by Sun.

The following is a shortened hs_err_<pid>.log. The SIGSEGV, problematic frame and the method being compiled by the CompileTask are always the same!

The following thread has some similarities: http://forums.sun.com/thread.jspa?threadID=5280098 however, the referenced bugs are marked as fix released.

Any help on how to track this down or fix this is greatly appreciated!

Thanks in advance
Kevin Göser
#
# An unexpected error has been detected by HotSpot Virtual Machine:
#
#  SIGSEGV (0xb) at pc=0x00007f9c94f68a40, pid=17857, tid=1087859024
#
# Java VM: Java HotSpot(TM) 64-Bit Server VM (1.5.0_17-b04 mixed mode)
# Problematic frame:
# V  [libjvm.so+0x286a40]
#

---------------  T H R E A D  ---------------

Current thread (0x00007f9c7c021c90):  JavaThread "CompilerThread1" daemon [_thread_in_native, id=17868]

siginfo:si_signo=11, si_errno=0, si_code=1, si_addr=0x0000000000000008

Registers:
RAX=0x0000000000000001, RBX=0x00007f9c7cf23c60, RCX=0x00007f9c7c021c90, RDX=0x00007f9c7cf23c70
RSP=0x0000000040d73e58, RBP=0x0000000040d73ea0, RSI=0x0000000000000010, RDI=0x0000000000000000
R8 =0x00007f9c7ce3f890, R9 =0x000000000000000b, R10=0x00007f9c7ca3d530, R11=0x00000000404d92d0
R12=0x00000000404d92d0, R13=0x0000000040d74540, R14=0x0000000040d74f00, R15=0x00007f9c7cf23940
RIP=0x00007f9c94f68a40, EFL=0x0000000000010293, CSGSFS=0x0000000000000033, ERR=0x0000000000000004
  TRAPNO=0x000000000000000e

Top of Stack: (sp=0x0000000040d73e58)
0x0000000040d73e58:   00007f9c952526af 0000000040d73ea0
0x0000000040d73e68:   00007f9c7cf23940 ffffffff40d73ea0
0x0000000040d73e78:   00007f9c7cf23940 0000000000000027
...

Instructions: (pc=0x00007f9c94f68a40)
0x00007f9c94f68a30:   55 48 c1 ff 03 48 89 e5 89 f8 c9 c3 66 66 66 90
0x00007f9c94f68a40:   48 8b 47 08 55 48 89 e5 c9 c3 66 66 90 66 66 90 

Stack: [0x0000000040c76000,0x0000000040d77000),  sp=0x0000000040d73e58,  free space=1015k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0x286a40]
V  [libjvm.so+0x570c8c]
.....

Current CompileTask:
opto: 13% !   org.apache.derby.impl.store.raw.data.StoredPage.logRow(IZI[Ljava/lang/Object;Lorg/apache/derby/iapi/services/io/FormatableBitSet;Lorg/apache/derby/iapi/services/io/DynamicByteArrayOutputStream;IBIII)I @ 511 (1279 bytes)


---------------  P R O C E S S  ---------------

Java Threads: ( => current thread )
....
=>0x00007f9c7c021c90 JavaThread "CompilerThread1" daemon [_thread_in_native, id=17868]
....

Other Threads:
  0x00000000405188d0 VMThread [id=17860]
  0x00007f9c7c0253c0 WatcherThread [id=17870]

VM state:not at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread: None

Heap
 def new generation   total 7616K, used 2279K [0x00007f9c835c0000, 0x00007f9c83e00000, 0x00007f9c851c0000)
  eden space 6784K,  21% used [0x00007f9c835c0000, 0x00007f9c83729f68, 0x00007f9c83c60000)
  from space 832K, 100% used [0x00007f9c83d30000, 0x00007f9c83e00000, 0x00007f9c83e00000)
  to   space 832K,   0% used [0x00007f9c83c60000, 0x00007f9c83c60000, 0x00007f9c83d30000)
 tenured generation   total 16776K, used 10180K [0x00007f9c851c0000, 0x00007f9c86222000, 0x00007f9c889c0000)
   the space 16776K,  60% used [0x00007f9c851c0000, 0x00007f9c85bb1060, 0x00007f9c85bb1200, 0x00007f9c86222000)
 compacting perm gen  total 27008K, used 26908K [0x00007f9c889c0000, 0x00007f9c8a420000, 0x00007f9c8ddc0000)
   the space 27008K,  99% used [0x00007f9c889c0000, 0x00007f9c8a4071a0, 0x00007f9c8a407200, 0x00007f9c8a420000)
No shared spaces configured.

....

VM Arguments:
jvm_args: -Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8000 ... (truncated)

---------------  S Y S T E M  ---------------

OS:5.0

uname:Linux 2.6.26-1-amd64 #1 SMP Mon Dec 15 17:25:36 UTC 2008 x86_64
libc:glibc 2.7 NPTL 2.7 
rlimit: STACK 8192k, CORE 0k, NPROC 14336, NOFILE 1024, AS infinity
load average:3.60 3.75 3.11

CPU:total 2 em64t

Memory: 4k page, physical 1803480k(1175480k free), swap 497972k(248992k free)

vm_info: Java HotSpot(TM) 64-Bit Server VM (1.5.0_17-b04) for linux-amd64, built on Nov 10 2008 01:04:27 by java_re with gcc 3.2.2 (SuSE Linux)
Edited by: kgoeser on Feb 12, 2009 1:54 AM
  • 1. Re: JVM crashes in CompilerThread CompileTask
    843829 Newbie
    Currently Being Moderated
    We have many of these too. I believe there is a defect in the JIT compiler causing this. It is not easy to make a simple test condition to demonstrate it.

    What we have done is to exclude certain classes from JIT compilation - for example:

    -XX:CompileCommand=exclude,java/util/Arrays.mergeSort

    However, eventually the same error crops up on a different class.
  • 2. Re: JVM crashes in CompilerThread CompileTask
    843829 Newbie
    Currently Being Moderated
    We also are infected by the same issue...

    #
    # An unexpected error has been detected by HotSpot Virtual Machine:
    #
    # SIGSEGV (0xb) at pc=0x00002b06d0bcc712, pid=22067, tid=1101850944
    #
    # Java VM: Java HotSpot(TM) 64-Bit Server VM (1.5.0_17-b04 mixed mode)

    Still no solution yet.. but just wondering if it has to do anything with the memory configuration (GC)

    Logs shows below 99% for space..
    PSYoungGen total 307840K, used 145482K [0x00002b070a3b0000, 0x00002b071f900000, 0x00002b071f900000)
    eden space 266176K, 39% used [0x00002b070a3b0000,0x00002b07109164a8,0x00002b071a7a0000)
    from space 41664K, 99% used [0x00002b071a7a0000,0x00002b071d04c380,0x00002b071d050000)
    to space 41664K, 0% used [0x00002b071d050000,0x00002b071d050000,0x00002b071f900000)

    Let me know if you find anything...
  • 3. Re: JVM crashes in CompilerThread CompileTask
    843829 Newbie
    Currently Being Moderated
    I found that when I moved from JDK 1.5.0_16 to JDK1.5.0_17 that I would occasionally get a JVM crash in the ComplieTask during the deployment of our applicatioin EAR in JBoss.

    I found that this occured if I configured "-Xdebug -Xrunjdwp:server=y,suspend=n,transport=dt_socket,address=7777".
    If I left out these parameters I would not get a crash.
  • 4. Re: JVM crashes in CompilerThread CompileTask
    843829 Newbie
    Currently Being Moderated
    So why don't you report it after all?