This discussion is archived
2 Replies Latest reply: Feb 17, 2012 4:22 PM by 263312 RSS

repeated jrockit crash

263312 Newbie
Currently Being Moderated
It doesn't happen too frequently and unfortunately I have no small reproducible test case, but I've experienced this crash a few times with the same signature.

Is it a known bug? Any thoughts on how to proceed?

I'll post the crash dumps as replies
  • 1. Re: repeated jrockit crash
    263312 Newbie
    Currently Being Moderated
    ===== BEGIN DUMP =============================================================
    JRockit dump produced after 0 days, 00:00:16 on Wed Jan 18 09:44:53 2012

    **************************************************************************
    * If you see this dump, please go to *
    * http://download.oracle.com/docs/cd/E15289_01/go2troubleshooting.html *
    * for troubleshooting information. *
    **************************************************************************

    Additional information is available in:
    /export1/cruise/jenkins/workspace/terracotta-session_trunk_tests_tomcat-7.0.16_JROCKIT_1.6_3.7.x/trunk/system-tests/target/temp/SynchWriteMultiThreadsTest/sandbox/server_1/jrockit.9896.dump
    No snapshot file (core dump) will be created because core dumps have been
    disabled. To enable core dumping, try "ulimit -c unlimited"
    before starting JRockit again.

    Error Message: Illegal memory access. [54]
    Signal info : si_signo=11, si_code=1 si_addr=0xfffffff8
    Version : Oracle JRockit(R) R28.1.4-7-144370-1.6.0_26-20110617-2130-linux-ia32
    CPU : AMD Opteron QC/Phenom SSE SSE2 SSE3 SSE4A AMD64
    Number CPUs : 8
    Tot Phys Mem : 6442627072 (6144 MB)
    OS version : SUSE Linux Enterprise Server 10 (x86_64)
    VERSION = 10
    PATCHLEVEL = 4
    Linux version 2.6.16.60-0.85.1-xen (geeko@buildhost) (gcc version 4.1.2 20070115 (SUSE Linux)) #1 SMP Thu Mar 17 11:45:06 UTC 2011 (x86_64)
    Hypervisor : Xen v3.4
    Thread System: Linux NPTL
    LibC release : 2.4-stable
    Java locking : Lazy unlocking enabled (class banning) (transfer banning)
    State : JVM is running
    Command Line : -XX:+HeapDumpOnOutOfMemoryError -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=63629 -Dcom.tc.session.debug.sessions=true -Drmi.registry.port=17295 -verbose:gc -Xverboselog:/export1/cruise/jenkins/workspace/terracotta-session_trunk_tests_tomcat-7.0.16_JROCKIT_1.6_3.7.x/trunk/system-tests/target/temp/SynchWriteMultiThreadsTest/sandbox/server_1-gc.log -Djvmroute=server_1 -Xms128m -Xmx192m -Djava.security.egd=file:/dev/./urandom -DNODE=server_1 -Dcatalina.home=/export1/cruise/.tc/appservers/tomcat-7.0.16 -Dcatalina.base=/export1/cruise/jenkins/workspace/terracotta-session_trunk_tests_tomcat-7.0.16_JROCKIT_1.6_3.7.x/trunk/system-tests/target/temp/SynchWriteMultiThreadsTest/sandbox/server_1 -Djava.io.tmpdir=/export1/cruise/jenkins/workspace/terracotta-session_trunk_tests_tomcat-7.0.16_JROCKIT_1.6_3.7.x/trunk/system-tests/target/temp/SynchWriteMultiThreadsTest/sandbox/server_1/temp -Dsun.java.launcher=SUN_STANDARD com.tc.lcp.CargoLinkedChildProcess org.apache.catalina.startup.Bootstrap 55239 /export1/cruise/jenkins/workspace/terracotta-session_trunk_tests_tomcat-7.0.16_JROCKIT_1.6_3.7.x/trunk/system-tests/target/temp/SynchWriteMultiThreadsTest/sandbox/server_1 start
    Repository : /export1/cruise/jenkins/workspace/terracotta-session_trunk_tests_tomcat-7.0.16_JROCKIT_1.6_3.7.x/trunk/system-tests/target/temp/SynchWriteMultiThreadsTest/sandbox/server_1/temp/2012_01_18_09_44_39_9896
    java.home : /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre
    j.class.path : /export1/cruise/.tc/appservers/tomcat-7.0.16/bin/tomcat-juli.jar:/export1/cruise/.tc/appservers/tomcat-7.0.16/bin/bootstrap.jar:/export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/lib/tools.jar:/export1/cruise/.m2/repository/org/terracotta/linked-child-process/1.0.6/linked-child-process-1.0.6.jar
    j.lib.path : /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386/jrockit:/export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386:/export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/../lib/i386:/shares/monkeyshare/setup/jdk/linux/hotspot1.6.0_30/jre/lib/i386/server:/shares/monkeyshare/setup/jdk/linux/hotspot1.6.0_30/jre/lib/i386:/shares/monkeyshare/setup/jdk/linux/hotspot1.6.0_30/jre/../lib/i386
    JAVA_HOME : /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre
    JAVAOPTIONS: <not set>
    LD_LIBRARY_PATH: /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386/jrockit:/export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386:/export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/../lib/i386:/shares/monkeyshare/setup/jdk/linux/hotspot1.6.0_30/jre/lib/i386/server:/shares/monkeyshare/setup/jdk/linux/hotspot1.6.0_30/jre/lib/i386:/shares/monkeyshare/setup/jdk/linux/hotspot1.6.0_30/jre/../lib/i386
    LD_ASSUME_KERNEL: <not set>
    LD_PRELOAD : <not set>
    StackOverFlow: 0 StackOverFlowErrors have occured
    OutOfMemory : 0 OutOfMemoryErrors have occured
    C Heap : Good; no memory allocations have failed
    GC Strategy : Mode: throughput, with strategy: genparpar (basic strategy: genparpar)
    GC Status : OC is not running. Last finished OC was OC#1.
    : YC is not running. Last finished YC was YC#9.
    YC Promotion : Last YC successfully promoted all objects
    YC History : Ran 0 YCs before OC#1.
    : Ran 9 YCs since last OC.
    Heap : 0xe9ebb000 - 0xf1ebb000 (Size: 128 MB)
    Compaction : (no compaction area)
    Allocation : TLA-min: 2048, TLA-preferred: 24576 TLA-waste limit: 2048
    NurseryList : 0xe9ec8958 - 0xecf8cef8
    KeepArea : 0xeb698720 - 0xec716a50
    KA Markers : [ 0xeb698720,  0xec716a50 , 0xecf8cef8 ]
    Forbidden A : (none)
    Previous KA : 0xec716a50 - 0xecf8cef8
    Previous FA : (none)
    CompRefs : References are 32-bit.

    Registers (from ThreadContext: 0xe663a7ac:
    eax = ffffffff ecx = e6618dc4 edx = 00000000 ebx = e663ab3c
    esp = e663aaa0 ebp = e663aad8 esi = fffffff8 edi = e663abe8
    es = 0000002b cs = 00000023 ss = 0000002b ds = 0000002b
    fs = 00000003 gs = 00000063
    eip = f74e100d eflags = 00200282

    Loaded modules:
    (* denotes the module where the exception occured)
    08048000-08057193 /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/bin/java
    ffffe000-ffffe7c7 /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/bin/java
    f7761000-f776304b /lib/libdl.so.2
    f774a000-f775cec7 /lib/libpthread.so.0
    f7608000-f774242d /lib/libc.so.6
    f7770000-f778b2d3 /lib/ld-linux.so.2
    f72ce000-f7590317 */export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386/jrockit/libjvm.so
    f72b1000-f72cb4eb /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386/libosal.so
    f72a5000-f72af5cf /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386/libutil.so
    f727e000-f72a2f37 /lib/libm.so.6
    f7274000-f727b0dc /lib/librt.so.1
    f6887000-f6893747 /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386/libjfr.so
    e9760000-e976a9bb /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386/libverify.so
    e973b000-e975df07 /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386/libjava.so
    e971b000-e972d20b /lib/libnsl.so.1
    f7767000-f776ce50 /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386/native_threads/libhpi.so
    e93c2000-e93d06e4 /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386/libzip.so
    e9734000-e9739a06 /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386/libmanagement.so
    e90cb000-e90de150 /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386/libnet.so
    e8d1c000-e8d24f93 /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386/libjmapi.so
    e8cb3000-e8cb8013 /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386/liborii.so
    e7ef2000-e7ef84e4 /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386/libnio.so

    Stack:
    (* marks the word pointed to by the stack pointer)
    e663aaa0: e663aac4* e6c66a30 0000000c f7632cfe e6c66af8 00000000
    e663aab8: 00000000 ffffffff ffffffff e6618dc4 04000009 ffffffff
    e663aad0: e663ab3c e663ab3c e663aae8 f74e10f5 e663ab3c 00000000
    e663aae8: e663ab28 f74e1507 e6c63cd8 f74e0df0 e663ab3c e663ab3c

    Code:
    (* marks the word pointed to by the instruction pointer)
    f74e0fdc: 1c4389ec 89f0458b 458b2043 0c4383f4 24438901 5b30c483
    f74e0ff4: 90c35d5e 097407a8 012843c7 8b000000 83c68901 7389f8e6
    f74e100c: a8068b14* 74c18901 fee08305 7983088b 40740344 8d48418b
    f74e1024: 43890604 90afeb18 0026748d 85e0558b 8b4174d2 4389e045


    Last optimized methods:
    No methods optimized.

    Thread:
    ""http-bio-14955"-exec-15" id=115 idx=0x1d8 tid=10067 lastJavaFrame=0xe663adbc
    Stack 0: start=0xe661a000, end=0xe663c000, guards=0xe661f000 (ok), forbidden=0xe661d000

    Thread Stack Trace:
    at findNext+253(refiter.c:409)@0xf74e100d
    at findNextToReturn+100(refiter.c:302)@0xf74e10f5
    at refIterFillFromFrame+342(refiter.c:123)@0xf74e1507
    at trProcessLocksForThread+73(roots.c:620)@0xf74ec4ea
    at get_all_locks+95(javalock.c:1265)@0xf73ea810
    at javaLockConvertLazyToThin+133(javalock.c:1760)@0xf73ea9f6
    at RJNI_jrockit_vm_Locks_checkAndTransferLazyLocked+486(javalock.c:2032)@0xf73eba27
    -- Java stack --
    at jrockit/vm/Locks.checkAndTransferLazyLocked(Ljava/lang/Object;)I(Native Method)
    at jrockit/vm/Locks.monitorEnterSecondStage(Locks.java:971)
    at jrockit/vm/Locks.monitorEnter(Locks.java:2179)
    at EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap.getTableForReading()[LEDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry;(Unknown Source)
        at EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap.get(Ljava/lang/Object;)Ljava/lang/Object;(Unknown Source)
        at com/tc/object/PortabilityImpl.isPortableClass(PortabilityImpl.java:122)
        at com/tc/object/ClientObjectManagerImpl.isPortableClass(ClientObjectManagerImpl.java:1225)
        at com/tc/object/TCClassImpl.createPortableFields(TCClassImpl.java:341)
        at com/tc/object/TCClassImpl.<init>(TCClassImpl.java:119)
        at com/tc/object/TCClassFactoryImpl.createTCClass(TCClassFactoryImpl.java:95)
        at com/tc/object/TCClassFactoryImpl.getOrCreate(TCClassFactoryImpl.java:72)
        at com/tc/object/ClientObjectManagerImpl.lookup(ClientObjectManagerImpl.java:567)
        at com/tc/object/ClientObjectManagerImpl.lookupObject(ClientObjectManagerImpl.java:463)
        at com/tc/object/ClientObjectManagerImpl.lookupObject(ClientObjectManagerImpl.java:451)
        at com/tc/object/bytecode/ManagerImpl.lookupObject(ManagerImpl.java:484)
        at com/tc/object/bytecode/ManagerUtil.lookupObject(ManagerUtil.java:389)
        at com/terracotta/toolkit/collections/ConcurrentDistributedMapDso.lookupAndFaultIn(ConcurrentDistributedMapDso.java:708)
        at com/terracotta/toolkit/collections/ConcurrentDistributedMapDso.lookupAndFaultIn(ConcurrentDistributedMapDso.java:702)
        at com/terracotta/toolkit/collections/ConcurrentDistributedMapDso.get(ConcurrentDistributedMapDso.java:198)
        at com/terracotta/toolkit/collections/ConcurrentDistributedMapDsoArray.get(ConcurrentDistributedMapDsoArray.java:249)
        at org/terracotta/collections/ConcurrentDistributedMap.get(ConcurrentDistributedMap.java:220)
        at com/terracotta/session/SessionDataStore.find(SessionDataStore.java:145)
        at com/terracotta/session/TerracottaSessionManager.getSessionIfExists(TerracottaSessionManager.java:573)
        at com/terracotta/session/SessionRequest.getTerracottaSession(SessionRequest.java:150)
        at com/terracotta/session/SessionRequest.getSession(SessionRequest.java:109)
        at org/apache/catalina/connector/SessionRequest55.getSession(SessionRequest55.java:57)
        at org/apache/catalina/connector/SessionRequest55.getSessionInternal(SessionRequest55.java:105)
        at org/apache/catalina/authenticator/AuthenticatorBase.invoke(AuthenticatorBase.java:417)
        at org/terracotta/modules/tomcat/tomcat_7_0/SessionValve70.tcInvoke(SessionValve70.java:97)
        at org/terracotta/modules/tomcat/tomcat_7_0/SessionValve70.invoke(SessionValve70.java:82)
        at org/terracotta/session/ModernTomcatSessionValve.invoke(ModernTomcatSessionValve.java:66)
        at org/apache/catalina/core/StandardHostValve.invoke(StandardHostValve.java:164)
        at org/apache/catalina/valves/ErrorReportValve.invoke(ErrorReportValve.java:100)
        at org/apache/catalina/valves/AccessLogValve.invoke(AccessLogValve.java:563)
        at org/apache/catalina/core/StandardEngineValve.invoke(StandardEngineValve.java:118)
        at org/apache/catalina/connector/CoyoteAdapter.service(CoyoteAdapter.java:403)
        at org/apache/coyote/http11/Http11Processor.process(Http11Processor.java:301)
        at org/apache/coyote/http11/Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:162)
        at org/apache/coyote/http11/Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:140)
        at org/apache/tomcat/util/net/JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:309)
        at java/util/concurrent/ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java/util/concurrent/ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java/lang/Thread.run(Thread.java:662)
        at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
        -- end of trace

    Memory usage report:
    Total mapped                   322484KB           (reserved=69932KB)
    -              Java heap       196608KB           (reserved=65536KB)
    -              GC tables         6588KB         
    -          Thread stacks        24204KB           (#threads=114)
    -          Compiled code         4480KB           (used=4366KB)
    -               Internal          968KB         
    -                     OS        25628KB         
    -                  Other        37640KB         
    -        Java class data        25344KB           (malloced=25087KB #35708 in 6424 classes)
                                                      Not tracing sites.

    - Native memory tracking         1024KB           (malloced=99KB #8)
                                                      Not tracing sites.


    Set the env variable TRACE_ALLOC_SITES=1 or use the print_memusage switch
    trace_alloc_sites=true to enable alloc site tracing.

        **************************************************************************
        *  If you see this dump, please go to                                    *
        *  http://download.oracle.com/docs/cd/E15289_01/go2troubleshooting.html  *
        *  for troubleshooting information.                                      *
        **************************************************************************

    ===== END DUMP ===============================================================                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           
  • 2. Re: repeated jrockit crash
    263312 Newbie
    Currently Being Moderated
    ===== BEGIN DUMP =============================================================
    JRockit dump produced after 0 days, 00:00:23 on Sat Dec 31 11:02:53 2011

    **************************************************************************
    * If you see this dump, please go to *
    * http://download.oracle.com/docs/cd/E15289_01/go2troubleshooting.html *
    * for troubleshooting information. *
    **************************************************************************

    Additional information is available in:
    /export1/cruise/jenkins/workspace/terracotta-session_trunk_tests_jetty-7.4.4_JROCKIT_1.6_3.7.x/trunk/system-tests/target/temp/SynchWriteMultiThreadsTest/sandbox/server_1/jrockit.19791.dump
    No snapshot file (core dump) will be created because core dumps have been
    disabled. To enable core dumping, try "ulimit -c unlimited"
    before starting JRockit again.

    Error Message: Illegal memory access. [54]
    Signal info : si_signo=11, si_code=1 si_addr=0xfffffff8
    Version : Oracle JRockit(R) R28.1.4-7-144370-1.6.0_26-20110617-2130-linux-ia32
    CPU : AMD Opteron QC/Phenom SSE SSE2 SSE3 SSE4A AMD64
    Number CPUs : 4
    Tot Phys Mem : 6442450944 (6144 MB)
    OS version : Red Hat Enterprise Linux Server release 5.6 (Tikanga)
    Linux version 2.6.18-238.9.1.el5xen (mockbuild@x86-002.build.bos.redhat.com) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-50)) #1 SMP Fri Mar 18 12:53:42 EDT 2011 (x86_64)
    Hypervisor : Xen v3.4
    Thread System: Linux NPTL
    LibC release : 2.5-stable
    Java locking : Lazy unlocking enabled (class banning) (transfer banning)
    State : JVM is running (Main thread has finished)
    Command Line : -XX:+HeapDumpOnOutOfMemoryError -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=4964 -Dcom.tc.session.debug.sessions=true -Drmi.registry.port=13087 -verbose:gc -Xverboselog:/export1/cruise/jenkins/workspace/terracotta-session_trunk_tests_jetty-7.4.4_JROCKIT_1.6_3.7.x/trunk/system-tests/target/temp/SynchWriteMultiThreadsTest/sandbox/server_1-gc.log -Djava.security.egd=file:/dev/./urandom -Djetty.home=/export1/cruise/.tc/appservers/jetty-7.4.4 -Djetty.port=14976 -Djava.io.tmpdir=/export1/cruise/jenkins/workspace/terracotta-session_trunk_tests_jetty-7.4.4_JROCKIT_1.6_3.7.x/trunk/system-tests/target/temp/SynchWriteMultiThreadsTest/sandbox/work -Dsun.java.launcher=SUN_STANDARD com.tc.lcp.CargoLinkedChildProcess org.eclipse.jetty.start.Main 51660 /export1/cruise/jenkins/workspace/terracotta-session_trunk_tests_jetty-7.4.4_JROCKIT_1.6_3.7.x/trunk/system-tests/target/temp/SynchWriteMultiThreadsTest/sandbox/server_1 STOP.PORT=21626 STOP.KEY=secret OPTIONS=All /export1/cruise/jenkins/workspace/terracotta-session_trunk_tests_jetty-7.4.4_JROCKIT_1.6_3.7.x/trunk/system-tests/target/temp/SynchWriteMultiThreadsTest/sandbox/server_1/jetty.xml
    Repository : /export1/cruise/jenkins/workspace/terracotta-session_trunk_tests_jetty-7.4.4_JROCKIT_1.6_3.7.x/trunk/system-tests/target/temp/SynchWriteMultiThreadsTest/sandbox/work/2011_12_31_11_02_31_19791
    java.home : /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre
    j.class.path : /export1/cruise/.tc/appservers/jetty-7.4.4/start.jar:/export1/cruise/.m2/repository/org/terracotta/linked-child-process/1.0.6/linked-child-process-1.0.6.jar
    j.lib.path : /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386/jrockit:/export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386:/export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/../lib/i386:/shares/monkeyshare/setup/jdk/linux/hotspot1.6.0_27/jre/lib/i386/server:/shares/monkeyshare/setup/jdk/linux/hotspot1.6.0_27/jre/lib/i386:/shares/monkeyshare/setup/jdk/linux/hotspot1.6.0_27/jre/../lib/i386
    JAVA_HOME : /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26
    JAVAOPTIONS: <not set>
    LD_LIBRARY_PATH: /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386/jrockit:/export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386:/export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/../lib/i386:/shares/monkeyshare/setup/jdk/linux/hotspot1.6.0_27/jre/lib/i386/server:/shares/monkeyshare/setup/jdk/linux/hotspot1.6.0_27/jre/lib/i386:/shares/monkeyshare/setup/jdk/linux/hotspot1.6.0_27/jre/../lib/i386
    LD_ASSUME_KERNEL: <not set>
    LD_PRELOAD : <not set>
    StackOverFlow: 0 StackOverFlowErrors have occured
    OutOfMemory : 0 OutOfMemoryErrors have occured
    C Heap : Good; no memory allocations have failed
    GC Strategy : Mode: throughput, with strategy: genparpar (basic strategy: genparpar)
    GC Status : OC is not running. Last finished OC was OC#16.
    : YC is not running. Last finished YC was YC#141.
    YC Promotion : Last YC successfully promoted all objects
    YC History : Ran 0 YCs before OC#12.
    : Ran 0 YCs before OC#13.
    : Ran 0 YCs before OC#14.
    : Ran 2 YCs before OC#15.
    : Ran 6 YCs before OC#16.
    : Ran 2 YCs since last OC.
    Heap : 0xb6605000 - 0xbeabd000 (Size: 133 MB)
    Heap History : OC#8 - 0xba605000 (64 MB -> 77 MB; +13108 KB)
    : OC#12 - 0xbb2d2000 (77 MB -> 92 MB; +15732 KB)
    : OC#15 - 0xbc22f000 (92 MB -> 111 MB; +18876 KB)
    : OC#16 - 0xbd49e000 (111 MB -> 133 MB; +22652 KB)
    Compaction : (no compaction area)
    Allocation : TLA-min: 2048, TLA-preferred: 65536 TLA-waste limit: 2048
    NurseryList : 0xb6ca5008 - 0xbbb0bf68
    KeepArea : 0xb9be8168 - 0xbaf15878
    KA Markers : [ 0xb919cb58,  0xb9be8168 , 0xbaf15878 ]
    Forbidden A : (none)
    Previous KA : 0xb919cb58 - 0xb9be8168
    Previous FA : (none)
    CompRefs : References are 32-bit.

    Registers (from ThreadContext: 0xb24e83dc:
    eax = ffffffff ecx = b22baa1c edx = 00000000 ebx = b24e876c
    esp = b24e86d0 ebp = b24e8708 esi = fffffff8 edi = b24e8818
    es = 0000002b cs = b2fb0023 ss = 0000002b ds = 0000002b
    fs = 00000003 gs = 00000063
    eip = f7e2c00d eflags = 00000282

    Loaded modules:
    (* denotes the module where the exception occured)
    08048000-08057193 /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/bin/java
    ffffe000-ffffe7bf /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/bin/java
    00674000-006760fb /lib/libdl.so.2
    00659000-0066df17 /lib/libpthread.so.0
    004fe000-00650723 /lib/libc.so.6
    004df000-004f9457 /lib/ld-linux.so.2
    f7c19000-f7edb317 */export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386/jrockit/libjvm.so
    f7bfc000-f7c164eb /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386/libosal.so
    f7bf0000-f7bfa5cf /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386/libutil.so
    f7bc7000-f7beda23 /lib/libm.so.6
    f7bbe000-f7bc4ebf /lib/librt.so.1
    f6fd1000-f6fdd747 /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386/libjfr.so
    b42d6000-b42e09bb /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386/libverify.so
    b42b1000-b42d3f07 /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386/libjava.so
    007d4000-007e87a7 /lib/libnsl.so.1
    f7f55000-f7f5ae50 /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386/native_threads/libhpi.so
    b3e13000-b3e216e4 /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386/libzip.so
    b3b8e000-b3b93a06 /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386/libmanagement.so
    b3b79000-b3b8c150 /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386/libnet.so
    b37ca000-b37d2f93 /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386/libjmapi.so
    b3721000-b3726013 /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386/liborii.so
    b3607000-b360d4e4 /export1/cruise/jenkins/tools/JROCKIT_1.6/jrockit-1.6.0_26/jre/lib/i386/libnio.so

    Stack:
    (* marks the word pointed to by the stack pointer)
    b24e86d0: b24e86f4* 097f5508 0000000c 0000000c 00000002 f7e2bdf0
    b24e86e8: 00000000 ffffffff ffffffff b22baa1c 04000009 ffffffff
    b24e8700: b24e876c b24e876c b24e8718 f7e2c0f5 b24e876c 00000000
    b24e8718: b24e8758 f7e2c507 097f5270 f7e2bdf0 b24e876c b24e876c

    Code:
    (* marks the word pointed to by the instruction pointer)
    f7e2bfdc: 1c4389ec 89f0458b 458b2043 0c4383f4 24438901 5b30c483
    f7e2bff4: 90c35d5e 097407a8 012843c7 8b000000 83c68901 7389f8e6
    f7e2c00c: a8068b14* 74c18901 fee08305 7983088b 40740344 8d48418b
    f7e2c024: 43890604 90afeb18 0026748d 85e0558b 8b4174d2 4389e045


    Last optimized methods:
    No methods optimized.

    Thread:
    "qtp386208759-54" id=54 idx=0xdc tid=19853 lastJavaFrame=0xb24e89ec
    Stack 0: start=0xb24c8000, end=0xb24ea000, guards=0xb24cd000 (ok), forbidden=0xb24cb000

    Thread Stack Trace:
    at findNext+253(refiter.c:409)@0xf7e2c00d
    at findNextToReturn+100(refiter.c:302)@0xf7e2c0f5
    at refIterFillFromFrame+342(refiter.c:123)@0xf7e2c507
    at trProcessLocksForThread+73(roots.c:620)@0xf7e374ea
    at get_all_locks+95(javalock.c:1265)@0xf7d35810
    at javaLockConvertLazyToThin+133(javalock.c:1760)@0xf7d359f6
    at RJNI_jrockit_vm_Locks_checkAndTransferLazyLocked+486(javalock.c:2032)@0xf7d36a27
    -- Java stack --
    at jrockit/vm/Locks.checkAndTransferLazyLocked(Ljava/lang/Object;)I(Native Method)
    at jrockit/vm/Locks.monitorEnterSecondStage(Locks.java:971)
    at jrockit/vm/Locks.monitorEnter(Locks.java:2179)
    at EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap.getTableForReading()[LEDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap$Entry;(Unknown Source)
        at EDU/oswego/cs/dl/util/concurrent/ConcurrentReaderHashMap.get(Ljava/lang/Object;)Ljava/lang/Object;(Unknown Source)
        at com/tc/object/PortabilityImpl.isClassPhysicallyInstrumented(PortabilityImpl.java:144)
        at com/tc/object/PortabilityImpl.isPortableClass(PortabilityImpl.java:127)
        at com/tc/object/ClientObjectManagerImpl.isPortableClass(ClientObjectManagerImpl.java:1225)
        at com/tc/object/TCClassImpl.createPortableFields(TCClassImpl.java:341)
        at com/tc/object/TCClassImpl.<init>(TCClassImpl.java:119)
        at com/tc/object/TCClassFactoryImpl.createTCClass(TCClassFactoryImpl.java:95)
        at com/tc/object/TCClassFactoryImpl.getOrCreate(TCClassFactoryImpl.java:72)
        at com/tc/object/ClientObjectManagerImpl.lookup(ClientObjectManagerImpl.java:567)
        at com/tc/object/ClientObjectManagerImpl.lookupObject(ClientObjectManagerImpl.java:463)
        at com/tc/object/ClientObjectManagerImpl.lookupObject(ClientObjectManagerImpl.java:451)
        at com/tc/object/bytecode/ManagerImpl.lookupObject(ManagerImpl.java:484)
        at com/tc/object/bytecode/ManagerUtil.lookupObject(ManagerUtil.java:389)
        at com/terracotta/toolkit/collections/ConcurrentDistributedMapDso.lookupAndFaultIn(ConcurrentDistributedMapDso.java:708)
        at com/terracotta/toolkit/collections/ConcurrentDistributedMapDso.lookupAndFaultIn(ConcurrentDistributedMapDso.java:702)
        at com/terracotta/toolkit/collections/ConcurrentDistributedMapDso.get(ConcurrentDistributedMapDso.java:198)
        at com/terracotta/toolkit/collections/ConcurrentDistributedMapDsoArray.get(ConcurrentDistributedMapDsoArray.java:251)
        at org/terracotta/collections/ConcurrentDistributedMap.get(ConcurrentDistributedMap.java:220)
        at com/terracotta/session/SessionDataStore.find(SessionDataStore.java:145)
        at com/terracotta/session/TerracottaSessionManager.doGetSession(TerracottaSessionManager.java:671)
        at com/terracotta/session/TerracottaSessionManager.getSession(TerracottaSessionManager.java:562)
        at com/terracotta/session/SessionRequest.getTerracottaSession(SessionRequest.java:150)
        at com/terracotta/session/SessionRequest.getSession(SessionRequest.java:109)
        at com/tctest/webapp/servlets/SynchWriteMultiThreadsTestServlet.doGet(SynchWriteMultiThreadsTestServlet.java:18)
        at javax/servlet/http/HttpServlet.service(HttpServlet.java:707)
        at javax/servlet/http/HttpServlet.service(HttpServlet.java:820)
        at org/eclipse/jetty/servlet/ServletHolder.handle(ServletHolder.java:538)
        at org/eclipse/jetty/servlet/ServletHandler$CachedChain.doFilter(ServletHandler.java:1352)
        at com/terracotta/session/SessionFilter.doHttpFilter(SessionFilter.java:81)
        at com/terracotta/session/SessionFilter.doFilter(SessionFilter.java:65)
        at org/terracotta/session/BaseExpressSessionFilter.doFilter(BaseExpressSessionFilter.java:44)
        at org/terracotta/session/TerracottaJetty74xSessionFilter.doFilter(TerracottaJetty74xSessionFilter.java:13)
        at org/eclipse/jetty/servlet/ServletHandler$CachedChain.doFilter(ServletHandler.java:1323)
        at org/eclipse/jetty/servlet/ServletHandler.doHandle(ServletHandler.java:476)
        at org/eclipse/jetty/server/handler/ScopedHandler.handle(ScopedHandler.java:119)
        at org/eclipse/jetty/security/SecurityHandler.handle(SecurityHandler.java:480)
        at org/eclipse/jetty/server/session/SessionHandler.doHandle(SessionHandler.java:225)
        at org/eclipse/jetty/server/handler/ContextHandler.doHandle(ContextHandler.java:937)
        at org/eclipse/jetty/servlet/ServletHandler.doScope(ServletHandler.java:406)
        at org/eclipse/jetty/server/session/SessionHandler.doScope(SessionHandler.java:183)
        at org/eclipse/jetty/server/handler/ContextHandler.doScope(ContextHandler.java:871)
        at org/eclipse/jetty/server/handler/ScopedHandler.handle(ScopedHandler.java:117)
        at org/eclipse/jetty/server/handler/ContextHandlerCollection.handle(ContextHandlerCollection.java:247)
        at org/eclipse/jetty/server/handler/HandlerCollection.handle(HandlerCollection.java:149)
        at org/eclipse/jetty/server/handler/HandlerWrapper.handle(HandlerWrapper.java:110)
        at org/eclipse/jetty/server/Server.handle(Server.java:346)
        at org/eclipse/jetty/server/HttpConnection.handleRequest(HttpConnection.java:589)
        at org/eclipse/jetty/server/HttpConnection$RequestHandler.headerComplete(HttpConnection.java:1048)
        at org/eclipse/jetty/http/HttpParser.parseNext(HttpParser.java:601)
        at org/eclipse/jetty/http/HttpParser.parseAvailable(HttpParser.java:214)
        at org/eclipse/jetty/server/HttpConnection.handle(HttpConnection.java:411)
        at org/eclipse/jetty/io/nio/SelectChannelEndPoint.handle(SelectChannelEndPoint.java:535)
        at org/eclipse/jetty/io/nio/SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:40)
        at org/eclipse/jetty/util/thread/QueuedThreadPool$3.run(QueuedThreadPool.java:529)
        at java/lang/Thread.run(Thread.java:662)
        at jrockit/vm/RNI.c2java(IIIII)V(Native Method)
        -- end of trace

    Memory usage report:
    Total mapped                  1217824KB           (reserved=944628KB)
    -              Java heap      1048576KB           (reserved=912672KB)
    -              GC tables        35084KB         
    -          Thread stacks        31284KB           (#threads=151)
    -          Compiled code         4352KB           (used=3823KB)
    -               Internal         1352KB         
    -                     OS         9272KB         
    -                  Other        59488KB         
    -        Java class data        27392KB           (malloced=27135KB #38974 in 6995 classes)
                                                      Not tracing sites.

    - Native memory tracking         1024KB           (malloced=120KB #8)
                                                      Not tracing sites.


    Set the env variable TRACE_ALLOC_SITES=1 or use the print_memusage switch
    trace_alloc_sites=true to enable alloc site tracing.

        **************************************************************************
        *  If you see this dump, please go to                                    *
        *  http://download.oracle.com/docs/cd/E15289_01/go2troubleshooting.html  *
        *  for troubleshooting information.                                      *
        **************************************************************************

    ===== END DUMP ===============================================================                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points