1 Reply Latest reply on Jul 22, 2010 12:40 AM by jschellSomeoneStoleMyAlias

    Access violation during compilation

      We have a customer that has been experiencing a failure in the HotSpot compiler while running our application. The client has managed to create a scenario where they can fairly consistently reproduce the problem. When the failure occurs they get a hs_err_pid*.log file with a EXCEPTION_ACCESS_VIOLATION (0xc0000005), the compilation task is always within the same method.

      They are running the YourKit Profiler and this seems to be a possible contributing factor. Turning this off or turning compilation of the method off does seem to resolve the problem, however as the issue first occurred after the application had been up for a few months they are concerned this has merely put the issue off and it may occur again or in another method compilation.

      We’ve also captured the output from –XX:+LogCompilation and this shows a fragment of the compile of the method with the optimizer and idealLoop incomplete.

      They are also running java 1.5 build, moving to java 1.6 is not really an option as this would involve an upgrading to a later version of our application.

      Is this a know problem with the 1.5 HotSpot compiler? We are being pushed by the client to explain why this has suddenly started happening and confirm that the issue or similar compiler issues will not occur again if they turn off the profiler and/or stop the method from compiling.

      Any insight greatly appreciated. I can supply either the complete fragament or the whole compile log but these are large.

      <task compile_id='37' compile_kind='osr' method='com/allustra/core/frameworks/bu
      sinesstran/distributed/common/BusinessTransactionEventConvertor convert (Ljava/l
      nt/common/AEvent;)Lcom/allustra/core/frameworks/event/common/AEvent;' bytes='110
      3' count='5000' backedge_count='5001' iicount='7082' decompiles='8' null_check_t
      raps='1' class_check_traps='4' osr_bci='171' stamp='1574.538'>
      <task_queued stamp='1574.538' comment='backedge_count' hot_count='4797'/>
      <phase name='parse' nodes='3' stamp='1574.538'>
      <klass id='505' name='com/allustra/core/frameworks/event/common/AEvent' flags='1
      <klass id='483' name='java/lang/String' flags='17'/>
      <klass id='502' name='com/allustra/core/frameworks/businesstran/distributed/comm
      on/BusinessTransactionEventConvertor' flags='1'/>
      <method id='507' holder='502' name='convert' return='505' arguments='483 483 483
       505' flags='1' bytes='1103' iicount='7082'/>
      <parse method='507' uses='7082' osr_bci='171' stamp='1574.539'>
      <observe trap='null_check' count='1' total='1'/>
      <observe trap='class_check' count='4' total='4'/>
      <observe that='has_exception_handlers'/>
      <OSR_mismatch local_index='9'/>
      <OSR_mismatch local_index='14'/>
      <OSR_mismatch local_index='15'/>
      <OSR_mismatch local_index='16'/>
      <cast_up reason='!has_subklass' from='502' to='(exact)'/>
      <uncommon_trap bci='171' reason='unreached' action='reinterpret'/>
      <bc code='185' bci='173'/>
      <klass id='537' name='java/util/AbstractList$ListItr' flags='2'/>
      <type id='435' name='boolean'/>
      <predicted_call bci='183' klass='537'/>
      <call method='557' count='11242' prof_factor='0.208647' inline='1'/>
      <call method='756' count='6795' prof_factor='1' inline='1'/>
      <parse method='756' uses='6795' osr_bci='171' stamp='1574.579'>
      <parse_done nodes='15559' memory='3990528' stamp='1574.579'/>
      <parse_done nodes='15559' memory='3990656' stamp='1574.579'/>
      <phase_done nodes='15561' stamp='1574.582'/>
      <phase name='optimizer' nodes='15561' stamp='1574.582'>
      <phase name='idealLoop' nodes='16345' stamp='1574.591'>
      Example of hs_err_pid*.log - edited to fit:
      # An unexpected error has been detected by HotSpot Virtual Machine:
      # EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00000000082a78cf, pid=5280, tid=5428
      # Java VM: Java HotSpot(TM) 64-Bit Server VM (1.5.0_20-b02 mixed mode)
      # Problematic frame:
      # V [jvm.dll+0x2a78cf]
      --------------- T H R E A D ---------------
      Current thread (0x0000000004ca8ed0): JavaThread "CompilerThread1" daemon [_thread_in_native, id=5428]

      siginfo: ExceptionCode=0xc0000005, reading address 0x0000000000000008

      EAX=0x0000000000000000, EBX=0x0000000009bec128, ECX=0x000000000a1cf718, EDX=0x0000000000000206
      ESP=0x000000000650e5d0, EBP=0x000000000a1cf718, ESI=0x0000000009df50e8, EDI=0x0000000000000000
      EIP=0x00000000082a78cf, EFLAGS=0x0000000000010202

      Top of Stack: (sp=0x000000000650e5d0)
      0x000000000650e5d0: 0000000000000000 0000000009ade660
      0x000000000650e5e0: 0000000009df50e8 0000000009bec128
      0x000000000650e5f0: 0000000009ade660 000000000000009f
      0x000000000650e600: 000000000650e730 0000000000000235
      0x000000000650e610: 0000000009a968a0 00000000082a9b3b
      0x000000000650e620: 0000000009dc2a40 000000000a5f36f8
      0x000000000650e630: 0000000000000200 0000000009bec128
      0x000000000650e640: 0000000009daa590 0000000000000002
      0x000000000650e650: 0000000000000020 00000000052846a0
      0x000000000650e660: 0000000005285d20 000000000650eba0
      0x000000000650e670: 0000000000000000 000000000650eb10
      0x000000000650e680: 0000000000000000 0000000009a968a0
      0x000000000650e690: 0000000000000001 00000000082aa3c7
      0x000000000650e6a0: 000000000650eb10 000000000000296a
      0x000000000650e6b0: 000000000000009f 000000000650eb10
      0x000000000650e6c0: 0000000000000000 000000000830dae4

      Instructions: (pc=0x00000000082a78cf)
      0x00000000082a78bf: 90 45 8b 43 28 4d 8b 8d b0 00 00 00 4f 8b 1c c1
      0x00000000082a78cf: 49 8b 43 08 48 39 38 75 2d 41 8b 55 28 66 66 66

      Stack: [0x0000000006410000,0x0000000006510000), sp=0x000000000650e5d0, free space=1017k
      Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
      V [jvm.dll+0x2a78cf]

      Current CompileTask:
      opto: 23% ! com.allustra.core.frameworks.businesstran.distributed.common.BusinessTransactionEventConvertor.convert(Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;Lcom/allustra/core/frameworks/event/common/AEvent;)Lcom/allustra/core/frameworks/event/common/AEvent; @ 147 (871 bytes)

      Dynamic libraries:

      0x0000000008000000 - 0x0000000008562000      C:\Program Files\Java\jdk1.5.0_20\jre\bin\server\jvm.dll

      VM Arguments:
      jvm_args: -Diid=18707 -Dcom.allustra.config=,,,,,server-config.properties -Dremote.profiler=true -agentpath:d:\ProtoColl\YourKit\yjpagent.dll=sampling -Xmx512m
      java_command: com.allustra.core.frameworks.application.server.RLServiceServer orderServer password OPS-PRCLLAPP:1100 OrderServer server/CustomResources.xml com/allustra/business/order/server/JdbcRmiOrderServerResources.xml com/allustra/core/frameworks/distributed/resources/ServiceAccessPeerRmiResources.xml com/allustra/core/frameworks/distributed/resources/ServiceAccessCommonRmiResources.xml com/allustra/application/server/SecurityResources.xml com/allustra/application/server/DBResources.xml com/allustra/application/server/ServerResources.xml com/allustra/core/CoreResources.xml
      Launcher Type: SUN_STANDARD

      --------------- S Y S T E M ---------------
      Memory: 4k page, physical 16771724k(11157020k free), swap 18241664k(12418664k free)

      vm_info: Java HotSpot(TM) 64-Bit Server VM (1.5.0_20-b02) for windows-amd64, built on Jun 18 2009 10:52:44 by "java_re" with unknown MS VC++:1400
        • 1. Re: Access violation during compilation
          # EXCEPTION_ACCESS_VIOLATION (0xc0000005)
          Most common cause of that is JNI or 3rd party libraries that do JNI.
          A bug in those can cause ill effects much later in an application.

          It would be very rare to see that originate solely based on the VM.
          I am not sure that I have seen that exception, as reported on these forums, originate from the VM in years (before 1.5 and perhaps before 1.4).
          They are running the YourKit Profiler
          I believe profiling requires using JNI interface - the debugging API.
          Misusing the API would cause problems.
          64-Bit Server VM (1.5.0_20-b02 mixed mode)
          That said the 64 bit implementation was rather new at the time. So it could be a bug in the VM.
          ...or in another method compilation
          Certainly a valid concern.