5 Replies Latest reply on May 9, 2007 5:38 AM by 807597

    Dump complete java VM state as core dump (not via OS) possible?

      Hi everyone,

      I've a problem debugging an application.

      Background: Sometimes my application comes to a very unlikely state, which at the moment results in an error message. The stack trace alone has no great value, since this state is cause by the interaction of more than one thread. The state is resolved throwing an exception, the program continues normally.

      Goal: If I reach this state, I want to suspend the application, dump the complete state of all java threads, objects, ... (complete java memory core dump) to analyse it later.

      Question: Is there a possibility to generate such dumps?

      Just thinking:

      One posibility would be, to send a signal from the vm to some external program (e.g. udp packet), this program attaches a standard OS-level debugger (e.g. gdb under linux), dumps the core, resumes app, detach. But I guess reconstruction of internal VM state from complete dump is rather hard, is it?

      I've looked at jdb, to see if it has some java-core-dump functions, but it seems not to be so. Are there alternative implementations, or helper scripts that make jdb loop over all java object addresses and dump them?

      Does someone known about the jdb remote debugging interface? Is the protocol public, can I implement such a feature on myself?
        • 1. Re: Dump complete java VM state as core dump (not via OS) possible?
          Thanks to Jim Holmlund from Sun I got it working the way I wanted to have it. It was all there in the linux jdk1.5 installation, he pointed out to me how to put the things together (sorry, works only for linux):

          $ cat << END > gdb.commands
          gcore tomcat.core
          $ gdb --pid 26003 -x gdb.commands
          $ jsadebugd ..../jdk1.5.0_06/bin/java tomcat.core DebugServer &
          $ jdb -connect

          You can inspect all the data (threads, stack frames, local variables,
          objects) just as if you would have attached to a live VM, only
          modifications do not work (step, continue, set...) because a VM core
          dump is dead.

          This is a really strange way to debug a java application, I never
          thought that it could work that way.
          • 2. Re: Dump complete java VM state as core dump (not via OS) possible?
            Hi everyone,

            I was looking for a way to dump the complete state of a JVM (all runing threads, objects, ...) and afterwards to analize the dumped data. I have found a thread(http://forum.java.sun.com/thread.jspa?threadID=735052) here in the forums with some explainations how this could be done using the Java 2 sdk 1.5.0 - troubleshooting tools (http://java.sun.com/j2se/1.5.0/docs/tooldocs/experimentaltools.html).

            I've made a small application, which starts a thread and just prints out continously some messages to the console.
            After that I attached with gdb to the running process as described, and generated a core file.
            Unfortunatelly on calling jdsadebugd with the core dump it throws the following exception:

            jsadebugd /..../jdk1.5.0_07/bin/java core.18294 DebServ
            Attaching to core core.18294 from executable /.../jdk1.5.0_07/bin/java and starting RMI services, please wait...
            Error attaching to core file or starting server: sun.jvm.hotspot.debugger.DebuggerException: Can't attach to the core file
            at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.attach0(Native Method)
            at sun.jvm.hotspot.debugger.linux.LinuxDebuggerLocal.attach(LinuxDebuggerLocal.java:258)
            at sun.jvm.hotspot.HotSpotAgent.attachDebugger(HotSpotAgent.java:624)
            at sun.jvm.hotspot.HotSpotAgent.setupDebuggerLinux(HotSpotAgent.java:610)
            at sun.jvm.hotspot.HotSpotAgent.setupDebugger(HotSpotAgent.java:323)
            at sun.jvm.hotspot.HotSpotAgent.go(HotSpotAgent.java:298)
            at sun.jvm.hotspot.HotSpotAgent.startServer(HotSpotAgent.java:234)
            at sun.jvm.hotspot.DebugServer.run(DebugServer.java:94)
            at sun.jvm.hotspot.DebugServer.main(DebugServer.java:29)
            at sun.jvm.hotspot.jdi.SADebugServer.main(SADebugServer.java:46)

            Even if I try to print the shared objects mapping with jmap it fails with the following message:

            jmap /.../jdk1.5.0_07/bin/java core.18294
            Attaching to core core.18294 from executable /.../jdk1.5.0_07/bin/java, please wait...
            Error attaching to core file: Can't attach to the core file

            Does anyone have an idea what goes wrong here?
            • 3. Re: Dump complete java VM state as core dump (not via OS) possible?
              Already solved it? If not how big is your core file, which file permissions are on it? Is gdb able to use it (I think gdb --core-file [filename] is the call). If not perhaps the core is broken
              • 4. Re: Dump complete java VM state as core dump (not via OS) possible?
                I am trying to do the same thing, only in Windows running Tomcat.

                I am running 1.5_07, and most of the good tools, are not available for Windows until 1.6 (jmap, ect).

                What can I use to get a full thread dump, memory dump, thread states?
                • 5. Re: Dump complete java VM state as core dump (not via OS) possible?
                  May be a very dumb question -
                  Given a java1.5 core generated by gcore on solaris2.9, is it possible to convert it
                  to a hprof binary recognized by java6 jhat? Tried jmap -dump from java6, no luck. hprof agent in java1.5 too heavy for serious production heap dump - 5g heap in our case.

                  Any better way to handle this?

                  Thanks in advance.