This discussion is archived
8 Replies Latest reply: Apr 22, 2011 6:08 PM by 856704 RSS

JVM heap layout (for garbage collector)

856704 Newbie
Currently Being Moderated
Hi,

I am using Java TI and BCI to keep track of java objects allocation/free. I have two questions as follows:
1. In my situation I would like to know the actual virtual address of the start and end of different regions on the heap (i.e. young generation, old generation, from region, to region, etc). Is there a way to know that information?
2. Java TI provides callbacks on Full garbage collection, I am also wondering if I can keep track of other minor collections (is that indicated by callbacks.ObjectFree)?

Thanks very much for any relevant information!

Yong
  • 1. Re: JVM heap layout (for garbage collector)
    802316 Pro
    Currently Being Moderated
    Why do you want to know this? What problem are you trying to solve? With some more detail, we might be able suggest a simpler way to solve your actual problem.

    Say the "young generation" starts at 0x00B6472363000000, what will that tell you?
  • 2. Re: JVM heap layout (for garbage collector)
    tschodt Pro
    Currently Being Moderated
    Any data [url http://java.sun.com/performance/jvmstat/visualgc.html]visualgc (part of [url http://java.sun.com/performance/jvmstat/]jvmstat) has access to
    you can get access to through jstat.
  • 3. Re: JVM heap layout (for garbage collector)
    856704 Newbie
    Currently Being Moderated
    Hi Peter,

    Thanks for your reply.

    Actually I am doing a project in which the OS needs to know the "unused virtual pages" in applications' page tables. By "unused virtual page" I mean a page whose
    content is irrelevant (all the objects in the pages are freed or deallocated). One of the optimizations we can do using this information is to avoid write back of such pages upon page swapping. We have other useful things to do with these unused virtual pages ...

    So I know the Java garbage collector frequently copies live objects from one region (say "from" region) to another ("to" region) and deallocate the whole "to" region and/or "eden" region. What I would like to know is how these freed regions correspond to virtual pages.

    Regards,

    Yong
  • 4. Re: JVM heap layout (for garbage collector)
    856704 Newbie
    Currently Being Moderated
    Thanks for your suggestion. I'll have a look at the tools. But I doubt the tool provide any APIs that I can use in an agent lib to get the information.
  • 5. Re: JVM heap layout (for garbage collector)
    jschellSomeoneStoleMyAlias Expert
    Currently Being Moderated
    Yong wrote:
    Hi Peter,

    Thanks for your reply.

    Actually I am doing a project in which the OS needs to know the "unused virtual pages" in applications' page tables. By "unused virtual page" I mean a page whose
    content is irrelevant (all the objects in the pages are freed or deallocated). One of the optimizations we can do using this information is to avoid write back of such pages upon page swapping. We have other useful things to do with these unused virtual pages ...
    That has nothing to do with 'java'.

    The OS is resposible for managing the physical memory and providing applications with a virtual address space.
    The application is responsible for managing its virtual memory

    An application, any application, requests virtual memory from the OS.
    When the application no longer wants to use that memory it gives it back to the OS.
    By giving it back the application is informing the OS that it no longer in use.

    That model works very well having been proven in many different OSes.

    There is one or more hotspot options on the Sun VM that controls when that VM decides to return memory to the OS.
    If you are using the Sun VM then that is the only option.

    If you have a VM that you own then you are resposible for the code that manages the VM heap. That code, not java code, would be responsible for deciding when to return memory to the OS.
  • 6. Re: JVM heap layout (for garbage collector)
    856704 Newbie
    Currently Being Moderated
    jschell wrote:
    Yong wrote:
    Hi Peter,

    Thanks for your reply.

    Actually I am doing a project in which the OS needs to know the "unused virtual pages" in applications' page tables. By "unused virtual page" I mean a page whose
    content is irrelevant (all the objects in the pages are freed or deallocated). One of the optimizations we can do using this information is to avoid write back of such pages upon page swapping. We have other useful things to do with these unused virtual pages ...
    That has nothing to do with 'java'.

    The OS is resposible for managing the physical memory and providing applications with a virtual address space.
    The application is responsible for managing its virtual memory

    An application, any application, requests virtual memory from the OS.
    When the application no longer wants to use that memory it gives it back to the OS.
    By giving it back the application is informing the OS that it no longer in use.

    That model works very well having been proven in many different OSes.

    There is one or more hotspot options on the Sun VM that controls when that VM decides to return memory to the OS.
    If you are using the Sun VM then that is the only option.

    If you have a VM that you own then you are resposible for the code that manages the VM heap. That code, not java code, would be responsible for deciding when to return memory to the OS.
    Thanks for your reply. I am using Sun Java HotSpot 1.5.0_01
    Yes, I agree with you that the OS and architecture determine the high level virtual address layout (where the heap starts, where the stack goes, etc). But they do not know about the detailed information inside a particular virtual segment. I don't think the OS knows the address for a particular section or an object inside the java heap. For C++ application, I can get that information using & operator in the application. But I don't know how to get any meaningful address in Java. Java object header may maintain some sort of pointers, but that is not exposed to the users.

    As for the memory return, generally, Java will give back memory to OS very "reluctantly" in case it needs the space in the near future. In other words, there are lots of unused space (e.g. eden section, from section) after a garbage collection that OS thinks in use by the application but actually not. What I am trying to do is to get the virtual address for these unused sections. I am now able to keep track of the object free event using Java TI and pass the handle/reference to my native library,
    it is now how to get an meaningful address stuck me.

    Edited by: Yong on Apr 21, 2011 2:45 PM
  • 7. Re: JVM heap layout (for garbage collector)
    jschellSomeoneStoleMyAlias Expert
    Currently Being Moderated
    Yong wrote:
    Yes, I agree with you that the OS and architecture determine the high level virtual address layout (where the heap starts, where the stack goes, etc). But they do not know about the detailed information inside a particular virtual segment. I don't think the OS knows the address for a particular section or an object inside the java heap. For C++ application, I can get that information using & operator in the application. But I don't know how to get any meaningful address in Java. Java object header may maintain some sort of pointers, but that is not exposed to the users.
    Regardless. If the page is in fact "unused" then because the OS manages writes the OS is the one that decides how it is handled. And the only way the OS does that is if you tell the OS you are no longer using the page.

    And it doesn't have anything to do with an "object" because the OS manages pages (or chunks, etc) not objects. And that is only at the level that swapping occurs.

    And that has nothing to do with language. Same thing is true in C++.
    As for the memory return, generally, Java will give back memory to OS very "reluctantly" in case it needs the space in the near future. In other words, there are lots of unused space (e.g. eden section, from section) after a garbage collection that OS thinks in use by the application but actually not. What I am trying to do is to get the virtual address for these unused sections. I am now able to keep track of the object free event using Java TI and pass the handle/reference to my native library,
    it is now how to get an meaningful address stuck me.
    I already pointed out your option then - there is a command line option that gives a limited control over more aggressively returning pages to the OS.

    You could get the source for the VM, figure out how it works, and add your own options. That is line with my second suggestion.

    However I seriously doubt you can manage that at the object level without adding extensively object management support into the language itself. Something along the lines of C++ allowing one to override operator new. In that case you would no longer be using Java.
  • 8. Re: JVM heap layout (for garbage collector)
    856704 Newbie
    Currently Being Moderated
    jschell wrote:
    Yong wrote:
    Yes, I agree with you that the OS and architecture determine the high level virtual address layout (where the heap starts, where the stack goes, etc). But they do not know about the detailed information inside a particular virtual segment. I don't think the OS knows the address for a particular section or an object inside the java heap. For C++ application, I can get that information using & operator in the application. But I don't know how to get any meaningful address in Java. Java object header may maintain some sort of pointers, but that is not exposed to the users.
    Regardless. If the page is in fact "unused" then because the OS manages writes the OS is the one that decides how it is handled. And the only way the OS does that is if you tell the OS you are no longer using the page.

    And it doesn't have anything to do with an "object" because the OS manages pages (or chunks, etc) not objects. And that is only at the level that swapping occurs.

    And that has nothing to do with language. Same thing is true in C++.
    As for the memory return, generally, Java will give back memory to OS very "reluctantly" in case it needs the space in the near future. In other words, there are lots of unused space (e.g. eden section, from section) after a garbage collection that OS thinks in use by the application but actually not. What I am trying to do is to get the virtual address for these unused sections. I am now able to keep track of the object free event using Java TI and pass the handle/reference to my native library,
    it is now how to get an meaningful address stuck me.
    I already pointed out your option then - there is a command line option that gives a limited control over more aggressively returning pages to the OS.

    You could get the source for the VM, figure out how it works, and add your own options. That is line with my second suggestion.

    However I seriously doubt you can manage that at the object level without adding extensively object management support into the language itself. Something along the lines of C++ allowing one to override operator new. In that case you would no longer be using Java.
    Anyway, really thanks for your explanation. I think I now understand better the tricks here. I'll be trying to get some workaround.

    Thanks,

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points