1 2 Previous Next 25 Replies Latest reply on May 4, 2011 11:51 PM by jschellSomeoneStoleMyAlias

    JVM Heap Memory Management

    858646
      Hi,

      In the real world, we have an application which use about 150m on heap after startup, but some task can make the memory grow up to 700m.
      The problem is that after the "memory consuming" task, the JVM don't release it.

      We made some test with Java Visual VM, with different garbage collector and -XX:MaxHeapFreeRatio= -XX:MinHeapFreeRatio= values, and finally write a little application to test various case.

      The mini app can be found here : http://pastebin.com/tuiEJaMA

      For testing purpose, we run it with very little value for heap-ratio here is our testing command line : -Xmx1224m -XX:MaxHeapFreeRatio=2 -XX:MinHeapFreeRatio=2

      For now, we was able to see a real allocated memory shrink in two conditions :
      - Manual call to System.gc() soon after use-memory/free clicking.
      - Spamming call to System.gc() if you wait about ten second after use-memory/free clicking.

      In other case, the GC seem to pass only on even space, doesn't see that the byte array is not referenced anymore, and real allocated memory grow up for each use-memory call.
      Worst, 1 click to call explicitly System.gc() free internally the use memory, but do not return unused memory to the system (doesn't follow the -XX:MaxHeapFreeRatio=2 directive).

      So, here is my question : How can I be sure after a memory consuming task that the JVM won't keep the memory for itself when the really used memory is about 150m and the allocated size is about 1gb ? is there any clean way to notify the JVM that this memory usage was really temporary and that it should release it for other application ?

      At this time, the only solution I see will be to use "the spam System.gc()" technique, because in our little testCase, it was the only way to reprodruce a memory freeing at system level. But this solution seems more than dirty to me.
        • 1. Re: JVM Heap Memory Management
          jschellSomeoneStoleMyAlias
          855643 wrote:
          In the real world, we have an application which use about 150m on heap after startup, but some task can make the memory grow up to 700m.
          The problem is that after the "memory consuming" task, the JVM don't release it.
          And why do you think this is a problem?
          • 2. Re: JVM Heap Memory Management
            858646
            It's a problem because we have about 10 user on a 8gb RAM server, and the consumed memory force swapping memory on disk which cause slowdown.
            • 3. Re: JVM Heap Memory Management
              EJP
              The problem is that after the "memory consuming" task, the JVM don't release it.
              Correct.
              So, here is my question : How can I be sure after a memory consuming task that the JVM won't keep the memory for itself when the really used memory is about 150m and the allocated size is about 1gb?
              You can't. It will.
              is there any clean way to notify the JVM that this memory usage was really temporary and that it should release it for other application ?
              There is no way at all, clean or otherwise.
              At this time, the only solution I see will be to use "the spam System.gc()" technique
              That will not release memory back to the operating system.

              You need to look into why there is so much memory being used by your application.
              • 4. Re: JVM Heap Memory Management
                802316
                Java allocates the maximum heap space as virtual memory as soon as it starts and it never gives it back.

                However, the OS can do a number of tricks so this virtual memory doesn't turn into physical memory immediately. e.g. it only allocates pages of memory actually used. So as the heap grows and more pages are accessed, more pages are allocated to your program by the OS.

                Say you don't need those pages any more? Unless another program or the OS needs those pages, nothing happens. (And it doesn't matter because nothing else needs the memory) If more pages are needed than the system has, the OS starts looking for pages it can swap to disk (or cached data it can discard), If you have a program which hasn't used pages for a long time, they will be swapped to disk until there is enough free pages. You will know this has happened because the page file usage grows. (But you won't know which pages from which programs were swapped to disk)

                In short, if your swap file is unused, don't worry about it. If you swap file is used, it might be a issue e.g. you will see a delay if swapped pages are used again. But I suspect it won't be if you say you won't use the memory again. If this is a issue, try to reduce the maximum memory your program uses, or buy more memory. You can buy a server with 16 GB costs for about $1000 (or upgrade the one you have)
                • 5. Re: JVM Heap Memory Management
                  858646
                  Thanks for your answer.

                  About the "spam System.gc()" technique, it does free memory (I don't understand why, but spamming it seems to provoque a more deep gc, otherwise, GC seems to pass only on even).
                  The code linked in paste bin show it (monitored with jvisualvm).


                  +"You need to look into why there is so much memory being used by your application."+
                  This is probably the core of the problem. Some task load a big graph of object from database (using hibernate). To allow this task running without out of memory exception, we add a big -Xmx. So for a while, the application is far far away of it's -Xmx and it seem it make the GC "lazy", it don't run often, and when it run, it pass only on newer object, not older one. Most of our unreferenced object are too much "old"...


                  I see at your answer that it should be normal to have about 80mb (from permgen) + 150 mb (from heap) really used by application which take about 1Gb on system, it's hard to believe/understand for me (and the server team of my compagny).

                  About the "java will release it when the system will need it", I tried to overload system memory until java refuse to start a new instance of my test app because jvm can't get a large enouth memory block, there is at this time more than 3gb of unused memory reserved by others JVM, they don't release it even if the system need it.

                  In other languages, memory usage is growing and shrinking as needed. Why java is "only" growing is a mystery to me, If someone has a link or can give me a real explication on that, I'll be happy to finally understand this issue.
                  • 6. Re: JVM Heap Memory Management
                    EJP
                    About the "spam System.gc()" technique, it does free memory
                    I have no idea what 'spam' has to do with it, but of course System.gc() frees memory, eventually. That's what it's for. But it doesn't release it to the operating system.
                    The code linked in paste bin show it (monitored with jvisualvm).
                    The code you linked to monitors free memory in the JVM. It doesn't show anything about memory released to the operating system. It has no way of knowing. There is no way of knowing in Java. It shows memory released to the JVM. That's all it can do. That's probably all jvisualVM can do.
                    I see at your answer that it should be normal to have about 80mb (from permgen) + 150 mb (from heap) really used by application which take about 1Gb on system
                    I don't know how you got all that from what I wrote. I certainly didn't say any of it, or provide any premisses from which you could logically deduce that conclusion.
                    it's hard to believe/understand for me (and the server team of my compagny).
                    It's hard for me to believe your application really needs all that memory.
                    they don't release it even if the system need it.
                    Exactly. That is precisely what I told you above.
                    In other languages, memory usage is growing and shrinking as needed.
                    'Other languages' such as what? Fortran, IBM 7044 assembler, Interdata 7/16 assembler, Basic-Plus, Basic-Plus-2, Pascal, PL/M, Intel 8080 assembler, MPL, C, Cobol, ML/1, Motorola 68xxx assembler, C++, Intel 808x assembler, Visual Basic, Java, ... don't release free memory to the operating system. No doubt there are languages that do, but I've never used one in several decades. You might be surprised how many decades that is.
                    • 7. Re: JVM Heap Memory Management
                      gimbal2
                      EJP wrote:
                      'Other languages' such as what? Fortran, IBM 7044 assembler, Interdata 7/16 assembler, Basic-Plus, Basic-Plus-2, Pascal, PL/M, Intel 8080 assembler, MPL, C, Cobol, ML/1, Motorola 68xxx assembler, C++, Intel 808x assembler, Visual Basic, Java, ... don't release free memory to the operating system. No doubt there are languages that do, but I've never used one in several decades. You might be surprised how many decades that is.
                      Yes we know you're an old geezer ;)

                      I never really cared about it, but free/delete actions in C/C++ do not release the released memory back to the OS? I could have sworn that they do.
                      • 8. Re: JVM Heap Memory Management
                        802316
                        855643 wrote:
                        About the "java will release it when the system will need it", I tried to overload system memory until java refuse to start a new instance of my test app because jvm can't get a large enouth memory block, there is at this time more than 3gb of unused memory reserved by others JVM, they don't release it even if the system need it.
                        This is a problem of not enough continuous memory, which is a different problem. AFAIK, this only occurs on a 32-bit OS and is a limitation of the OS (as a requirement of Java that it allocate continuous virtual memory at the start)
                        In other languages, memory usage is growing and shrinking as needed.
                        You appear to be confused as to the difference between virtual memory (which is not released by an language I know) and physical memory. which is retrieved by the OS when it is not used.
                        Why java is "only" growing is a mystery to me,
                        It doesn't grow. The virtual memory of the heap is the same from startup, its only how much of this which used which grows.
                        If someone has a link or can give me a real explication on that, I'll be happy to finally understand this issue.
                        We have been trying to help you, but you appear to want to hold on to incorrect assumption. I am not sure a link will help you there.
                        • 9. Re: JVM Heap Memory Management
                          EJP
                          I never really cared about it, but free/delete actions in C/C++ do not release the released memory back to the OS? I could have sworn that they do.
                          Not in any version of C/C++ I have ever used. malloc() takes it from the OS with sbrk(). There is no system call to give it back. ergo no giving back. There probably is an API on Windows but still no giving back from any C/C++ RTS I am aware of.
                          • 10. Re: JVM Heap Memory Management
                            858646
                            Sorry, if any of my answer hurt you, English is not my native language and maybe my question can be taken for agressive or so, it's not my will.

                            I'm trying to understand my problem and the answer people give to me. And it is possible that I misunderstood memory management and difference between virtual memory and physical one.

                            I'll try to explain how I'm understanding this so you can correct me where I'm wrong.

                            - System memory = Virtual memory = Physical RAM + Swap
                            - Use of system memory by a java process = Java Heap Size + PermGen size => approximation of what I can see for javaw.exe in windows task manager.
                            - Use of java memory by a java app = Java Heap usage + PermGen usage.
                            - Task manager display memory used by a process from an OS point of view.

                            What come from testing :
                            - Change in java heap usage has no effect in taskmanager (if heap size do not change).
                            - A call of System.gc() only change the used heap.
                            - Change of Java Heap size is visible in taskmanager.
                            - The spam GC (which only a loop with multiple call to System.gc()) do shrink java heap size (at least in visualvm monitor and in windows taskmanager).

                            The server team called the dev team saying : Hey, your application use huge amount of memory all the time, raising swap alert on our side.

                            When I said other language, I thinked of C application and CPP/Qt application. It was not a blame for Java or the JVM, just a way to show what I do not understand, because, in task manager, I can see the used ram shrinking, and here, I can only see it shrinking when we spam call to GC.

                            Now, what I said is I do not understand why there is big difference between (heap use+permgen use) and (heap size+permgen size), and if consumed memory from OS point of view equals to (heap size+permgen size), is there a way to minimize this difference ? And I'm open to any explaination.
                            • 11. Re: JVM Heap Memory Management
                              EJP
                              The server team called the dev team saying : Hey, your application use huge amount of memory all the time, raising swap alert on our side.
                              I agree. It does. I would call them too. Your application uses a huge amount of memory. Why? Why does it need a huge amount of memory even some of the time? This is the issue you need to address. Futzing around with definitions isn't getting you anywhere. A real problem exists.
                              • 12. Re: JVM Heap Memory Management
                                800381
                                EJP wrote:
                                I never really cared about it, but free/delete actions in C/C++ do not release the released memory back to the OS? I could have sworn that they do.
                                Not in any version of C/C++ I have ever used. malloc() takes it from the OS with sbrk(). There is no system call to give it back. ergo no giving back. There probably is an API on Windows but still no giving back from any C/C++ RTS I am aware of.
                                Actually, you CAN use sbrk() to reduce your heap size, since all sbrk() does is take the current break value and add to it. Just pass sbrk() a negative value and your heap will shrink.

                                You just can't use anything like malloc() or free() or new or delete without replacing the underlying memory management system if you try something like that.
                                • 13. Re: JVM Heap Memory Management
                                  800381
                                  EJP wrote:
                                  The server team called the dev team saying : Hey, your application use huge amount of memory all the time, raising swap alert on our side.
                                  I agree. It does. I would call them too. Your application uses a huge amount of memory. Why? Why does it need a huge amount of memory even some of the time? This is the issue you need to address. Futzing around with definitions isn't getting you anywhere. A real problem exists.
                                  How much would it cost to add memory to the servers?

                                  How much would it cost to figure out how to make the apps use less memory, then change the code to do that, then fix everything you broke when you did that?
                                  • 14. Re: JVM Heap Memory Management
                                    EJP
                                    Actually, you CAN use sbrk() to reduce your heap size
                                    Thanks for the reminder. It's been a very long time. However the only use a C/C++ RTL could make of that would be to release the topmost memory in the stack, which absent GC isn't at all likely to be possible given how malloc() and free() work and what they are used for. Hence no use is made of the feature.
                                    1 2 Previous Next