1 2 Previous Next 15 Replies Latest reply on Dec 20, 2019 4:23 PM by Carlos Cortez

    Weblogic 10.3.6 heap memory usage is high

    user3897984

      Hi Team,

       

      Currenlty I do have deployed my application in webloigc 10.3.6

       

      Current physical memory is around 20gb , we have allocated 5gb of heap space for the JVM,

      When the application is not used still the heap memory reach to 5gb but GC was able to clear the heap to 300mb from 5gb,

      just want to under stand why the heap space goes till 5gb when the application is not used at all.

       

      Is the behavior for the web logic application.

       

      My minimum and max setting of heap is 5gb  for JVM.

       

      Please help in resolving the issue.

       

      Is it ok to set the heap for JVM to 5GB ? How to fine tune the GC ?

        • 1. Re: Weblogic 10.3.6 heap memory usage is high
          Carlos Cortez

          Hi,

           

          Is it ok to set the heap for JVM to 5GB ? How to fine tune the GC ?

          That will depend on your application, each application is different.

           

          You should first understand what is happening with your application, why is taking so much heap, I recommend you take a heapdump to see why this behavior:

           

          https://docs.oracle.com/javase/7/docs/technotes/tools/share/jhat.html

           

          Then you can open this heapdump in a tool like MAT to analy it.

          1 person found this helpful
          • 2. Re: Weblogic 10.3.6 heap memory usage is high
            user3897984

            Thanks for the response.

            we are currently using the weblgic 010.3.6 with jrockit JVM 1.6, I do have captured the heap dump.

            We do have enabled the JRMC port and captured the flight recording.. still not able to find the exact root cause.

             

            Do do have alert mechanism when the Heap reach 5gb it triggers a alert to us.

            Alert is not happening frequently, randomly for every 3 to 4 hrs it happens.

             

            The above  are for production environment.

             

            1.) We do have un-deployed the app fro the weblogic in UAT environment weblogic is still using 1.2 gb

                 For UAT we have heap as 2GB, the min and max level is 2gb application is not there why weblogic goes to 1.2gb ? Any thoughts ?

            2.) We don't receive request to our application after 6pm but still the heap grows till 1.5 gb , GC was able to recollect the heap space up to 400 MB in UAT, bu again it gradually grows and reaches 1.5 to 1.8 gb..

             

            We are using - genpar (Jrockit JVM)for GC, do we need to use some other algorithm for this  ?

             

            What are parameters we need to look into ?

             

            Waiting for your thoughts.

            • 3. Re: Weblogic 10.3.6 heap memory usage is high
              user3897984

              Thanks for the response.

              we are currently using the weblgic 010.3.6 with jrockit JVM 1.6, I do have captured the heap dump.

              We do have enabled the JRMC port and captured the flight recording.. still not able to find the exact root cause.

               

              Do do have alert mechanism when the Heap reach 5gb it triggers a alert to us.

              Alert is not happening frequently, randomly for every 3 to 4 hrs it happens.

               

              The above  are for production environment.

               

              1.) We do have un-deployed the app fro the weblogic in UAT environment weblogic is still using 1.2 gb

                   For UAT we have heap as 2GB, the min and max level is 2gb application is not there why weblogic goes to 1.2gb ? Any thoughts ?

              2.) We don't receive request to our application after 6pm but still the heap grows till 1.5 gb , GC was able to recollect the heap space up to 400 MB in UAT, bu again it gradually grows and reaches 1.5 to 1.8 gb..

               

              We are using - genpar (Jrockit JVM)for GC, do we need to use some other algorithm for this  ?

               

              What are parameters we need to look into ?

               

              Waiting for your thoughts.

              • 4. Re: Weblogic 10.3.6 heap memory usage is high
                Steff2-Oracle

                Hello,

                 

                Is it the same behavior if you start with an initial heap size smaller than the max heap size ?

                for example : -Xms512mb -Xmx5GB

                 

                Also, WLS 10.3.6 is certified and supported against JDK 7.

                What if you upgrade your JDK to 1.7 , do you experience the same ?

                (cf: https://support.oracle.com/epmos/faces/DocumentDisplay?id=1439822.1 )

                and

                (cf: https://support.oracle.com/epmos/faces/DocumentDisplay?id=1309855.1 )

                 

                 

                With Best Regards,

                Stephan

                • 5. Re: Weblogic 10.3.6 heap memory usage is high
                  Carlos Cortez

                  Hi,

                   

                  In the heap dump you should have seen a table of what objects are claiming the heap space and then dig into to find out the part of the application which is causing this behavior.

                   

                  If this is the normal behavior of the application then you should increase the heap size, as Steff2 said, you could try changing JRockit to JDK 1.7 which has a better memory management.

                   

                  There are other parameters you could try depending of what your application does like:

                   

                  - Changing GC algorithm: CMS (low pause) or parallel (throughput).

                  - Is this physical or virtual environment

                   

                  An example of some options you may try (without knowing your application and assuming you are using JDK 1.7):

                  -server -XX:+DoEscapeAnalysis -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC -XX:+ExplicitGCInvokesConcurrent -XX:CMSInitiatingOccupancyFraction=80 -XX:CMSIncrementalSafetyFactor=20 -XX:+UseCMSInitiatingOccupancyOnly -XX:ParallelGCThreads=2 -Xmx5072M -Xms5072M -XX:+UseCompressedOops -XX:PermSize=1024M -XX:MaxPermSize=1024M

                  • 6. Re: Weblogic 10.3.6 heap memory usage is high
                    user3897984

                    Thanks for the reply.

                     

                    My current production heap size is 5GB, when I get the heap dump and open in MAT tool is shows the total 230 MB, why is that so ?

                     

                    I read one of the article that GC will trigger before heap dump we taken is that so ?

                     

                    - Changing GC algorithm: CMS (low pause) or parallel (throughput) - Will effect effect the application performance - (Currently we are using  - genpar algorithm )  with Jrockit JDK 1.6 - 64bit

                    - Is this physical or virtual environment - (Virtual Environment  - VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform)

                     

                    We are using the genpar algorithm   - with Jrockit jdk 1.6 - 64 bit

                    -Xgc: option

                    Generational

                    Mark

                    Sweep

                    genparcon

                    Yes

                    parallel

                    concurrent

                    • 7. Re: Weblogic 10.3.6 heap memory usage is high
                      user3897984

                      Thanks for the reply

                      If we set the -Xms512mb -Xmx5GB, I read a article that the rest of the memory will be used by the OS for other process. - Is this statement correct ?

                      And is the recommended that -Xms5GB -Xmx5GB always for better performance ? Is that correct ?

                       

                      Also, WLS 10.3.6 is certified and supported against JDK 7. -- So the web-logic recommend Jdk1.7 for better performance, currently we are sing JRockit Jdk1.6 64 bit.

                       

                      If we set the -Xms5GB -Xmx5GB as heap size why the GC goes till the 5gb mark and then GC is triggered (When application is ideal state)?

                      We are using the JRMC - for flight recording done analysis on that .. but clues.

                       

                      Please let me know your thoughts.

                       

                      • 8. Re: Weblogic 10.3.6 heap memory usage is high
                        Carlos Cortez

                        Hi,

                         

                        If we set the -Xms512mb -Xmx5GB, I read a article that the rest of the memory will be used by the OS for other process. - Is this statement correct ?

                        Could be, if the OS requires more memory for other processes it will get it from the available memory, your JVM will grow dynamically because you are not setting the same value. The problem of this is when the JVM requires more memory and there is no available memory.

                         

                        And is the recommended that -Xms5GB -Xmx5GB always for better performance ? Is that correct ?

                        Yes, because with this you reserve the required memory for JVM and also for a better performance.

                         

                        If we set the -Xms5GB -Xmx5GB as heap size why the GC goes till the 5gb mark and then GC is triggered (When application is ideal state)?

                        We are using the JRMC - for flight recording done analysis on that .. but clues.

                        This could be because you heap is almost full and GC is trying to release memory, it doesn't release more objects because maybe they still referenced... you have to determine why your heap is always full

                        • 9. Re: Weblogic 10.3.6 heap memory usage is high
                          user3897984

                          Thanks for  the response.

                           

                          My current production heap size is -Xms5GB -Xmx5GB as heap size, when I get the heap dump and open in MAT tool is shows the total 230 MB, why is that so ?

                          • 10. Re: Weblogic 10.3.6 heap memory usage is high
                            Steff2-Oracle

                            Hello,

                             

                            Are you running any other product  (or running another Oracle Software) on that same server, that is using quite some big memory from your total of 20 GB ?

                            If that's the case, then the heapsize cannot reach the maximum 5GB because it is not available.

                            Also, make sure you've got enough free harddisk space (ex: # df -h), so that the swap space can be used too.

                             

                            You can get the swap space available by running the command : grep SwapTotal /proc/meminfo

                            or /sbin/swapon -s  or simply use the "free" command, for instance : free -m

                             

                            Hope this will help you too.

                             

                            With Best Regards,

                            Stephan

                            • 11. Re: Weblogic 10.3.6 heap memory usage is high
                              Carlos Cortez

                              Can you run the following command to see all the running java processes your server has:

                               

                              ps -fea | grep java

                              and the output of:

                              df -h

                               

                              and:

                              free -m

                              1 person found this helpful
                              • 12. Re: Weblogic 10.3.6 heap memory usage is high
                                user3897984

                                Thanks

                                Here is the details from production environment :

                                 

                                Command : free -m

                                               total       used       free     shared    buffers     cached

                                Mem:   19696      15658       4038          0        502       2689

                                -/+ buffers/cache:      12466       7230

                                Swap:    2047          0       2047

                                 

                                1.) Since production heap size is Min and Max value is 5GB, still we will face issue if we have other java processing running in the same Linux machine. ( -Xms5GB -Xmx5GB )

                                2.) Is the below statement matter

                                • Twice the size of RAM if RAM is less than 2 GB
                                • Size of RAM + 2 GB if RAM size is more than 2 GB i.e. 5GB of swap for 3GB of RAM

                                 

                                Thanks

                                • 13. Re: Weblogic 10.3.6 heap memory usage is high
                                  user3897984

                                     I do capture the details from the UAT, I do see 10+ java process running in the same host, Since production heap size is Min and Max value is set to same value, still we will face issue if we have other java processing running in the same machine, please confirm.

                                  How the Swap it related to the RAM ?

                                   

                                  2019-12-20_125733.jpg

                                  • 14. Re: Weblogic 10.3.6 heap memory usage is high
                                    user3897984

                                      I do capture the details from the PROD, I do see only 2 java process running in the same host, Since production heap size is Min and Max value is set to same value, still we will face issue if we have other java processing running in the same machine, please confirm.

                                    2019-12-20_153031.jpg

                                    1 2 Previous Next