0 Replies Latest reply: Apr 12, 2013 1:13 AM by 1002669 RSS

    Why OLD GC always FULL(not fluctuating)

    1002669
      Hi All,

      We are getting response issue for portal.
      After seeing all, I found ''CMS old Gen heap memory was always FULL''.
      Present situation is: Suddenly java process id takes >=100% CPU and after some time it drops to minimum.


      Please go through all stats which I have collected during this issue. Always OC=OU. Is this memory leak?

      1) home/jboss $ /usr/java/jdk1.6.0_29/bin/jstat -gc 28678 3000 10
      S0C S1C S0U S1U EC EU OC OU PC PU YGC YGCT FGC FGCT GCT
      26176.0 26176.0 26176.0 0.0 209792.0 209792.0 2908160.0 2908159.7 786432.0 588167.4 38327 5341.012 10887 46163.805 51504.817
      26176.0 26176.0 26176.0 0.0 209792.0 209792.0 2908160.0 2908159.7 786432.0 588167.4 38327 5341.012 10887 46163.805 51504.817
      26176.0 26176.0 0.0 0.0 209792.0 94779.9 2908160.0 2908159.7 786432.0 588167.4 38327 5341.012 10888 46176.629 51517.641
      26176.0 26176.0 0.0 0.0 209792.0 102283.0 2908160.0 2908159.7 786432.0 588167.4 38327 5341.012 10888 46176.629 51517.641
      26176.0 26176.0 0.0 0.0 209792.0 111346.8 2908160.0 2908159.7 786432.0 588167.4 38327 5341.012 10888 46176.629 51517.641
      26176.0 26176.0 0.0 0.0 209792.0 118928.1 2908160.0 2908159.7 786432.0 588167.4 38327 5341.012 10888 46176.629 51517.641
      26176.0 26176.0 0.0 0.0 209792.0 128037.0 2908160.0 2908159.7 786432.0 588167.4 38327 5341.012 10889 46176.629 51517.641
      26176.0 26176.0 0.0 0.0 209792.0 163246.9 2908160.0 2908159.7 786432.0 588203.4 38327 5341.012 10889 46176.950 51517.962
      26176.0 26176.0 0.0 0.0 209792.0 180997.0 2908160.0 2908159.7 786432.0 588206.6 38327 5341.012 10889 46176.950 51517.962
      26176.0 26176.0 0.0 0.0 209792.0 188662.2 2908160.0 2908159.7 786432.0 588206.6 38327 5341.012 10889 46176.950 51517.962
      2) home/jboss $ /usr/java/jdk1.6.0_29/bin/jstat -gc 28678 3000 10
      S0C S1C S0U S1U EC EU OC OU PC PU YGC YGCT FGC FGCT GCT
      26176.0 26176.0 26176.0 0.0 209792.0 209792.0 2908160.0 2908160.0 786432.0 588256.2 38353 5341.013 10965 46507.824 51848.836
      26176.0 26176.0 26176.0 0.0 209792.0 209792.0 2908160.0 2908160.0 786432.0 588256.2 38353 5341.013 10965 46507.824 51848.836
      26176.0 26176.0 26176.0 0.0 209792.0 209792.0 2908160.0 2908160.0 786432.0 588256.2 38353 5341.013 10965 46507.824 51848.836
      26176.0 26176.0 26176.0 0.0 209792.0 209792.0 2908160.0 2908160.0 786432.0 588256.2 38353 5341.013 10965 46507.824 51848.836
      26176.0 26176.0 0.0 0.0 209792.0 94390.9 2908160.0 2908160.0 786432.0 588266.6 38353 5341.013 10966 46520.655 51861.668
      26176.0 26176.0 0.0 0.0 209792.0 101762.3 2908160.0 2908160.0 786432.0 588266.6 38353 5341.013 10966 46520.655 51861.668
      26176.0 26176.0 0.0 0.0 209792.0 111417.3 2908160.0 2908160.0 786432.0 588266.6 38353 5341.013 10966 46520.655 51861.668
      26176.0 26176.0 0.0 0.0 209792.0 118899.7 2908160.0 2908160.0 786432.0 588266.6 38353 5341.013 10966 46520.655 51861.668
      26176.0 26176.0 0.0 0.0 209792.0 126927.6 2908160.0 2908160.0 786432.0 588266.6 38353 5341.013 10967 46520.971 51861.984
      26176.0 26176.0 0.0 0.0 209792.0 135804.1 2908160.0 2908160.0 786432.0 588266.6 38353 5341.013 10967 46520.971 51861.984



      3. Remaining Memory pools are seems fine. I could see some freed/utilization of memory. Collected today only.
      Pool: Code Cache (Non-heap memory)
      Peak Usage : init:2555904, used:47488576, committed:47906816, max:50331648
      Current Usage : init:2555904, used:47480128, committed:47906816, max:50331648
      Pool: Par Eden Space (Heap memory)
      Peak Usage : init:107479040, used:214827008, committed:214827008, max:214827008
      Current Usage : init:107479040, used:163173688, committed:214827008, max:214827008
      Pool: Par Survivor Space (Heap memory)
      Peak Usage : init:13369344, used:26804224, committed:26804224, max:26804224
      Current Usage : init:13369344, used:0, committed:26804224, max:26804224

      Pool: CMS Perm Gen (Non-heap memory)
      Peak Usage : init:805306368, used:602522624, committed:805306368, max:805306368
      Current Usage : init:805306368, used:602046224, committed:805306368, max:805306368
      4. SWAP usage is FINE
      Swap: 2.38 GB of 3.91 GB used (61.05%)

      SWAP Usage: 4709.81 MB --> PID: 28678 --> /usr/java/jdk1.6.0_30/bin/java
      SWAP Usage: 3308.27 MB --> PID: 31314 --> /usr/java/jdk1.6.0_30/bin/java
      SWAP Usage: 3130.38 MB --> PID: 31651 --> /usr/java/jdk1.6.0_30/bin/java
      SWAP Usage: 2333.38 MB --> PID: 11335 --> /opt/savideploy51/jre/bin/java
      SWAP Usage: 1442.82 MB --> PID: 26346 --> /opt/savideploy51/jre/bin/java

      5. PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
      28678 jboss 18 0 4709m 3.9g 11m S 94.5 50.1 1606:54 java


      Please could you share your observations.

      Regards,
      Prabhu