9 Replies Latest reply: Mar 13, 2013 2:10 PM by hyperionEPM RSS

    Oracle JRockit R28.2.2. OutOfMemoryError: classblock allocation

    Josue Barrero
      Hi Folks,

      Well, we have a production Weblogic environment over Oracle JRockit Virtual Machine. The problem is that we are trying to avoid an " OutOfMemoryError" that is thrown everyday by a SOA Managed Server and causes a failure and stop of the managed server which then has to be restarted, causing applications downtime.

      Platform:
      Windows 64bits Server 2008 R2 Standard. 32GB RAM

      Products:
      Oracle JRockit R28.2.2
      Oracle Weblogic Server 10.3.2
      Oracle SOA and BPM Suite 11g R1 11.1.1.5 Features Pack
      Oracle Services Bus 11g 11.1.1.5


      The message error is the following:

      java.lang.OutOfMemoryError: classblock allocation, 110787528 loaded, 84224K footprint, in check_alloc (src/jvm/model/classload/classalloc.c:215).

      Attempting to allocate 7G bytes

      There is insufficient native memory for the Java
      Runtime Environment to continue.

      Possible reasons:
      The system is out of physical RAM or swap space
      In 32 bit mode, the process size limit was hit

      Possible solutions:
      Reduce memory load on the system
      Increase physical memory or swap space
      Check if swap backing store is full
      Use 64 bit Java on a 64 bit OS
      Decrease Java heap size (-Xmx/-Xms)
      Decrease number of Java threads
      Decrease Java thread stack sizes (-Xss)
      Disable compressed references (-XXcompressedRefs=false)


      The following are the conclusions I have made about this problem:

      - It is clear in the message error that this is not a Java Heap Size issue since those kind of errors usually look like: (Java Heap Space, getNewTLA, etc...) and the message clearly states: "There is insufficient native memory for the Java Runtime Environment to continue". Besides, the MissionControl recordings show that the heap is constantly being garbage collected and there are not memory leaks.
      - The process size is about 10GB (7GB Heap Size and 4GB Native memory) but the platform consists of a 64 bit Windows 2008 Server with 32GB, so this is neither a "process size limit issue".
      -According to this, it could be clear that the problem is that "The system is out of physical RAM or swap space" but this isn't true either, since the system has a 48GB max paging file (wich in fact is not being used) and most importantly, the system has a total of 16GB RAM being used by all processes, leaving 16GB memory free for use.
      I have come to think that either JRockit is failing and throwing the exception without a good reason, or the message displayed is wrong and it is not letting me find the right cause of the issue we are facing.

      Any kind of advice will be appreciated.
        • 1. Re: Oracle JRockit R28.2.2. OutOfMemoryError: classblock allocation
          913518
          Hi,
          Do you think you could give me your JAVA_OPTS? 7G of HEAP is too large for a JVM. If you have this amount of memory and still out of it. May be, you have a develop issue. Next time, do a thread Dump to analyze the problem. If you want to, you can post it and I'll help you.
          • 2. Re: Oracle JRockit R28.2.2. OutOfMemoryError: classblock allocation
            664991
            Hi

            Perhaps you can make use of the -XX:MaxClassBlockMemory option? JRockit can run out of memory when a large number of classes are loaded. Set it to aproxx 75mb

            kind regards


            Michel
            • 3. Re: Oracle JRockit R28.2.2. OutOfMemoryError: classblock allocation
              Kalyan Pasupuleti-Oracle
              Hi Josue,

              When ever you see java.lang.OutOfMemoryError: classblock allocation, while using Jrockit older version then it deftly sure shot a BUG.

              I would only suggest to upgrade your Jrockit version.

              Still you face this issue then please raise ticket with Oracle Jrockit support you will get Fix for it.

              Regards,
              Kal
              • 4. Re: Oracle JRockit R28.2.2. OutOfMemoryError: classblock allocation
                960946
                Hello,

                I am running into a similar issue with Jrockit 28.2.3. Following is the stacktrace:

                2012-09-08 18:08:56,341 WARN [STDERR] (Load thread:) Exception in thread "Load thread" java.lang.OutOfMemoryError: classblock allocation, 2814576 loaded, 2816K footprint, in check_alloc (src/jvm/model/classload/classalloc.c:215).

                Attempting to allocate 4320M bytes

                There is insufficient native memory for the Java
                Runtime Environment to continue.

                Possible reasons:
                The system is out of physical RAM or swap space
                In 32 bit mode, the process size limit was hit

                Possible solutions:
                Reduce memory load on the system
                Increase physical memory or swap space
                Check if swap backing store is full
                Use 64 bit Java on a 64 bit OS
                Decrease Java heap size (-Xmx/-Xms)
                Decrease number of Java threads
                Decrease Java thread stack sizes (-Xss)
                Disable compressed references (-XXcompressedRefs=false)
                2012-09-08 18:08:56,341 WARN [STDERR] (Load thread:)
                2012-09-08 18:08:56,341 WARN [STDERR] (Load thread:) at java.lang.ClassLoader.defineClass1(Native Method)
                2012-09-08 18:08:56,341 WARN [STDERR] (Load thread:) at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
                2012-09-08 18:08:56,341 WARN [STDERR] (Load thread:) at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
                2012-09-08 18:08:56,341 WARN [STDERR] (Load thread:) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
                2012-09-08 18:08:56,341 WARN [STDERR] (Load thread:) at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
                2012-09-08 18:08:56,341 WARN [STDERR] (Load thread:) at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
                2012-09-08 18:08:56,341 WARN [STDERR] (Load thread:) at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
                2012-09-08 18:08:56,341 WARN [STDERR] (Load thread:) at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
                2012-09-08 18:08:56,341 WARN [STDERR] (Load thread:) at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
                2012-09-08 18:08:56,341 WARN [STDERR] (Load thread:) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
                2012-09-08 18:08:56,341 WARN [STDERR] (Load thread:) at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
                2012-09-08 18:08:56,341 WARN [STDERR] (Load thread:) at org.apache.log4j.spi.ThrowableInformation.getThrowableStrRep(ThrowableInformation.java:87)
                2012-09-08 18:08:56,341 WARN [STDERR] (Load thread:) at org.apache.log4j.spi.LoggingEvent.getThrowableStrRep(LoggingEvent.java:413)
                2012-09-08 18:08:56,341 WARN [STDERR] (Load thread:) at org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:313)
                2012-09-08 18:08:56,341 WARN [STDERR] (Load thread:) at org.apache.log4j.RollingFileAppender.subAppend(RollingFileAppender.java:276)
                2012-09-08 18:08:56,341 WARN [STDERR] (Load thread:) at org.apache.log4j.WriterAppender.append(WriterAppender.java:162)
                2012-09-08 18:08:56,341 WARN [STDERR] (Load thread:) at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
                2012-09-08 18:08:56,341 WARN [STDERR] (Load thread:) at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
                2012-09-08 18:08:56,341 WARN [STDERR] (Load thread:) at org.apache.log4j.Category.callAppenders(Category.java:206)
                2012-09-08 18:08:56,341 WARN [STDERR] (Load thread:) at org.apache.log4j.Category.forcedLog(Category.java:391)
                2012-09-08 18:08:56,341 WARN [STDERR] (Load thread:) at org.apache.log4j.Category.warn(Category.java:1060)
                2012-09-08 18:08:56,341 WARN [STDERR] (Load thread:) at com.teneo.esa.common.util.Logger.printStack(Logger.java:584)
                • 5. Re: Oracle JRockit R28.2.2. OutOfMemoryError: classblock allocation
                  Kalyan Pasupuleti-Oracle
                  Hi,

                  Well this kind of error OutOfMemoryError: classblock allocation, can occur either you have less physical memory or lack of swap space.

                  Try to increase your swap space mostly this will solve your problem.

                  Regards,
                  Kal
                  • 6. Re: Oracle JRockit R28.2.2. OutOfMemoryError: classblock allocation
                    960946
                    Kalyan,

                    Thanks for the prompt reply. Like Josue Barrero issue, the swap size in my case 33GB and the physical memory is 32GB. There was almost 10GB of free physical memory available.

                    Also, I took a heap dump at the time of OOM issue. The application was hardly using 700MB of heap space, so I wonder if the issue is something else.

                    -Snehal
                    • 7. Re: Oracle JRockit R28.2.2. OutOfMemoryError: classblock allocation
                      Kalyan Pasupuleti-Oracle
                      Hi Snehal,

                      As Mike mentioned earlier try to increase

                      -XX:MaxClassBlockMemory to 75M.

                      use following option.

                      -XX:+UnlockDiagnosticVMOptions -XX:MaxClassBlockMemory=75M

                      Regards,
                      Kal
                      • 8. Re: Oracle JRockit R28.2.2. OutOfMemoryError: classblock allocation
                        Josue Barrero
                        Hi,

                        As Mike and Kal mentioned:

                        -XX:+UnlockDiagnosticVMOptions -XX:MaxClassBlockMemory=75M

                        Solved the issue.

                        Regards,

                        Josue
                        • 9. Re: Oracle JRockit R28.2.2. OutOfMemoryError: classblock allocation
                          hyperionEPM
                          Hi Kalyan,

                          I am having a similar kind of entries in Hyperion Planning log

                          Where should I make the entry that you have specified, I mean to which file? Is that startweblogic.cmd
                          Please let me know, so that I can make the changes to weblogic on our production server.

                          Thanks,
                          hyperionEPM.