6 Replies Latest reply: Oct 31, 2010 8:01 PM by 799380 RSS

    ORA-29516: Bulk load of method failed; insufficient shm-object space

    508952
      Hello,

      Just installed 11.2.0.1.0 on CentOS 5.5 64-bit. All dependencies satisfied, installation/linking went without a problem.
      Server has 32GB RAM, using AMM with target set at 29GB, no swapping is occuring.

      No matter what i do when loading Java code (loadjava with JARs or "create and compile java source") I keep getting the error:
      ----------------------
      ORA-29516: Error in module Aurora: Assertion failure at joez.c:3311
      Bulk load of method java/lang/Object.<init> failed; insufficient shm-object space
      ----------------------

      Checked shm-related kernel params, all seems to be normal:
      ----------------------
      # Controls the maximum size of a message, in bytes
      kernel.msgmnb = 65536
      # Controls the default maxmimum size of a mesage queue
      kernel.msgmax = 65536
      # Controls the maximum shared segment size, in bytes
      kernel.shmmax = 68719476736
      # Controls the maximum number of shared memory segments, in pages
      kernel.shmall = 4294967296

      kernel.shmmni = 4096
      kernel.sem = 250 32000 100 128

      net.core.rmem_default = 262144
      net.core.rmem_max = 4194304
      net.core.wmem_default = 262144
      net.core.wmem_max = 1048576
      ----------------------

      Please help.
        • 1. Re: ORA-29516: Bulk load of method failed; insufficient shm-object space
          MarceloF.Ochoa
          Hi:
          First check your init.ora parameters, special the parameters that have a key java, for example java_pool_size. If all of them are 0 means that are automatically tunned by the RDBMS.
          Then look at your shell limits, and ensure that your stack is set to less than two gigabytes:

          ulimit -a
          time(seconds) unlimited
          file(blocks) unlimited
          data(kbytes) 131072
          stack(kbytes) 2048
          memory(kbytes) 245504
          coredump(blocks) unlimited
          nofiles(descriptors) 4096
          vmemory(kbytes) 1048576

          Also if you found a .trc file associated to your oracle process which cause the dump it will have more information, check the directory:
          $ORACLE_BASE/diag/rdbms/test/test/trace
          Best regards, Marcelo
          • 2. Re: ORA-29516: Bulk load of method failed; insufficient shm-object space
            508952
            Hi, Marcelo!

            Thanks for you answer, checked it, all params seem to be fine:

            SQL> show parameter java

            java_jit_enabled boolean TRUE
            java_max_sessionspace_size integer 0
            java_pool_size big integer 0
            java_soft_sessionspace_limit integer 0

            $ ulimit -a
            core file size (blocks, -c) 0
            data seg size (kbytes, -d) unlimited
            scheduling priority (-e) 0
            file size (blocks, -f) unlimited
            pending signals (-i) 270336
            max locked memory (kbytes, -l) 32
            max memory size (kbytes, -m) unlimited
            open files (-n) 65536
            pipe size (512 bytes, -p) 8
            POSIX message queues (bytes, -q) 819200
            real-time priority (-r) 0
            stack size (kbytes, -s) 10240
            cpu time (seconds, -t) unlimited
            max user processes (-u) 16384
            virtual memory (kbytes, -v) unlimited
            file locks (-x) unlimited

            There seems to be no trace files generated on that error.
            • 3. Re: ORA-29516: Bulk load of method failed; insufficient shm-object space
              MarceloF.Ochoa
              Hi:
              Did you follow all the steps defined into this guide (specially step 6):
              http://download.oracle.com/docs/cd/E11882_01/install.112/e10860/toc.htm
              also try to set a little value for your memory target, for example 16Gb out of 32Gb , 900Mb is enough to create the DB.
              Then you can increase the SGA target when your application requires more memory to work.
              Best regards, Marcelo.
              • 4. Re: ORA-29516: Bulk load of method failed; insufficient shm-object space
                799380
                Hi there,

                I've stumbled into exactly the same issue for 11g. After I start the database up and I ran loadjava on an externally
                compiled class (Hello.class in my instance) I got the following error:

                Error while testing for existence of dbms_java.handleMd5
                ORA-29516: Aurora assertion failure: Assertion failure at joez.c:3311
                Bulk load of method java/lang/Object.<init> failed; insufficient shm-object space
                ORA-06512: at "SYS.DBMS_JAVA", line 679

                Error while creating class Hello
                ORA-29516: Aurora assertion failure: Assertion failure at joez.c:3311
                Bulk load of method java/lang/Object.<init> failed; insufficient shm-object space
                ORA-06512: at line 1

                The following operations failed
                class Hello: creation (createFailed)
                exiting : Failures occurred during processing


                After this, I checked the trace file and saw the following error message:

                peshmmap_Create_Memory_Map:
                Map_Length = 4096
                Map_Protection = 7
                Flags = 1
                File_Offset = 0
                mmap failed with error 1
                error message:Operation not permitted
                ORA-04035: unable to allocate 4096 bytes of shared memory in shared object cache "JOXSHM" of size "134217728"
                peshmmap_Create_Memory_Map:
                Map_Length = 4096
                Map_Protection = 7
                Flags = 1
                File_Offset = 0
                mmap failed with error 1
                error message:Operation not permitted
                ORA-04035: unable to allocate 4096 bytes of shared memory in shared object cache "JOXSHM" of size "134217728"
                Assertion failure at joez.c:3311
                Bulk load of method java/lang/Object.<init> failed; insufficient shm-object space

                It seems as though that "JOXSHM" of size "134217728" (which is 128MB) corresponds to the java_pool_size setting in my init.ora file:

                ..........
                memory_target=1000M
                memory_max_target=2000M
                java_pool_size=128M
                shared_pool_size=256M
                ..........

                Whenever I change that size it propagates to the trace file. I also picked up that only 592MB of shm memory gets used. My df -h dump:

                Filesystem Size Used Avail Use% Mounted on
                /dev/sda7 39G 34G 4.6G 89% /
                udev 10M 288K 9.8M 3% /dev
                /dev/sda5 63M 43M 21M 69% /boot
                /dev/sda4 59G 45G 11G 81% /mnt/data
                shm 2.0G 592M 1.5G 29% /dev/shm

                The only way in which I could get loadjava to work was to remove java from the database by calling the rmjvm.sql script.
                After this I installed java again by calling the initjvm.sql script. I noticed that after these scripts my shm-memory usage
                increased to about 624MB which is 32MB larger than before:

                Filesystem Size Used Avail Use% Mounted on
                /dev/sda7 39G 34G 4.6G 89% /
                udev 10M 288K 9.8M 3% /dev
                /dev/sda5 63M 43M 21M 69% /boot
                /dev/sda4 59G 45G 11G 81% /mnt/data
                shm 2.0G 624M 1.4G 31% /dev/shm

                However, after I stopped the database and started it again my Java was broken again and calling loadjava produced
                the same error message as before. The shm memory usage would also return to 592MB again. Is there something I
                need to do in terms of persisting the changes that initjvm and rmjvm does to the database? Or is there something else
                wrong that I'm overlooking like the memory management settings or something?

                Regards,

                Wiehann
                • 5. Re: ORA-29516: Bulk load of method failed; insufficient shm-object space
                  808682
                  Applies to:
                  Oracle Server - Enterprise Edition - Version: 11.1.0.6 to 11.2.0.2.0 [Release: 11.1 to 11.2]
                  Linux x86
                  Linux x86-64
                  JAVA_JIT_ENABLED
                  Symptoms
                  This type of error may appear in the alert log upon instance start-up on Linux:


                  Wed Jun 23 16:44:13 2010
                  Errors in file /app/ORCL/product/orcldb/11.2.0/admin/ORCL/diag/rdbms/orcl/ORCL/trace/ORCL_m000_20219.trc:
                  ORA-29516: Aurora assertion failure: Assertion failure at joez.c:3311
                  Bulk load of method java/lang/Object.<init> failed; insufficient shm-object space

                  It may also occur when attempting to reference a Java class method or a function within the DBMS_JAVA package, such as LONGNAME.


                  Changes
                  You have set JAVA_JIT_ENABLED=TRUE (or not set it all, in which case it defaults to TRUE).

                  Cause
                  You have set additional mount options on the shared memory device, /dev/shm, other than just the option named defaults.

                  Solution
                  If you see any other mount options in file /etc/fstab after the word "defaults" for /dev/shm,
                  such as nosuid, nodev, or noexec:

                  none /dev/shm tmpfs defaults,nosuid,nodev,noexec 0 0

                  then remove these options from the entry in /etc/fstab:

                  none /dev/shm tmpfs defaults 0 0

                  shut down the database instance, and reboot the system. Restart the database instance. The errors should no longer appear.


                  References
                  BUG:9294055 - GETTING ORA-29516 WHEN JAVA_JIT_ENABLED=TRUE

                  Related



                  --------------------------------------------------------------------------------
                  Products
                  --------------------------------------------------------------------------------

                  Oracle Database Products > Oracle Database > Oracle Database > Oracle Server - Enterprise Edition
                  Errors
                  --------------------------------------------------------------------------------
                  ORA-29516; AURORA; 29516 ERROR; 2010 ERROR
                  • 6. Re: ORA-29516: Bulk load of method failed; insufficient shm-object space
                    799380
                    Hi there,

                    Thank you very much, the solution works.

                    Regards,

                    Wiehann