1 Reply Latest reply on May 15, 2017 1:13 PM by 1244000

    Out-of-memory

    3290470

      Hi guys, I'm looking in all websites and I can't found informations and solution about this error.

       

      Look my environment.

      Linux version 3.8.13-118.10.2.el7uek.x86_64 (mockbuild@x86-ol7-builder-02) (gcc version 4.8.3 20140911 (Red Hat 4.8.3-9) (GCC) )

       

      [code]

      [oracledb@srv-3189 ~]$ free -g

                    total        used        free      shared  buff/cache   available

      Mem:             17           1           1           5          15          10

      Swap:             3           0           3

       

      tmpfs                              128G  5.6G  123G   5% /dev/shm

      [/code]

      [code]

      [oracledb@srv-3189 ~]$ cat /proc/meminfo

      MemTotal:       18498276 kB

      MemFree:         1754072 kB

      Buffers:            1664 kB

      Cached:         14657604 kB

      SwapCached:            0 kB

      Active:          9891564 kB

      Inactive:        5947832 kB

      Active(anon):    5854248 kB

      Inactive(anon):  1131984 kB

      Active(file):    4037316 kB

      Inactive(file):  4815848 kB

      Unevictable:           0 kB

      Mlocked:               0 kB

      SwapTotal:       4194300 kB

      SwapFree:        4194040 kB

      Dirty:             13348 kB

      Writeback:             0 kB

      AnonPages:       1180128 kB

      Mapped:          4272172 kB

      Shmem:           5806104 kB

      Slab:             342952 kB

      SReclaimable:     302772 kB

      SUnreclaim:        40180 kB

      KernelStack:        2496 kB

      PageTables:       342392 kB

      NFS_Unstable:          0 kB

      Bounce:                0 kB

      WritebackTmp:          0 kB

      CommitLimit:    13443436 kB

      Committed_AS:    7665120 kB

      VmallocTotal:   34359738367 kB

      VmallocUsed:       59116 kB

      VmallocChunk:   34359666948 kB

      HardwareCorrupted:     0 kB

      HugePages_Total:       0

      HugePages_Free:        0

      HugePages_Rsvd:        0

      HugePages_Surp:        0

      Hugepagesize:       2048 kB

      DirectMap4k:        8192 kB

      DirectMap2M:     2088960 kB

      DirectMap1G:    16777216 kB

      [/code]

      [code]

      SYS@dsdsds AS SYSDBA> select * from V$RESOURCE_LIMIT;

       

      RESOURCE_NAME                  CURRENT_UTILIZATION MAX_UTILIZATION INITIAL_AL LIMIT_VALU     CON_ID

      ------------------------------ ------------------- --------------- ---------- ---------- ----------

      processes                                       63              74        600        600          0

      sessions                                        74              94        928        928          0

      enqueue_locks                                   79             133      10700      10700          0

      enqueue_resources                               56              56       4300  UNLIMITED          0

      ges_procs                                        0               0          0          0          0

      ges_ress                                         0               0          0  UNLIMITED          0

      ges_locks                                        0               0          0  UNLIMITED          0

      ges_cache_ress                                   0               0          0  UNLIMITED          0

      ges_reg_msgs                                     0               0          0  UNLIMITED          0

      ges_big_msgs                                     0               0          0  UNLIMITED          0

      ges_rsv_msgs                                     0               0          0          0          0

      gcs_resources                                    0               0          0  UNLIMITED          0

      gcs_shadows                                      0               0          0  UNLIMITED          0

      smartio_overhead_memory                          0               0          0  UNLIMITED          0

      smartio_buffer_memory                            0               0          0  UNLIMITED          0

      smartio_metadata_memory                          0               0          0  UNLIMITED          0

      smartio_sessions                                 0               0          0  UNLIMITED          0

      dml_locks                                        0               0       4080  UNLIMITED          0

      temporary_table_locks                            0               5  UNLIMITED  UNLIMITED          0

      transactions                                     0               0       1020  UNLIMITED          0

      branches                                         0               1       1020  UNLIMITED          0

      cmtcallbk                                        0               6       1020  UNLIMITED          0

      max_rollback_segments                           11              11       1020      65535          0

      sort_segment_locks                              24              49  UNLIMITED  UNLIMITED          0

      k2q_locks                                        0               0       1856  UNLIMITED          0

      max_shared_servers                               1               1  UNLIMITED  UNLIMITED          0

      parallel_max_servers                             0               0          0      32767          0

      [/code]

       

      [code]

      SYS@dsdsds AS SYSDBA> show parameter memory

       

      NAME                                 TYPE        VALUE

      ------------------------------------ ----------- ------------------------------

      hi_shared_memory_address             integer     0

      inmemory_clause_default              string

      inmemory_force                       string      DEFAULT

      inmemory_max_populate_servers        integer     0

      inmemory_query                       string      ENABLE

      inmemory_size                        big integer 0

      inmemory_trickle_repopulate_servers_ integer     1

      percent

      memory_max_target                    big integer 12G

      memory_target                        big integer 12G

      optimizer_inmemory_aware             boolean     TRUE

      shared_memory_address                integer     0

      SYS@dsdsds AS SYSDBA> show parameter ga

       

      NAME                                 TYPE        VALUE

      ------------------------------------ ----------- ------------------------------

      enable_goldengate_replication        boolean     FALSE

      enable_pluggable_database            boolean     FALSE

      lock_sga                             boolean     FALSE

      pga_aggregate_limit                  big integer 5G

      pga_aggregate_target                 big integer 0

      pre_page_sga                         boolean     TRUE

      sga_max_size                         big integer 12G

      sga_target                           big integer 0

      unified_audit_sga_queue_size         integer     1048576

      [/code]

      Now, my problem.

       

      Sometimes I'm unable to connect on database with sysdba (instance idle) and the Oracle shows a message like ORA-27102: out of memory when I try to startup, sometimes I can connect on database normaly but APEX applications doesn't work as well, out-of-memory when I try create Jasper reports or try to generate a SNAPSHOT using SMO (NETAPP)

       

      Alert.log

      [code]

      2016-11-10 16:57:57.541000 -02:00

      Process J000 died, see its trace file

      kkjcre1p: unable to spawn jobq slave process

      Errors in file /opt/app/oracle/diag/rdbms/dsdsds/dsdsds/trace/dsdsds_cjq0_22940.trc:

      2016-11-10 16:57:59.541000 -02:00

      Process J000 died, see its trace file

      kkjcre1p: unable to spawn jobq slave process

      Errors in file /opt/app/oracle/diag/rdbms/dsdsds/dsdsds/trace/dsdsds_cjq0_22940.trc:

      2016-11-10 16:58:01.541000 -02:00

      Process J000 died, see its trace file

      kkjcre1p: unable to spawn jobq slave process

      Errors in file /opt/app/oracle/diag/rdbms/dsdsds/dsdsds/trace/dsdsds_cjq0_22940.trc:

      2016-11-10 16:58:03.541000 -02:00

      Process J000 died, see its trace file

      kkjcre1p: unable to spawn jobq slave process

      Errors in file /opt/app/oracle/diag/rdbms/dsdsds/dsdsds/trace/dsdsds_cjq0_22940.trc:

      2016-11-10 16:58:04.903000 -02:00[/code]

       

       

      trace

      [code]

      Trace file /opt/app/oracle/diag/rdbms/dsdsds/dsdsds/trace/dsdsds_cjq0_22940.trc

      Oracle Database 12c Standard Edition Release 12.1.0.2.0 - 64bit Production

      ORACLE_HOME = /opt/app/oracle/product/12r102/db

      System name:    Linux

      Node name:      srv-3189

      Release:        3.8.13-118.10.2.el7uek.x86_64

      Version:        #2 SMP Fri Aug 12 15:08:18 PDT 2016

      Machine:        x86_64

      VM name:        Xen Version: 4.4 (HVM)

      Instance name: dsdsds

      Redo thread mounted by this instance: 1

      Oracle process number: 37

      Unix process pid: 22940, image: oracle@srv-3189 (CJQ0)

       

       

      *** 2016-11-10 16:06:19.023

      *** SESSION ID:(587.25250) 2016-11-10 16:06:19.023

      *** CLIENT ID:() 2016-11-10 16:06:19.023

      *** SERVICE NAME:(SYS$BACKGROUND) 2016-11-10 16:06:19.023

      *** MODULE NAME:() 2016-11-10 16:06:19.023

      *** CLIENT DRIVER:() 2016-11-10 16:06:19.023

      *** ACTION NAME:() 2016-11-10 16:06:19.023

       

      Process J000 is dead (pid=7856 req_ver=177 cur_ver=177 state=KSOSP_SPAWNED).

       

      *** 2016-11-10 16:19:07.059

      Process J000 is dead (pid=8655 req_ver=137 cur_ver=137 state=KSOSP_SPAWNED).

       

      *** 2016-11-10 16:19:09.059

      Process J000 is dead (pid=8657 req_ver=138 cur_ver=138 state=KSOSP_SPAWNED).

       

      *** 2016-11-10 16:22:21.061

      Process J000 is dead (pid=8925 req_ver=241 cur_ver=241 state=KSOSP_SPAWNED).

       

      *** 2016-11-10 16:27:31.269

      Process J000 is dead (pid=9248 req_ver=248 cur_ver=248 state=KSOSP_SPAWNED).

       

      *** 2016-11-10 16:28:19.477

      Process J000 is dead (pid=9422 req_ver=275 cur_ver=275 state=KSOSP_SPAWNED).

       

      *** 2016-11-10 16:28:21.477

      Process J000 is dead (pid=9424 req_ver=276 cur_ver=276 state=KSOSP_SPAWNED).

       

      *** 2016-11-10 16:28:25.479

      Process J000 is dead (pid=9428 req_ver=278 cur_ver=278 state=KSOSP_SPAWNED).

       

      *** 2016-11-10 16:28:49.514

      Process J002 is dead (pid=9470 req_ver=12 cur_ver=12 state=KSOSP_SPAWNED).

      [/code]

       

      What mean "Process J002 is dead" and how can I revise my config to avoid this issue?

       

      I've searched this link https://community.oracle.com/message/11046127#11046127 the problem is the same of mine. When I alter the memory target 12G to 6G to 12G again works fine, but what's the reason?

        • 1. Re: Out-of-memory
          1244000

          HI

           

          I had the same Issue. (exact the same symtoms) Its a Oracle Bug if you are using AMM. See --> Bug 24921392

          For me it helped to disable MEMORY_TARGET and set all memory settings by hand. Bug will be fixed in Ora12.2.01

           

          Hope this advice will help someone

           

          Regards