9 Replies Latest reply on Feb 28, 2014 8:43 AM by Tom321

    OEM-HIgh Memory Utilization (>94%) every hour

    egkua

      Hi,

       

      I noticed that on OEM that there is high memory utilization (>94%) every hour. I am using Oracle 11g.

      At other time there is still lots of free memory left.

      What is actually running on 11g every hour that utilize such high memory ?

       

      CriticalFeb 24, 2014 8:58:39 AMMemory Utilization is 95.67%, crossed warning (80) or critical (95) threshold.-
      WarningFeb 24, 2014 8:53:39 AMMemory Utilization is 94.88%, crossed warning (80) or critical (95) threshold.-
      CriticalFeb 24, 2014 7:58:29 AMMemory Utilization is 95.65%, crossed warning (80) or critical (95) threshold.-
      WarningFeb 24, 2014 7:53:29 AMMemory Utilization is 94.84%, crossed warning (80) or critical (95) threshold.-
      CriticalFeb 24, 2014 6:58:19 AMMemory Utilization is 95.65%, crossed warning (80) or critical (95) threshold

      .......

       

      Regards,

      Eye Gee

        • 1. Re: OEM-HIgh Memory Utilization (>94%) every hour
          Rahul-Em-Oracle

          Hi Eye Gee,

           

          As the high memory is reported every hour , it might be due to any scheduled job which is running

          Would recommend to review the below doc to understand how this value is calculated

          How Memory Utilization (%) is Calculated for Linux Hosts Monitored via Enterprise Manager (Doc ID 730104.1)

           

          Regards,

          Rahul

          • 2. Re: OEM-HIgh Memory Utilization (>94%) every hour
            egkua

            Rahul,

             

            Thank you for the reply.

            I am using Solaris Sparc and those patches are not available for Solaris.

             

            I tried the following sql and found the Memory Utilization captured is high and in fact the top shows 16G of free mem.:

            select to_char(COLLECTION_TIMESTAMP,'dd-mm-yyyy hh24:mi:ss'),

            substr(value,1,15)

            from mgmt$metric_details

            where column_label = 'Memory Utilization (%)'

            and COLLECTION_TIMESTAMP > sysdate -1 order by COLLECTION_TIMESTAMP;

             

             

            24-02-2014 13:59:33 95.7

            24-02-2014 14:04:33 96.06

            24-02-2014 14:09:33 96.19

            24-02-2014 14:14:33 96.24

            24-02-2014 14:19:33 96.36

            24-02-2014 14:24:33 96.47

            24-02-2014 14:29:33 96.48

             

            However it does happen that the free memory will reduce to 1GB+ sometimes, it could be some the oem scheduled job running.

             

            Any patches available for Solaris ?

             

            Regards,

            Eye Gee

            • 3. Re: OEM-HIgh Memory Utilization (>94%) every hour
              Tom321

              Hi,

               

              can you confirm with e.g. top, that you indeed have low memory on the system every hour? What processes does top show to have high memory usage during those peaks?

              Is this the agent process reporting high memory or the solaris host memory?

              There are some occasions where oem reports low mem on solaris, e.g. if you are running a rman backup every hour (archivelogs?) the database (shared) memory get's mapped to the agent process during the backup. OEM then reports high memory utilization during the rman backup, which however isn't there in reality.

               

              I'm running oem (12.1.0.3) on solaris 11.1 too and i don't see those spikes. What version do you run?

               

              Regards

              Thomas

              • 4. Re: OEM-HIgh Memory Utilization (>94%) every hour
                egkua

                Hi,

                 

                It is the number of Oracle connections causing with only 1.2GB of free memory.

                I am running Oracle 11.2.0.4 on Solaris with 32GB of memory with only 50 connections.

                I was using 9i on AIX with 24GB of memory with 500 connections and I just got to know the differences on platform.

                 

                Regards,

                Eye Gee

                • 5. Re: OEM-HIgh Memory Utilization (>94%) every hour
                  Tom321

                  Hi,

                   

                  solaris shows the memory usage of the processes including shared memory segement of the db instance for each process, that may be missleading. During next spike of your ram as root (on the global zone):

                   

                  mdb -k

                  ::memstat

                   

                  That gets you an output like this, which shows the  memory distribution of the system:

                   

                  Page Summary            Pages            MB  %Tot
                  ------------ ----------------  ----------------  ----
                  Kernel                1671477         13058   10%
                  ZFS File Data          524288          40963%
                  Anon                 10295801         80435   62%
                  Exec and libs          354692          27712%
                  Page cache              83249           6500%
                  Free (cachelist)       138801          10841%
                  Free (freelist)       3643372         28463   22%
                  Total                16711680        130560

                   

                   

                   

                  Hit control D for exit from the shell.

                   

                  What does your system show?

                   

                  Regards

                  Thomas

                  • 6. Re: OEM-HIgh Memory Utilization (>94%) every hour
                    egkua

                    Thomas,

                     

                    Thank you for the reply.

                     

                    It seems that the ZFS File Data is consuming high memory.

                     

                    Page Summary            Pages            MB  %Tot
                    ZFS File Data         1900544         14848   46%

                     

                     

                    Regards,

                    Eye Gee

                     

                    • 7. Re: OEM-HIgh Memory Utilization (>94%) every hour
                      Tom321

                      Hi,

                       

                      you can limit the file cache with an entry in

                      /etc/system:

                      set zfs:zfs_arc_max=4294967296

                       

                      That will limit the file cache to 4 GB.

                      You could go even lower with just 32 GB ram available in the server.The optimal size of zfs_arc depends on how you are using this server.

                      If it's primary role is db server and db's are using ASM then you could make this rather small like 1-2 GB. If databases are on filesystem the could get some performance from the cache, but most likely you still get more performance if you directly allocate the memory to the database via SGA and PGA.

                       

                      Regards

                      Thomas

                      • 8. Re: OEM-HIgh Memory Utilization (>94%) every hour
                        sam995972

                        good info Tom....

                        Tom321 wrote:

                         

                        Hi,

                         

                        solaris shows the memory usage of the processes including shared memory segement of the db instance for each process, that may be missleading. During next spike of your ram as root (on the global zone):

                         

                        mdb -k

                        ::memstat

                         

                        That gets you an output like this, which shows the  memory distribution of the system:

                         

                        Page Summary            Pages            MB  %Tot
                        ------------ ----------------  ----------------  ----
                        Kernel                1671477         13058   10%
                        ZFS File Data          524288          4096 3%
                        Anon                 10295801         80435   62%
                        Exec and libs          354692          2771 2%
                        Page cache              83249           650 0%
                        Free (cachelist)       138801          1084 1%
                        Free (freelist)       3643372         28463   22%
                        Total                16711680        130560

                         

                         

                         

                        Hit control D for exit from the shell.

                         

                        What does your system show?

                         

                        Regards

                        Thomas

                        please share us how to get the above info like in Linux system...

                        thanks,

                        Sam.

                        • 9. Re: OEM-HIgh Memory Utilization (>94%) every hour
                          Tom321

                          Hi Sam,

                           

                          i don't think there is a tool like this available in linux. You will have to use top, sar, vmstat, etc. to gather the information.

                          If you are concerned about free memory and file cache run:

                          free -m

                          That will give you an overview of the ram allocation.

                           

                          Regards

                          Thomas