8 Replies Latest reply on Nov 14, 2019 2:06 PM by Beauty_and_dBest

    EBS 12.1.3 - Weird Performance Behavior

    Beauty_and_dBest

      EBS 12.1.3

      11g

      OL6

       

       

      Hi ALL,

       

       

      We have been encountering this weird performance behavior in our EBS 12.1.3 instance for long time.

      It is a single node server where both apps & db are housed on.

      The weird behavior is that we need to bounce it every midnight, because if we do not, forms will surely hung the next day, or the apps is going to be very slow.

      The users always describe it as in "circling & cicling" state, meaning forms is waiting a long time to open.

      But if you check the "top" linux status command, the server is 99% idle   Meaning the request  is not being able to reach the server?

      I do not know where the bottleneck is? because if we bounce the apps & db , the performance is getting okay again.

       

      Please help how to capture the cause of this unusual hung behavior?

       

      Our temporary solution is to schedule a script to bounce apps & database every midnight.

       

       

      Kind regards,

      jc

        • 1. Re: EBS 12.1.3 - Weird Performance Behavior
          mdtaylor

          Hi jc,

           

          This sounds like a memory leak.  How much RAM does the server have, and how many users is it supporting?  Please type

           

          free -m

           

          at the end of the day before rebooting to see how much swapping is happening.

           

          Regards,

          Michael

          • 2. Re: EBS 12.1.3 - Weird Performance Behavior
            Beauty_and_dBest

            Thanks Michael,

             

            But I think there is no issue with swap or memory? Our RAM is 60Gb and 8cpu, Our swap is not even used?

             

            # free -m

                         total       used       free     shared    buffers     cached

            Mem:         60134      18091      42043      12978        558      13944

            -/+ buffers/cache:       3588      56546

            Swap:        27723          0      27723

             

             

             

            Could it be a problem with OACORE? need to increase its value?

            What other causes to why forms hung up?

             

             

            Please help.....

             

             

            Kind regards,

            • 3. Re: EBS 12.1.3 - Weird Performance Behavior
              lmu

              What are the values of processes and sessions in the database? What are the results of this?

              select resource_name, current_utilization, max_utilization

              from v$resource_limit where resource_name in ('processes','sessions');

              • 4. Re: EBS 12.1.3 - Weird Performance Behavior
                mdtaylor

                Are there anything in the 10.1.2 or 10.1.3 home logs?

                 

                EBS R12.0.x / R12.1.x : How To Find Location Of Install, Autoconfig, Patching , Clone And Other Logs ? (Doc ID 804603.1)

                • 5. Re: EBS 12.1.3 - Weird Performance Behavior
                  Kanda-Oracle

                  Hi

                   

                  screeshot looks fine. was it taken before bounce ?

                   

                  when its hung, did you check memory and top process

                   

                  Thanks!

                  • 6. Re: EBS 12.1.3 - Weird Performance Behavior
                    Beauty_and_dBest

                    Hi Kanda and ALL,

                     

                     

                    Yes  I checked the top session when its hung,  and all you can see is its has minimal activity, or CPU and Memory is 90% idle like the screenshot above.

                     

                    Kind regards,

                    • 7. Re: EBS 12.1.3 - Weird Performance Behavior
                      Beauty_and_dBest

                      Hi ALL,

                       

                      Is it okay to increase javaforms memory? or is it related to my issue?

                       

                      1. Edit application tier context file, change from:

                      <forms_jvm_start_options oa_var="s_forms_jvm_start_options">-server -verbose:gc -Xmx256M -Xms64M

                      <forms_jvm_stop_options oa_var="s_forms_jvm_stop_options">-server -verbose:gc -Xmx256M -Xms64M

                      <oacore_jvm_start_options oa_var="s_oacore_jvm_start_options">-server -verbose:gc -Xmx512M -Xms128M

                      <oacore_jvm_stop_options oa_var="s_oacore_jvm_stop_options">-server -verbose:gc -Xmx512M

                      To:

                      <forms_jvm_start_options oa_var="s_forms_jvm_start_options">-server -verbose:gc -Xmx1024M -Xms1024M

                      <forms_jvm_stop_options oa_var="s_forms_jvm_stop_options">-server -verbose:gc -Xmx1024M -Xms1024M

                      <oacore_jvm_start_options oa_var="s_oacore_jvm_start_options">-server -verbose:gc -Xmx1024M -Xms1024M

                      <oacore_jvm_stop_options oa_var="s_oacore_jvm_stop_options">-server -verbose:gc -Xmx1024M -Xms1024M

                      2. Run autoconfig.

                      3. Restart server and retest.

                       

                      Please share your comment/advise.

                       

                      Kind regards,

                      • 8. Re: EBS 12.1.3 - Weird Performance Behavior
                        Beauty_and_dBest

                        Hi Imu and ALL,

                         

                        SQL> select resource_name, current_utilization, max_utilization
                        from v$resource_limit where resource_name in ('processes','sessions');  2

                        RESOURCE_NAME                  CURRENT_UTILIZATION MAX_UTILIZATION
                        ------------------------------ ------------------- ---------------
                        processes                                      155             183
                        sessions                                       158             200

                         

                         

                        Kind regards,