1 2 Previous Next 25 Replies Latest reply on Nov 3, 2019 11:27 AM by Beauty_and_dBest

    EBS Performance Issue

    Beauty_and_dBest

      ESB R12.2.6

      12c

      OL6

       

       

      Hi ALL,

       

      Our PROD EBS has performance issue.

      But  we are off-limits to connect from there directly, hence we need only to ask the PROD operator/admin to extract some performance logs for analysis of the issue.

      Can you help what scripts to run to gather performance logs for analysis?

       

      Can I  export/import AWR data to a dmp file  and import it to our TEST instance?

       

      Please help...

       

       

      Kind regards,

      jc

        • 1. Re: EBS Performance Issue
          VishnuVinnakota

          Hi,

           

            Can you define the performance issue? Is it only Concurrent Processing or Web Pages, Forms etc.? or Workflow?

           

            You need to collect below to start with:

           

          • DB Alert Logfile
          • AWR report for 1-2 hours during the time of the issue
          • OACORE Logs
          • Forms Server Logs

           

            You can also ask them to run the below advisors to get a quick overview of the configuration:

           

          E-Business Suite Support Analyzer Bundle Menu Tool (Doc ID 1939637.1)

          • Concurrent Processing Analyzer
          • Database Performance and Statistics Analyzer

           

          Also WebLogic Configuration using :

           

          Thanks,

          Vishnu

          • 2. Re: EBS Performance Issue
            John_K

            What is the "Performance Issue"? Is it a specific task/screen? A specific concurrent request? Logging in/general usage?

            • 3. Re: EBS Performance Issue
              Beauty_and_dBest

              Thanks Vishnu & John,

               

              Users are just saying the system in general is slow, in everything they do

               

              How do I check if there is bottleneck on the disk?  or if there is problem with the disk controllers?  or network?

               

              Kind regards,

              • 4. Re: EBS Performance Issue
                John_K

                Welcome to Oracle EBS.

                 

                on a serious note though - they need to produce a test case. Saying "everything is slow" isn't very defined - slow compared to what? They need to give a specific example of something that is slow. I.e.

                 

                "Updating this invoice takes 2 minutes, whereas it took 30 seconds last week. Here are the parameters I'm using and the steps I'm taking on the screen".

                • 5. Re: EBS Performance Issue
                  Beauty_and_dBest

                  Hi ALL,

                   

                  Based on the users data gatherings, the following are the main issues.

                   

                  1. Workflow does not run smoothly.  intermittently  email is stucked or not sent or  to the approver.  There are more than 1000+ PO being emailed to the approver daily, only 50% is processed or gets emailed.

                  2.  In the email exchange server or SMTP,   all the non-oracle emails are processed successfully (this 80% of the email volumes),

                       but the Oracle WF PO approval emails (comprised 20% of the email volumes), only 10% are successful. the 10% are stucked or have errors.

                  3.  The Weblogic java memory is exhausting the server RAM, because it is not being freed-up after use.

                  4.  Users take very long time to login the apps, sometimes one user can login fast, while the 2nd user takes very long time. then alternately next time the 2nd user can login fast, and the other users takes too long.

                       Then the user that is currently login will be disconnected or timeout suddenly.

                  5.  User login cache  is weirdo, maybe some security setup failed? The wierd behavior is, everytime I logged in to my pc in Oracle EBS, the login in cache should be my username, but I see different login name being      

                       chache to my login page in my own PC.

                  6.  We have lots of personalization forms.

                  7.  One INV form update action takes very long time process >  UPDATE Locator Item.

                   

                   

                  Please help what logs to extract to get the issue above?

                   

                   

                  Kind regards,

                  • 6. Re: EBS Performance Issue
                    Maaz Khan

                    Hello Jc,

                     

                    Looking at list, it seems ebs environment was far from being administered and monitored by Apps dba. You will have to pick one performance issue at a time and deep-dive further as these issues cannot be solved by one-liner solutions.

                    It will be better if you can get access of OS users which are running apps services and database.

                     

                    Regards,

                    Maaz

                    • 7. Re: EBS Performance Issue
                      John_K

                      I'd guess they aren't likely to fall down to the same cause... For example, point 7 could be completely independent of the others. For that I'd suggest applying a 10046 trace to see where the time is being taken. The personalizations could affect the performance, but likely only if you're calling out to database procedures. Enabling/disabling fields etc generally adds minimal overhead.

                       

                      "User login cache  is weirdo, maybe some security setup failed? The wierd behavior is, everytime I logged in to my pc in Oracle EBS, the login in cache should be my username, but I see different login name being     

                           chache to my login page in my own PC"

                       

                      Where are you seeing this?

                      • 8. Re: EBS Performance Issue
                        Beauty_and_dBest

                        Hi John.

                         

                        The users see this in their login screen  on thier, for example,  the user id is sysadmin, they usually see the cached login as sysadmin,

                        but other username was being displayed everytime they logout and login.

                         

                        Kind regards,

                        • 9. Re: EBS Performance Issue
                          Beauty_and_dBest

                          Hi Maaz,

                           

                          For now the priority is the  Workflow approval, why the email notification does not send properly?

                          Where do I extract logs or hint or info why email stuck?

                           

                          Please help...

                           

                          Kind regards,

                          • 10. Re: EBS Performance Issue
                            Kanda-Oracle

                            Hi

                             

                            Can you get

                            $ sar 2 20 (from DB & Apps tier )

                             

                            Thanks!

                            • 11. Re: EBS Performance Issue
                              Maaz Khan

                              In that case, workflow analyzer should be starting point of troubleshooting.
                              EBS Workflow (WF) Analyzer (Doc ID 1369938.1)

                              Please check for below -
                              1. Schedule of Purge Obsolete Workflow Runtime Data
                              2. Review and handle the errored open notifications and open notifications
                              3. Any email address assigned to sysadmin to receive errored notifications?

                               

                              Regards,

                              Maaz

                               

                               

                               

                              • 12. Re: EBS Performance Issue
                                Beauty_and_dBest

                                Thanks Kanda, Maaz, and John,

                                 

                                I will try to all your suggestions.

                                 

                                But do below tasks help?

                                 

                                1. gather ashrpt every hour

                                2. gather addmrpt  every hour

                                3. gather awrrpt  every hour

                                 

                                A friend also suggested to insert into a backup_table every our  as select * from $active_session_history  where sample_id>  <previous_hour_max_sample_id>

                                 

                                 

                                Please help...

                                 

                                 

                                Kind regards,

                                • 13. Re: EBS Performance Issue
                                  Beauty_and_dBest

                                  Hi Kanda,

                                   

                                  $ sar 2 20 (from DB & Apps tier )

                                   

                                  Do I run it on peak period ? or for wholeday every 30 minutes and saved it?

                                   

                                   

                                  Kind regards,

                                  • 14. Re: EBS Performance Issue
                                    Kanda-Oracle

                                    Hi

                                     

                                    you can analyze history for the whole day.. you get understand the trend.

                                     

                                    To get history data, you may try this.

                                     

                                    The below command displays  19th CPU status....

                                     

                                    $ cd /var/log/sa

                                    $ ls

                                    sa19  sa20  sa21  sa22  sa23  sa24  sa25  sa26  sa27  sar19  sar20  sar21  sar22  sar23  sar24  sar25  sar26

                                     

                                    $ sar -f /var/log/sa/sa19

                                     

                                    Thanks!

                                     

                                    N Kandasamy

                                    1 2 Previous Next