Can you define the performance issue? Is it only Concurrent Processing or Web Pages, Forms etc.? or Workflow?
You need to collect below to start with:
- DB Alert Logfile
- AWR report for 1-2 hours during the time of the issue
- OACORE Logs
- Forms Server Logs
You can also ask them to run the below advisors to get a quick overview of the configuration:
E-Business Suite Support Analyzer Bundle Menu Tool (Doc ID 1939637.1)
- Concurrent Processing Analyzer
- Database Performance and Statistics Analyzer
Also WebLogic Configuration using :
- E-Business Suite: WLS (WebLogic Server) Utility (Doc ID 2230225.1)
What is the "Performance Issue"? Is it a specific task/screen? A specific concurrent request? Logging in/general usage?
Thanks Vishnu & John,
Users are just saying the system in general is slow, in everything they do
How do I check if there is bottleneck on the disk? or if there is problem with the disk controllers? or network?
Welcome to Oracle EBS.
on a serious note though - they need to produce a test case. Saying "everything is slow" isn't very defined - slow compared to what? They need to give a specific example of something that is slow. I.e.
"Updating this invoice takes 2 minutes, whereas it took 30 seconds last week. Here are the parameters I'm using and the steps I'm taking on the screen".
Based on the users data gatherings, the following are the main issues.
1. Workflow does not run smoothly. intermittently email is stucked or not sent or to the approver. There are more than 1000+ PO being emailed to the approver daily, only 50% is processed or gets emailed.
2. In the email exchange server or SMTP, all the non-oracle emails are processed successfully (this 80% of the email volumes),
but the Oracle WF PO approval emails (comprised 20% of the email volumes), only 10% are successful. the 10% are stucked or have errors.
3. The Weblogic java memory is exhausting the server RAM, because it is not being freed-up after use.
4. Users take very long time to login the apps, sometimes one user can login fast, while the 2nd user takes very long time. then alternately next time the 2nd user can login fast, and the other users takes too long.
Then the user that is currently login will be disconnected or timeout suddenly.
5. User login cache is weirdo, maybe some security setup failed? The wierd behavior is, everytime I logged in to my pc in Oracle EBS, the login in cache should be my username, but I see different login name being
chache to my login page in my own PC.
6. We have lots of personalization forms.
7. One INV form update action takes very long time process > UPDATE Locator Item.
Please help what logs to extract to get the issue above?
Looking at list, it seems ebs environment was far from being administered and monitored by Apps dba. You will have to pick one performance issue at a time and deep-dive further as these issues cannot be solved by one-liner solutions.
It will be better if you can get access of OS users which are running apps services and database.
I'd guess they aren't likely to fall down to the same cause... For example, point 7 could be completely independent of the others. For that I'd suggest applying a 10046 trace to see where the time is being taken. The personalizations could affect the performance, but likely only if you're calling out to database procedures. Enabling/disabling fields etc generally adds minimal overhead.
"User login cache is weirdo, maybe some security setup failed? The wierd behavior is, everytime I logged in to my pc in Oracle EBS, the login in cache should be my username, but I see different login name being
chache to my login page in my own PC"
Where are you seeing this?
For now the priority is the Workflow approval, why the email notification does not send properly?
Where do I extract logs or hint or info why email stuck?
Can you get
$ sar 2 20 (from DB & Apps tier )
In that case, workflow analyzer should be starting point of troubleshooting.
EBS Workflow (WF) Analyzer (Doc ID 1369938.1)
Please check for below -
1. Schedule of Purge Obsolete Workflow Runtime Data
2. Review and handle the errored open notifications and open notifications
3. Any email address assigned to sysadmin to receive errored notifications?
Thanks Kanda, Maaz, and John,
I will try to all your suggestions.
But do below tasks help?
1. gather ashrpt every hour
2. gather addmrpt every hour
3. gather awrrpt every hour
A friend also suggested to insert into a backup_table every our as select * from $active_session_history where sample_id> <previous_hour_max_sample_id>
$ sar 2 20 (from DB & Apps tier )
Do I run it on peak period ? or for wholeday every 30 minutes and saved it?
you can analyze history for the whole day.. you get understand the trend.
To get history data, you may try this.
The below command displays 19th CPU status....
$ cd /var/log/sa
sa19 sa20 sa21 sa22 sa23 sa24 sa25 sa26 sa27 sar19 sar20 sar21 sar22 sar23 sar24 sar25 sar26
$ sar -f /var/log/sa/sa19