In the last few days, I have noticed the load on the production server as follows:
load averages: 4.66, 4.86, 4.92 14:34:16
333 processes: 327 sleeping, 6 on cpu
CPU states: % idle, % user, % kernel, % iowait, % swap
Memory: 32G real, 7591M free, 23G swap in use, 14G swap free
PID USERNAME LWP PRI NICE SIZE RES STATE TIME CPU COMMAND
20045 appsdapo 1 0 0 4073M 4029M cpu/2 165.0H 12.49% f60webmx
1854 appsdapo 1 10 0 4082M 4060M cpu/0 240.0H 12.48% FNDLIBR
2504 appsdapo 1 10 0 1872M 1869M cpu/19 1980.2 12.46% FNDLIBR
2496 appsdapo 1 0 0 1876M 1873M cpu/1 1957.5 11.68% FNDLIBR
18641 appsdapo 1 10 0 1512M 1509M cpu/16 1080.3 8.64% FNDLIBR
I have come across Metalink note ID 114380.1 which says identify the C.M but how do I relate the FNDLIBR processes above with the specific C.Managers.
Please post the details of the application release, database version and OS.
Any changes have been done recently (patches, upgrade, installation, ..etc)?
Concurrent Processing - How to Find Database Session & Process Associated with a Concurrent Program Which is Currently Running. [ID 735119.1]
How to Retrieve SID Information For a Running Request [ID 280391.1]
Concurrent Processing (CP) / APPS Reporting Scripts [ID 213021.1]
Concurrent Processing - CCM.sql Diagnostic Script to Diagnose Common Concurrent Manager Issues [ID 171855.1]
Scheduled Requests Running for END_DATED Users 100% CPU and Giving ORA-01002 [ID 1080047.1]
Concurrent Processing - Troubleshooting Concurrent Manager Issues (Unix specific) [ID 104452.1]
Also, search the forum for "Concurrent Manager" and "Tuning".
Nothing has changed that I know as the dba.
I have looked through some of the notes and none is saying how I can relate the FNDLIBR process with a c.manager etc
Thanks Hussein, as always
If the users who scheduled the concurrent requests are end-dated, then the concurrent managers will consume high cpu and memory. You can see the error and the concurrent request id in Concurrent Manager logs
Thanks for the Post. This post helped me to solve my production issue.
My production application started hanging suddenly and no clue why it is behaving like this.
The symptom was FNDLIBR processes were on top cpu consumer on application server.
The database server also experienced heavy system i/o and cpu usage.
The database base log switches every minutes (500 MB) and there are no concurrent requests running.
No concurrent request errors. No OS errors (Cleaned the GNOME and restarted). No database errors.
But in sql area i saw lot of fnd service requests keep coming.
At the end the application server hangs because of high cpu and it caused the production down.
The problem was one of Our System Administrator disabled one responsibility with scheduled request.
One interesting thing we noticed, even you put the request in question Hold, it will start running while you restart the concurrent request.
You have to cancel the concurrent request and restart the concurrent manager can solve the issue.