This discussion is archived
6 Replies Latest reply: Sep 16, 2011 7:49 AM by pouler RSS

Concurrent manager using lots of cpu & memory

user15639 Newbie
Currently Being Moderated
In the last few days, I have noticed the load on the production server as follows:

load averages: 4.66, 4.86, 4.92 14:34:16
333 processes: 327 sleeping, 6 on cpu
CPU states: % idle, % user, % kernel, % iowait, % swap
Memory: 32G real, 7591M free, 23G swap in use, 14G swap free

PID USERNAME LWP PRI NICE SIZE RES STATE TIME CPU COMMAND
20045 appsdapo 1 0 0 4073M 4029M cpu/2 165.0H 12.49% f60webmx
1854 appsdapo 1 10 0 4082M 4060M cpu/0 240.0H 12.48% FNDLIBR
2504 appsdapo 1 10 0 1872M 1869M cpu/19 1980.2 12.46% FNDLIBR
2496 appsdapo 1 0 0 1876M 1873M cpu/1 1957.5 11.68% FNDLIBR
18641 appsdapo 1 10 0 1512M 1509M cpu/16 1080.3 8.64% FNDLIBR

I have come across Metalink note ID 114380.1 which says identify the C.M but how do I relate the FNDLIBR processes above with the specific C.Managers.

Thanks
  • 1. Re: Concurrent manager using lots of cpu & memory
    Hussein Sawwan-Oracle Employee ACE
    Currently Being Moderated
    Please post the details of the application release, database version and OS.

    Any changes have been done recently (patches, upgrade, installation, ..etc)?

    Concurrent Processing - How to Find Database Session & Process Associated with a Concurrent Program Which is Currently Running. [ID 735119.1]
    How to Retrieve SID Information For a Running Request [ID 280391.1]
    Concurrent Processing (CP) / APPS Reporting Scripts [ID 213021.1]
    Concurrent Processing - CCM.sql Diagnostic Script to Diagnose Common Concurrent Manager Issues [ID 171855.1]
    Scheduled Requests Running for END_DATED Users 100% CPU and Giving ORA-01002 [ID 1080047.1]
    Concurrent Processing - Troubleshooting Concurrent Manager Issues (Unix specific) [ID 104452.1]

    Also, search the forum for "Concurrent Manager" and "Tuning".

    Thanks,
    Hussein
  • 2. Re: Concurrent manager using lots of cpu & memory
    user15639 Newbie
    Currently Being Moderated
    Details:

    11.5.10.2 apps
    10.2.0.4 database.
    Solaris 10

    Nothing has changed that I know as the dba.

    I have looked through some of the notes and none is saying how I can relate the FNDLIBR process with a c.manager etc

    Thanks Hussein, as always
  • 3. Re: Concurrent manager using lots of cpu & memory
    Hussein Sawwan-Oracle Employee ACE
    Currently Being Moderated
    From the OS, issue "ps -ef | grep FNDLIBR" -- This will return many processes, so pick the PID of the process with a high CPU utilization and run this query (let's say PID is "495732"):
    SQL> SELECT d.sid, d.serial# ,d.osuser,d.process
    FROM apps.fnd_concurrent_processes b,
    v$process c,
    v$session d
    WHERE c.pid = b.oracle_process_id
    AND b.session_id=d.audsid
    AND d.process = '434372';
    Thanks,
    Hussein
  • 4. Re: Concurrent manager using lots of cpu & memory
    864708 Newbie
    Currently Being Moderated
    Hi,

    Is this happening in your test instance also?

    Did you find if there is any major change in your query execution plan of the concurrent request ?


    Regards
    Sridhar M
  • 5. Re: Concurrent manager using lots of cpu & memory
    877189 Newbie
    Currently Being Moderated
    If the users who scheduled the concurrent requests are end-dated, then the concurrent managers will consume high cpu and memory. You can see the error and the concurrent request id in Concurrent Manager logs
  • 6. Re: Concurrent manager using lots of cpu & memory
    pouler Newbie
    Currently Being Moderated
    Hello,

    Thanks for the Post. This post helped me to solve my production issue.
    My production application started hanging suddenly and no clue why it is behaving like this.
    The symptom was FNDLIBR processes were on top cpu consumer on application server.
    The database server also experienced heavy system i/o and cpu usage.
    The database base log switches every minutes (500 MB) and there are no concurrent requests running.
    No concurrent request errors. No OS errors (Cleaned the GNOME and restarted). No database errors.

    But in sql area i saw lot of fnd service requests keep coming.

    At the end the application server hangs because of high cpu and it caused the production down.

    The problem was one of Our System Administrator disabled one responsibility with scheduled request.

    One interesting thing we noticed, even you put the request in question Hold, it will start running while you restart the concurrent request.

    You have to cancel the concurrent request and restart the concurrent manager can solve the issue.


    Thx
    Pouler

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points