This discussion is archived
1 2 Previous Next 24 Replies Latest reply: Jan 27, 2013 5:46 PM by Hemant K Chitale RSS

rman process and CPU usage

441858 Newbie
Currently Being Moderated
Oracle 11gR2 RHEL5

I have rman backups scheduled to run through cron. I noticed that after the backup has completed successfully (and exited from RMAN) the session remains connected. I see several rman processes and all consuming 95-98% CPU. Any idea why this would be happening? And we do have a catalog database as well.

In addition, I see a lot of audit files (.aud) are generated.

JrOraDBA
  • 1. Re: rman process and CPU usage
    441858 Newbie
    Currently Being Moderated
    It seems like it leaves a channel open even after completing the backup.

    Program: rman@db1 (TNS V1 V3)
    Client Info: Channel ORA_DISK_2
    backup archivelog
  • 2. Re: rman process and CPU usage
    mseberg Guru
    Currently Being Moderated
    Can you post your backup script and OS?

    Best Regards

    mseberg
  • 3. Re: rman process and CPU usage
    441858 Newbie
    Currently Being Moderated
    The OS is RHEL5 64 bit (2.6.32-300.11.1.el5)

    Here is the script. Note that I added the 'allocate channel...' and 'release channel' clauses afterwards to test explicitly closing the channels after the backup completed. Even though Oracle says
    it does it automatically....I just wanted to do a test. It seems to have worked but not sure. I will see how tonight's backup goes.
    run {
       sql 'alter system switch logfile';
       configure device type disk parallelism 4;
       configure default device type to disk;
       configure compression algorithm 'LOW' as of release 'default' optimize for load true;
       allocate channel t1 device type disk;
       allocate channel t2 device type disk;
       allocate channel t3 device type disk;
       allocate channel t4 device type disk;
       backup as compressed backupset database format '/opt/oracle/backups/datafile_%d_%U_%s_%T.bkp';
       backup as compressed backupset current controlfile format '/opt/oracle/backups/ctrl_%d_%U_%s_%T.bkp';
       sql 'alter system archive log current';
       backup as compressed backupset archivelog all format '/opt/oracle/backups/arch_%d_%U_%s_%T.bkp' delete input;
       release channel t1;
       release channel t2;
       release channel t3;
       release channel t4;
       }
  • 4. Re: rman process and CPU usage
    mseberg Guru
    Currently Being Moderated
    Running same OS and I don't see anything in your script that would cause it.

    Is it possible you don't have an exit from RMAN in the script ?
    run {
       sql 'alter system switch logfile';
       configure device type disk parallelism 4;
       configure default device type to disk;
       configure compression algorithm 'LOW' as of release 'default' optimize for load true;
       allocate channel t1 device type disk;
       allocate channel t2 device type disk;
       allocate channel t3 device type disk;
       allocate channel t4 device type disk;
       backup as compressed backupset database format '/opt/oracle/backups/datafile_%d_%U_%s_%T.bkp';
       backup as compressed backupset current controlfile format '/opt/oracle/backups/ctrl_%d_%U_%s_%T.bkp';
       sql 'alter system archive log current';
       backup as compressed backupset archivelog all format '/opt/oracle/backups/arch_%d_%U_%s_%T.bkp' delete input;
       release channel t1;
       release channel t2;
       release channel t3;
       release channel t4;
       }
       
     exit
    EOF
    
    echo `date`
    #
    echo
    echo "RMAN Full on ${DBNAME} Complete"
      
    Best Regards

    mseberg
  • 5. Re: rman process and CPU usage
    441858 Newbie
    Currently Being Moderated
    No, I do have an 'exit' command from the RMAN script. Just like you posted.

    After the 'exit' command I have some error checking..whether the backup was successful or unsuccessful and that's it.
  • 6. Re: rman process and CPU usage
    mseberg Guru
    Currently Being Moderated
    Can you post the list of RMAN processes hanging around after a backup?

    And what command you use the list them?

    Best Regards

    mseberg
  • 7. Re: rman process and CPU usage
    Hemant K Chitale Oracle ACE
    Currently Being Moderated
    Do you happen to have another RMAN job (e.g. scheduled through CRON or some other scheduler) that is specifically written to backup archivelogs ?

    Check the logon_time for the RMAN session that you find open. See if it matches the time the backp script started.


    Hemant K Chitale
  • 8. Re: rman process and CPU usage
    441858 Newbie
    Currently Being Moderated
    Well, I checked today and the backups ran fine and there were no left over processes. I guess in this case explicitly closing the channels did the trick, perhaps?

    To check I usually just use the 'ps -ef | grep rman' command
  • 9. Re: rman process and CPU usage
    mseberg Guru
    Currently Being Moderated
    OK.

    That should work. ( the grep command ).

    You have seemed to have ruled everything else out. No exit, another RMAN process etc.

    This is the only note I can think of that might help :

    RMAN: Quick Debugging Guide [ID 1198753.1]

    Best Regards

    mseberg
  • 10. Re: rman process and CPU usage
    441858 Newbie
    Currently Being Moderated
    You have seemed to have ruled everything else out. No exit, another RMAN process etc.
    I don't understand. Could you please clarify?

    Thanks.
  • 11. Re: rman process and CPU usage
    mseberg Guru
    Currently Being Moderated
    Hemant asked you if you had another RMAN job and you ruled that out.

    I asked you if you had an "EXIT" in your script and you ruled that out.

    Then you add the releases so I don't know of anything else.

    Other than this :

    RMAN: Quick Debugging Guide [ID 1198753.1]

    Best Regards

    mseberg
  • 12. Re: rman process and CPU usage
    441858 Newbie
    Currently Being Moderated
    Okay, thank you.

    I don't know what else it could be. I only have one backup job that runs twice (once at night and once in the early morning). Yesterday and today it went through that cycle and I see nothing that is left over.

    I will take a look at that note though.

    Thank you all for your help.

    JrOraDBA
  • 13. Re: rman process and CPU usage
    441858 Newbie
    Currently Being Moderated
    Ok, something strange happened again.

    The second backup that we run in the morning, is just sitting at the part where it is deleting archivelogs...it is not moving forward. THe whole backup script only takes about 20-30 mins to run. It's been it this state for 4 hrs.

    Here is where it is hanging:
    ...
    ...
    ...
    channel t6: backup set complete, elapsed time: 00:02:39
    channel t6: deleting archived log(s)
    archived log file name=+FLASH/oradb/archivelog/2013_01_24/thread_1_seq_1855.272.805515367 RECID=1854 STAMP=805515369
    archived log file name=+FLASH/oradb/archivelog/2013_01_24/thread_1_seq_1856.273.805515535 RECID=1855 STAMP=805515537
    archived log file name=+FLASH/oradb/archivelog/2013_01_24/thread_1_seq_1857.270.805515905 RECID=1856 STAMP=805515907
    archived log file name=+FLASH/oradb/archivelog/2013_01_24/thread_1_seq_1858.285.805518005 RECID=1857 STAMP=805518005
    archived log file name=+FLASH/oradb/archivelog/2013_01_24/thread_1_seq_1859.260.805518955 RECID=1858 STAMP=805518955
    archived log file name=+FLASH/oradb/archivelog/2013_01_24/thread_1_seq_1860.276.805518957 RECID=1859 STAMP=805518958
    And again I see a process consuming high CPU. It is the first entry in 'top'
    PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
    
    6629 oracle    20   0 12.2g 103m  20m R 98.9  0.1 252:18.19 oracle
    Doing some more digging, the wait event for this process is 'Disk file operations I/O', P1 is Fileoperation +DATA/oradb/datafile/tbsidx1.dbf. The 'Current Client info' is 'rman channel=t4'

    There are also four open cursors with the following SQLs:

    select nvl(max(cpmid),0) from x$kcccp
    SELECT TO_CHAR(SYSDATE , 'YYYY' , 'NLS_CALENDAR=Gregorian'
    SELECT SYSDATE FROM SYS.DUAL
    SELECT SYSDATE - :b1 FROM X$DUAL
  • 14. Re: rman process and CPU usage
    mseberg Guru
    Currently Being Moderated
    Hello;

    Great information on the issue.

    This jumps out :
    archived log file name=+FLASH/oradb/archivelog/2013_01_24/thread_1_seq_1855.272.805515367 RECID=1854 STAMP=805515369
    Since you are not keeping Archive around very long the database might be holding the lock on the archivelog file you are trying to delete.

    I would try a deleting the archive in a way that newer Archive is not in the job. That would leave more Archive around, but then you'll know what the exact issue is.

    Best Regards

    mseberg
1 2 Previous Next

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points