9 Replies Latest reply: Oct 6, 2012 3:54 PM by user12878327 RSS

    Rman incremental long time

    Fayn_dba
      My production database rman backups are taking so long ?

      RMAN>
      RMAN> RUN {
      2> BACKUP INCREMENTAL LEVEL 0 DATABASE;
      3> crosscheck backup;
      4> crosscheck backupset;
      5> delete noprompt obsolete;
      6> }
      7> EXIT;
      Starting backup at 04-OCT-12
      using target database control file instead of recovery catalog
      allocated channel: ORA_DISK_1
      channel ORA_DISK_1: SID=508 device type=DISK
      channel ORA_DISK_1: starting incremental level 0 datafile backup set
      channel ORA_DISK_1: specifying datafile(s) in backup set
      input datafile file number=00021 name=/u02/app/oracle/oradata/orcl/users02.dbf
      input datafile file number=00022 name=/u02/app/oracle/oradata/orcl/users03.dbf
      input datafile file number=00030 name=/u02/app/oracle/oradata/orcl/users05.dbf
      input datafile file number=00033 name=/u02/app/oracle/oradata/orcl/users07.dbf
      input datafile file number=00034 name=/u02/app/oracle/oradata/orcl/users08.dbf
      input datafile file number=00004 name=/u02/app/oracle/oradata/orcl/users01.dbf
      input datafile file number=00029 name=/u02/app/oracle/oradata/orcl/users04.dbf
      input datafile file number=00032 name=/u02/app/oracle/oradata/orcl/users06.dbf
      input datafile file number=00020 name=/u02/app/oracle/oradata/orcl/system02.dbf
      input datafile file number=00002 name=/u02/app/oracle/oradata/orcl/sysaux01.dbf
      input datafile file number=00005 name=/u02/app/oracle/oradata/orcl/DEV_DATA01.dbf
      input datafile file number=00003 name=/u02/app/oracle/oradata/orcl/undotbs01.dbf
      input datafile file number=00037 name=/u02/app/oracle/oradata/orcl/users10.dbf
      input datafile file number=00031 name=/u02/app/oracle/oradata/orcl/mmi_prod04.dbf
      input datafile file number=00016 name=/u02/app/oracle/oradata/orcl/DEV_DATA02.dbf
      input datafile file number=00017 name=/u02/app/oracle/oradata/orcl/DEV_DATA03.dbf
      input datafile file number=00018 name=/u02/app/oracle/oradata/orcl/DEV_DATA04.dbf
      input datafile file number=00019 name=/u02/app/oracle/oradata/orcl/DEV_DATA05.dbf
      input datafile file number=00027 name=/u02/app/oracle/oradata/orcl/mmi_prod02.dbf
      input datafile file number=00028 name=/u02/app/oracle/oradata/orcl/mmi_prod03.dbf
      input datafile file number=00026 name=/u02/app/oracle/oradata/orcl/mmi_prod01.dbf
      input datafile file number=00024 name=/u02/app/oracle/oradata/orcl/all_idx01.dbf
      input datafile file number=00025 name=/u02/app/oracle/oradata/orcl/all_idx02.dbf
      input datafile file number=00035 name=/u02/app/oracle/oradata/orcl/users09.dbf
      input datafile file number=00001 name=/u02/app/oracle/oradata/orcl/system01.dbf
      input datafile file number=00006 name=/u02/app/oracle/oradata/orcl/DEV_INDX01.dbf
      input datafile file number=00023 name=/u02/app/oracle/oradata/orcl/DEV_INDX02.dbf
      input datafile file number=00011 name=/u02/app/oracle/oradata/orcl/PATH_TBL01.dbf
      input datafile file number=00036 name=/u02/app/oracle/oradata/orcl/all_idx03.dbf
      input datafile file number=00007 name=/u02/app/oracle/oradata/orcl/DEV_TEMP01.dbf
      input datafile file number=00008 name=/u02/app/oracle/oradata/orcl/KYA_IDX01.dbf
      input datafile file number=00009 name=/u02/app/oracle/oradata/orcl/LOB_IMAGE01.dbf
      input datafile file number=00010 name=/u02/app/oracle/oradata/orcl/PATH_IDX01.dbf
      input datafile file number=00012 name=/u02/app/oracle/oradata/orcl/RIS_DATA01.dbf
      input datafile file number=00013 name=/u02/app/oracle/oradata/orcl/RIS_INDX01.dbf
      input datafile file number=00014 name=/u02/app/oracle/oradata/orcl/TS_FREE01.dbf
      input datafile file number=00015 name=/u02/app/oracle/oradata/orcl/TS_RV11201.dbf
      channel ORA_DISK_1: starting piece 1 at 04-OCT-12
      RMAN-00571: ===========================================================
      RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
      RMAN-00571: ===========================================================
      RMAN-03009: failure of backup command on ORA_DISK_1 channel at 10/04/2012 03:59:09
      ORA-19502: write error on file "/media/disk-1/ora_df795751210_s780_s1", block number 13957633 (block size=8192)
      ORA-27072: File I/O error
      Linux-x86_64 Error: 2: No such file or directory
      Additional information: 4
      Additional information: 13957633
      Additional information: 778240

      Recovery Manager complete.


      about elapsed time: 03:59:09 >> (for Incremental level 0)
      and elapsed time: 01:42:51 >> (for level 1)

      what can I possibly do to reduce time
      I have checked block change tracking is not enabled

      Good suggestion will be highly appreciated


      & how can i use allocation of channels & if i take compressed incremental backups will it take more time than Current time

      Edited by: Fayn_dba on Oct 5, 2012 3:50 AM
        • 1. Re: Rman incremental long time
          kuljeet singh -
          Fayn_dba wrote:
          My production database rman backups are taking so long ?
          but its failing ! "Linux-x86_64 Error: 2: No such file or directory"
          >
          about elapsed time: 03:59:09 >> (for Incremental level 0)
          and elapsed time: 01:42:51 >> (for level 1)

          what can I possibly do to reduce time
          I have checked block change tracking is not enabled
          you can enable blocking change tracking but it has additional overhead aswell.
          >

          >
          & how can i use allocation of channels &
          http://docs.oracle.com/cd/B28359_01/backup.111/b28273/rcmsynta004.htm
          if i take compressed incremental backups will it take more time than Current time
          yes, and it consume more CPU as it has to compress the backup.
          >
          • 2. Re: Rman incremental long time
            mseberg
            Hello;

            Try changing this :
             RUN {
             BACKUP INCREMENTAL LEVEL 0 DATABASE;
             crosscheck backup;
             crosscheck backupset;
             delete noprompt obsolete;
             }
            to this :
             RUN {
             allocate channel d1 type disk;
             allocate channel d2 type disk;
             allocate channel d3 type disk;
              BACKUP INCREMENTAL LEVEL 0 DATABASE;
              crosscheck backup;
              crosscheck backupset;
              delete noprompt obsolete;
              release channel d1; 
              release channel d2; 
              release channel d3; 
             }
            For RMAN-03009 ORA-27072 ORA-19502

            Check the OS and storage.

            Best Regards

            mseberg
            • 3. Re: Rman incremental long time
              AJ
              If time is an issue I would consider: allocate more RMAN channels to do the work and enabling block change tracking.

              Hopefully you have a test system that is somewhat similar to production and you can test the effects there before doing the implementation in production.

              Another thing would be to check if there are possibilites to set some of the tablespaces read-only and configure backup optimization.
              • 4. Re: Rman incremental long time
                Shivananda Rao
                ORA-19502: write error on file "/media/disk-1/ora_df795751210_s780_s1", block number 13957633 (block size=8192)
                ORA-27072: File I/O error
                Linux-x86_64 Error: 2: No such file or directory
                Additional information: 4
                Additional information: 13957633
                Additional information: 778240
                check if the mount point "/media/disk-1/"exists or not.
                • 5. Re: Rman incremental long time
                  Levi Pereira
                  ORA-19502: write error on file "/media/disk-1/ora_df795751210_s780_s1", block number 13957633 (block size=8192)
                  ORA-27072: File I/O error
                  Linux-x86_64 Error: 2: No such file or directory
                  This mensagem means Oracle already backed some data (block number 13957633) when this failure happens.

                  This problem is related to the OS. This filesystem "/media/disk-1/" is a NFS or LOCAL or Removable Device.

                  I believe you are performing backup to a removable device, look for msg errors on OS. Maybe this device is disconnecting or hanging for a moment, so Oracle hang for a moment and abort.

                  If its a removable check troughput on that device.

                  All of this can be related to a OS/hw configuration.
                  • 6. Trace files Size !
                    Fayn_dba
                    Hi Experts:

                    My production database is producing a huge amount of trace files. Is it safe to delete them.
                    I have notices that 190GB size if the disk is occupied by these files (.trc & .trm)

                    I will not touch alert.log as it contains very useful info

                    Please provide me good solution !

                    Regards
                    • 7. Re: Trace files Size !
                      Shivananda Rao
                      My production database is producing a huge amount of trace files. Is it safe to delete them.
                      I have notices that 190GB size if the disk is occupied by these files (.trc & .trm)
                      Yes, you can delete them and have only a week's trace files. You can schedule a script which cleans up the .trc and .trm files and retains them of just 1 week.


                      http://dba-oracle.com/t_clean_up_oracle_trace_dump_archive_log_files.htm
                      http://oradba4all.blogspot.in/2011/05/shell-script-to-remove-trace-or-log.html

                      You can customize and have your scripts set up
                      • 8. Re: Trace files Size !
                        mseberg
                        Hello;

                        On Linux you can do something like this : ( Change path and dates as needed )
                        #!/bin/bash
                        
                        #-------------------------------------------------------------------------------
                        #-
                        #-------------------------------------------------------------------------------
                        
                        echo "Trace file in bdump older than 60 days from  current date" > /tmp/cleanup_trace.out
                        echo
                        
                        
                        
                        
                        echo "-----------------------------------------------------------------------------" >> /tmp/cleanup_trace.out
                        echo " List of old trace files on PRIMARY        "                                       >> /tmp/cleanup_trace.out
                        echo "-----------------------------------------------------------------------------" >> /tmp/cleanup_trace.out
                        find /u01/app/oracle/admin/PRIMARY/adump -name "*.aud" -ctime +60 -exec \ls -l {} \; >> /tmp/cleanup_trace.out
                        find /u01/app/oracle/admin/PRIMARY/adump -name "*.aud" -ctime +60 -exec \rm {} \;
                        find /u01/app/oracle/diag/rdbms/primary/PRIMARY/trace -name "*.trc" -ctime +60 -exec \ls -l {} \; >> /tmp/cleanup_trace.out
                        find /u01/app/oracle/diag/rdbms/primary/PRIMARY/trace -name "*.trc" -ctime +60 -exec \rm {} \;
                        find /u01/app/oracle/diag/rdbms/primary/PRIMARY/trace -name "*.trm" -ctime +60 -exec \ls -l {} \; >> /tmp/cleanup_trace.out
                        find /u01/app/oracle/diag/rdbms/primary/PRIMARY/trace -name "*.trm" -ctime +60 -exec \rm {} \;
                        
                        
                        
                        
                        mailx -s "Cleanup of TRACE on all databases " youremail@ < /tmp/cleanup_trace.out
                        Best Regards

                        mseberg
                        • 9. Re: Rman incremental long time
                          user12878327
                          channel ORA_DISK_1: starting piece 1 at 04-OCT-12
                          RMAN-00571: ===========================================================
                          RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
                          RMAN-00571: ===========================================================
                          RMAN-03009: failure of backup command on ORA_DISK_1 channel at 10/04/2012 03:59:09
                          ORA-19502: write error on file "/media/disk-1/ora_df795751210_s780_s1", block number 13957633 (block size=8192)
                          ORA-27072: File I/O error
                          Linux-x86_64 Error: 2: No such file or directory
                          Additional information: 4
                          Additional information: 13957633
                          Additional information: 778240


                          Make some free space in the backup location by deleting obsolete backups or by deleting unwanted tracefiles and rerun the backup again.

                          Allocate more channels in backup script to improve the backup speed.

                          Thanks
                          Amruta.