9 Replies Latest reply on Mar 24, 2012 4:50 PM by Marcelo Marques - ESRI

    dgutil.pl line 99 ERROR when creating 11.2.0.3 data guard via GRID CONTROL

    Marcelo Marques - ESRI
      Oracle Grid Control 11g Release 1 (11.1.0.1.0)
      (64bit)
      Oracle Enterprise Linux Server release 5.4 x86_64
      Oracle Database 11.2.0.3 (64bit)

      1. Grid Control>Targets>Databases
      2.Click on pc01prmy (Primary)
      3.Availability > Data Guard > Setup and Manage
      4.Add Standby Database
      5.Create a new physical standby database.

      Fails with message:
      38706 at /u01/app/oracle/agent11g/sysman/admin/scripts/db/dg/dgutil.pl line 99.

      I can create a standby with a rman script without problems, but via the Grid Control page fails.

      Have you seen this before?
        • 1. Re: dgutil.pl line 99 ERROR when creating 11.2.0.3 data guard via GRID CONTROL
          mseberg
          Hello;

          Not that exact error but a very similar error :

          Creating Standby Database With Enterprise Manager Failing [ID 1400482.1]

          or

          Error Type Collection Failure On Metric Data Guard Performance [ID 1082151.1]

          or

          Metric Collection Error With Ha_dg_target_summary [ID 1302678.1]

          In any case its a metric collection error.

          I wonder if this is possible too :

          Flashback 'ON' fails with ORA-38706 and ORA-38714 [ID 342764.1]

          Don't see the Grid Control method much, the RMAN method seems to be the way to go.

          Best Regards

          mseberg

          Edited by: mseberg on Mar 17, 2012 4:05 PM

          Edited by: mseberg on Mar 17, 2012 4:36 PM
          • 2. Re: dgutil.pl line 99 ERROR when creating 11.2.0.3 data guard via GRID CONTROL
            Marcelo Marques - ESRI
            Job Status: Failed
            ...
            Step: Destination Preparation: Succeded
            Step: Duplicate Database: Succeded
            Step: Re-create Control File: Succeded
            Step: Recover Database: Succeded
            Step: Create Standby Database: Failed
            --------------------------------------------
            open the failed step to see the log
            --------------------------------------------
            SQL*Plus: Release 11.2.0.3.0 Production on Sat Mar 17 14:29:04 2012
            Copyright (c) 1982, 2011, Oracle. All rights reserved.
            SQL> SQL> Connected.
            SQL> SQL> SQL> ORA-01109: database not open
            Database dismounted.
            ORACLE instance shut down.
            SQL> SQL> Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
            With the Partitioning, Oracle Label Security, OLAP, Data Mining,
            Oracle Database Vault and Real Application Testing options
            SQL*Plus: Release 11.2.0.3.0 Production on Sat Mar 17 14:29:12 2012
            Copyright (c) 1982, 2011, Oracle. All rights reserved.
            SQL> SQL> Connected to an idle instance.
            SQL> SQL> ORACLE instance started.
            Total System Global Area 1252663296 bytes
            Fixed Size          2227944 bytes
            Variable Size          738197784 bytes
            Database Buffers     503316480 bytes
            Redo Buffers          8921088 bytes
            SQL> Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
            With the Partitioning, Oracle Label Security, OLAP, Data Mining,
            Oracle Database Vault and Real Application Testing options
            SQL*Plus: Release 11.2.0.3.0 Production on Sat Mar 17 14:29:14 2012
            Copyright (c) 1982, 2011, Oracle. All rights reserved.
            SQL> SQL> Connected.
            SQL> SQL> SQL> ORA-01507: database not mounted
            ORACLE instance shut down.
            SQL> SQL> Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
            With the Partitioning, Oracle Label Security, OLAP, Data Mining,
            Oracle Database Vault and Real Application Testing options
            SQL*Plus: Release 11.2.0.3.0 Production on Sat Mar 17 14:29:20 2012
            Copyright (c) 1982, 2011, Oracle. All rights reserved.
            SQL> SQL> Connected to an idle instance.
            SQL> SQL> ORACLE instance started.
            Total System Global Area 1252663296 bytes
            Fixed Size          2227944 bytes
            Variable Size          738197784 bytes
            Database Buffers     503316480 bytes
            Redo Buffers          8921088 bytes
            Database mounted.
            SQL> Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
            With the Partitioning, Oracle Label Security, OLAP, Data Mining,
            Oracle Database Vault and Real Application Testing options
            SQL*Plus: Release 11.2.0.3.0 Production on Sat Mar 17 14:29:27 2012
            Copyright (c) 1982, 2011, Oracle. All rights reserved.
            SQL> SQL> Connected.
            SQL> SQL> SQL> ORA-01109: database not open
            Database dismounted.
            ORACLE instance shut down.
            SQL> SQL> Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
            With the Partitioning, Oracle Label Security, OLAP, Data Mining,
            Oracle Database Vault and Real Application Testing options
            SQL*Plus: Release 11.2.0.3.0 Production on Sat Mar 17 14:29:42 2012
            Copyright (c) 1982, 2011, Oracle. All rights reserved.
            SQL> SQL> Connected to an idle instance.
            SQL> SQL> ORACLE instance started.
            Total System Global Area 1252663296 bytes
            Fixed Size          2227944 bytes
            Variable Size          738197784 bytes
            Database Buffers     503316480 bytes
            Redo Buffers          8921088 bytes
            Database mounted.
            SQL> Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
            With the Partitioning, Oracle Label Security, OLAP, Data Mining,
            Oracle Database Vault and Real Application Testing options
            dgcreate.DGgetRecoverySCN: SELECT MAX(NEXT_CHANGE#) FROM V$ARCHIVED_LOG
            dgcreate.DGcreateSpfile: CREATE SPFILE FROM PFILE='/u01/app/oracle/product/11.2.0/dbhome_1/dbs/initpc01sby2f.ora'
            SQL*Plus: Release 11.2.0.3.0 Production on Sat Mar 17 14:29:49 2012
            Copyright (c) 1982, 2011, Oracle. All rights reserved.
            SQL> SQL> Connected.
            SQL> SQL> SQL> ORA-01109: database not open
            Database dismounted.
            ORACLE instance shut down.
            SQL> SQL> Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
            With the Partitioning, Oracle Label Security, OLAP, Data Mining,
            Oracle Database Vault and Real Application Testing options
            SQL*Plus: Release 11.2.0.3.0 Production on Sat Mar 17 14:30:04 2012
            Copyright (c) 1982, 2011, Oracle. All rights reserved.
            SQL> SQL> Connected to an idle instance.
            SQL> SQL> ORACLE instance started.
            Total System Global Area 1252663296 bytes
            Fixed Size          2227944 bytes
            Variable Size          738197784 bytes
            Database Buffers     503316480 bytes
            Redo Buffers          8921088 bytes
            Database mounted.
            SQL> Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
            With the Partitioning, Oracle Label Security, OLAP, Data Mining,
            Oracle Database Vault and Real Application Testing options
            Use of uninitialized value $db_password in substitution (s///) at - line 6378.
            Recovery Manager: Release 11.2.0.3.0 - Production on Sat Mar 17 14:30:11 2012
            Copyright (c) 1982, 2011, Oracle and/or its affiliates. All rights reserved.
            RMAN>
            connected to target database: PC01PRMY (DBID=2646667080, not open)
            using target database control file instead of recovery catalog
            RMAN>
            echo set on
            RMAN> CONFIGURE ARCHIVELOG DELETION POLICY TO APPLIED ON STANDBY;
            new RMAN configuration parameters:
            CONFIGURE ARCHIVELOG DELETION POLICY TO APPLIED ON STANDBY;
            new RMAN configuration parameters are successfully stored
            exit;
            Recovery Manager complete.
            dgcreate.DGsetFlashbackDatabase: alter database flashback on
            38706 at /u01/app/oracle/agent11g/sysman/admin/scripts/db/dg/dgutil.pl line 99.

            Edited by: Marcelo Marques - ESRI on Mar 17, 2012 2:37 PM
            • 3. Re: dgutil.pl line 99 ERROR when creating 11.2.0.3 data guard via GRID CONTROL
              mseberg
              OK

              I came back and was looking at a note on that
              dgcreate.DGsetFlashbackDatabase: alter database flashback on
              38706 at /u01/app/oracle/agent11g/sysman/admin/scripts/db/dg/dgutil.pl line 99.
              Check out :

              Flashback 'ON' fails with ORA-38706 and ORA-38714 [ID 342764.1]

              I think its harder to track down because this happens on older versions and you are current on the database version.

              Best Regards

              mseberg
              • 4. Re: dgutil.pl line 99 ERROR when creating 11.2.0.3 data guard via GRID CONTROL
                Marcelo Marques - ESRI
                You are right, the "alter database flashback on" failed. The oracle alert log also shows the same.
                [oracle@burbank3 ~]$ export ORACLE_SID=pc01sby2
                [oracle@burbank3 ~]$ sqlplus /nolog
                SQL> connect /as sysdba
                Connected.
                SQL> select * from v$instance;
                SQL> archive log list
                Database log mode Archive Mode
                Automatic archival Enabled
                Archive destination USE_DB_RECOVERY_FILE_DEST
                Oldest online log sequence 0
                Next log sequence to archive 0
                Current log sequence 0
                SQL> select flashback_on from v$database;
                FLASHBACK_ON
                ------------------
                NO
                SQL> alter database flashback on;
                alter database flashback on
                *
                ERROR at line 1:
                ORA-38706: Cannot turn on FLASHBACK DATABASE logging.
                ORA-38788: More standby database recovery is needed
                • 5. Re: dgutil.pl line 99 ERROR when creating 11.2.0.3 data guard via GRID CONTROL
                  Shivananda Rao
                  Hi,

                  Refer this on how to enable flashback on the database http://shivanandarao.wordpress.com/2012/03/02/enabling-flashback-database-in-oracle/
                  • 6. Re: dgutil.pl line 99 ERROR when creating 11.2.0.3 data guard via GRID CONTROL
                    Marcelo Marques - ESRI
                    Found the bug, it's fixed in Grid Control 12c :( bummer!!!!

                    Creating Standby Database With Enterprise Manager Failing [ID 1400482.1]
                    Symptoms
                    === ODM Issue Clarification ===
                    Standby database creation getting error:
                    ORA-38706: Cannot turn on FLASHBACK DATABASE logging.
                    ORA-38788: More standby database recovery is needed
                    ORA-38706 signalled during: alter database flashback on...
                    Cause
                    Bug 12923814 FLASHBACK AND ARL DELETION OPTIONS IGNORED IN ADD STANDBY DATABASE WIZARD - this bug is unpublished and is fixed in 12g grid.
                    Solution
                    By default a standby database is created without DORECOVER, when the backup is online the files will be inconsistent. With a standby database, this is OK as managed recovery will take care of making files consistent. However, in order to turn 'flashback ON', the files must be consistent.
                    Thus, once it fails:
                    1. recover standby database applying enough redo to make files consistent.
                    2. cancel recovery and turn on flashback database
                    3. operate standby database normally
                    • 7. Re: dgutil.pl line 99 ERROR when creating 11.2.0.3 data guard via GRID CONTROL
                      Marcelo Marques - ESRI
                      I tried to recover but still fails. I gave up. I'll use the RMAN script to create the standby until I move to Grid Control 12g.

                      SQL>RECOVER DATABASE;
                      ORA-00283: recovery session canceled due to errors
                      ORA-01610: recovery using the BACKUP CONTROLFILE option must be done

                      SQL> RECOVER DATABASE USING BACKUP CONTROLFILE;

                      SQL> ALTER DATABASE FLASHBACK ON;
                      alter database flashback on
                      *
                      ERROR at line 1:
                      ORA-38706: Cannot turn on FLASHBACK DATABASE logging.
                      ORA-38788: More standby database recovery is needed

                      SQL> RECOVER DATABASE USING BACKUP CONTROLFILE;
                      ORA-00283: recovery session canceled due to errors
                      ORA-01666: control file is for a standby database

                      SQL> RECOVER DATABASE USING BACKUP CONTROLFILE UNTIL CANCEL;
                      ORA-00283: recovery session canceled due to errors
                      ORA-01666: control file is for a standby database
                      • 8. Re: dgutil.pl line 99 ERROR when creating 11.2.0.3 data guard via GRID CONTROL
                        Marcelo Marques - ESRI
                        I contact Oracle Support on this issue, here are the details.

                        Generic Note
                        ------------------------
                        Hi Marcelo, the note you sited Creating Standby Database With Enterprise Manager Failing [ID 1400482.1
                        Says something a little different than what you tried.
                        You did RECOVER DATABASE for manual recover and the error says use backup control file.
                        For a standby to do manual recovery it's
                        *alter database recover standby database ;*
                        It may behave better.

                        But the note implies to just apply enough redo to have the standby consistent enough to turn on flashback.
                        This can be done by starting managed recovery and applying some redo log sequences.
                        So the flashback being turned on is too soon, but should have worked anyway. I think it would turn on managed recovery though.

                        This also happens since the job doesn't use dorecover since recovery can be done with managed recovery.
                        So most likely there isn't enough activity on the primary and the online redo has not been archived yet.
                        So some log switches on the primary will send enough redo to get consistent so you can turn on flashback and finish it.

                        *So why did it happen?*

                        Possibly
                        Bug 13250486 - ADD STANDBY DATABASE FOR TARGET WITH FLASHBACK ON FAILS WITH ERROR
                        Base bug 12923814 FLASHBACK AND ARL DELETION OPTIONS IGNORED IN ADD STANDBY DATABASE WIZARD
                        fixed in Grid Control 12.1c
                        There is currently no patch available for 11.1.0.1 Grid Control. We would have to open a bug to confirm you hit this in 11.1.0.1 and get a patch.

                        *A workaround would be to turn off flashback at the primary then try to create the standby.*
                        *Once the standby is created you can turn flashback back on for the primary and if required, the standby.*

                        You can try again to see, or finish the standby manually as I stated above.

                        I will still need the diagnostic information below.

                        Action Plan
                        =========

                        The product verion in this SR is for 12.1.0.1 Grid Control. I assume it should be 11.1 since you stated it is.

                        Grid Control job log that shows the errors.
                        alert logs from the primary and standby.

                        Please run these two diagnostic scripts and upload the output.
                        Script to Collect Data Guard Primary Site Diagnostic (Doc ID 241374.1)
                        Script to Collect Data Guard Physical Standby Diagnostic (Doc ID 241438.1)

                        Edited by: Marcelo Marques - ESRI on Mar 24, 2012 9:51 AM                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           
                        • 9. Re: dgutil.pl line 99 ERROR when creating 11.2.0.3 data guard via GRID CONTROL
                          Marcelo Marques - ESRI
                          Workaround 1: alter database recover standby database;

                          Workaround 2: A workaround would be to turn off flashback at the primary then try to create the standby.
                          Once the standby is created you can turn flashback back on for the primary and if required, the standby.

                          the workaround 2 worked for me and I was able to create the stand by using Grid Control