1 2 3 Previous Next 32 Replies Latest reply on Oct 16, 2014 7:40 AM by Jr.raj Go to original post
      • 15. Re: Error 1033 received logging on to the standby
        Jr.raj

        Hi CKPT,

        i have copied password files from primary nodes to standby nodes respectively.

         

        current content of primary's alert logfile(last 100 lines)

        $ tail alert_test1.log -n 100

        Error 1033 received logging on to the standby

        Mon Oct 13 07:53:46 2014

        Error 1033 received logging on to the standby

        Mon Oct 13 07:54:48 2014

        Error 1033 received logging on to the standby

        Mon Oct 13 07:55:50 2014

        Error 1033 received logging on to the standby

        Mon Oct 13 07:56:53 2014

        Error 1033 received logging on to the standby

        Mon Oct 13 07:57:55 2014

        Error 1033 received logging on to the standby

        Mon Oct 13 07:58:57 2014

        Error 1033 received logging on to the standby

        Mon Oct 13 07:59:59 2014

        Error 1033 received logging on to the standby

        Mon Oct 13 08:01:02 2014

        Error 1033 received logging on to the standby

        Mon Oct 13 08:02:04 2014

        Error 1033 received logging on to the standby

        Mon Oct 13 08:02:27 2014

        Error 1033 received logging on to the standby

        Mon Oct 13 08:02:28 2014

        Thread 1 advanced to log sequence 2115 (LGWR switch)

          Current log# 1 seq# 2115 mem# 0: +DATA/test/onlinelog/group_1.312.849710537

          Current log# 1 seq# 2115 mem# 1: +RECO/test/onlinelog/group_1.901.849710537

        Mon Oct 13 08:02:28 2014

        Archived Log entry 8029 added for thread 1 sequence 2114 ID 0xf5850205 dest 1:

        Mon Oct 13 08:05:38 2014

         

         

         

         

        ***********************************************************************

         

         

        Fatal NI connect error 12170.

         

         

          VERSION INFORMATION:

                TNS for Linux: Version 11.2.0.4.0 - Production

                Oracle Bequeath NT Protocol Adapter for Linux: Version 11.2.0.4.0 - Production

                TCP/IP NT Protocol Adapter for Linux: Version 11.2.0.4.0 - Production

          Time: 13-OCT-2014 08:05:38

          Tracing not turned on.

          Tns error struct:

            ns main err code: 12535

         

         

        TNS-12535: TNS:operation timed out

            ns secondary err code: 12606

            nt main err code: 0

            nt secondary err code: 0

            nt OS err code: 0

          Client address: (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.1.30)(PORT=27908))

        WARNING: inbound connection timed out (ORA-3136)

        Mon Oct 13 09:27:07 2014

        Thread 1 advanced to log sequence 2116 (LGWR switch)

          Current log# 2 seq# 2116 mem# 0: +DATA/test/onlinelog/group_2.313.849710539

          Current log# 2 seq# 2116 mem# 1: +RECO/test/onlinelog/group_2.902.849710539

        Mon Oct 13 09:27:07 2014

        ******************************************************************

        LGWR: Setting 'active' archival for destination LOG_ARCHIVE_DEST_2

        ******************************************************************

        Mon Oct 13 09:27:07 2014

        Archived Log entry 8030 added for thread 1 sequence 2115 ID 0xf5850205 dest 1:

        Mon Oct 13 09:27:08 2014

        Error 1033 received logging on to the standby

        Error 1033 received logging on to the standby

        Errors in file /u01/app/oracle/diag/rdbms/test/test1/trace/test1_nsa2_47003.trc:

        ORA-01033: ORACLE initialization or shutdown in progress

        Mon Oct 13 09:27:09 2014

        Error 1033 received logging on to the standby

        FAL[server, ARC3]: FAL archive failed, see trace file.

        ARCH: FAL archive failed. Archiver continuing

        ORACLE Instance test1 - Archival Error. Archiver continuing.

        Mon Oct 13 09:33:07 2014

        Error 1033 received logging on to the standby

        Mon Oct 13 09:34:10 2014

        Error 1033 received logging on to the standby

        Mon Oct 13 09:35:12 2014

        Error 1033 received logging on to the standby

        Mon Oct 13 09:36:14 2014

        Error 1033 received logging on to the standby

        Mon Oct 13 09:37:16 2014

        Error 1033 received logging on to the standby

        Mon Oct 13 09:38:19 2014

        Error 1033 received logging on to the standby

        Mon Oct 13 09:39:21 2014

        Error 1033 received logging on to the standby

        Mon Oct 13 09:40:23 2014

        Error 1033 received logging on to the standby

        Mon Oct 13 09:41:25 2014

        Error 1033 received logging on to the standby

        Mon Oct 13 09:42:28 2014

        Error 1033 received logging on to the standby

        Mon Oct 13 09:43:30 2014

        Error 1033 received logging on to the standby

        Mon Oct 13 09:44:32 2014

        Error 1033 received logging on to the standby

        Mon Oct 13 09:45:34 2014

        Error 1033 received logging on to the standby

        Mon Oct 13 09:46:37 2014

        Error 1033 received logging on to the standby

        Mon Oct 13 09:47:39 2014

        Error 1033 received logging on to the standby

        Thanks & Regards

        raj

        • 16. Re: Error 1033 received logging on to the standby
          CKPT

          Looks like i do not have much options even with me. Few points i want you to review more carefully

          1) Any Firewall settings?

          2) Any information from primary after enabling the destination?

          3) check permissions of password file on all the nodes

          4) Recreate password file on primary 1 node, then copy to 2nd node and also all standby nodes

          1 person found this helpful
          • 17. Re: Error 1033 received logging on to the standby
            Jr.raj

            Hi CKPT,

             

            i have changed permissions for all password files and  i bounced the standby database. still same issue

             

            Thanks & Regards,

            raj

            • 18. Re: Error 1033 received logging on to the standby
              Jr.raj

              Hi CKPT,

              we have firewall.

              i will create new password file , will copy to all nodes accordingly.

               

              Thanks & Regards,

              raj

              • 19. Re: Error 1033 received logging on to the standby
                Jr.raj

                Hi CKPT,

                 

                i have created password file in primary node1 as follows..

                orapwd file=orapw$ORACLE_SID password=welcome1 entries=30 force=y

                i have changed the file permissions and copied to all nodes

                ]$ chmod 755 orapwtest

                ]$ scp orapwtest oracle@192.168.1.41:/u01/app/oracle/product/11.2.0.4/dbhome_1/dbs/

                orapwtest                                                                                                                            100% 5120     5.0KB/s   00:00

                ]$  scp orapwtest oracle@192.168.1.30:/u01/app/oracle/product/11.2.0.4/dbhome_1/dbs/

                oracle@192.168.1.30's password:

                orapwtest                                                                                                                            100% 5120     5.0KB/s   00:00

                ]$  scp orapwtest oracle@192.168.1.31:/u01/app/oracle/product/11.2.0.4/dbhome_1/dbs/

                oracle@192.168.1.31's password:

                orapwtest                                                                                                                            100% 5120     5.0KB/s   00:00

                ]$

                still same issue.

                 

                Thanks & Regards,

                raj

                • 20. Re: Error 1033 received logging on to the standby
                  BPeaslandDBA

                  The password file was my first guess.

                   

                  Have you verified that the instance is in MOUNT mode? Query V$INSTANCE on the standby to know for sure.

                   

                   

                  Cheers,
                  Brian

                  • 21. Re: Error 1033 received logging on to the standby
                    Jr.raj

                    Thank you bpeasland!

                     

                    SQL> select status from v$instance; --- standby database

                     

                     

                    STATUS

                    ------------

                    MOUNTED

                    Thanks & Regards,

                    raj

                    • 22. Re: Error 1033 received logging on to the standby
                      CKPT

                      I hope on primary and standby both ORACLE_SID is same "test" ?

                      From standby, Can you tell me what is value of parameter "service_names" and also "select * from gv$pwfile_users;"

                      1 person found this helpful
                      • 23. Re: Error 1033 received logging on to the standby
                        Jr.raj

                        Thank you CKPT,

                         

                        SQL> sho parameter service_name

                         

                         

                        NAME                                 TYPE        VALUE

                        ------------------------------------ ----------- ------------------------------

                        service_names                        string      testdg

                         

                        SQL> select * from gv$pwfile_users;

                         

                         

                           INST_ID USERNAME                       SYSDB  SYSOP  SYSAS

                        ---------- ------------------------------              -----       -----         -----

                                 1                SYS                            TRUE   TRUE     FALSE

                                 2                SYS                            TRUE   TRUE     FALSE

                         

                        Thanks & Regards,

                        raj

                        • 24. Re: Re: Error 1033 received logging on to the standby
                          Jr.raj

                          Hi CKPT,

                           

                          i have noticed one issue, listner status in standby node2 , testdg2 is blocked,

                           

                          $ lsnrctl status

                           

                           

                          LSNRCTL for Linux: Version 11.2.0.4.0 - Production on 14-OCT-2014 03:15:05

                           

                           

                          Copyright (c) 1991, 2013, Oracle.  All rights reserved.

                           

                           

                          Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))

                          STATUS of the LISTENER

                          ------------------------

                          Alias                    LISTENER

                          Version                  TNSLSNR for Linux: Version 11.2.0.4.0 - Production

                          Start Date                21-JUN-2014 18:44:08

                          Uptime                    114 days 8 hr. 30 min. 57 sec

                          Trace Level              off

                          Security                  ON: Local OS Authentication

                          SNMP                      OFF

                          Listener Parameter File  /u01/app/11.2.0.4/grid/network/admin/listener.ora

                          Listener Log File        /u01/app/grid/diag/tnslsnr/enet-dr2/listener/alert/log                                                                                        .xml

                          Listening Endpoints Summary...

                            (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER)))

                            (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.1.31)(PORT=1521)))

                            (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.1.33)(PORT=1521)))

                          Services Summary...

                          Service "+ASM" has 1 instance(s).

                            Instance "+ASM2", status READY, has 1 handler(s) for this service...

                          Service "pnetdg" has 1 instance(s).

                            Instance "pnetdg2", status READY, has 1 handler(s) for this service...

                          Service "pnetdgXDB" has 1 instance(s).

                            Instance "pnetdg2", status READY, has 1 handler(s) for this service...

                          Service "pnetdg_DGB" has 1 instance(s).

                            Instance "pnetdg2", status READY, has 1 handler(s) for this service...

                          Service "testdg" has 1 instance(s).

                            Instance "testdg2", status BLOCKED, has 1 handler(s) for this service...

                          The command completed successfully

                           

                          Thanks & Regards,

                          raj

                          • 25. Re: Error 1033 received logging on to the standby
                            CKPT
                            Yeah. I was keep asking you somethings needs to look at service_name and it looks as per listener status it's blocked, verify the configuration what is differing from node 1 to 2
                            • 26. Re: Re: Error 1033 received logging on to the standby
                              Jr.raj

                              Thank you CKPT for your prompt response,

                              A

                              i have enabled dest_state_2 in standby node and i have mounted the instance in node2 and started MRP,

                              still same

                              Am new to RAC, so confused with nodes.

                              Thanks & Regards,

                              raj

                              • 27. Re: Error 1033 received logging on to the standby
                                BPeaslandDBA

                                 

                                Service "testdg" has 1 instance(s).

                                  Instance "testdg2", status BLOCKED, has 1 handler(s) for this service...

                                The command completed successfully

                                 

                                 

                                 

                                You you tried to restart just instance testdg2? Bounce the instance and wait a few minutes for it to reregister with the listener. Then check the status again.

                                 

                                Also...look in $GRID_HOME/network/admin to see if the two nodes have different configurations in the listener.ora.

                                 

                                Have you verified the /etc/hosts file does not have the IP addresses in their for these systems? Names should be resolved through DNS.

                                 

                                Do both standby instances have the same values for REMOTE_LISTENER? and for LOCAL_LISTENER?

                                 

                                 

                                HTH,

                                Brian

                                1 person found this helpful
                                • 28. Re: Re: Error 1033 received logging on to the standby
                                  Jr.raj

                                  Thank you bpeasland,

                                  in standby node 1

                                  SQL> sho parameter local_listener

                                   

                                   

                                  NAME                                 TYPE        VALUE

                                  ------------------------------------ ----------- ------------------------------

                                  local_listener                       string       (ADDRESS=(PROTOCOL=TCP)(HOST=

                                                                                   192.168.1.33)(PORT=1521))

                                  SQL> sho parameter remote_listener

                                   

                                   

                                  NAME                                 TYPE        VALUE

                                  ------------------------------------ ----------- ------------------------------

                                  remote_listener                      string      enet-dr-scan:1521

                                  SQL> exit

                                   

                                  in standby node2

                                  SQL> sho parameter local_listener

                                   

                                   

                                  NAME                                 TYPE        VALUE

                                  ------------------------------------ ----------- ------------------------------

                                  local_listener                       string       (ADDRESS=(PROTOCOL=TCP)(HOST=

                                                                                   192.168.1.33)(PORT=1521))

                                  SQL> sho parameter remote_listener

                                   

                                   

                                  NAME                                 TYPE        VALUE

                                  ------------------------------------ ----------- ------------------------------

                                  remote_listener                      string      enet-dr-scan:1521

                                  SQL> exit

                                   

                                   

                                   

                                  Thanks & Regards,

                                  raj

                                  • 29. Re: Error 1033 received logging on to the standby
                                    BPeaslandDBA

                                    The LOCAL_LISTENER parameter is pointing to the same IP address for both instances. They should be pointing to the IP address of the local db server. I bet that 192.168.1.33 is the IP address of the node for instance1. Instance2 is not allowed to use that as its local listener, hence the service for that instance is blocked. Change instance2's LOCAL_LISTENER to use the IP address of node2.

                                     

                                    Also note that you probably have this configuration because this parameter is defined as "*.LOCAL_LISTENER" in the spfile. The star-dot means it will apply to all instances. You should have instance1.LOCAL_LISTENER and instance2.LOCAL_LISTENER so that each instance gets its own unique value.

                                     

                                     

                                    Cheers,
                                    Brian

                                    1 person found this helpful