7 Replies Latest reply on Jun 4, 2014 7:59 PM by mseberg

    11gR2 physical standby stops applying logs - ora-00020, ora-16146


      I have 2 physical standbys for an 11gR2 RAC database. Standby 1 uses standby logs and has no problems. Standby 2 does not use standby logs and about once a week stops applying logs. Both standbys are using maximum performance.  Everything runs on Windows 2008 R2 EE.


      I checked the alert log on standby 2 and I found ora-00020 and ora-16146 errors.


      ORA-16146: standby control file enqueue unavailable

      ORA-00020: maximum number of processes 500 exceeded


      I suspect the ORA-00020 is indicating the root cause. I checked and all week long the # of processes is growing by 2 every 30 minutes as the logs from the primary are shipped to it. Why would these processes not close after the log gets shipped to standby?


      FWIW v$session shows the username as PUBLIC, the program as ORACLE.EXE, and the machine is always node 2 of the RAC cluster.


      As a temporary fix I've started killing sessions on the standby that have been inactive for over 1 hour. I think that will prevent it from happening but the question is why is it happening in the first place?


      On the primary node 2 I see I lot of warnings and errors in v$dataguard_status with error numbers 3135 and 16141 for the affected destination. Node 1 shows only a handful of warnings with error 16198 "hung on a network operation".