9 Replies Latest reply: Jul 18, 2012 2:54 AM by user13110695 RSS

    Collector gows down and audit data aren't in OAV Console

    762264
      Hi,

      I have OAV 10.2.3.2, DBAUD collector, collection agent is on windows. When I run this collector, it starts but after a while it is stopped again. I can retrieve politics, create politics and provision them. Audit data are stored in AUD$ but I can't see them in Audit Vault console.

      In C:/oracle/product/10.2.3.2/av_agent_1/av/log/DBAUD_Collector_av_db2_2 are following:

           ***** Started logging for 'AUD$ Audit Collector' *****


      INFO @ '21/04/2010 12:04:45 02:00':
      ***** Collector Name = DBAUD_Collector

      INFO @ '21/04/2010 12:04:45 02:00':
      ***** Source Name = av_db2

      INFO @ '21/04/2010 12:04:45 02:00':
      ***** Av Name = AV

      INFO @ '21/04/2010 12:04:45 02:00':
      ***** Initialization done OK

      INFO @ '21/04/2010 12:04:45 02:00':
      ***** Starting CB

      INFO @ '21/04/2010 12:04:46 02:00':
      Getting parameter |AUDAUDIT_DELAY_TIME|, got |20|

      INFO @ '21/04/2010 12:04:46 02:00':
      Getting parameter |AUDAUDIT_SLEEP_TIME|, got |5000|

      INFO @ '21/04/2010 12:04:46 02:00':
      Getting parameter |AUDAUDIT_ACTIVE_SLEEP_TIME|, got |1000|

      INFO @ '21/04/2010 12:04:46 02:00':
      Getting parameter |AUDAUDIT_MAX_PROCESS_RECORDS|, got |1000|

      INFO @ '21/04/2010 12:04:46 02:00':
      ***** CSDK inited OK + 1

      INFO @ '21/04/2010 12:04:46 02:00':
      ***** Src alias = SRCDB2

      INFO @ '21/04/2010 12:04:46 02:00':
      ***** SRC connected OK

      INFO @ '21/04/2010 12:04:46 02:00':
      ***** SRC data retrieved OK

      INFO @ '21/04/2010 12:04:46 02:00':
      ***** Recovery done OK

      ERROR @ '21/04/2010 12:05:07 02:00':
      On line 1287; OAV-46599: internal error ORA-1882 count(DWFACT_P20100420_TMP) =
      ORA-06512: na "AVSYS.DBMS_AUDIT_VAULT", line 6
      ORA-06512: na "AVSYS.AV$DW", line 1022
      ORA-01882: oblast časového pásma nebyla nalezena
      ORA-06512: na "AVSYS.AV$DW", line 1290
      ORA-06512: na line 1

      ERROR @ '21/04/2010 12:05:08 02:00':
      Collecting thread died with status 46821

      INFO @ '21/04/2010 12:06:01 02:00':
      Could not call Listener, NS error 12541, 12560, 511, 2, 0
      INFO @ '21/04/2010 12:07:01 02:00':
      Could not call Listener, NS error 12541, 12560, 511, 2, 0
      INFO @ '21/04/2010 12:08:01 02:00':
      Could not call Listener, NS error 12541, 12560, 511, 2, 0
      INFO @ '21/04/2010 12:09:01 02:00':
      Could not call Listener, NS error 12541, 12560, 511, 2, 0
      INFO @ '21/04/2010 12:10:01 02:00':
      Could not call Listener, NS error 12541, 12560, 511, 2, 0
      INFO @ '21/04/2010 12:11:01 02:00':
      Could not call Listener, NS error 12541, 12560, 511, 2, 0
      INFO @ '21/04/2010 12:12:01 02:00':
      Could not call Listener, NS error 12541, 12560, 511, 2, 0
      INFO @ '21/04/2010 12:13:01 02:00':
      Could not call Listener, NS error 12541, 12560, 511, 2, 0
      INFO @ '21/04/2010 12:13:41 02:00':
      Could not call Listener, NS error 12541, 12560, 511, 2, 0
      INFO @ '21/04/2010 12:13:41 02:00':

           ***** Started logging for 'AUD$ Audit Collector' *****


      INFO @ '21/04/2010 12:13:41 02:00':
      ***** Collector Name = DBAUD_Collector

      INFO @ '21/04/2010 12:13:41 02:00':
      ***** Source Name = av_db2

      INFO @ '21/04/2010 12:13:41 02:00':
      ***** Av Name = AV

      INFO @ '21/04/2010 12:13:41 02:00':
      ***** Initialization done OK

      INFO @ '21/04/2010 12:13:41 02:00':
      ***** Starting CB

      INFO @ '21/04/2010 12:13:42 02:00':
      Getting parameter |AUDAUDIT_DELAY_TIME|, got |20|

      INFO @ '21/04/2010 12:13:42 02:00':
      Getting parameter |AUDAUDIT_SLEEP_TIME|, got |5000|

      INFO @ '21/04/2010 12:13:42 02:00':
      Getting parameter |AUDAUDIT_ACTIVE_SLEEP_TIME|, got |1000|

      INFO @ '21/04/2010 12:13:42 02:00':
      Getting parameter |AUDAUDIT_MAX_PROCESS_RECORDS|, got |1000|

      INFO @ '21/04/2010 12:13:42 02:00':
      ***** CSDK inited OK + 1

      INFO @ '21/04/2010 12:13:42 02:00':
      ***** Src alias = SRCDB2

      INFO @ '21/04/2010 12:13:42 02:00':
      ***** SRC connected OK

      INFO @ '21/04/2010 12:13:42 02:00':
      ***** SRC data retrieved OK

      INFO @ '21/04/2010 12:13:42 02:00':
      ***** Recovery done OK

      ERROR @ '21/04/2010 12:14:03 02:00':
      On line 1287; OAV-46599: internal error ORA-1882 count(DWFACT_P20100420_TMP) =
      ORA-06512: na "AVSYS.DBMS_AUDIT_VAULT", line 6
      ORA-06512: na "AVSYS.AV$DW", line 1022
      ORA-01882: oblast časového pásma nebyla nalezena
      ORA-06512: na "AVSYS.AV$DW", line 1290
      ORA-06512: na line 1

      ERROR @ '21/04/2010 12:14:05 02:00':
      Error on get metric callback: 46821

      ERROR @ '21/04/2010 12:14:05 02:00':
      Collecting thread died with status 46821

      ERROR @ '21/04/2010 12:14:06 02:00':
      Receive error. NS error 12537
      ERROR @ '21/04/2010 12:14:06 02:00':
      Timeout for getting metrics reply!

      INFO @ '21/04/2010 12:15:01 02:00':
      Could not call Listener, NS error 12541, 12560, 511, 2, 0
      INFO @ '21/04/2010 12:16:01 02:00':
      Could not call Listener, NS error 12541, 12560, 511, 2, 0
      INFO @ '21/04/2010 12:17:01 02:00':
      Could not call Listener, NS error 12541, 12560, 511, 2, 0


      I've already follow help in administrator guide for Problem: Cannot start the DBAUD collector and the log file shows an error. There is that if I run command
      $ sqlplus /@SRCDB1

      successful, my source database is set up correctly. I run this command successful, but my problem didn't solve.

      Any advice, please?
        • 1. Re: Collector gows down and audit data aren't in OAV Console
          damorgan
          Others may have a different answer but mine is to get off Windows.

          Windows is just not ready for the data center. It is getting better but it still isn't there.
          • 2. Re: Collector gows down and audit data aren't in OAV Console
            706614
            Looks like you've hit a bug on the AV server. Please work with Oracle Support to follow up and resolve the issue.
            • 3. Re: Collector gows down and audit data aren't in OAV Console
              764417
              What's version on AV agent? Did you patch 10.2.3.2 on your AV agent?

              Regards,
              New
              • 4. Re: Collector gows down and audit data aren't in OAV Console
                762264
                AV agent is 10.2.3.2
                • 5. Re: Collector gows down and audit data aren't in OAV Console
                  762264
                  I reinstall AVS and AV Collection agent and again patchset 10.2.3.2, again add source database, agent, collectors,... but still have the same problem... collector starts successfully, but after wihile its stopped and in AV console aren't any data for reports (I create some audit policy and alerts).
                  I test connection with sqlplus /@SRCDB1 and it is ok, log for collector is:

                  May 11, 2010 8:11:15 AM Thread-43 FINE: return cached metric , name=RECORDS_PER_SEC value=0.00
                  May 11, 2010 8:11:38 AM Thread-44 FINE: timer task interval calculated = 60000
                  May 11, 2010 8:11:38 AM Thread-44 FINE: Going to start collector, m_finalCommand=/bin/sh -c $ORACLE_HOME/bin/a
                  vaudcoll hostname="avtest.zcu.cz" sourcename="stag_db" collectorname="DBAUD_Collector" avname="AV" loglevel="I
                  NFO" command=START
                  May 11, 2010 8:11:46 AM Thread-44 FINE: collector started, exitval=0
                  May 11, 2010 8:11:49 AM Thread-44 FINE: return cached metric , name=IS_ALIVE value=true
                  May 11, 2010 8:11:49 AM Thread-44 FINE: return cached metric , name=BYTES_PER_SEC value=0.0000
                  May 11, 2010 8:11:49 AM Thread-44 FINE: return cached metric , name=RECORDS_PER_SEC value=0.0000
                  May 11, 2010 8:11:49 AM Thread-44 FINE: timer task going to be started...
                  May 11, 2010 8:12:15 AM Thread-43 FINE: return cached metric , name=IS_ALIVE value=false
                  May 11, 2010 8:12:15 AM Thread-43 FINE: return cached metric , name=BYTES_PER_SEC value=0.00
                  May 11, 2010 8:12:15 AM Thread-43 FINE: return cached metric , name=RECORDS_PER_SEC value=0.00
                  May 11, 2010 8:13:15 AM Thread-43 FINE: return cached metric , name=IS_ALIVE value=false
                  May 11, 2010 8:13:15 AM Thread-43 FINE: return cached metric , name=BYTES_PER_SEC value=0.00
                  May 11, 2010 8:13:15 AM Thread-43 FINE: return cached metric , name=RECORDS_PER_SEC value=0.00
                  May 11, 2010 8:14:15 AM Thread-43 FINE: return cached metric , name=IS_ALIVE value=false
                  May 11, 2010 8:14:15 AM Thread-43 FINE: return cached metric , name=BYTES_PER_SEC value=0.00
                  May 11, 2010 8:14:15 AM Thread-43 FINE: return cached metric , name=RECORDS_PER_SEC value=0.00
                  May 11, 2010 8:15:15 AM Thread-43 FINE: return cached metric , name=IS_ALIVE value=false
                  May 11, 2010 8:15:15 AM Thread-43 FINE: return cached metric , name=BYTES_PER_SEC value=0.00
                  May 11, 2010 8:15:15 AM Thread-43 FINE: return cached metric , name=RECORDS_PER_SEC value=0.00
                  May 11, 2010 8:16:15 AM Thread-43 FINE: return cached metric , name=IS_ALIVE value=false
                  May 11, 2010 8:16:15 AM Thread-43 FINE: return cached metric , name=BYTES_PER_SEC value=0.00
                  May 11, 2010 8:16:15 AM Thread-43 FINE: return cached metric , name=RECORDS_PER_SEC value=0.00
                  May 11, 2010 8:17:15 AM Thread-43 FINE: return cached metric , name=IS_ALIVE value=false
                  May 11, 2010 8:17:15 AM Thread-43 FINE: return cached metric , name=BYTES_PER_SEC value=0.00
                  May 11, 2010 8:17:15 AM Thread-43 FINE: return cached metric , name=RECORDS_PER_SEC value=0.00
                  May 11, 2010 8:18:15 AM Thread-43 FINE: return cached metric , name=IS_ALIVE value=false
                  May 11, 2010 8:18:15 AM Thread-43 FINE: return cached metric , name=BYTES_PER_SEC value=0.00
                  May 11, 2010 8:18:15 AM Thread-43 FINE: return cached metric , name=RECORDS_PER_SEC value=0.00
                  May 11, 2010 8:19:15 AM Thread-43 FINE: return cached metric , name=IS_ALIVE value=false
                  May 11, 2010 8:19:15 AM Thread-43 FINE: return cached metric , name=BYTES_PER_SEC value=0.00
                  May 11, 2010 8:19:15 AM Thread-43 FINE: return cached metric , name=RECORDS_PER_SEC value=0.00
                  May 11, 2010 8:20:15 AM Thread-43 FINE: return cached metric , name=IS_ALIVE value=false
                  May 11, 2010 8:20:15 AM Thread-43 FINE: return cached metric , name=BYTES_PER_SEC value=0.00
                  May 11, 2010 8:20:15 AM Thread-43 FINE: return cached metric , name=RECORDS_PER_SEC value=0.00
                  May 11, 2010 8:21:15 AM Thread-43 FINE: return cached metric , name=IS_ALIVE value=false
                  May 11, 2010 8:21:15 AM Thread-43 FINE: return cached metric , name=BYTES_PER_SEC value=0.00
                  May 11, 2010 8:21:15 AM Thread-43 FINE: return cached metric , name=RECORDS_PER_SEC value=0.00
                  May 11, 2010 8:21:15 AM Thread-43 FINE: Stop caching since it is NOT alive for 10 times of query

                  And in table av$rads_flat are collected audit data...

                  Pls, know any one where could be a problem??
                  • 6. Re: Collector gows down and audit data aren't in OAV Console
                    762264
                    But the AUDIT_EVENT_FACT table is empty...?
                    • 7. Re: Collector gows down and audit data aren't in OAV Console
                      621199
                      Did you manage to solve this problem?
                      • 8. Re: Collector gows down and audit data aren't in OAV Console
                        797962
                        I had a similar problem: DBAUD Collector starts and then falls after some time. In the logs:
                        ERROR @ '29/04/2011 13:48:22 04:00':
                        On line 1291; OAV-46599: internal error ORA-14080 count(DWFACT_P20070710_TMP) =
                        ORA-06512: at "AVSYS.DBMS_AUDIT_VAULT", line 6
                        ORA-06512: at "AVSYS.AV$DW", line 1082
                        ORA-14080: partition cannot be split along the specified high bound
                        ORA-06512: at "AVSYS.AV$DW", line 1563
                        ORA-06512: at "AVSYS.AV$DW", line 1686
                        ORA-06512: at line 1

                        ERROR @ '29/04/2011 13:48:22 04:00':
                        Collecting thread died with status 46821

                        The problem was: AUD$ table in the source DB was moved to the system scheme, and synonim to it was made in the sys scheme.
                        After we`moved it to the sys scheme the problem was solved

                        Edited by: tsibir on 29.04.2011 9:14
                        • 9. Re: Collector goes down after successful start
                          user13110695
                          We had same problem. After a while, we found a record in sys.aud$ which includes an sqlbind column value over 4000 chars. Deleted record and bingo! It works well now.


                          /home/oracle/oracle/product/10.3.0/av_agent_1/av/log/DBAUD_Collector_avorionx_81.log