7 Replies Latest reply: May 11, 2012 12:48 PM by Aerosmith RSS

    Oracle enterprise manager 11g - Internal Error has occurred. Check the log

    Aerosmith
      Hi.

      It used to work to fine before, but since yesterday, I am getting this error after logging on to the em site -> https://hostname:1158/em

      SQL> SELECT * from v$version
      2 ;

      BANNER
      --------------------------------------------------------------------------------
      Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
      PL/SQL Release 11.2.0.1.0 - Production
      CORE     11.2.0.1.0     Production
      TNS for Linux: Version 11.2.0.1.0 - Production
      NLSRTL Version 11.2.0.1.0 - Production

      emagent.trc
      2012-05-11 09:31:09,025 Thread-233563904 ERROR http: snmehl_allocateListenerPort: failed to create socket: Address family not supported by protocol (error =
      97)
      2012-05-11 09:31:09,027 Thread-233563904 WARN http: nmehl_connect_internal: connect failed to ( servername.domainname:1158): Address family not supported by pro
      tocol (error = 97)
      2012-05-11 09:31:09,027 Thread-233563904 ERROR pingManager: nmepm_pingReposURL: Cannot connect to https:// servername.domainname:1158/em/upload/: retStatus=-1
      2012-05-11 09:31:09,027 Thread-233563904 WARN command: Job Subsystem Timeout set at 600 seconds
      2012-05-11 09:31:09,029 Thread-233563904 WARN upload: Upload manager has no Failure script: disabled
      2012-05-11 09:31:09,037 Thread-233563904 WARN metadata: Metric VirtualHosts does not have any data columns
      2012-05-11 09:31:09,095 Thread-233563904 WARN metadata: Metric hung_system_traces does not have any data columns
      2012-05-11 09:31:09,174 Thread-108107520 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,174 Thread-101807872 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,174 Thread-104957696 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,174 Thread-98658048 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,174 Thread-101807872 WARN TargetManager: Query returned 0 rows (only one expected) for the dynamic property from_cluster
      2012-05-11 09:31:09,175 Thread-101807872 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,178 Thread-101807872 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,178 Thread-101807872 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,210 Thread-104957696 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,211 Thread-104957696 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,211 Thread-101807872 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,211 Thread-104957696 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,215 Thread-104957696 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,215 Thread-104957696 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,216 Thread-104957696 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,216 Thread-104957696 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,217 Thread-104957696 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,256 Thread-101807872 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,274 Thread-104957696 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,337 Thread-104957696 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,349 Thread-104957696 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,351 Thread-104957696 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,353 Thread-104957696 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,354 Thread-104957696 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,355 Thread-104957696 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,359 Thread-104957696 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,359 Thread-104957696 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,415 Thread-104957696 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,418 Thread-104957696 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,419 Thread-104957696 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,501 Thread-104957696 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,501 Thread-104957696 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,597 Thread-104957696 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,690 Thread-104957696 ERROR : (nmecmgr.c,3230):Memory 0x0 encountered, expect struct_id=11011
      2012-05-11 09:31:09,799 Thread-233563904 WARN collector: the column name first_message_age in this condition does not exist in metric aq_msgs_persistentq_pe
      r_subscriber
      2012-05-11 09:31:09,801 Thread-233563904 WARN collector: the column name first_message_age in this condition does not exist in metric aq_msgs_persistentq_pe
      r_subscriber
      2012-05-11 09:31:09,835 Thread-233563904 WARN collector: the column name first_message_age in this condition does not exist in metric aq_msgs_persistentq_pe
      r_subscriber
      2012-05-11 09:31:09,841 Thread-101807872 WARN http: nmehl_connect_internal: connect failed to ( servername.domainname:1158): Address family not supported by pro
      tocol (error = 97)
      2012-05-11 09:31:09,841 Thread-101807872 ERROR pingManager: nmepm_pingReposURL: Cannot connect to https:// servername.domainname:1158/em/upload/: retStatus=-1
      2012-05-11 09:31:09,843 Thread-101807872 WARN http: nmehl_connect_internal: connect failed to ( servername.domainname:1158): Address family not supported by pro
      tocol (error = 97)
      2012-05-11 09:31:09,843 Thread-101807872 ERROR pingManager: nmepm_pingReposURL: Cannot connect to https:// servername.domainname:1158/em/upload/: retStatus=-1
      2012-05-11 09:31:10,029 Thread-124647168 WARN http: nmehl_connect_internal: connect failed to ( servername.domainname:1158): Address family not supported by pro
      tocol (error = 97)
      2012-05-11 09:31:10,029 Thread-124647168 ERROR command: nmejcn: failed http connection to https:// servername.domainname:1158/em/upload/: retStatus=-1
      2012-05-11 09:31:12,031 Thread-124647168 WARN http: nmehl_connect_internal: connect failed to ( servername.domainname:1158): Address family not supported by pro
      tocol (error = 97)
      2012-05-11 09:31:12,031 Thread-124647168 ERROR command: nmejcn: failed http connection to https:// servername.domainname:1158/em/upload/: retStatus=-1
      2012-05-11 09:31:16,033 Thread-124647168 WARN http: nmehl_connect_internal: connect failed to ( servername.domainname:1158): Address family not supported by pro
      tocol (error = 97)
      2012-05-11 09:31:16,033 Thread-124647168 ERROR command: nmejcn: failed http connection to https:// servername.domainname:1158/em/upload/: retStatus=-1
      2012-05-11 09:31:24,035 Thread-124647168 WARN http: nmehl_connect_internal: connect failed to ( servername.domainname:1158): Address family not supported by pro
      tocol (error = 97)
      2012-05-11 09:31:24,035 Thread-124647168 ERROR command: nmejcn: failed http connection to https:// servername.domainname:1158/em/upload/: retStatus=-1

      Can anyone tell me what may be going on? Yesterday I had configured channel bonding for network interfaces. Did firmware update on RAID adapter and updated linux packages including kernel. Going to old kernel didn't help. I tried restarting processes, that also didn't help.
        • 1. Re: Oracle enterprise manager 11g - Internal Error has occurred. Check the log
          sb92075
          does this system obtain IP# via DHCP?
          • 2. Re: Oracle enterprise manager 11g - Internal Error has occurred. Check the log
            Aerosmith
            Hi , thanks for the reply.

            Answer is no. It's statically configured.
            • 3. Re: Oracle enterprise manager 11g - Internal Error has occurred. Check the log
              sb92075
              Aerosmith wrote:
              Hi.

              It used to work to fine before, but since yesterday, I am getting this error after logging on to the em site -> https://hostname:1158/em
              what results when you do as below

              http://hostname:1158/em
              • 4. Re: Oracle enterprise manager 11g - Internal Error has occurred. Check the log
                Aerosmith
                http://hostname:1158/em shows a ? on the webpage.

                However,
                https://hostname:1158/em , does get redirected to https://hostname:1158/em/console/logon/logon where it asks me fore crdentials.

                * User Name          
                * Password          
                Connect As     Normal/SYSDBA     

                I login using sys and its password, and "I get Internal Error has occurred. Check the log file for details" Database     |      Help     |      Logout and OK button.
                • 5. Re: Oracle enterprise manager 11g - Internal Error has occurred. Check the log
                  Aerosmith
                  Aerosmith wrote:
                  http://hostname:1158/em shows a ? on the webpage.

                  However,
                  https://hostname:1158/em , does get redirected to https://hostname:1158/em/console/logon/logon where it asks me fore crdentials.

                  * User Name          
                  * Password          
                  Connect As     Normal/SYSDBA     

                  I login using sys and its password, and "I get Internal Error has occurred. Check the log file for details" Database     |      Help     |      Logout and OK button.
                  This is what I see in emoms.trc

                  2012-05-11 09:42:29,294 [XMLLoader0 00000000004.xml] WARN jdbc.ConnectionCache _getConnection.352 - Closed Connection: OraclePooledConnection.getConnection() - SQLException Ocurred:Invalid or Stale Connection found in the Connection Cache
                  java.sql.SQLException: Closed Connection: OraclePooledConnection.getConnection() - SQLException Ocurred:Invalid or Stale Connection found in the Connection Cache
                       at oracle.jdbc.driver.SQLStateMapping.newSQLException(SQLStateMapping.java:73)
                       at oracle.jdbc.driver.DatabaseError.newSQLException(DatabaseError.java:111)
                       at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:174)
                       at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:230)
                       at oracle.jdbc.pool.OraclePooledConnection.getConnection(OraclePooledConnection.java:467)
                       at oracle.jdbc.pool.OracleImplicitConnectionCache.getConnection(OracleImplicitConnectionCache.java:508)
                       at oracle.jdbc.pool.OracleDataSource.getConnection(OracleDataSource.java:544)
                       at oracle.jdbc.pool.OracleDataSource.getConnection(OracleDataSource.java:480)
                       at oracle.sysman.util.jdbc.ConnectionCache._getConnection(ConnectionCache.java:336)
                       at oracle.sysman.util.jdbc.ConnectionCache._getConnection(ConnectionCache.java:322)
                       at oracle.sysman.util.jdbc.ConnectionCache.getUnwrappedConnection(ConnectionCache.java:575)
                       at oracle.sysman.emSDK.svc.conn.FGAConnectionCache.getFGAConnection(FGAConnectionCache.java:207)
                       at oracle.sysman.emSDK.svc.conn.ConnectionService.getSystemConnection(ConnectionService.java:1383)
                       at oracle.sysman.emdrep.dbjava.loader.XMLLoaderContext.getConnection(XMLLoaderContext.java:997)
                       at oracle.sysman.emdrep.dbjava.loader.XMLLoaderContext.loadFromStream(XMLLoaderContext.java:1812)
                       at oracle.sysman.emdrep.dbjava.loader.XMLLoader.LoadFile(XMLLoader.java:578)
                       at oracle.sysman.emdrep.dbjava.loader.XMLLoader.LoadFiles(XMLLoader.java:669)
                       at oracle.sysman.emdrep.dbjava.loader.XMLLoader.run(XMLLoader.java:1316)
                       at java.lang.Thread.run(Thread.java:595)
                  2012-05-11 09:42:29,294 [XMLLoader0 00000000004.xml] WARN jdbc.ConnectionCache _getConnection.353 - Got a fatal exeption when getting a connection; Error code = 17008; Cleaning up cache and retrying
                  2012-05-11 09:42:29,406 [XMLLoader0 00000000010.xml] ERROR eml.XMLLoader LoadFiles.696 - Error while processing 00000000010.xml:
                  java.sql.SQLException: ORA-20101: Metric PhysAddress for target type netapp_filer not configured
                  ORA-06512: at "SYSMAN.ECM_CT", line 718
                  ORA-06512: at line 1
                  ORA-06512: at "SYSMAN.EMD_LOADER", line 4931
                  ORA-06512: at line 1

                       at oracle.jdbc.driver.SQLStateMapping.newSQLException(SQLStateMapping.java:73)
                       at oracle.jdbc.driver.DatabaseError.newSQLException(DatabaseError.java:111)
                       at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:174)
                       at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:472)
                       at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:422)
                       at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:1089)
                       at oracle.jdbc.driver.T4CCallableStatement.doOall8(T4CCallableStatement.java:200)
                       at oracle.jdbc.driver.T4CCallableStatement.executeForRows(T4CCallableStatement.java:1029)
                       at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1347)
                       at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3556)
                       at oracle.jdbc.driver.OraclePreparedStatement.execute(OraclePreparedStatement.java:3727)
                       at oracle.jdbc.driver.OracleCallableStatement.execute(OracleCallableStatement.java:8130)
                       at oracle.jdbc.driver.OraclePreparedStatementWrapper.execute(OraclePreparedStatementWrapper.java:1209)
                       at oracle.sysman.util.jdbc.PreparedStatementWrapper.execute(PreparedStatementWrapper.java:173)
                       at oracle.sysman.util.jdbc.CallableStatementWrapper.execute(CallableStatementWrapper.java:135)
                       at oracle.sysman.emdrep.dbjava.loader.XMLDocumentSplitter.postLoadProc(XMLDocumentSplitter.java:1709)
                       at oracle.sysman.emdrep.dbjava.loader.XMLDocumentSplitter.endElement(XMLDocumentSplitter.java:1009)
                       at oracle.xml.parser.v2.NonValidatingParser.parseElement(NonValidatingParser.java:1318)
                       at oracle.xml.parser.v2.NonValidatingParser.parseRootElement(NonValidatingParser.java:336)
                       at oracle.xml.parser.v2.NonValidatingParser.parseDocument(NonValidatingParser.java:303)
                       at oracle.xml.parser.v2.XMLParser.parse(XMLParser.java:206)
                       at oracle.sysman.emdrep.dbjava.loader.XMLDocumentSplitter.split(XMLDocumentSplitter.java:477)
                       at oracle.sysman.emdrep.dbjava.loader.XMLLoaderContext.loadFromStream(XMLLoaderContext.java:1824)
                       at oracle.sysman.emdrep.dbjava.loader.XMLLoader.LoadFile(XMLLoader.java:578)
                       at oracle.sysman.emdrep.dbjava.loader.XMLLoader.LoadFiles(XMLLoader.java:669)
                       at oracle.sysman.emdrep.dbjava.loader.XMLLoader.run(XMLLoader.java:1316)
                       at java.lang.Thread.run(Thread.java:595)
                  2012-05-11 09:42:37,681 [JobWorker 5792:Thread-24] ERROR em.linux_provision getHardwareInterface_Name_MacAddress.173 - SQLException:ProvisionDeviceDetailsDAO:getHardwareInterface_Name_MacAddress: ORA-00600: internal error code, arguments: [kdsgrp1], [], [], [], [], [], [], [], [], [], [], []

                  2012-05-11 09:42:37,682 [JobWorker 5792:Thread-24] ERROR em.linux_provision_jobs doProvision.135 - UpdateProvisioningStatusCommand:HardwareException: error is
                  oracle.sysman.emgc.provision.hardware.HardwareException
                       at oracle.sysman.emgc.provision.impl.hardware.dao.ProvisionDeviceDetailsDAO.getHardwareInterface_Name_MacAddress(ProvisionDeviceDetailsDAO.java:175)
                       at oracle.sysman.emgc.provision.impl.hardware.HardwareMgrImpl.getDeviceDetails(HardwareMgrImpl.java:309)
                       at oracle.sysman.emgc.provision.impl.hardware.HardwareMgrImpl.getOrCreateHardwareServer(HardwareMgrImpl.java:346)
                       at oracle.sysman.emgc.provision.jobs.commands.UpdateProvisioningStatusCommand.doProvision(UpdateProvisioningStatusCommand.java:131)
                       at oracle.sysman.emgc.provision.jobs.commands.UpdateProvisioningStatusCommand.executeProvisioningCommand(UpdateProvisioningStatusCommand.java:372)
                       at oracle.sysman.emgc.provision.jobs.commands.AbstractProvisioningCommand.executeCommand(AbstractProvisioningCommand.java:141)
                       at oracle.sysman.emdrep.jobs.JobWorker.runCommand(Dispatcher.java:830)
                       at oracle.sysman.emdrep.jobs.JobWorker.run(Dispatcher.java:713)
                       at oracle.sysman.util.threadPoolManager.WorkerThread.run(Worker.java:260)
                  Caused by: java.sql.SQLException: ORA-00600: internal error code, arguments: [kdsgrp1], [], [], [], [], [], [], [], [], [], [], []

                       at oracle.jdbc.driver.SQLStateMapping.newSQLException(SQLStateMapping.java:73)
                       at oracle.jdbc.driver.DatabaseError.newSQLException(DatabaseError.java:111)
                       at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:174)
                       at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:472)
                       at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:422)
                       at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:1089)
                       at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:204)
                       at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:861)
                       at oracle.jdbc.driver.T4CPreparedStatement.executeMaybeDescribe(T4CPreparedStatement.java:945)
                       at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1303)
                       at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3556)
                       at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3608)
                       at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1341)
                       at oracle.sysman.util.jdbc.PreparedStatementWrapper.executeQuery(PreparedStatementWrapper.java:211)
                       at oracle.sysman.emgc.provision.impl.hardware.dao.ProvisionDeviceDetailsDAO.getHardwareInterface_Name_MacAddress(ProvisionDeviceDetailsDAO.java:146)
                       ... 8 more
                  2012-05-11 09:42:37,683 [JobWorker 5792:Thread-24] ERROR em.linux_provision_jobs executeCommand.172 - Got SQLException in executeCommand
                  java.sql.SQLException: Closed Connection
                       at oracle.jdbc.driver.SQLStateMapping.newSQLException(SQLStateMapping.java:73)
                       at oracle.jdbc.driver.DatabaseError.newSQLException(DatabaseError.java:111)
                       at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:174)
                       at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:230)
                       at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:445)
                       at oracle.jdbc.driver.PhysicalConnection.rollback(PhysicalConnection.java:4483)
                       at oracle.jdbc.OracleConnectionWrapper.rollback(OracleConnectionWrapper.java:129)
                       at oracle.jdbc.OracleConnectionWrapper.rollback(OracleConnectionWrapper.java:129)
                       at oracle.sysman.emgc.provision.jobs.commands.AbstractProvisioningCommand.executeCommand(AbstractProvisioningCommand.java:150)
                       at oracle.sysman.emdrep.jobs.JobWorker.runCommand(Dispatcher.java:830)
                       at oracle.sysman.emdrep.jobs.JobWorker.run(Dispatcher.java:713)
                       at oracle.sysman.util.threadPoolManager.WorkerThread.run(Worker.java:260)
                  2012-05-11 09:42:37,683 [JobWorker 5792:Thread-24] ERROR em.linux_provision_jobs executeCommand.179 - Got SQLException in executeCommand
                  java.sql.SQLException: Closed Connection
                       at oracle.jdbc.driver.SQLStateMapping.newSQLException(SQLStateMapping.java:73)
                       at oracle.jdbc.driver.DatabaseError.newSQLException(DatabaseError.java:111)
                       at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:174)
                       at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:230)
                       at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:445)
                       at oracle.jdbc.driver.PhysicalConnection.rollback(PhysicalConnection.java:4483)
                       at oracle.jdbc.OracleConnectionWrapper.rollback(OracleConnectionWrapper.java:129)
                       at oracle.jdbc.OracleConnectionWrapper.rollback(OracleConnectionWrapper.java:129)
                       at oracle.sysman.emgc.provision.jobs.commands.AbstractProvisioningCommand.executeCommand(AbstractProvisioningCommand.java:175)
                       at oracle.sysman.emdrep.jobs.JobWorker.runCommand(Dispatcher.java:830)
                       at oracle.sysman.emdrep.jobs.JobWorker.run(Dispatcher.java:713)
                       at oracle.sysman.util.threadPoolManager.WorkerThread.run(Worker.java:260)
                  2012-05-11 09:42:37,684 [JobWorker 5792:Thread-24] WARN jdbc.ConnectionCache _getConnection.352 - Closed Connection: OraclePooledConnection.getConnection() - SQLException Ocurred:Invalid or Stale Connection found in the Connection Cache
                  java.sql.SQLException: Closed Connection: OraclePooledConnection.getConnection() - SQLException Ocurred:Invalid or Stale Connection found in the Connection Cache
                       at oracle.jdbc.driver.SQLStateMapping.newSQLException(SQLStateMapping.java:73)
                       at oracle.jdbc.driver.DatabaseError.newSQLException(DatabaseError.java:111)
                       at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:174)
                       at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:230)
                       at oracle.jdbc.pool.OraclePooledConnection.getConnection(OraclePooledConnection.java:467)
                       at oracle.jdbc.pool.OracleImplicitConnectionCache.getConnection(OracleImplicitConnectionCache.java:508)
                       at oracle.jdbc.pool.OracleDataSource.getConnection(OracleDataSource.java:544)
                       at oracle.jdbc.pool.OracleDataSource.getConnection(OracleDataSource.java:480)
                       at oracle.sysman.util.jdbc.ConnectionCache._getConnection(ConnectionCache.java:336)
                       at oracle.sysman.util.jdbc.ConnectionCache._getConnection(ConnectionCache.java:322)
                       at oracle.sysman.util.jdbc.ConnectionCache.getUnwrappedConnection(ConnectionCache.java:575)
                       at oracle.sysman.emSDK.svc.conn.FGAConnectionCache.getFGAConnection(FGAConnectionCache.java:207)
                       at oracle.sysman.emSDK.svc.conn.ConnectionService.getPrivateConnectionWait(ConnectionService.java:1231)
                       at oracle.sysman.emdrep.jobs.JobWorker.getPrivateConnectionWait(Dispatcher.java:316)
                       at oracle.sysman.emdrep.jobs.JobWorker.updateStepStatus(Dispatcher.java:987)
                       at oracle.sysman.emdrep.jobs.JobWorker.runCommand(Dispatcher.java:894)
                       at oracle.sysman.emdrep.jobs.JobWorker.run(Dispatcher.java:713)
                       at oracle.sysman.util.threadPoolManager.WorkerThread.run(Worker.java:260)
                  2012-05-11 09:42:37,685 [JobWorker 5792:Thread-24] WARN jdbc.ConnectionCache _getConnection.353 - Got a fatal exeption when getting a connection; Error code = 17008; Cleaning up cache and retrying
                  2012-05-11 09:49:19,583 [EMUI_09_49_19_/console/database/instance/sitemap] ERROR db.featureUsage logFeatureUsage.136 - java.sql.SQLException: ORA-01873: the leading precision of the interval is too small
                  ORA-06512: at "DBSNMP.MGMT_UPDATE_DB_FEATURE_LOG", line 13
                  ORA-06512: at line 1

                  java.sql.SQLException: ORA-01873: the leading precision of the interval is too small
                  ORA-06512: at "DBSNMP.MGMT_UPDATE_DB_FEATURE_LOG", line 13
                  ORA-06512: at line 1

                       at oracle.jdbc.driver.SQLStateMapping.newSQLException(SQLStateMapping.java:77)
                       at oracle.jdbc.driver.DatabaseError.newSQLException(DatabaseError.java:111)
                       at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:174)
                       at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:472)
                       at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:422)
                       at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:1089)
                       at oracle.jdbc.driver.T4CCallableStatement.doOall8(T4CCallableStatement.java:200)
                       at oracle.jdbc.driver.T4CCallableStatement.executeForRows(T4CCallableStatement.java:1029)
                       at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1347)
                       at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3556)
                       at oracle.jdbc.driver.OraclePreparedStatement.execute(OraclePreparedStatement.java:3727)
                       at oracle.jdbc.driver.OracleCallableStatement.execute(OracleCallableStatement.java:8130)
                       at oracle.jdbc.driver.OraclePreparedStatementWrapper.execute(OraclePreparedStatementWrapper.java:1209)
                       at oracle.sysman.util.jdbc.PreparedStatementWrapper.execute(PreparedStatementWrapper.java:173)
                       at oracle.sysman.util.jdbc.CallableStatementWrapper.execute(CallableStatementWrapper.java:135)
                       at oracle.sysman.db.adm.util.FeatureUsageUtils.logFeatureUsage(FeatureUsageUtils.java:127)
                       at oracle.sysman.db.adm.util.FeatureUsageUtils.logFeatureUsage(FeatureUsageUtils.java:93)
                       at oracle.sysman.db.adm.util.FeatureUsageUtils.logFeatureUsage(FeatureUsageUtils.java:73)
                       at oracle.sysman.db.adm.inst.SitemapController.logFeatureUsageHomePage(SitemapController.java:6393)
                       at oracle.sysman.db.adm.inst.SitemapController.onDoLoad(SitemapController.java:3058)
                       at oracle.sysman.db.adm.BaseController.handleEvent(BaseController.java:1306)
                       at oracle.sysman.emSDK.svlt.PageHandler.handleRequest(PageHandler.java:577)
                       at oracle.sysman.db.adm.RootController.handleRequest(RootController.java:235)
                       at oracle.sysman.db.adm.BaseController.handleRequest(BaseController.java:1238)
                       at oracle.sysman.db.adm.DBControllerResolver.handleRequest(DBControllerResolver.java:139)
                       at oracle.sysman.emSDK.svlt.EMServlet.myDoGet(EMServlet.java:787)
                       at oracle.sysman.emSDK.svlt.EMServlet.doGet(EMServlet.java:343)
                       at oracle.sysman.eml.app.Console.doGet(Console.java:319)
                       at javax.servlet.http.HttpServlet.service(HttpServlet.java:743)
                       at javax.servlet.http.HttpServlet.service(HttpServlet.java:856)
                       at com.evermind.server.http.ResourceFilterChain.doFilter(ResourceFilterChain.java:64)
                       at oracle.sysman.eml.app.EMRepLoginFilter.doFilter(EMRepLoginFilter.java:208)
                       at com.evermind.server.http.EvermindFilterChain.doFilter(EvermindFilterChain.java:15)
                       at oracle.sysman.db.adm.inst.HandleRepDownFilter.doFilter(HandleRepDownFilter.java:165)
                       at com.evermind.server.http.EvermindFilterChain.doFilter(EvermindFilterChain.java:17)
                       at oracle.sysman.eml.app.BrowserVersionFilter.doFilter(BrowserVersionFilter.java:122)
                       at com.evermind.server.http.EvermindFilterChain.doFilter(EvermindFilterChain.java:17)
                       at oracle.sysman.emSDK.svlt.EMRedirectFilter.doFilter(EMRedirectFilter.java:102)
                       at com.evermind.server.http.EvermindFilterChain.doFilter(EvermindFilterChain.java:17)
                       at oracle.sysman.eml.app.ContextInitFilter.doFilter(ContextInitFilter.java:357)
                       at com.evermind.server.http.ServletRequestDispatcher.invoke(ServletRequestDispatcher.java:621)
                       at com.evermind.server.http.ServletRequestDispatcher.forwardInternal(ServletRequestDispatcher.java:370)
                       at com.evermind.server.http.HttpRequestHandler.doProcessRequest(HttpRequestHandler.java:871)
                       at com.evermind.server.http.HttpRequestHandler.processRequest(HttpRequestHandler.java:453)
                       at com.evermind.server.http.HttpRequestHandler.serveOneRequest(HttpRequestHandler.java:221)
                       at com.evermind.server.http.HttpRequestHandler.run(HttpRequestHandler.java:122)
                       at com.evermind.server.http.HttpRequestHandler.run(HttpRequestHandler.java:111)
                       at oracle.oc4j.network.ServerSocketReadHandler$SafeRunnable.run(ServerSocketReadHandler.java:260)
                       at com.evermind.util.ReleasableResourcePooledExecutor$MyWorker.run(ReleasableResourcePooledExecutor.java:303)
                       at java.lang.Thread.run(Thread.java:595)
                  2012-05-11 09:49:20,513 [EMUI_09_49_19_/console/database/instance/sitemap] ERROR inv.EcmInvSW createTopLevelInvTable.850 - ORA-00600: internal error code, arguments: [kdsgrp1], [], [], [], [], [], [], [], [], [], [], []
                  ORA-06512: at "SYSMAN.ECM_INV", line 563
                  ORA-06512: at line 1

                  java.sql.SQLException: ORA-00600: internal error code, arguments: [kdsgrp1], [], [], [], [], [], [], [], [], [], [], []
                  ORA-06512: at "SYSMAN.ECM_INV", line 563
                  ORA-06512: at line 1

                       at oracle.jdbc.driver.SQLStateMapping.newSQLException(SQLStateMapping.java:73)
                       at oracle.jdbc.driver.DatabaseError.newSQLException(DatabaseError.java:111)
                       at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:174)
                       at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:472)
                       at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:422)
                       at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:1089)
                       at oracle.jdbc.driver.T4CCallableStatement.doOall8(T4CCallableStatement.java:200)
                       at oracle.jdbc.driver.T4CCallableStatement.executeForRows(T4CCallableStatement.java:1029)
                       at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1347)
                       at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3556)
                       at oracle.jdbc.driver.OraclePreparedStatement.execute(OraclePreparedStatement.java:3727)
                       at oracle.jdbc.driver.OracleCallableStatement.execute(OracleCallableStatement.java:8130)
                       at oracle.jdbc.driver.OraclePreparedStatementWrapper.execute(OraclePreparedStatementWrapper.java:1209)
                       at oracle.sysman.util.jdbc.PreparedStatementWrapper.execute(PreparedStatementWrapper.java:173)
                       at oracle.sysman.util.jdbc.CallableStatementWrapper.execute(CallableStatementWrapper.java:135)
                       at oracle.sysman.eml.ecm.inv.EcmInvSW.createTopLevelInvTable(EcmInvSW.java:831)
                       at oracle.sysman.eml.ecm.inv.EcmInvSW.createInventoryTable(EcmInvSW.java:419)
                       at oracle.sysman.eml.ecm.inv.EcmInvBean.createInventoryTable(EcmInvBean.java:1449)
                       at oracle.sysman.db.adm.inst.SitemapController.setRequestAttributes(SitemapController.java:1405)
                       at oracle.sysman.db.adm.BaseController.handleEvent(BaseController.java:1391)
                       at oracle.sysman.emSDK.svlt.PageHandler.handleRequest(PageHandler.java:577)
                       at oracle.sysman.db.adm.RootController.handleRequest(RootController.java:235)
                       at oracle.sysman.db.adm.BaseController.handleRequest(BaseController.java:1238)
                       at oracle.sysman.db.adm.DBControllerResolver.handleRequest(DBControllerResolver.java:139)
                       at oracle.sysman.emSDK.svlt.EMServlet.myDoGet(EMServlet.java:787)
                       at oracle.sysman.emSDK.svlt.EMServlet.doGet(EMServlet.java:343)
                       at oracle.sysman.eml.app.Console.doGet(Console.java:319)
                       at javax.servlet.http.HttpServlet.service(HttpServlet.java:743)
                       at javax.servlet.http.HttpServlet.service(HttpServlet.java:856)
                       at com.evermind.server.http.ResourceFilterChain.doFilter(ResourceFilterChain.java:64)
                       at oracle.sysman.eml.app.EMRepLoginFilter.doFilter(EMRepLoginFilter.java:208)
                       at com.evermind.server.http.EvermindFilterChain.doFilter(EvermindFilterChain.java:15)
                       at oracle.sysman.db.adm.inst.HandleRepDownFilter.doFilter(HandleRepDownFilter.java:165)
                       at com.evermind.server.http.EvermindFilterChain.doFilter(EvermindFilterChain.java:17)
                       at oracle.sysman.eml.app.BrowserVersionFilter.doFilter(BrowserVersionFilter.java:122)
                       at com.evermind.server.http.EvermindFilterChain.doFilter(EvermindFilterChain.java:17)
                       at oracle.sysman.emSDK.svlt.EMRedirectFilter.doFilter(EMRedirectFilter.java:102)
                       at com.evermind.server.http.EvermindFilterChain.doFilter(EvermindFilterChain.java:17)
                       at oracle.sysman.eml.app.ContextInitFilter.doFilter(ContextInitFilter.java:357)
                       at com.evermind.server.http.ServletRequestDispatcher.invoke(ServletRequestDispatcher.java:621)
                       at com.evermind.server.http.ServletRequestDispatcher.forwardInternal(ServletRequestDispatcher.java:370)
                       at com.evermind.server.http.HttpRequestHandler.doProcessRequest(HttpRequestHandler.java:871)
                       at com.evermind.server.http.HttpRequestHandler.processRequest(HttpRequestHandler.java:453)
                       at com.evermind.server.http.HttpRequestHandler.serveOneRequest(HttpRequestHandler.java:221)
                       at com.evermind.server.http.HttpRequestHandler.run(HttpRequestHandler.java:122)
                       at com.evermind.server.http.HttpRequestHandler.run(HttpRequestHandler.java:111)
                       at oracle.oc4j.network.ServerSocketReadHandler$SafeRunnable.run(ServerSocketReadHandler.java:260)
                       at com.evermind.util.ReleasableResourcePooledExecutor$MyWorker.run(ReleasableResourcePooledExecutor.java:303)
                       at java.lang.Thread.run(Thread.java:595)
                  2012-05-11 09:49:20,521 [EMUI_09_49_19_/console/database/instance/sitemap] ERROR svlt.PageHandler handleRequest.639 - java.lang.NullPointerException
                  java.lang.NullPointerException
                       at oracle.sysman.emSDK.license.LicenseUtil.isTargetLicensedForPacks(LicenseUtil.java:1648)
                       at oracle.sysman.emo.smap.DBInstance.isDiagPackEnabled(DBInstance.java:2954)
                       at oracle.sysman.db.adm.inst.SitemapController.setRequestAttributes(SitemapController.java:1544)
                       at oracle.sysman.db.adm.BaseController.handleEvent(BaseController.java:1391)
                       at oracle.sysman.emSDK.svlt.PageHandler.handleRequest(PageHandler.java:577)
                       at oracle.sysman.db.adm.RootController.handleRequest(RootController.java:235)
                       at oracle.sysman.db.adm.BaseController.handleRequest(BaseController.java:1238)
                       at oracle.sysman.db.adm.DBControllerResolver.handleRequest(DBControllerResolver.java:139)
                       at oracle.sysman.emSDK.svlt.EMServlet.myDoGet(EMServlet.java:787)
                       at oracle.sysman.emSDK.svlt.EMServlet.doGet(EMServlet.java:343)
                       at oracle.sysman.eml.app.Console.doGet(Console.java:319)
                       at javax.servlet.http.HttpServlet.service(HttpServlet.java:743)
                       at javax.servlet.http.HttpServlet.service(HttpServlet.java:856)
                       at com.evermind.server.http.ResourceFilterChain.doFilter(ResourceFilterChain.java:64)
                       at oracle.sysman.eml.app.EMRepLoginFilter.doFilter(EMRepLoginFilter.java:208)
                       at com.evermind.server.http.EvermindFilterChain.doFilter(EvermindFilterChain.java:15)
                       at oracle.sysman.db.adm.inst.HandleRepDownFilter.doFilter(HandleRepDownFilter.java:165)
                       at com.evermind.server.http.EvermindFilterChain.doFilter(EvermindFilterChain.java:17)
                       at oracle.sysman.eml.app.BrowserVersionFilter.doFilter(BrowserVersionFilter.java:122)
                       at com.evermind.server.http.EvermindFilterChain.doFilter(EvermindFilterChain.java:17)
                       at oracle.sysman.emSDK.svlt.EMRedirectFilter.doFilter(EMRedirectFilter.java:102)
                       at com.evermind.server.http.EvermindFilterChain.doFilter(EvermindFilterChain.java:17)
                       at oracle.sysman.eml.app.ContextInitFilter.doFilter(ContextInitFilter.java:357)
                       at com.evermind.server.http.ServletRequestDispatcher.invoke(ServletRequestDispatcher.java:621)
                       at com.evermind.server.http.ServletRequestDispatcher.forwardInternal(ServletRequestDispatcher.java:370)
                       at com.evermind.server.http.HttpRequestHandler.doProcessRequest(HttpRequestHandler.java:871)
                       at com.evermind.server.http.HttpRequestHandler.processRequest(HttpRequestHandler.java:453)
                       at com.evermind.server.http.HttpRequestHandler.serveOneRequest(HttpRequestHandler.java:221)
                       at com.evermind.server.http.HttpRequestHandler.run(HttpRequestHandler.java:122)
                       at com.evermind.server.http.HttpRequestHandler.run(HttpRequestHandler.java:111)
                       at oracle.oc4j.network.ServerSocketReadHandler$SafeRunnable.run(ServerSocketReadHandler.java:260)
                       at com.evermind.util.ReleasableResourcePooledExecutor$MyWorker.run(ReleasableResourcePooledExecutor.java:303)
                       at java.lang.Thread.run(Thread.java:595)
                  2012-05-11 09:50:19,162 [MetricCollector:HOMETAB_THREAD600:60] WARN jdbc.ConnectionCache _getConnection.352 - Closed Connection: OraclePooledConnection.getConnection() - SQLException Ocurred:Invalid or Stale Connection found in the Connection Cache
                  java.sql.SQLException: Closed Connection: OraclePooledConnection.getConnection() - SQLException Ocurred:Invalid or Stale Connection found in the Connection Cache
                       at oracle.jdbc.driver.SQLStateMapping.newSQLException(SQLStateMapping.java:73)
                       at oracle.jdbc.driver.DatabaseError.newSQLException(DatabaseError.java:111)
                       at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:174)
                       at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:230)
                       at oracle.jdbc.pool.OraclePooledConnection.getConnection(OraclePooledConnection.java:467)
                       at oracle.jdbc.pool.OracleImplicitConnectionCache.getConnection(OracleImplicitConnectionCache.java:508)
                       at oracle.jdbc.pool.OracleDataSource.getConnection(OracleDataSource.java:544)
                       at oracle.jdbc.pool.OracleDataSource.getConnection(OracleDataSource.java:480)
                       at oracle.sysman.util.jdbc.ConnectionCache._getConnection(ConnectionCache.java:336)
                       at oracle.sysman.util.jdbc.ConnectionCache._getConnection(ConnectionCache.java:322)
                       at oracle.sysman.util.jdbc.ConnectionCache.getUnwrappedConnection(ConnectionCache.java:575)
                       at oracle.sysman.emSDK.svc.conn.FGAConnectionCache.getFGAConnection(FGAConnectionCache.java:207)
                       at oracle.sysman.emSDK.svc.conn.ConnectionService.getPrivateConnection(ConnectionService.java:1138)
                       at oracle.sysman.emSDK.svc.conn.ConnectionService.getPrivateConnection(ConnectionService.java:1172)
                       at oracle.sysman.emo.perf.metric.rt.DbHomeTab.getData(DbHomeTab.java:135)
                       at oracle.sysman.emo.perf.metric.eng.MetricCached.collectCachedData(MetricCached.java:416)
                       at oracle.sysman.emo.perf.metric.eng.MetricCollectorThread._collectCachedData(MetricCollectorThread.java:605)
                       at oracle.sysman.emo.perf.metric.eng.MetricCollectorThread.run(MetricCollectorThread.java:325)
                       at java.lang.Thread.run(Thread.java:595)
                  2012-05-11 09:50:19,162 [MetricCollector:HOMETAB_THREAD600:60] WARN jdbc.ConnectionCache _getConnection.353 - Got a fatal exeption when getting a connection; Error code = 17008; Cleaning up cache and retrying                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       
                  • 6. Re: Oracle enterprise manager 11g - Internal Error has occurred. Check the log
                    Aerosmith
                    Below is list of packages I installed yesterday.

                    sysstat-9.0.4-18.el6 Thu 10 May 2012 07:25:24 PM CDT
                    pam-1.1.1-10.el6_2.1 Thu 10 May 2012 04:28:56 PM CDT
                    libxml2-devel-2.7.6-4.el6_2.4 Thu 10 May 2012 04:28:56 PM CDT
                    libxml2-2.7.6-4.el6_2.4 Thu 10 May 2012 04:28:55 PM CDT
                    glibc-devel-2.12-1.47.el6_2.12 Thu 10 May 2012 04:28:55 PM CDT
                    glibc-2.12-1.47.el6_2.12 Thu 10 May 2012 04:28:55 PM CDT
                    strace-4.5.19-1.11.el6_2.1 Thu 10 May 2012 04:28:54 PM CDT
                    python-slip-gtk-0.2.20-1.el6_2 Thu 10 May 2012 04:28:54 PM CDT
                    python-slip-dbus-0.2.20-1.el6_2 Thu 10 May 2012 04:28:54 PM CDT
                    perf-2.6.32-220.13.1.el6 Thu 10 May 2012 04:28:54 PM CDT
                    libvorbis-1.2.3-4.el6_2.1 Thu 10 May 2012 04:28:54 PM CDT
                    gmp-4.3.1-7.el6_2.2 Thu 10 May 2012 04:28:54 PM CDT
                    epel-release-6-6 Thu 10 May 2012 04:28:54 PM CDT
                    java-1.6.0-openjdk-javadoc-1.6.0.0-1.43.1.10.6.el6_2 Thu 10 May 2012 04:28:53 PM CDT
                    sudo-1.7.4p5-9.el6_2 Thu 10 May 2012 04:28:39 PM CDT
                    pam-devel-1.1.1-10.el6_2.1 Thu 10 May 2012 04:28:39 PM CDT
                    ntsysv-1.3.49.3-1.el6_2 Thu 10 May 2012 04:28:39 PM CDT
                    ksh-20100621-12.el6_2.1 Thu 10 May 2012 04:28:39 PM CDT
                    irqbalance-0.55-30.el6_2 Thu 10 May 2012 04:28:39 PM CDT
                    ipmiutil-2.8.2-1.el6 Thu 10 May 2012 04:28:39 PM CDT
                    cvs-1.11.23-11.el6_2.1 Thu 10 May 2012 04:28:39 PM CDT
                    passwd-0.77-4.el6_2.2 Thu 10 May 2012 04:28:38 PM CDT
                    libxml2-devel-2.7.6-4.el6_2.4 Thu 10 May 2012 04:28:38 PM CDT
                    libpng-devel-1.2.49-1.el6_2 Thu 10 May 2012 04:28:38 PM CDT
                    libarchive-2.8.3-4.el6_2 Thu 10 May 2012 04:28:38 PM CDT
                    freetype-devel-2.3.11-6.el6_2.9 Thu 10 May 2012 04:28:38 PM CDT
                    at-3.1.10-43.el6_2.1 Thu 10 May 2012 04:28:38 PM CDT
                    libtasn1-devel-2.3-3.el6_2.1 Thu 10 May 2012 04:28:37 PM CDT
                    libsmbclient-3.5.10-116.el6_2 Thu 10 May 2012 04:28:37 PM CDT
                    libipa_hbac-python-1.5.1-66.el6_2.3 Thu 10 May 2012 04:28:37 PM CDT
                    glibc-devel-2.12-1.47.el6_2.12 Thu 10 May 2012 04:28:37 PM CDT
                    texlive-latex-2007-57.el6_2 Thu 10 May 2012 04:28:29 PM CDT
                    java-1.6.0-openjdk-devel-1.6.0.0-1.43.1.10.6.el6_2 Thu 10 May 2012 04:28:29 PM CDT
                    gnutls-devel-2.8.5-4.el6_2.2 Thu 10 May 2012 04:28:29 PM CDT
                    spice-client-0.8.2-7.el6_2.2 Thu 10 May 2012 04:28:27 PM CDT
                    nss-tools-3.13.1-7.el6_2 Thu 10 May 2012 04:28:27 PM CDT
                    nss-devel-3.13.1-7.el6_2 Thu 10 May 2012 04:28:27 PM CDT
                    libcurl-devel-7.19.7-26.el6_2.4 Thu 10 May 2012 04:28:27 PM CDT
                    libvirt-python-0.9.4-23.el6_2.8 Thu 10 May 2012 04:28:26 PM CDT
                    libvirt-devel-0.9.4-23.el6_2.8 Thu 10 May 2012 04:28:26 PM CDT
                    libvirt-0.9.4-23.el6_2.8 Thu 10 May 2012 04:28:26 PM CDT
                    yelp-2.28.1-13.el6_2 Thu 10 May 2012 04:28:25 PM CDT
                    firefox-10.0.4-1.el6_2 Thu 10 May 2012 04:28:25 PM CDT
                    yum-3.2.29-22.el6_2.2 Thu 10 May 2012 04:28:23 PM CDT
                    rpm-build-4.8.0-19.el6_2.1 Thu 10 May 2012 04:28:23 PM CDT
                    gdm-user-switch-applet-2.30.4-33.el6_2 Thu 10 May 2012 04:28:23 PM CDT
                    gdm-plugin-fingerprint-2.30.4-33.el6_2 Thu 10 May 2012 04:28:23 PM CDT
                    xorg-x11-drv-qxl-0.0.14-13.el6_2 Thu 10 May 2012 04:28:22 PM CDT
                    sos-2.2-17.el6_2.3 Thu 10 May 2012 04:28:22 PM CDT
                    ruby-ri-1.8.7.352-7.el6_2 Thu 10 May 2012 04:28:22 PM CDT
                    mysql-devel-5.1.61-1.el6_2.1 Thu 10 May 2012 04:28:22 PM CDT
                    bind-utils-9.7.3-8.P3.el6_2.2 Thu 10 May 2012 04:28:22 PM CDT
                    emacs-23.1-21.el6_2.3 Thu 10 May 2012 04:28:13 PM CDT
                    cups-1.4.2-44.el6_2.3 Thu 10 May 2012 04:28:12 PM CDT
                    sssd-1.5.1-66.el6_2.3 Thu 10 May 2012 04:28:11 PM CDT
                    qemu-kvm-0.12.1.2-2.209.el6_2.4 Thu 10 May 2012 04:28:10 PM CDT
                    nss-pam-ldapd-0.7.5-14.el6_2.1 Thu 10 May 2012 04:28:10 PM CDT
                    httpd-2.2.15-15.el6_2.1 Thu 10 May 2012 04:28:10 PM CDT
                    openssh-server-5.3p1-70.el6_2.2 Thu 10 May 2012 04:28:09 PM CDT
                    openssh-askpass-5.3p1-70.el6_2.2 Thu 10 May 2012 04:28:09 PM CDT
                    autofs-5.0.5-39.el6_2.1 Thu 10 May 2012 04:28:09 PM CDT
                    selinux-policy-targeted-3.7.19-126.el6_2.10 Thu 10 May 2012 04:27:49 PM CDT
                    samba-client-3.5.10-116.el6_2 Thu 10 May 2012 04:27:48 PM CDT
                    kexec-tools-2.0.0-209.el6_2.5 Thu 10 May 2012 04:27:47 PM CDT
                    systemtap-1.6-5.el6_2 Thu 10 May 2012 04:27:46 PM CDT
                    fail2ban-0.8.4-28.el6 Thu 10 May 2012 04:27:46 PM CDT
                    kernel-devel-2.6.32-220.13.1.el6 Thu 10 May 2012 04:27:43 PM CDT
                    samba-common-3.5.10-116.el6_2 Thu 10 May 2012 04:27:36 PM CDT
                    python-inotify-0.9.1-1.el6 Thu 10 May 2012 04:27:36 PM CDT
                    kpartx-0.4.9-46.el6_2.2 Thu 10 May 2012 04:27:36 PM CDT
                    selinux-policy-3.7.19-126.el6_2.10 Thu 10 May 2012 04:27:34 PM CDT
                    openssh-clients-5.3p1-70.el6_2.2 Thu 10 May 2012 04:27:34 PM CDT
                    systemtap-runtime-1.6-5.el6_2 Thu 10 May 2012 04:27:33 PM CDT
                    kernel-2.6.32-220.13.1.el6 Thu 10 May 2012 04:27:33 PM CDT
                    policycoreutils-2.0.83-19.21.el6_2 Thu 10 May 2012 04:27:29 PM CDT
                    openssh-5.3p1-70.el6_2.2 Thu 10 May 2012 04:27:29 PM CDT
                    dracut-kernel-004-256.el6_2.1 Thu 10 May 2012 04:27:29 PM CDT
                    dracut-004-256.el6_2.1 Thu 10 May 2012 04:27:29 PM CDT
                    net-tools-1.60-110.el6_2 Thu 10 May 2012 04:27:28 PM CDT
                    initscripts-9.03.27-1.el6_2.1 Thu 10 May 2012 04:27:28 PM CDT
                    emacs-common-23.1-21.el6_2.3 Thu 10 May 2012 04:27:27 PM CDT
                    ruby-rdoc-1.8.7.352-7.el6_2 Thu 10 May 2012 04:27:23 PM CDT
                    ruby-libs-1.8.7.352-7.el6_2 Thu 10 May 2012 04:27:23 PM CDT
                    ruby-irb-1.8.7.352-7.el6_2 Thu 10 May 2012 04:27:23 PM CDT
                    ruby-1.8.7.352-7.el6_2 Thu 10 May 2012 04:27:23 PM CDT
                    nscd-2.12-1.47.el6_2.12 Thu 10 May 2012 04:27:23 PM CDT
                    openssl-devel-1.0.0-20.el6_2.4 Thu 10 May 2012 04:27:22 PM CDT
                    httpd-tools-2.2.15-15.el6_2.1 Thu 10 May 2012 04:27:21 PM CDT
                    xorg-x11-server-Xorg-1.10.4-6.el6_2.3 Thu 10 May 2012 04:27:20 PM CDT
                    sssd-client-1.5.1-66.el6_2.3 Thu 10 May 2012 04:27:20 PM CDT
                    libxml2-python-2.7.6-4.el6_2.4 Thu 10 May 2012 04:27:20 PM CDT
                    gdm-2.30.4-33.el6_2 Thu 10 May 2012 04:27:20 PM CDT
                    bind-libs-9.7.3-8.P3.el6_2.2 Thu 10 May 2012 04:27:20 PM CDT
                    rpm-python-4.8.0-19.el6_2.1 Thu 10 May 2012 04:27:19 PM CDT
                    libxklavier-4.0-7.el6_2.1 Thu 10 May 2012 04:27:19 PM CDT
                    xulrunner-10.0.4-1.el6_2 Thu 10 May 2012 04:27:18 PM CDT
                    rpm-libs-4.8.0-19.el6_2.1 Thu 10 May 2012 04:27:18 PM CDT
                    rpm-4.8.0-19.el6_2.1 Thu 10 May 2012 04:27:18 PM CDT
                    curl-7.19.7-26.el6_2.4 Thu 10 May 2012 04:27:18 PM CDT
                    libvirt-client-0.9.4-23.el6_2.8 Thu 10 May 2012 04:27:17 PM CDT
                    nss-sysinit-3.13.1-7.el6_2 Thu 10 May 2012 04:27:16 PM CDT
                    nss-3.13.1-7.el6_2 Thu 10 May 2012 04:27:16 PM CDT
                    libcurl-7.19.7-26.el6_2.4 Thu 10 May 2012 04:27:16 PM CDT
                    java-1.6.0-openjdk-1.6.0.0-1.43.1.10.6.el6_2 Thu 10 May 2012 04:27:14 PM CDT
                    texlive-utils-2007-57.el6_2 Thu 10 May 2012 04:27:09 PM CDT
                    nss-util-devel-3.13.1-3.el6_2 Thu 10 May 2012 04:27:09 PM CDT
                    mysql-5.1.61-1.el6_2.1 Thu 10 May 2012 04:27:09 PM CDT
                    libgcrypt-devel-1.4.5-9.el6_2.2 Thu 10 May 2012 04:27:09 PM CDT
                    iptables-ipv6-1.4.7-5.1.el6_2 Thu 10 May 2012 04:27:09 PM CDT
                    gnutls-utils-2.8.5-4.el6_2.2 Thu 10 May 2012 04:27:09 PM CDT
                    ghostscript-8.70-11.el6_2.6 Thu 10 May 2012 04:27:09 PM CDT
                    qemu-img-0.12.1.2-2.209.el6_2.4 Thu 10 May 2012 04:27:07 PM CDT
                    mysql-libs-5.1.61-1.el6_2.1 Thu 10 May 2012 04:27:07 PM CDT
                    libtiff-3.9.4-5.el6_2 Thu 10 May 2012 04:27:07 PM CDT
                    libssh2-1.2.2-7.el6_2.3 Thu 10 May 2012 04:27:07 PM CDT
                    libipa_hbac-1.5.1-66.el6_2.3 Thu 10 May 2012 04:27:07 PM CDT
                    glibc-headers-2.12-1.47.el6_2.12 Thu 10 May 2012 04:27:07 PM CDT
                    gdm-libs-2.30.4-33.el6_2 Thu 10 May 2012 04:27:07 PM CDT
                    cups-libs-1.4.2-44.el6_2.3 Thu 10 May 2012 04:27:07 PM CDT
                    texlive-dvips-2007-57.el6_2 Thu 10 May 2012 04:27:06 PM CDT
                    nspr-devel-4.8.9-3.el6_2 Thu 10 May 2012 04:27:06 PM CDT
                    libtasn1-2.3-3.el6_2.1 Thu 10 May 2012 04:27:06 PM CDT
                    gnutls-2.8.5-4.el6_2.2 Thu 10 May 2012 04:27:06 PM CDT
                    texlive-2007-57.el6_2 Thu 10 May 2012 04:26:59 PM CDT
                    kpathsea-2007-57.el6_2 Thu 10 May 2012 04:26:58 PM CDT
                    samba-winbind-clients-3.5.10-116.el6_2 Thu 10 May 2012 04:26:57 PM CDT
                    nss-util-3.13.1-3.el6_2 Thu 10 May 2012 04:26:57 PM CDT
                    module-init-tools-3.9-19.el6_2 Thu 10 May 2012 04:26:57 PM CDT
                    libpng-1.2.49-1.el6_2 Thu 10 May 2012 04:26:57 PM CDT
                    libgcrypt-1.4.5-9.el6_2.2 Thu 10 May 2012 04:26:57 PM CDT
                    iptables-1.4.7-5.1.el6_2 Thu 10 May 2012 04:26:57 PM CDT
                    freetype-2.3.11-6.el6_2.9 Thu 10 May 2012 04:26:57 PM CDT
                    pam-1.1.1-10.el6_2.1 Thu 10 May 2012 04:26:56 PM CDT
                    openssl-1.0.0-20.el6_2.4 Thu 10 May 2012 04:26:56 PM CDT
                    nspr-4.8.9-3.el6_2 Thu 10 May 2012 04:26:56 PM CDT
                    libxml2-2.7.6-4.el6_2.4 Thu 10 May 2012 04:26:56 PM CDT
                    chkconfig-1.3.49.3-1.el6_2 Thu 10 May 2012 04:26:55 PM CDT
                    glibc-common-2.12-1.47.el6_2.12 Thu 10 May 2012 04:26:53 PM CDT
                    glibc-2.12-1.47.el6_2.12 Thu 10 May 2012 04:26:41 PM CDT
                    tzdata-2012b-3.el6 Thu 10 May 2012 04:26:40 PM CDT
                    bash-4.1.2-9.el6_2 Thu 10 May 2012 04:26:40 PM CDT
                    tzdata-java-2012b-3.el6 Thu 10 May 2012 04:26:38 PM CDT
                    kernel-firmware-2.6.32-220.13.1.el6 Thu 10 May 2012 04:26:38 PM CDT
                    xorg-x11-server-common-1.10.4-6.el6_2.3 Thu 10 May 2012 04:26:37 PM CDT
                    python-slip-0.2.20-1.el6_2 Thu 10 May 2012 04:26:37 PM CDT
                    kernel-headers-2.6.32-220.13.1.el6 Thu 10 May 2012 04:26:37 PM CDT
                    • 7. Re: Oracle enterprise manager 11g - Internal Error has occurred. Check the log
                      Aerosmith
                      Issue resolved by doing below,

                      "emca -config dbcontrol db -repos recreate -ORACLE_HOSTNAME hostname -SID orcl -PORT 1521 -ORACLE_HOME $ORACLE_HOME -DBDNMP_PWD sys"


                      emca -config dbcontrol db -repos recreate <--- Didn't work.

                      I can login fine in to EM site and do all kinds of stuff. Wish I had not wasted my time downgrading yesterday's updates on RHEL.