1 Reply Latest reply on May 14, 2019 7:43 PM by user6865098

    Uncaught exception in command updateARUTables: GC overhead limit exceeded

    Vlad P.

      The "Refresh_From_My_Oracle_Support" job on enterprise manager 13c  fails with the following error: Uncaught exception in command updateARUTables: GC overhead limit exceeded

       

      My OMS_HEAP_MAX is configured for 4096M. I started with 1024M and raised it in 1024M increments up until 4096M with no luck.

       

      The emoms.log file also has the following when this happens.

       

      2019-04-18 10:48:45,690 [RJob Step 18172] ERROR em.jobs logMsg.579 - [RFMPlugIn_Cert] storedChecksum :e94e9fc30424e523c022db90eed55214 new checksum=e94e9fc30424e523c022db90eed55214 componentType=ARU_CERT_PRODUCT timestamp=2019-04-18 10:48:38.26
      2019-04-18 10:49:21,013 [RJob Step 18172] ERROR em.jobs logMsg.579 - [RFMPlugIn_Cert] storedChecksum :c41167c602102869df9a921a6562f36b new checksum=c41167c602102869df9a921a6562f36b componentType=ARU_CERT_CERTIFICATION timestamp=2019-04-18 10:48:38.26
      2019-04-18 10:54:00,174 [CCCThreadMgr:AnnotationCheckerThread] ERROR workmanager.Work logp.251 - WorkAnnotationCheckerThreadencountered severe error
      java.lang.OutOfMemoryError: GC overhead limit exceeded
      2019-04-18 10:54:41,423 [Job System Daemons:JobStatusChangeListenerService] ERROR emdrep.jobs logp.251 - Critical error occured in Job Status Change Listener id=24: 
      2019-04-18 10:54:43,323 [CCCThreadMgr:PostloadThread_DoReconciliation] WARN  jdbc.ConnectionCache _getConnection.424 - IO Error: Software caused connection abort: socket write error
      java.sql.SQLRecoverableException: IO Error: Software caused connection abort: socket write error
      at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:818)
      at oracle.jdbc.driver.PhysicalConnection.connect(PhysicalConnection.java:793)
      at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:33)
      at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:614)
      at oracle.jdbc.pool.OracleDataSource.getPhysicalConnection(OracleDataSource.java:449)
      at oracle.jdbc.pool.OracleDataSource.getConnection(OracleDataSource.java:330)
      at oracle.jdbc.pool.OracleConnectionPoolDataSource.getPhysicalConnection(OracleConnectionPoolDataSource.java:260)
      at oracle.jdbc.pool.OracleConnectionPoolDataSource.getPooledConnection(OracleConnectionPoolDataSource.java:173)
      at oracle.jdbc.pool.OracleImplicitConnectionCache.makeCacheConnection(OracleImplicitConnectionCache.java:1820)
      at oracle.jdbc.pool.OracleImplicitConnectionCache.makeOneConnection(OracleImplicitConnectionCache.java:670)
      at oracle.jdbc.pool.OracleImplicitConnectionCache.getCacheConnection(OracleImplicitConnectionCache.java:618)
      at oracle.jdbc.pool.OracleImplicitConnectionCache.getConnection(OracleImplicitConnectionCache.java:482)
      at oracle.jdbc.pool.OracleDataSource.getConnection(OracleDataSource.java:644)
      at oracle.jdbc.pool.OracleDataSource.getConnection(OracleDataSource.java:562)
      at oracle.sysman.emSDK.core.util.jdbc.ConnectionCache._getConnection(ConnectionCache.java:408)
      at oracle.sysman.emSDK.core.util.jdbc.ConnectionCache._getConnection(ConnectionCache.java:369)
      at oracle.sysman.emSDK.core.util.jdbc.ConnectionCache.getUnwrappedConnection(ConnectionCache.java:789)
      at oracle.sysman.emSDK.svc.conn.FGAConnectionCache.getFGAConnection(FGAConnectionCache.java:208)
      at oracle.sysman.emSDK.svc.conn.ConnectionService.getConnectionFromCache(ConnectionService.java:1524)
      at oracle.sysman.emSDK.svc.conn.ConnectionService.getConnectionFromCache(ConnectionService.java:1516)
      at oracle.sysman.emSDK.svc.conn.ConnectionService.getPrivateConnectionWaitccc(ConnectionService.java:1406)
      at oracle.sysman.core.gccompliance.model.backend.ccc.thread.CCCThreadBase.getDbConnection(CCCThreadBase.java:110)
      at oracle.sysman.core.gccompliance.model.backend.ccc.reconciliation.PostloadThread_DoReconciliation.doBusiness(PostloadThread_DoReconciliation.java:37)
      at oracle.sysman.core.gccompliance.model.backend.ccc.thread.CCCThreadBase.doWork(CCCThreadBase.java:35)
      at oracle.sysman.core.common.workmanager.Work.handleScheduledWork(Work.java:439)
      at oracle.sysman.core.common.workmanager.Work.call(Work.java:284)
      at oracle.sysman.core.common.workmanager.Work.call(Work.java:50)
      at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622)
      at java.lang.Thread.run(Thread.java:745)
      Caused by: java.net.SocketException: Software caused connection abort: socket write error
      at java.net.SocketOutputStream.socketWrite0(Native Method)
      at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:115)
      at java.net.SocketOutputStream.write(SocketOutputStream.java:161)
      at oracle.net.ns.DataPacket.send(DataPacket.java:210)
      at oracle.net.ns.NetOutputStream.flush(NetOutputStream.java:216)
      at oracle.net.ano.AnoComm.a(Unknown Source)
      at oracle.net.ano.Ano.negotiation(Unknown Source)
      at oracle.net.ns.NSProtocol.connect(NSProtocol.java:310)
      at oracle.jdbc.driver.T4CConnection.connect(T4CConnection.java:1608)
      at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:541)
      ... 30 more
      2019-04-18 10:54:43,911 [CCCThreadMgr:PostloadThread_DoReconciliation] WARN  jdbc.ConnectionCache _getConnection.425 - Got a fatal exeption when getting a connection; Error code = 17002; Cleaning up cache and retrying
      2019-04-18 10:54:44,687 [RJob Step 18172] WARN  httpconn.SupportConnectionFactory logp.251 - Unable to destroy support connection, ignoring Exception: Index: 0, Size: 0
      2019-04-18 10:54:44,688 [RJob Step 18172] ERROR emdrep.jobs logp.251 - [EM-00163] Error while running command/step updateARUTables/Refresh_From_My_Oracle_Support (18172) for job REFRESH_FROM_MY_ORACLE_SUPPORT_JOB:SYSMAN: GC overhead limit exceeded