13c Agents Generates Many DispatchingTaskSchedule alerts

Venkata Thiruveedhi-Oracle

We have the following issue introduced in 13c.

Bug 22684160 - Provide an option to disable duplicate task checking

Email notification:

Host=host.domain.com
Target type=Agent
Target name=host.domain.com:3872
Incident creation time=Jan 31, 2016 7:23:21 AM EST
Last updated time=Jan 31, 2016 7:23:43 AM EST
Message=Internal error detected: java.lang.Throwable:oracle.sysman.gcagent.task.scheduler.DispatchingTaskScheduler:1071.
Severity=Critical

When these alerts are triggered, incidents are generated under '[Agent_Instance_Home]/diag/ofm/emagent/emagent/incident/'

Stack observed from incident dump:

java.lang.Throwable: Duplicate Scheduled Tasks
at oracle.sysman.gcagent.tmmain.diag.DiagManager.createIncident(DiagManager.java:770)
at oracle.sysman.gcagent.task.scheduler.DispatchingTaskScheduler.checkForDuplicateScheduledTasks(DispatchingTaskScheduler.java:1071)
at oracle.sysman.gcagent.tmmain.lifecycle.TMComponentSvc$SystemMonitoring$CheckForDuplicateScheduledTasksRunnable.run(TMComponentSvc.java:1998)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at oracle.sysman.gcagent.util.system.GCAThread$RunnableWrapper.run(GCAThread.java:198)
at java.lang.Thread.run(Thread.java:745)

Many customers are hitting this issue after upgrade & with new install.

This issue is fixed now and patch is included in Agent May Bundle patch.

You can fix this case by applying the below patch:

Patch 23208577: EM-AGENT Bundle Patch 13.1.0.0.160531

The following document was created for this issue,

EM 13c : Enterprise Manager 13c Cloud Control Agent Generates Many Alerts: oracle.sysman.gcagent.task.scheduler.DispatchingTaskSchedule (Doc ID 2108408.1)