After Network Outage, Clustered Targets All Status Pending
Hi
We have mostly RAC clusters, 11.2 and 12.1, most on Linux x86_64, either Red Hat 6 or Oracle Linux 6. Last Sunday our environment experienced a network outage that caused most of the hosts in the clusters to lose sight of their storage. Each host cycled several times, some needed to be cold booted, sometime the storage was detected as read-only, etc. In short, a pretty thorough mess.
Enterprise Manager (12.1.0.4, also Linux Oracle O/S) itself was affected, and after service was restored and all the agents restarted, there are now dozens of clustered targets in permanent pending status. By clustered targets, I mean those of type cluster, cluster ASM, and RAC databases. The actual hosts, ASM and database instances and listeners are all detected as up.