This content has been marked as final. Show 7 replies
Oracle Support has now requested LMS stacktraces during the problematic period. Moreover, I found out that the "GC Average CR/Curr Get Time" is more than 50-500 ms instead of 1-2 ms during normal operation.
Perhaps this is interconnect issue ? Maybe those commands reveal something interesting (spikes, fragmentations, etc. ) ?
ip -s link
sar -n DEV
sar -n EDEV
sar -n SOCK
I've read somewhere that DRM can be deactivated per object_id (via oradebug lkdebug or something similiar), not database-wide.
Unfortunately, there is no possibility to deactivate DRM on an object level.
Interconnect behaviour is normal.
Oracle Support has requested stacktraces of lms processes during the period of performance degradation. We decided to enable OSWatcher to get systemwide linux data and procwatcher to get lms process stacktraces. We created a Grid Control User Defined Metric to check whether the symptoms of a DRM performance problem is taking place. Then we triggered the lms stacktraces with a Grid Control Response Action script of the UDM.
Oracle Support has also requested global hanganalyze and system state dumps but we decided not to collect system state dumps because of the big additional performance impact.
The oswatcher data showed that during the drm period, the lms processes had very high CPU resource utilization.
In the meantime Oracle Support has confirmed that we are hitting 6960699. We have received patch 8516675 which includes the bugfix and have installed it. Now, we are waiting to see whether this indeed fixes the issue.
We had the same issue on a 2-node 10.2.0.4 RAC (HP-UX).
2 error 481 instance crashes and GRD freezes due to DRM operations.
The biggest one recorded, lasted 48 minutes.
In our case, the object being remastered was not always the same and not relevant to its size.
For instance, the 48 minute freeze was caused by a DRM on a 800mb table, and the database contains tables in the scale of Terabytes (12 TB total db size).
Since, the local Oracle branch and Metalink had not suggested any patches, we decided to disable DRM and, since then, these issues have not reappeared.
Interesting; How do you switch off DRM - Ive been digging about in Metalink but cant find anything?
You can disable DRM using below settings however it's not advisable until Oracle support is asking specifically to do it.
To disable DRM, set
Please check metalink note 401627.1.