Hi,
I have seen OutOfMemoryError in some of the nodes of my Coherence Cluster like below. However this problem did not cause the JVM to crash as the node only used 6G out of 26G when it happened. I had to forcibly restart the node to get it re-join the cluster. The Coherence version is 12.2.1. Can anyone perhaps advise what could be the cause? Thanks very much!
ERROR Coherence - 2017-07-27 14:07:03.840/16917.262 Oracle Coherence GE 12.2.1.0.0 <Error> (thread=Invocation:Management, member=15):
(Wrapped) java.lang.OutOfMemoryError
at com.tangosol.util.Base.ensureRuntimeException(Base.java:296)
at com.tangosol.util.Base.ensureRuntimeException(Base.java:277)
at com.tangosol.coherence.component.net.MessageHandler.serializeMessage(MessageHandler.CDB:26)
at com.tangosol.coherence.component.net.MessageHandler.post(MessageHandler.CDB:36)
at com.tangosol.coherence.component.net.Message.dispatch(Message.CDB:76)
at com.tangosol.coherence.component.net.Message.post(Message.CDB:1)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.post(Grid.CDB:2)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.send(Grid.CDB:1)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.InvocationService.sendResponse(InvocationService.CDB:11)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.InvocationService$InvocationRequest.proceed(InvocationService.CDB:53)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.InvocationService.onInvocationRequest(InvocationService.CDB:22)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.InvocationService$InvocationRequest.onReceived(InvocationService.CDB:40)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.onMessage(Grid.CDB:38)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.onNotify(Grid.CDB:23)
at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:45)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError
at sun.misc.Unsafe.allocateMemory(Native Method)
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:127)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
at com.oracle.common.io.BufferManagers$DirectManagerHolder$1.allocate(BufferManagers.java:213)
at com.oracle.common.internal.io.SlabBufferManager$SlabSegment$Slab.ensure(SlabBufferManager.java:397)
at com.oracle.common.internal.io.SlabBufferManager$SlabSegment.allocateGenerationBuffers(SlabBufferManager.java:109)
at com.oracle.common.internal.io.SegmentedBufferManager$Segment.allocateGeneration(SegmentedBufferManager.java:804)
at com.oracle.common.internal.io.SegmentedBufferManager$Segment.acquireComplex(SegmentedBufferManager.java:758)
at com.oracle.common.internal.io.SegmentedBufferManager$Segment.acquire(SegmentedBufferManager.java:544)
at com.oracle.common.internal.io.SegmentedBufferManager.ensureBuffer(SegmentedBufferManager.java:365)
at com.oracle.common.internal.io.SegmentedBufferManager.acquireSum(SegmentedBufferManager.java:188)
at com.oracle.common.internal.io.WrapperBufferManager.acquireSum(WrapperBufferManager.java:58)
at com.tangosol.internal.io.BufferSequenceWriteBufferPool.allocate(BufferSequenceWriteBufferPool.java:59)
at com.tangosol.io.MultiBufferWriteBuffer.<init>(MultiBufferWriteBuffer.java:50)
at com.tangosol.coherence.component.net.MessageHandler.serializeMessage(MessageHandler.CDB:19)
... 13 more