OutOfMemoryError from DirectByteBuffer

Zyd08

Hi,

I have seen OutOfMemoryError in some of the nodes of my Coherence Cluster like below. However this problem did not cause the JVM to crash as the node only used 6G out of 26G when it happened. I had to forcibly restart the node to get it re-join the cluster. The Coherence version is 12.2.1. Can anyone perhaps advise what could be the cause? Thanks very much!

ERROR Coherence - 2017-07-27 14:07:03.840/16917.262 Oracle Coherence GE 12.2.1.0.0 <Error> (thread=Invocation:Management, member=15):

(Wrapped) java.lang.OutOfMemoryError

  at com.tangosol.util.Base.ensureRuntimeException(Base.java:296)

  at com.tangosol.util.Base.ensureRuntimeException(Base.java:277)

  at com.tangosol.coherence.component.net.MessageHandler.serializeMessage(MessageHandler.CDB:26)

  at com.tangosol.coherence.component.net.MessageHandler.post(MessageHandler.CDB:36)

  at com.tangosol.coherence.component.net.Message.dispatch(Message.CDB:76)

  at com.tangosol.coherence.component.net.Message.post(Message.CDB:1)

  at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.post(Grid.CDB:2)

  at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.send(Grid.CDB:1)

  at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.InvocationService.sendResponse(InvocationService.CDB:11)

  at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.InvocationService$InvocationRequest.proceed(InvocationService.CDB:53)

  at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.InvocationService.onInvocationRequest(InvocationService.CDB:22)

  at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.InvocationService$InvocationRequest.onReceived(InvocationService.CDB:40)

  at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.onMessage(Grid.CDB:38)

  at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.onNotify(Grid.CDB:23)

  at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:45)

  at java.lang.Thread.run(Thread.java:745)

Caused by: java.lang.OutOfMemoryError

  at sun.misc.Unsafe.allocateMemory(Native Method)

  at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:127)

  at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)

  at com.oracle.common.io.BufferManagers$DirectManagerHolder$1.allocate(BufferManagers.java:213)

  at com.oracle.common.internal.io.SlabBufferManager$SlabSegment$Slab.ensure(SlabBufferManager.java:397)

  at com.oracle.common.internal.io.SlabBufferManager$SlabSegment.allocateGenerationBuffers(SlabBufferManager.java:109)

  at com.oracle.common.internal.io.SegmentedBufferManager$Segment.allocateGeneration(SegmentedBufferManager.java:804)

  at com.oracle.common.internal.io.SegmentedBufferManager$Segment.acquireComplex(SegmentedBufferManager.java:758)

  at com.oracle.common.internal.io.SegmentedBufferManager$Segment.acquire(SegmentedBufferManager.java:544)

  at com.oracle.common.internal.io.SegmentedBufferManager.ensureBuffer(SegmentedBufferManager.java:365)

  at com.oracle.common.internal.io.SegmentedBufferManager.acquireSum(SegmentedBufferManager.java:188)

  at com.oracle.common.internal.io.WrapperBufferManager.acquireSum(WrapperBufferManager.java:58)

  at com.tangosol.internal.io.BufferSequenceWriteBufferPool.allocate(BufferSequenceWriteBufferPool.java:59)

  at com.tangosol.io.MultiBufferWriteBuffer.<init>(MultiBufferWriteBuffer.java:50)

  at com.tangosol.coherence.component.net.MessageHandler.serializeMessage(MessageHandler.CDB:19)

  ... 13 more

Tmiddlet-Oracle

Please upgrade to latest 12.2.1.0.X patch-set which is currently 12.2.1.0.3 (Oracle Coherence Java 12.2.1.0.3)

as there is a known issues related to DirectByteBuffer fixed in 12.2.1.0.2.

If you are able to, the latest coherence version is 12.2.1.2.0 which will include this fix as well.

Thanks

Tim

Zyd08

Thanks very much for your answer. However I don't seem to be able to download the patch pointed by your link, the warning says "You do not have the required access privilege to download this patch." How would I be able to access it please?

Tmiddlet-Oracle

You need to login to your Oracle support account at support.oracle.com.

Thanks

Tim

Zyd08

We have actually managed to install the latest version 12.2.1.2.0. Thanks very much!

Tmiddlet-Oracle

Thats great.

Did that resolve your problem?

Zyd08

Sorry for the late reply. Yes version upgrade has resolved the problem.

Tmiddlet-Oracle

Thanks, Glad to hear.