被管服务器与控制台频繁失联 — oracle-tech

    Forum Stats

  • 3,708,676 Users
  • 2,241,108 Discussions
  • 7,840,528 Comments

Discussions

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

被管服务器与控制台频繁失联

user10758122user10758122 Posts: 21 Blue Ribbon
edited July 2019 in 中间件

被管服务器与控制台频繁失联。然后业务处理变慢,控制台也无法打开,只能重启失联的节点才能恢复。但是过了一段时间后又会在不同的节点出现同样的问题。

日志里显示服务超时,同时也在做垃圾回收,线程快照里有大量的stuck线程,都在wait:0x0000000402d424d0;

"[ACTIVE] ExecuteThread: '932' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon prio=10 tid=0x00007ff0f48bd000 nid=0x95fc waiting for monitor entry [0x00007fec6e4a5000]

   java.lang.Thread.State: BLOCKED (on object monitor)

at weblogic.diagnostics.archive.wlstore.PersistentStoreDataArchive.insertRecord(PersistentStoreDataArchive.java:841)

- waiting to lock <0x0000000402d424d0> (a weblogic.diagnostics.archive.wlstore.HarvestedPersistentStoreDataArchive)

at weblogic.diagnostics.archive.wlstore.PersistentStoreDataArchive.writeData(PersistentStoreDataArchive.java:1488)

at weblogic.diagnostics.harvester.internal.MetricArchiver.archive(MetricArchiver.java:581)

at weblogic.diagnostics.harvester.internal.MetricArchiver.access$000(MetricArchiver.java:54)

at weblogic.diagnostics.harvester.internal.MetricArchiver$1.run(MetricArchiver.java:612)

at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:548)

at weblogic.work.ExecuteThread.execute(ExecuteThread.java:311)

at weblogic.work.ExecuteThread.run(ExecuteThread.java:263)

然后这把锁0x0000000402d424d0锁在了

"[ACTIVE] ExecuteThread: '958' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon prio=10 tid=0x00007ff0f48f3000 nid=0x9616 runnable [0x00007fec6ca8a000]

   java.lang.Thread.State: RUNNABLE

at java.lang.System.arraycopy(Native Method)

at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:146)

at weblogic.utils.io.ByteBufferInputStream.read(ByteBufferInputStream.java:42)

at java.io.ObjectInputStream$PeekInputStream.read(ObjectInputStream.java:2310)

at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2323)

at java.io.ObjectInputStream$BlockDataInputStream.readInt(ObjectInputStream.java:2818)

at java.io.ObjectInputStream.readHandle(ObjectInputStream.java:1452)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1330)

at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)

at weblogic.diagnostics.accessor.DataRecord.readExternal(DataRecord.java:58)

at java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:1837)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)

at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)

at weblogic.diagnostics.archive.wlstore.Snapshot.readExternal(Snapshot.java:64)

at java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:1837)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)

at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)

at weblogic.utils.io.ByteBufferObjectInputStream.readObject(ByteBufferObjectInputStream.java:38)

at weblogic.diagnostics.archive.CustomObjectHandler.readObject(CustomObjectHandler.java:20)

at weblogic.store.internal.PersistentStoreRecordImpl.initBody(PersistentStoreRecordImpl.java:99)

at weblogic.store.internal.PersistentStoreRecordImpl.getData(PersistentStoreRecordImpl.java:125)

at weblogic.diagnostics.archive.wlstore.PersistentStoreDataArchive.readRecord(PersistentStoreDataArchive.java:696)

- locked <0x0000000402d424d0> (a weblogic.diagnostics.archive.wlstore.HarvestedPersistentStoreDataArchive)

at weblogic.diagnostics.archive.wlstore.PersistentStoreDataArchive.readRecordObject(PersistentStoreDataArchive.java:674)

at weblogic.diagnostics.archive.wlstore.RecordWrapper.getDataObject(RecordWrapper.java:70)

at weblogic.diagnostics.archive.wlstore.PersistentStoreDataArchive.getWrapper(PersistentStoreDataArchive.java:1772)

at weblogic.diagnostics.archive.wlstore.PersistentStoreDataArchive.removeGarbageInPage(PersistentStoreDataArchive.java:1813)

- locked <0x0000000402d424d0> (a weblogic.diagnostics.archive.wlstore.HarvestedPersistentStoreDataArchive)

at weblogic.diagnostics.archive.wlstore.PersistentStoreDataArchive.cleanupPages(PersistentStoreDataArchive.java:1697)

at weblogic.diagnostics.archive.wlstore.PersistentStoreDataArchive.cleanup(PersistentStoreDataArchive.java:1645)

at weblogic.diagnostics.archive.wlstore.PersistentStoreDataArchive.deleteDataRecords(PersistentStoreDataArchive.java:1361)

at weblogic.diagnostics.archive.wlstore.PersistentStoreDataArchive.retireOldestRecords(PersistentStoreDataArchive.java:1211)

at weblogic.diagnostics.archive.DataRetirementByQuotaTaskImpl.performDataRetirement(DataRetirementByQuotaTaskImpl.java:92)

at weblogic.diagnostics.archive.DataRetirementByQuotaTaskImpl.run(DataRetirementByQuotaTaskImpl.java:49)

at weblogic.diagnostics.archive.DataRetirementTaskImpl.run(DataRetirementTaskImpl.java:261)

at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:548)

at weblogic.work.ExecuteThread.execute(ExecuteThread.java:311)

at weblogic.work.ExecuteThread.run(ExecuteThread.java:263)

Sign In or Register to comment.