This discussion is archived
1 Reply Latest reply: Mar 8, 2013 6:29 AM by Tom B RSS

weblogic.rjvm.PeerGoneException while executing JMS schedler thread(WLS8.1)

993391 Newbie
Currently Being Moderated
I have a web application deployed on weblogic server 8.1 SP4 and scheduler jobs implemented with standalone jms client which communicates with weblogic server.
Multiple threads get scheduled after every hour throughout the day.

I get the following error in server log file, occasionally after say 20 days

weblogic.rjvm.PeerGoneException: ; nested exception is:
     java.io.EOFException
     at weblogic.rjvm.BasicOutboundRequest.sendReceive(BasicOutboundRequest.java:108)
     at weblogic.rmi.cluster.ReplicaAwareRemoteRef.invoke(ReplicaAwareRemoteRef.java:290)
     at weblogic.rmi.cluster.ReplicaAwareRemoteRef.invoke(ReplicaAwareRemoteRef.java:247)
     at com.netechinc.rflow.webapp.ejbs.session.rfImportSessionEJB_fywk0g_EOImpl_814_WLStub.importFile(Unknown Source)
     at com.netechinc.rflow.webapp.transmission.Scheduler.run(Scheduler.java:140)
     at java.lang.Thread.run(Thread.java:595)
Caused by: java.io.EOFException
     at weblogic.rjvm.t3.T3JVMConnection.endOfStream(T3JVMConnection.java:874)
     at weblogic.socket.SocketMuxer.deliverExceptionAndCleanup(SocketMuxer.java:583)
     at weblogic.socket.SocketMuxer.deliverEndOfStream(SocketMuxer.java:533)
     at weblogic.socket.SocketMuxer.readReadySocketOnce(SocketMuxer.java:691)
     at weblogic.socket.SocketMuxer.readReadySocket(SocketMuxer.java:638)
     at weblogic.socket.JavaSocketMuxer.processSockets(JavaSocketMuxer.java:282)
     at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:32)
     at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:219)

<BEA-000513> <Failure in heartbeat trigger for RJVM: 7876994111581326024C:172.30.29.84R:-6676648650237697870S:ucabclg1pp:[7001,7001,-1,-1,7001,-1,-1,0,0]:rflowdomain:rflowserver
java.io.IOException: The connection manager to ConnectionManager for: 'weblogic.rjvm.RJVMImpl@15eb24c - id: '7876994111581326024C:172.30.29.84R:-6676648650237697870S:ucabclg1pp:[7001,7001,-1,-1,7001,-1,-1,0,0]:rflowdomain:rflowserver' connect time: 'Mon Feb 18 02:15:02 MST 2013'' has already been shut down.
java.io.IOException: The connection manager to ConnectionManager for: 'weblogic.rjvm.RJVMImpl@15eb24c - id: '7876994111581326024C:172.30.29.84R:-6676648650237697870S:ucabclg1pp:[7001,7001,-1,-1,7001,-1,-1,0,0]:rflowdomain:rflowserver' connect time: 'Mon Feb 18 02:15:02 MST 2013'' has already been shut down
     at weblogic.rjvm.ConnectionManager.getOutputStream(ConnectionManager.java:1589)
     at weblogic.rjvm.ConnectionManager.createHeartbeatMsg(ConnectionManager.java:1538)
     at weblogic.rjvm.ConnectionManager.sendHeartbeatMsg(ConnectionManager.java:555)
     at weblogic.rjvm.RJVMImpl$HeartbeatChecker.trigger(RJVMImpl.java:1496)
     at weblogic.time.common.internal.ScheduledTrigger.run(ScheduledTrigger.java:243)
     at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
     at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:121)
     at weblogic.time.common.internal.ScheduledTrigger.executeLocally(ScheduledTrigger.java:229)
     at weblogic.time.common.internal.ScheduledTrigger.execute(ScheduledTrigger.java:223)
     at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:219)
     at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:178)

Also get the following exception for some threads

Could not establish a connection with -6676648650237697870S:ucabclg1pp:[7001,7001,-1,-1,7001,-1,-1,0,0]:rflowdomain:rflowserver, java.io.IOException: Destination unreachable; nested exception is:
     java.io.IOException: Empty server reply; No available router to destination
java.rmi.ConnectException: Could not establish a connection with -6676648650237697870S:ucabclg1pp:[7001,7001,-1,-1,7001,-1,-1,0,0]:rflowdomain:rflowserver, java.io.IOException: Destination unreachable; nested exception is:
     java.io.IOException: Empty server reply; No available router to destination
     at weblogic.rjvm.RJVMImpl.getOutputStream(RJVMImpl.java:376)
     at weblogic.rjvm.RJVMImpl.getRequestStream(RJVMImpl.java:551)
     at weblogic.rjvm.RJVMImpl.getOutboundRequest(RJVMImpl.java:683)
     at weblogic.rmi.internal.BasicRemoteRef.getOutboundRequest(BasicRemoteRef.java:90)
     at weblogic.rmi.cluster.ReplicaAwareRemoteRef.invoke(ReplicaAwareRemoteRef.java:282)
     at weblogic.rmi.cluster.ReplicaAwareRemoteRef.invoke(ReplicaAwareRemoteRef.java:247)
     at com.netechinc.rflow.webapp.ejbs.session.rfExportSessionEJB_gkn6xe_EOImpl_814_WLStub.export(Unknown Source)
     at com.netechinc.rflow.webapp.transmission.Scheduler.run(Scheduler.java:127)
     at java.lang.Thread.run(Thread.java:595)

All the scheduled threads throw the above exception and scheduled job get scheduled to next interval after the exception.

I have checked if its network failure, but was nt.

how to fix this issue.

Please advise, thanx
  • 1. Re: weblogic.rjvm.PeerGoneException while executing JMS schedler thread(WLS8.1)
    Tom B Expert
    Currently Being Moderated
    I don't see any JMS in the stack. The "Destination" in the exception in this case is not a JMS destination, but an IP address that an EJB stub is attempting to use. The exception may be normal -- perhaps your EJB code is trying to connect to a WebLogic Server that is down. If you think all servers are up, then I think you have two basic things to look into that might help (1) closing your JNDI context and looking up your EJB again, or (2) checking if your cluster is is healthy (I think there are FAQs for cluster troubleshooting - at least in more recent versions of the doc -- 8.1 is many years old now...).

    If you need further help, I recommend trying to post to an EJB newsgroup or some-such.

    Hope this helps,

    Tom

    c

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points