0 Replies Latest reply on Apr 8, 2019 6:05 AM by MoinAhmed

    Massage deliver issue from MQ to Weblogic

    MoinAhmed

      Hi All,

       

      Following are the Environment Details:

       

      Websphere MQ server Details:

      Name: WebSphere MQ

      Version: 7.5.0.7

       

      WebLogic 12.2.1.3.0:

      we have setup a bridge between IBM Websphere MQ to WebLogic with "TestDQ" distributed queue. The source and target destinations are setup as eis.jms.WLSConnectionFactoryJNDIXA adapters. Start-up the bridge connects to both destinations without problems.

      we had send a massage at <Apr 4, 2019 4:26:42,896 PM PKT> from Websphere MQ to Weblogic in distributed Queue, after that we are getting an continuously error message in both manage servers logs.

      you can see an error in attached log files COM_MS1.log and COM_MS2

       

      please help me on this issue

       

      COM_MS1.log

       

      Description

      --------------

      Error during test

       

      Relevant Information Collection

      ---------------------------------------

      ####<Apr 4, 2019 4:25:51,578 PM PKT> <Info> <WorkManager> <OSM_Test_APP> <COM_MS1> <Timer-2> <<WLS Kernel>> <> <02cf2452-406c-4838-a6e1-98f9ecd2abbc-00000004> <1554377151578> <[severity-value: 64] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-002959> <Self-tuning thread pool contains 2 running threads, 3 idle threads, and 54 standby threads> 

      ####<Apr 4, 2019 4:26:42,896 PM PKT> <Debug> <MessagingBridgeRuntime> <OSM_Test_APP> <COM_MS1> <[ACTIVE] ExecuteThread: '21' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <BEA1-2B3D18448726B3DB3271> <02cf2452-406c-4838-a6e1-98f9ecd2abbc-00000f8f> <1554377202896> <[severity-value: 128] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-000000> <Bridge: MQToOSM-UATBridge@COM_MS1 (processMessages()) received message: 

        JMS Message Class: TextMessage

        JMSMessageID: ID:414d51205054434c514d3031202020205c9a16a5200fc702

        JMSCorrelationID: null

        JMSDeliveryMode: PERSISTENT

        JMSDestination: null

        JMSExpiration: 0

        JMSPriority: 0

        JMSRedelivered: false

        JMSReplyTo: null

        JMSTimestamp: 1554376513620 (Thu Apr 04 16:15:13 PKT 2019)

        JMSType: null

        Transaction Id: BEA1-2B3D18448726B3DB3271

        Hello to the future

      isWLS?false> 

      ####<Apr 4, 2019 4:27:52,033 PM PKT> <Info> <WorkManager> <OSM_Test_APP> <COM_MS1> <Timer-2> <<WLS Kernel>> <> <02cf2452-406c-4838-a6e1-98f9ecd2abbc-00000004> <1554377272033> <[severity-value: 64] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-002959> <Self-tuning thread pool contains 2 running threads, 3 idle threads, and 54 standby threads> 

      ####<Apr 4, 2019 4:29:52,490 PM PKT> <Info> <WorkManager> <OSM_Test_APP> <COM_MS1> <Timer-2> <<WLS Kernel>> <> <02cf2452-406c-4838-a6e1-98f9ecd2abbc-00000004> <1554377392490> <[severity-value: 64] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-002959> <Self-tuning thread pool contains 3 running threads, 2 idle threads, and 54 standby threads> 

      ####<Apr 4, 2019 4:30:40,860 PM PKT> <Error> <MessagingBridge> <OSM_Test_APP> <COM_MS1> <[ACTIVE] ExecuteThread: '21' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <BEA1-2B3D18448726B3DB3271> <02cf2452-406c-4838-a6e1-98f9ecd2abbc-00001136> <1554377440860> <[severity-value: 8] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-200015> <An error occurred in bridge "MQToOSM-UATBridge@COM_MS1" during the transfer of messages (java.lang.Exception: javax.resource.ResourceException: Error creating producer or sending message

        at weblogic.jms.adapter.JMSBaseConnection.throwResourceException(JMSBaseConnection.java:1750)

        at weblogic.jms.adapter.JMSBaseConnection.sendInternal(JMSBaseConnection.java:977)

        at weblogic.jms.adapter.JMSBaseConnection.access$300(JMSBaseConnection.java:92)

        at weblogic.jms.adapter.JMSBaseConnection$9.run(JMSBaseConnection.java:847)

        at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:370)

        at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:163)

        at weblogic.jms.adapter.JMSBaseConnection.send(JMSBaseConnection.java:844)

        at weblogic.jms.adapter.JMSConnectionHandle.send(JMSConnectionHandle.java:144)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)

        at java.lang.reflect.Method.invoke(Method.java:508)

        at weblogic.connector.security.layer.AdapterLayer.invoke(AdapterLayer.java:1474)

        at weblogic.connector.outbound.ConnectionWrapper.invoke(ConnectionWrapper.java:227)

        at com.sun.proxy.$Proxy367.send(Unknown Source)

        at weblogic.jms.bridge.internal.MessagingBridge.sendWithWorkContext(MessagingBridge.java:1735)

        at weblogic.jms.bridge.internal.MessagingBridge.processMessages(MessagingBridge.java:1565)

        at weblogic.jms.bridge.internal.MessagingBridge.beginForwarding(MessagingBridge.java:1079)

        at weblogic.jms.bridge.internal.MessagingBridge.run(MessagingBridge.java:1160)

        at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:678)

        at weblogic.invocation.ComponentInvocationContextManager._runAs(ComponentInvocationContextManager.java:352)

        at weblogic.invocation.ComponentInvocationContextManager.runAs(ComponentInvocationContextManager.java:337)

        at weblogic.work.LivePartitionUtility.doRunWorkUnderContext(LivePartitionUtility.java:57)

        at weblogic.work.PartitionUtility.runWorkUnderContext(PartitionUtility.java:41)

        at weblogic.work.SelfTuningWorkManagerImpl.runWorkUnderContext(SelfTuningWorkManagerImpl.java:652)

        at weblogic.work.ExecuteThread.execute(ExecuteThread.java:420)

        at weblogic.work.ExecuteThread.run(ExecuteThread.java:360)

      -------------- Linked Exception ------------

      weblogic.jms.common.JMSException: weblogic.messaging.kernel.KernelException: Error enlisting GXA transaction

        at weblogic.jms.dispatcher.DispatcherAdapter.convertToJMSExceptionAndThrow(DispatcherAdapter.java:127)

        at weblogic.jms.dispatcher.DispatcherAdapter.dispatchSyncTran(DispatcherAdapter.java:60)

        at weblogic.messaging.dispatcher.CrossPartitionDispatcher.dispatchSyncTran(CrossPartitionDispatcher.java:58)

        at weblogic.jms.client.JMSProducer.toFEProducer(JMSProducer.java:1549)

        at weblogic.jms.client.JMSProducer.deliveryInternalUnified(JMSProducer.java:943)

        at weblogic.jms.client.JMSProducer.sendInternal(JMSProducer.java:614)

        at weblogic.jms.client.JMSProducer.sendWithListenerUnified(JMSProducer.java:433)

        at weblogic.jms.client.JMSProducer.sendWithListener(JMSProducer.java:421)

        at weblogic.jms.client.JMSProducer.send(JMSProducer.java:414)

        at weblogic.jms.client.WLProducerImpl.send(WLProducerImpl.java:974)

        at weblogic.jms.adapter.JMSBaseConnection.sendInternal(JMSBaseConnection.java:949)

        at weblogic.jms.adapter.JMSBaseConnection.access$300(JMSBaseConnection.java:92)

        at weblogic.jms.adapter.JMSBaseConnection$9.run(JMSBaseConnection.java:847)

        at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:370)

        at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:163)

        at weblogic.jms.adapter.JMSBaseConnection.send(JMSBaseConnection.java:844)

        at weblogic.jms.adapter.JMSConnectionHandle.send(JMSConnectionHandle.java:144)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)

        at java.lang.reflect.Method.invoke(Method.java:508)

        at weblogic.connector.security.layer.AdapterLayer.invoke(AdapterLayer.java:1474)

        at weblogic.connector.outbound.ConnectionWrapper.invoke(ConnectionWrapper.java:227)

        at com.sun.proxy.$Proxy367.send(Unknown Source)

        at weblogic.jms.bridge.internal.MessagingBridge.sendWithWorkContext(MessagingBridge.java:1735)

        at weblogic.jms.bridge.internal.MessagingBridge.processMessages(MessagingBridge.java:1565)

        at weblogic.jms.bridge.internal.MessagingBridge.beginForwarding(MessagingBridge.java:1079)

        at weblogic.jms.bridge.internal.MessagingBridge.run(MessagingBridge.java:1160)

        at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:678)

        at weblogic.invocation.ComponentInvocationContextManager._runAs(ComponentInvocationContextManager.java:352)

        at weblogic.invocation.ComponentInvocationContextManager.runAs(ComponentInvocationContextManager.java:337)

        at weblogic.work.LivePartitionUtility.doRunWorkUnderContext(LivePartitionUtility.java:57)

        at weblogic.work.PartitionUtility.runWorkUnderContext(PartitionUtility.java:41)

        at weblogic.work.SelfTuningWorkManagerImpl.runWorkUnderContext(SelfTuningWorkManagerImpl.java:652)

        at weblogic.work.ExecuteThread.execute(ExecuteThread.java:420)

        at weblogic.work.ExecuteThread.run(ExecuteThread.java:360)

      Caused by: weblogic.jms.common.JMSException: weblogic.messaging.kernel.KernelException: Error enlisting GXA transaction

        at weblogic.jms.dispatcher.Request.handleThrowable(Request.java:99)

        at weblogic.jms.dispatcher.Request.getResult(Request.java:64)

        at weblogic.messaging.dispatcher.Request.wrappedFiniteStateMachine(Request.java:1175)

        at weblogic.messaging.dispatcher.DispatcherImpl.syncRequest(DispatcherImpl.java:243)

        at weblogic.messaging.dispatcher.DispatcherImpl.dispatchSyncTran(DispatcherImpl.java:296)

        at weblogic.jms.dispatcher.DispatcherAdapter.dispatchSyncTran(DispatcherAdapter.java:58)

        ... 34 more

      Caused by: weblogic.jms.common.JMSException: weblogic.messaging.kernel.KernelException: Error enlisting GXA transaction

        at weblogic.messaging.dispatcher.Request.wrappedFiniteStateMachine(Request.java:1124)

        ... 37 more

      Caused by: weblogic.jms.common.JMSException: weblogic.messaging.kernel.KernelException: Error enlisting GXA transaction

        at weblogic.jms.dispatcher.Request.handleThrowable(Request.java:99)

        at weblogic.jms.dispatcher.Request.getResult(Request.java:64)

        at weblogic.jms.frontend.FEProducer.sendRetryDestination(FEProducer.java:1166)

        at weblogic.jms.frontend.FEProducer.send(FEProducer.java:1590)

        at weblogic.jms.frontend.FEProducer.invoke(FEProducer.java:1664)

        at weblogic.messaging.dispatcher.Request.wrappedFiniteStateMachine(Request.java:1015)

        ... 37 more

      Caused by: weblogic.jms.common.JMSException: weblogic.messaging.kernel.KernelException: Error enlisting GXA transaction

        at weblogic.messaging.dispatcher.Request.getResult(Request.java:513)

        at weblogic.jms.dispatcher.Request.getResult(Request.java:62)

        ... 41 more

      Caused by: weblogic.jms.common.JMSException: weblogic.messaging.kernel.KernelException: Error enlisting GXA transaction

        at weblogic.utils.StackTraceDisabled.unknownMethod()

      Caused by: weblogic.messaging.kernel.KernelException: Error enlisting GXA transaction

        ... 1 more

      Caused by: weblogic.store.gxa.GXAException: Transaction already rolled back, or the transaction manager has timed out this resource.

        ... 1 more

      Caused by: weblogic.transaction.RollbackException: Tx marked rollback due to unexpected internal exception

      start() failed on resource 'eis/jms/internal/WLSConnectionFactoryJNDIXA': XAER_PROTO : Routine was invoked in an improper context

      javax.transaction.xa.XAException: The method 'xa_start' has failed with errorCode '-6'.

        at com.ibm.mq.jmqi.JmqiXAResource.start(JmqiXAResource.java:946)

        at weblogic.connector.security.layer.AdapterLayer.start(AdapterLayer.java:573)

        at weblogic.connector.transaction.outbound.XAWrapper.start(XAWrapper.java:517)

        at weblogic.connector.transaction.outbound.RecoveryOnlyXAWrapper.start(RecoveryOnlyXAWrapper.java:35)

        at weblogic.transaction.internal.XAServerResourceInfo.start(XAServerResourceInfo.java:1395)

        at weblogic.transaction.internal.XAServerResourceInfo.xaStart(XAServerResourceInfo.java:1326)

        at weblogic.transaction.internal.XAServerResourceInfo.enlist(XAServerResourceInfo.java:298)

        at weblogic.transaction.internal.XAServerResourceInfo.enlistIfStatic(XAServerResourceInfo.java:197)

        at weblogic.transaction.internal.ServerTransactionImpl.enlistStaticallyEnlistedResources(ServerTransactionImpl.java:1962)

        at weblogic.transaction.internal.ServerTransactionManagerImpl.dispatchRequest(ServerTransactionManagerImpl.java:557)

        at weblogic.messaging.dispatcher.DispatcherServerRef.handleRequest(DispatcherServerRef.java:301)

        at weblogic.messaging.dispatcher.DispatcherServerRef.access$000(DispatcherServerRef.java:41)

        at weblogic.messaging.dispatcher.DispatcherServerRef$2.run(DispatcherServerRef.java:238)

        at weblogic.invocation.ComponentInvocationContextManager._runAs(ComponentInvocationContextManager.java:352)

        at weblogic.invocation.ComponentInvocationContextManager.runAs(ComponentInvocationContextManager.java:337)

        at weblogic.work.LivePartitionUtility.doRunWorkUnderContext(LivePartitionUtility.java:57)

        at weblogic.work.PartitionUtility.runWorkUnderContext(PartitionUtility.java:41)

        at weblogic.work.SelfTuningWorkManagerImpl.runWorkUnderContext(SelfTuningWorkManagerImpl.java:652)

        at weblogic.work.ExecuteThread.execute(ExecuteThread.java:420)

        at weblogic.work.ExecuteThread.run(ExecuteThread.java:360)

       

        ... 1 more

      Caused by: javax.transaction.SystemException: start() failed on resource 'eis/jms/internal/WLSConnectionFactoryJNDIXA': XAER_PROTO : Routine was invoked in an improper context

      javax.transaction.xa.XAException: The method 'xa_start' has failed with errorCode '-6'.

        at com.ibm.mq.jmqi.JmqiXAResource.start(JmqiXAResource.java:946)

        at weblogic.connector.security.layer.AdapterLayer.start(AdapterLayer.java:573)

        at weblogic.connector.transaction.outbound.XAWrapper.start(XAWrapper.java:517)

        at weblogic.connector.transaction.outbound.RecoveryOnlyXAWrapper.start(RecoveryOnlyXAWrapper.java:35)

        at weblogic.transaction.internal.XAServerResourceInfo.start(XAServerResourceInfo.java:1395)

        at weblogic.transaction.internal.XAServerResourceInfo.xaStart(XAServerResourceInfo.java:1326)

        at weblogic.transaction.internal.XAServerResourceInfo.enlist(XAServerResourceInfo.java:298)

        at weblogic.transaction.internal.XAServerResourceInfo.enlistIfStatic(XAServerResourceInfo.java:197)

        at weblogic.transaction.internal.ServerTransactionImpl.enlistStaticallyEnlistedResources(ServerTransactionImpl.java:1962)

        at weblogic.transaction.internal.ServerTransactionManagerImpl.dispatchRequest(ServerTransactionManagerImpl.java:557)

        at weblogic.messaging.dispatcher.DispatcherServerRef.handleRequest(DispatcherServerRef.java:301)

        at weblogic.messaging.dispatcher.DispatcherServerRef.access$000(DispatcherServerRef.java:41)

        at weblogic.messaging.dispatcher.DispatcherServerRef$2.run(DispatcherServerRef.java:238)

        at weblogic.invocation.ComponentInvocationContextManager._runAs(ComponentInvocationContextManager.java:352)

        at weblogic.invocation.ComponentInvocationContextManager.runAs(ComponentInvocationContextManager.java:337)

        at weblogic.work.LivePartitionUtility.doRunWorkUnderContext(LivePartitionUtility.java:57)

        at weblogic.work.PartitionUtility.runWorkUnderContext(PartitionUtility.java:41)

        at weblogic.work.SelfTuningWorkManagerImpl.runWorkUnderContext(SelfTuningWorkManagerImpl.java:652)

        at weblogic.work.ExecuteThread.execute(ExecuteThread.java:420)

        at weblogic.work.ExecuteThread.run(ExecuteThread.java:360)

       

        ... 1 more

      -------------- Linked Exception 2 ------------

      weblogic.jms.common.JMSException: weblogic.messaging.kernel.KernelException: Error enlisting GXA transaction

        at weblogic.jms.dispatcher.Request.handleThrowable(Request.java:99)

        at weblogic.jms.dispatcher.Request.getResult(Request.java:64)

        at weblogic.messaging.dispatcher.Request.wrappedFiniteStateMachine(Request.java:1175)

        at weblogic.messaging.dispatcher.DispatcherImpl.syncRequest(DispatcherImpl.java:243)

        at weblogic.messaging.dispatcher.DispatcherImpl.dispatchSyncTran(DispatcherImpl.java:296)

        at weblogic.jms.dispatcher.DispatcherAdapter.dispatchSyncTran(DispatcherAdapter.java:58)

        at weblogic.messaging.dispatcher.CrossPartitionDispatcher.dispatchSyncTran(CrossPartitionDispatcher.java:58)

        at weblogic.jms.client.JMSProducer.toFEProducer(JMSProducer.java:1549)

        at weblogic.jms.client.JMSProducer.deliveryInternalUnified(JMSProducer.java:943)

        at weblogic.jms.client.JMSProducer.sendInternal(JMSProducer.java:614)

        at weblogic.jms.client.JMSProducer.sendWithListenerUnified(JMSProducer.java:433)

        at weblogic.jms.client.JMSProducer.sendWithListener(JMSProducer.java:421)

        at weblogic.jms.client.JMSProducer.send(JMSProducer.java:414)

        at weblogic.jms.client.WLProducerImpl.send(WLProducerImpl.java:974)

        at weblogic.jms.adapter.JMSBaseConnection.sendInternal(JMSBaseConnection.java:949)

        at weblogic.jms.adapter.JMSBaseConnection.access$300(JMSBaseConnection.java:92)

        at weblogic.jms.adapter.JMSBaseConnection$9.run(JMSBaseConnection.java:847)

        at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:370)

        at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:163)

        at weblogic.jms.adapter.JMSBaseConnection.send(JMSBaseConnection.java:844)

        at weblogic.jms.adapter.JMSConnectionHandle.send(JMSConnectionHandle.java:144)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)

        at java.lang.reflect.Method.invoke(Method.java:508)

        at weblogic.connector.security.layer.AdapterLayer.invoke(AdapterLayer.java:1474)

        at weblogic.connector.outbound.ConnectionWrapper.invoke(ConnectionWrapper.java:227)

        at com.sun.proxy.$Proxy367.send(Unknown Source)

        at weblogic.jms.bridge.internal.MessagingBridge.sendWithWorkContext(MessagingBridge.java:1735)

        at weblogic.jms.bridge.internal.MessagingBridge.processMessages(MessagingBridge.java:1565)

        at weblogic.jms.bridge.internal.MessagingBridge.beginForwarding(MessagingBridge.java:1079)

        at weblogic.jms.bridge.internal.MessagingBridge.run(MessagingBridge.java:1160)

        at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:678)

        at weblogic.invocation.ComponentInvocationContextManager._runAs(ComponentInvocationContextManager.java:352)

        at weblogic.invocation.ComponentInvocationContextManager.runAs(ComponentInvocationContextManager.java:337)

        at weblogic.work.LivePartitionUtility.doRunWorkUnderContext(LivePartitionUtility.java:57)

        at weblogic.work.PartitionUtility.runWorkUnderContext(PartitionUtility.java:41)

        at weblogic.work.SelfTuningWorkManagerImpl.runWorkUnderContext(SelfTuningWorkManagerImpl.java:652)

        at weblogic.work.ExecuteThread.execute(ExecuteThread.java:420)

        at weblogic.work.ExecuteThread.run(ExecuteThread.java:360)

      Caused by: weblogic.jms.common.JMSException: weblogic.messaging.kernel.KernelException: Error enlisting GXA transaction

        at weblogic.messaging.dispatcher.Request.wrappedFiniteStateMachine(Request.java:1124)

        ... 37 more

      Caused by: weblogic.jms.common.JMSException: weblogic.messaging.kernel.KernelException: Error enlisting GXA transaction

        at weblogic.jms.dispatcher.Request.handleThrowable(Request.java:99)

        at weblogic.jms.dispatcher.Request.getResult(Request.java:64)

        at weblogic.jms.frontend.FEProducer.sendRetryDestination(FEProducer.java:1166)

        at weblogic.jms.frontend.FEProducer.send(FEProducer.java:1590)

        at weblogic.jms.frontend.FEProducer.invoke(FEProducer.java:1664)

        at weblogic.messaging.dispatcher.Request.wrappedFiniteStateMachine(Request.java:1015)

        ... 37 more

      Caused by: weblogic.jms.common.JMSException: weblogic.messaging.kernel.KernelException: Error enlisting GXA transaction

        at weblogic.messaging.dispatcher.Request.getResult(Request.java:513)

        at weblogic.jms.dispatcher.Request.getResult(Request.java:62)

        ... 41 more

      Caused by: weblogic.jms.common.JMSException: weblogic.messaging.kernel.KernelException: Error enlisting GXA transaction

        at weblogic.utils.StackTraceDisabled.unknownMethod()

      Caused by: weblogic.messaging.kernel.KernelException: Error enlisting GXA transaction

        ... 1 more

      Caused by: weblogic.store.gxa.GXAException: Transaction already rolled back, or the transaction manager has timed out this resource.

        ... 1 more