This discussion is archived
9 Replies Latest reply: Sep 9, 2011 4:12 PM by ak RSS

Destination Q:mq.sys.dmq could not be found in the store

807581 Newbie
Currently Being Moderated
I tried to Google these errors but nothing really came up.
[26/Jan/2009:09:56:02 CET] ERROR [B3013]: Destination Q:mq.sys.dmq could not be found in the store
[26/Jan/2009:09:56:02 CET] WARNING Update consumer [consumer:0, type=AUTO_ACKNOWLEDGE] state failed for message 31327-10.13.1.149(ab:2d:72:8:5b:9a)-1558-1232959980070: Destination Q:mq.sys.dmq could not be found in the store
[26/Jan/2009:09:56:02 CET] ERROR Error in processing ack on 31327-10.13.1.149(ab:2d:72:8:5b:9a)-1558-1232959980070 for [consumer:5284726850505289224, type=AUTO_ACKNOWLEDGE]:
com.sun.messaging.jmq.jmsserver.util.BrokerException: Destination Q:mq.sys.dmq could not be found in the store
     at com.sun.messaging.jmq.jmsserver.persist.file.MsgStore.getDstMsgStore(MsgStore.java:777)
     at com.sun.messaging.jmq.jmsserver.persist.file.MsgStore.updateInterestState(MsgStore.java:669)
     at com.sun.messaging.jmq.jmsserver.persist.file.FileStore.updateInterestState(FileStore.java:929)
     at com.sun.messaging.jmq.jmsserver.core.PacketReference.acknowledged(PacketReference.java:1697)
     at com.sun.messaging.jmq.jmsserver.core.Session$ackEntry.acknowledged(Session.java:521)
     at com.sun.messaging.jmq.jmsserver.core.Session.ackMessage(Session.java:1309)
     at com.sun.messaging.jmq.jmsserver.core.Session.ackMessage(Session.java:1294)
     at com.sun.messaging.jmq.jmsserver.data.handlers.AckHandler.handleAcks(AckHandler.java:413)
     at com.sun.messaging.jmq.jmsserver.data.handlers.AckHandler.handle(AckHandler.java:246)
     at com.sun.messaging.jmq.jmsserver.data.PacketRouter.handleMessage(PacketRouter.java:181)
     at com.sun.messaging.jmq.jmsserver.service.imq.IMQIPConnection.readData(IMQIPConnection.java:1489)
     at com.sun.messaging.jmq.jmsserver.service.imq.IMQIPConnection.process(IMQIPConnection.java:644)
     at com.sun.messaging.jmq.jmsserver.service.imq.OperationRunnable.process(OperationRunnable.java:170)
     at com.sun.messaging.jmq.jmsserver.util.pool.BasicRunnable.run(BasicRunnable.java:493)
     at java.lang.Thread.run(Unknown Source)
[26/Jan/2009:09:56:02 CET] ERROR [B3100]: Unexpected Broker Internal Error : [-------------------------------------------Internal Error: Invalid Acknowledge Packet processing
  notifying client
      Packet: ACKNOWLEDGE(24):37103-10.13.1.149(82:29:9b:6d:53:83)-1475-1232960162481
  Magic/Version: 469754818/301     Size: 116      Type: ACKNOWLEDGE(24)
     Expiration: 0                 Timestamp: 1232960162481
      Source IP: 10.13.1.149(82:29:9b:6d:53:83)  Port: 1475     Sequence: 37103
Property Offset: 76               Property Size: 0
     Encryption: 0     Priority: 5
          Flags: A                  consumerID: 1354
   TransactionID: 0
       MessageID: 37103-10.13.1.149(82:29:9b:6d:53:83)-1475-1232960162481
      Properties: null
   Message Body: 40 bytes [5284726850505289224:31327-10.13.1.149(ab:2d:72:8:5b:9a)-1558-1232959980070]
Internal Buffers (useDirect=false):
Fixed Header Buffer:java.nio.HeapByteBuffer[pos=0 lim=72 cap=72]
--------------------------------------------]:
com.sun.messaging.jmq.jmsserver.util.BrokerException: Destination Q:mq.sys.dmq could not be found in the store
     at com.sun.messaging.jmq.jmsserver.persist.file.MsgStore.getDstMsgStore(MsgStore.java:777)
     at com.sun.messaging.jmq.jmsserver.persist.file.MsgStore.updateInterestState(MsgStore.java:669)
     at com.sun.messaging.jmq.jmsserver.persist.file.FileStore.updateInterestState(FileStore.java:929)
     at com.sun.messaging.jmq.jmsserver.core.PacketReference.acknowledged(PacketReference.java:1697)
     at com.sun.messaging.jmq.jmsserver.core.Session$ackEntry.acknowledged(Session.java:521)
     at com.sun.messaging.jmq.jmsserver.core.Session.ackMessage(Session.java:1309)
     at com.sun.messaging.jmq.jmsserver.core.Session.ackMessage(Session.java:1294)
     at com.sun.messaging.jmq.jmsserver.data.handlers.AckHandler.handleAcks(AckHandler.java:413)
     at com.sun.messaging.jmq.jmsserver.data.handlers.AckHandler.handle(AckHandler.java:246)
     at com.sun.messaging.jmq.jmsserver.data.PacketRouter.handleMessage(PacketRouter.java:181)
     at com.sun.messaging.jmq.jmsserver.service.imq.IMQIPConnection.readData(IMQIPConnection.java:1489)
     at com.sun.messaging.jmq.jmsserver.service.imq.IMQIPConnection.process(IMQIPConnection.java:644)
     at com.sun.messaging.jmq.jmsserver.service.imq.OperationRunnable.process(OperationRunnable.java:170)
     at com.sun.messaging.jmq.jmsserver.util.pool.BasicRunnable.run(BasicRunnable.java:493)
     at java.lang.Thread.run(Unknown Source)
They seem to occur at regular intervals, they occur about 500 times every minute or so, we do send thousands of messages with a timeout of 100 seconds.

Any pointers as to what we might be doing wrong would be very much appreciated, thanks.
  • 1. Re: Destination Q:mq.sys.dmq could not be found in the store
    3004 Newbie
    Currently Being Moderated
    This looks like bug 6722509. Should be fixed 4.3

    Tom
  • 2. Re: Destination Q:mq.sys.dmq could not be found in the store
    807581 Newbie
    Currently Being Moderated
    We are using 4.3 build 7-g as downloaded last Saturday.

    Do I need to provide more information, if so, which?

    We use the RefFSContextFactory, with the bindings on a share.

    OpenMQ is on one server, and two other servers connect to it using providerurl \\servername\OpenMQ\objectstoredir

    Thanks again!

    Edited by: brecht.yperman on Jan 26, 2009 6:03 AM
  • 3. Re: Destination Q:mq.sys.dmq could not be found in the store
    807581 Newbie
    Currently Being Moderated
    We don't really need the dead letter queue, so disabling the dead letter queue for all queues might be a workaround not to get these errors anymore?
  • 4. Re: Destination Q:mq.sys.dmq could not be found in the store
    3004 Newbie
    Currently Being Moderated
    Well, looks like it may be a new problem. Do you have a test case ? Could you set the property imq.log.level=DEBUGHIGH and try to reproduce the problem and then forward the broker's log file to OpenMQ mailing list users@mq.dev.java.net ?

    Tom
  • 5. Re: Destination Q:mq.sys.dmq could not be found in the store
    807581 Newbie
    Currently Being Moderated
    Unfortunately I do not have a test case.

    I have been able to reproduce it today, and I'm sending it to the mailinglist.
  • 6. Re: Destination Q:mq.sys.dmq could not be found in the store
    807581 Newbie
    Currently Being Moderated
    When disabling the dead letter queue for this queue, I get other errors:
    [30/Jan/2009:16:39:11 CET] Unable to process acknowledgement, Ignoring:
    java.lang.NullPointerException
         at com.sun.messaging.jmq.jmsserver.core.Session$ackEntry.acknowledged(Session.java:514)
         at com.sun.messaging.jmq.jmsserver.core.Session.ackMessage(Session.java:1309)
         at com.sun.messaging.jmq.jmsserver.core.Session.ackMessage(Session.java:1294)
         at com.sun.messaging.jmq.jmsserver.data.handlers.AckHandler.handleAcks(AckHandler.java:413)
         at com.sun.messaging.jmq.jmsserver.data.handlers.AckHandler.handle(AckHandler.java:246)
         at com.sun.messaging.jmq.jmsserver.data.PacketRouter.handleMessage(PacketRouter.java:181)
         at com.sun.messaging.jmq.jmsserver.service.imq.IMQIPConnection.readData(IMQIPConnection.java:1489)
         at com.sun.messaging.jmq.jmsserver.service.imq.IMQIPConnection.process(IMQIPConnection.java:644)
         at com.sun.messaging.jmq.jmsserver.service.imq.OperationRunnable.process(OperationRunnable.java:170)
         at com.sun.messaging.jmq.jmsserver.util.pool.BasicRunnable.run(BasicRunnable.java:493)
         at java.lang.Thread.run(Thread.java:595)
    [30/Jan/2009:16:39:11 CET] ERROR [B3100]: Unexpected Broker Internal Error : [-------------------------------------------Internal Error: Invalid Acknowledge Packet processing
      notifying client
          Packet: ACKNOWLEDGE(24):303-192.168.0.13(df:d9:a5:56:19:89)-2236-1233329951902
      Magic/Version: 469754818/301     Size: 116      Type: ACKNOWLEDGE(24)
         Expiration: 0                 Timestamp: 1233329951902
          Source IP: 192.168.0.13(df:d9:a5:56:19:89)  Port: 2236     Sequence: 303
    Property Offset: 76               Property Size: 0
         Encryption: 0     Priority: 5
              Flags: A                  consumerID: 75
       TransactionID: 0
           MessageID: 303-192.168.0.13(df:d9:a5:56:19:89)-2236-1233329951902
          Properties: null
       Message Body: 40 bytes [90950674893657346:27-192.168.0.13(8d:35:45:62:c8:10)-2375-1233329761245]
    Internal Buffers (useDirect=false):
    Fixed Header Buffer:java.nio.HeapByteBuffer[pos=0 lim=72 cap=72]
    --------------------------------------------]:
    com.sun.messaging.jmq.jmsserver.util.BrokerException: Unable to process ack
         at com.sun.messaging.jmq.jmsserver.core.Session$ackEntry.acknowledged(Session.java:530)
         at com.sun.messaging.jmq.jmsserver.core.Session.ackMessage(Session.java:1309)
         at com.sun.messaging.jmq.jmsserver.core.Session.ackMessage(Session.java:1294)
         at com.sun.messaging.jmq.jmsserver.data.handlers.AckHandler.handleAcks(AckHandler.java:413)
         at com.sun.messaging.jmq.jmsserver.data.handlers.AckHandler.handle(AckHandler.java:246)
         at com.sun.messaging.jmq.jmsserver.data.PacketRouter.handleMessage(PacketRouter.java:181)
         at com.sun.messaging.jmq.jmsserver.service.imq.IMQIPConnection.readData(IMQIPConnection.java:1489)
         at com.sun.messaging.jmq.jmsserver.service.imq.IMQIPConnection.process(IMQIPConnection.java:644)
         at com.sun.messaging.jmq.jmsserver.service.imq.OperationRunnable.process(OperationRunnable.java:170)
         at com.sun.messaging.jmq.jmsserver.util.pool.BasicRunnable.run(BasicRunnable.java:493)
         at java.lang.Thread.run(Thread.java:595)
    Caused by: java.lang.NullPointerException
         at com.sun.messaging.jmq.jmsserver.core.Session$ackEntry.acknowledged(Session.java:514)
         ... 10 more
    We also sometimes receive expired messages from this queue... (the consumer is on the same server as the broker, so no time sync issues here, I guess).
  • 7. Re: Destination Q:mq.sys.dmq could not be found in the store
    807581 Newbie
    Currently Being Moderated
    If we send all messages using DeliveryMode.PERSISTENT, we don't seem to get the errors...
  • 8. Re: Destination Q:mq.sys.dmq could not be found in the store
    807581 Newbie
    Currently Being Moderated
    After a while, other errors do occur though...
    [31/Jan/2009:12:10:53 CET] Unable to process acknowledgement, Ignoring:
    java.lang.NullPointerException
         at com.sun.messaging.jmq.jmsserver.core.Session$ackEntry.acknowledged(Session.java:514)
         at com.sun.messaging.jmq.jmsserver.core.Session.ackMessage(Session.java:1309)
         at com.sun.messaging.jmq.jmsserver.core.Session.ackMessage(Session.java:1294)
         at com.sun.messaging.jmq.jmsserver.data.handlers.AckHandler.handleAcks(AckHandler.java:413)
         at com.sun.messaging.jmq.jmsserver.data.handlers.AckHandler.handle(AckHandler.java:246)
         at com.sun.messaging.jmq.jmsserver.data.PacketRouter.handleMessage(PacketRouter.java:181)
         at com.sun.messaging.jmq.jmsserver.service.imq.IMQIPConnection.readData(IMQIPConnection.java:1489)
         at com.sun.messaging.jmq.jmsserver.service.imq.IMQIPConnection.process(IMQIPConnection.java:644)
         at com.sun.messaging.jmq.jmsserver.service.imq.OperationRunnable.process(OperationRunnable.java:170)
         at com.sun.messaging.jmq.jmsserver.util.pool.BasicRunnable.run(BasicRunnable.java:493)
         at java.lang.Thread.run(Thread.java:595)
    [31/Jan/2009:12:10:53 CET] ERROR [B3100]: Unexpected Broker Internal Error : [-------------------------------------------Internal Error: Invalid Acknowledge Packet processing
      notifying client
          Packet: ACKNOWLEDGE(24):46429-192.168.0.13(f1:ee:30:25:f8:65)-1274-1233400253342
      Magic/Version: 469754818/301     Size: 116      Type: ACKNOWLEDGE(24)
         Expiration: 0                 Timestamp: 1233400253342
          Source IP: 192.168.0.13(f1:ee:30:25:f8:65)  Port: 1274     Sequence: 46429
    Property Offset: 76               Property Size: 0
         Encryption: 0     Priority: 5
              Flags: A                  consumerID: 13128
       TransactionID: 0
           MessageID: 46429-192.168.0.13(f1:ee:30:25:f8:65)-1274-1233400253342
          Properties: null
       Message Body: 40 bytes [863880962091099396:8429-192.168.0.13(fe:3b:74:b9:d0:73)-1344-1233397287253]
    Internal Buffers (useDirect=false):
    Fixed Header Buffer:java.nio.HeapByteBuffer[pos=0 lim=72 cap=72]
    --------------------------------------------]:
    com.sun.messaging.jmq.jmsserver.util.BrokerException: Unable to process ack
         at com.sun.messaging.jmq.jmsserver.core.Session$ackEntry.acknowledged(Session.java:530)
         at com.sun.messaging.jmq.jmsserver.core.Session.ackMessage(Session.java:1309)
         at com.sun.messaging.jmq.jmsserver.core.Session.ackMessage(Session.java:1294)
         at com.sun.messaging.jmq.jmsserver.data.handlers.AckHandler.handleAcks(AckHandler.java:413)
         at com.sun.messaging.jmq.jmsserver.data.handlers.AckHandler.handle(AckHandler.java:246)
         at com.sun.messaging.jmq.jmsserver.data.PacketRouter.handleMessage(PacketRouter.java:181)
         at com.sun.messaging.jmq.jmsserver.service.imq.IMQIPConnection.readData(IMQIPConnection.java:1489)
         at com.sun.messaging.jmq.jmsserver.service.imq.IMQIPConnection.process(IMQIPConnection.java:644)
         at com.sun.messaging.jmq.jmsserver.service.imq.OperationRunnable.process(OperationRunnable.java:170)
         at com.sun.messaging.jmq.jmsserver.util.pool.BasicRunnable.run(BasicRunnable.java:493)
         at java.lang.Thread.run(Thread.java:595)
    Caused by: java.lang.NullPointerException
         at com.sun.messaging.jmq.jmsserver.core.Session$ackEntry.acknowledged(Session.java:514)
         ... 10 more
    This is a queue with 7 producer which produce a heartbeat every five seconds and 1 consumer reading all those heartbeats every five seconds.

    We have had so many problems with this, I'm guessing we must be doing something wrong?
  • 9. Re: Destination Q:mq.sys.dmq could not be found in the store
    ak Explorer
    Currently Being Moderated
    These appear to be caused by bugs 6886390 (fixed in 4.4.1 and later releases) and 6886391 (fixed in 4.4x and later releases)