This discussion is archived
11 Replies Latest reply: Jul 16, 2011 7:04 PM by EJP RSS

Any way to stream a TextMessage? OutofMemory exception...

361137 Newbie
Currently Being Moderated
I am receiving TextMessage from another application, but sometimes when I attempt to read the text the size of the string causes my application to fail on OutOfMemory error.

Ideally, I should be able to stream that textmessage sort of like reading from an InputStream.

Is this at all possible?
  • 1. Re: Any way to stream a TextMessage? OutofMemory exception...
    EJP Guru
    Currently Being Moderated
    Not unless you can find it in the API. I couldn't.

    This would have to be a seriously large message, hundreds of megabytes. Is the sender really sending you something that big?
  • 2. Re: Any way to stream a TextMessage? OutofMemory exception...
    874155 Newbie
    Currently Being Moderated
    You can try using ByteArrayInputStream like for TextMessage you can use it as shown below:

    ByteArrayInputStream stream = new ByteArrayInputStream(message.getText().getBytes());


    Thanks and Regards,
    -------------------------------------------------------------------------------------
    Nilesh Sahni | REMOVED
    -------------------------------------------------------------------------------------
  • 3. Re: Any way to stream a TextMessage? OutofMemory exception...
    EJP Guru
    Currently Being Moderated
    And how exactly is that going to solve an OutOfMemoryError?

    As a matter of fact it is twice as likely to cause one as just calling Message.getText(), because it requires both the String result of getText() and the byte[] array resulting from getBytes() to exist at the same time.

    Please.
  • 4. Re: Any way to stream a TextMessage? OutofMemory exception...
    gimbal2 Guru
    Currently Being Moderated
    Nilesh wrote:
    You can try using ByteArrayInputStream like for TextMessage you can use it as shown below:

    ByteArrayInputStream stream = new ByteArrayInputStream(message.getText().getBytes());


    Thanks and Regards,
    -------------------------------------------------------------------------------------
    Nilesh Sahni | REMOVED | REMOVED
    -------------------------------------------------------------------------------------
    Not only are you about as wrong as you possibly can be (the hard truth, no offense intended), but posting your email address in a public forum really is not a smart thing to do, unless you really want spam.
  • 5. Re: Any way to stream a TextMessage? OutofMemory exception...
    gimbal2 Guru
    Currently Being Moderated
    trant wrote:
    Ideally, I should be able to stream that textmessage sort of like reading from an InputStream.
    It makes no sense to be able to do that. We're talking about messages here. A message by definition should be compact. If you try to force a pilot's flight manual in one go you'll be doing it wrong.

    In this case, the message could have been "Hey system, I have a big chunk of data that you can fetch at your own leasure through a mutually agreed interface -here-" in stead of "hey system, here is a big chunk of data for you! BLURB".

    Note: this is following the assumption that this is about huge messages and not a badly configured system environment / memory leakage.
  • 6. Re: Any way to stream a TextMessage? OutofMemory exception...
    EJP Guru
    Currently Being Moderated
    A message by definition should be compact.
    ++. You have to be able to process the entire message at once and form a reply. If somebody really is sending you hundreds of megabytes in a message they are using the wrong technology. If they're not, you have a bug somewhere.
  • 7. Re: Any way to stream a TextMessage? OutofMemory exception...
    874671 Newbie
    Currently Being Moderated
    I think it should be dealt more of logically. Read only prefered fixed sized chunk at a time using buffered streams.
    You can use pagination techniques to display message in part by part but shouldn't hold all the message in the memory.

    We might never need to hold entire message in memory, even we can't watch the whole message once. On some event (ex: scroll down) u can get the remaining part of the message.

    Or still you have good configuration system you can increase the JVM memory settings using Java command.

    Edited by: skakita on Jul 14, 2011 2:39 AM

    Edited by: skakita on Jul 14, 2011 2:41 AM
  • 8. Re: Any way to stream a TextMessage? OutofMemory exception...
    EJP Guru
    Currently Being Moderated
    I think it should be dealt more of logically.
    Read only prefered fixed sized chunk at a time using buffered streams.
    There is nothing 'more logical' about that. It's not a stream, it's a message. It is designed to be retrieved via TextMessage.getText(). If it can't, it must be too big, by hundreds of megabytes. There is something seriously wrong with a system that behaves that way.
    You can use pagination techniques to display message in part by part
    Assuming the only purpose of the message is to be displayed, and assuming a pagination system can cope with hundreds ofmegabytes of message. These assumptions are preposterous.
    but shouldn't hold all the message in the memory.
    Why not? You have to act on the entire message. The last message-based system I worked on was a rewrite of a 20-year-old system whose messages were 200 bytes. Not 200 megabytes. 200MB is not a message. It's certainly not a text message.
    We might never need to hold entire message in memory
    Or we might. Is this your idea of 'more logical'?
    even we can't watch the whole message once. On some event (ex: scroll down) u can get the remaining part of the message.
    You seem to think (a) that the purpose of JMS messages is to be displayed in scrolling GUIs and (b) that a scrolling GUI can cope with hundreds of megabytes of data.

    You are violently mistaken. And not 'more logical'.
  • 9. Re: Any way to stream a TextMessage? OutofMemory exception...
    nigeldeakin Explorer
    Currently Being Moderated
    As others have said, the only way to obtain the payload of a TextMessage is to call getText(), which returns the whole String on one go.

    There are other JMS message types, BytesMessage or StreamMessage, which do provide methods to read the payload in chunks. Obviously if the incoming message is a TextMessage and you can't change that then this isn't any help to you.

    But also note that even if are able to use a BytesMessage or StreamMessage, it is up to the JMS provider whether they handle very large messages in a memory-efficient manner (i.e. by not holding the entire message in memory on the client).

    One other thought: many JMS servers don't send messages to a consuming client one at a time. Instead they send multiple messages and save them in the JMS client prior to them actually being delivered to the application. This is done to improve performance, but increases the memory usage on the client. If your messages are large you may want to tune your provider so that the number of messages it pre-sends to the consuming client doesn't cause your client JVM to run out of memory. Obviously, if the size of the message is greater than your client heap size this won't help.

    Nigel
  • 10. Re: Any way to stream a TextMessage? OutofMemory exception...
    361137 Newbie
    Currently Being Moderated
    Thanks for all the replies gentlemen.

    Yes it is a very large message, it is over 100 MB.

    We agreed to try transmitting data this way because it is much more convenient then going through all the painful hassles of setting up permissions and accounts for a more "traditional" way of transmitting this data such as FTP
  • 11. Re: Any way to stream a TextMessage? OutofMemory exception...
    EJP Guru
    Currently Being Moderated
    Much more convenient except that it doesn't work, makes inordinate demands on memory, and exhibits a major misunderstanding of the technology.

    I suggest you either break it up or use a streaming technology, such as sockets or indeed the dreaded FTP.

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points