This discussion is archived
1 2 Previous Next 27 Replies Latest reply: Feb 17, 2013 4:32 PM by EJP Go to original post RSS
  • 15. Re: Need an example of read a file into ByteArrayInputStream
    EJP Guru
    Currently Being Moderated
    I don't know why you're asking. You're the one who wants the ByteArrayInputStream. Only you know why. If you don't want the whole file in memory you don't needn't at all. Just use the FileInputStream. I've already said all this.
  • 16. Re: Need an example of read a file into ByteArrayInputStream
    baftos Expert
    Currently Being Moderated
    After reading all this thread, I understand that the 3rd party API (can we know what that API is?) requires an InputStream that supports mark() and reset(). Their reason is probably that, once you give them the InputStream, they need to mark() and reset() it for their own reasons. You decided that ByteArrayInputStream is the solution. Yes, it needs the whole file in memory, which limits you to 2GB files. But, according to DrClap, if I understand well, there is a better solution: PushbackInputStream. You construct it around your FileInputStream, give it to the API and the 3rd party code takes care of the rest. I have no hands-on experience with PushbackInputStream, but it looks to me that it should do the job. If this did not work for you, it would be nice if you could tell us why.
  • 17. Re: Need an example of read a file into ByteArrayInputStream
    EJP Guru
    Currently Being Moderated
    Or BufferedInputStream.
  • 18. Re: Need an example of read a file into ByteArrayInputStream
    jtahlborn Expert
    Currently Being Moderated
    baftos wrote:
    After reading all this thread, I understand that the 3rd party API (can we know what that API is?) requires an InputStream that supports mark() and reset(). Their reason is probably that, once you give them the InputStream, they need to mark() and reset() it for their own reasons. You decided that ByteArrayInputStream is the solution. Yes, it needs the whole file in memory, which limits you to 2GB files. But, according to DrClap, if I understand well, there is a better solution: PushbackInputStream. You construct it around your FileInputStream, give it to the API and the 3rd party code takes care of the rest. I have no hands-on experience with PushbackInputStream, but it looks to me that it should do the job. If this did not work for you, it would be nice if you could tell us why.
    PushbackInputStream does not support mark/reset, so i'm not sure what that has been offered as a solution.

    (as EJP mentioned) BufferedInputStream is one of the few InputStreams which does, however it is limited. until the OP indicates the real need, it's tough to give more advice (other than how to properly read the entire file into memory).
  • 19. Re: Need an example of read a file into ByteArrayInputStream
    DrClap Expert
    Currently Being Moderated
    jtahlborn wrote:
    PushbackInputStream does not support mark/reset, so i'm not sure what that has been offered as a solution.
    I offered it as a solution because I thought it was one. But I was wrong, because now that I look at the API docs, they explicitly say it doesn't support mark/reset. Sorry about that.
    (as EJP mentioned) BufferedInputStream is one of the few InputStreams which does, however it is limited. until the OP indicates the real need, it's tough to give more advice (other than how to properly read the entire file into memory).
    In which case the solution is to just wrap the existing InputStream, whatever it is, with a BufferedInputStream. And yes, it would be helpful for the OP to explain the real problem.
  • 20. Re: Need an example of read a file into ByteArrayInputStream
    EJP Guru
    Currently Being Moderated
    A BufferedInputStream can reset() back to a point as far back as the buffer size, which is a parameter to the constructor, by default 8192 in the current source code, although not specified.

    But if you are dealing with an API that requires mark() and reset() take my word for it, it is poorly designed. If you can build compilers for programming languages without more than one byte of pushback, which you certainly can, and which has been done since the 1960s, you can analyze any simpler protocol without it as well.
  • 21. Re: Need an example of read a file into ByteArrayInputStream
    Aacc Newbie
    Currently Being Moderated
    Here is stituation:
    all I need to do is pass a File as stream to an API method. the API take InputStream as parameter.
    However, the API implementation IS poor but this is situation I have to deal with. in the implemetation, it call reset()

    Ideally, I expect to pass a file as stream without loading entire file into memory.

    But this sounds imposible

    So far I've read following ideas through this discussion

    -- BufferredStream. it can only reset() back to a point as far back as the buffer size. unless I make huge buffer which basically loading entire file into buffer(memeory)
    -- BatyArryInputStream - no matter how to construct this this, e.g. File->bufferredStream -> ByteArrayOutputStream -> ByteArryInputStream. it still need to take entire file into memory


    Any better way I can deal with the situation?
    Thanks
  • 22. Re: Need an example of read a file into ByteArrayInputStream
    jtahlborn Expert
    Currently Being Moderated
    Aacc wrote:
    Here is stituation:
    all I need to do is pass a File as stream to an API method. the API take InputStream as parameter.
    However, the API implementation IS poor but this is situation I have to deal with. in the implemetation, it call reset()

    Ideally, I expect to pass a file as stream without loading entire file into memory.

    But this sounds imposible

    So far I've read following ideas through this discussion

    -- BufferredStream. it can only reset() back to a point as far back as the buffer size. unless I make huge buffer which basically loading entire file into buffer(memeory)
    -- BatyArryInputStream - no matter how to construct this this, e.g. File->bufferredStream -> ByteArrayOutputStream -> ByteArryInputStream. it still need to take entire file into memory


    Any better way I can deal with the situation?
    here's the million dollar question (oft repeated in this thread). why does this API method call reset(). if the answer is "no good reason", then just use a custom subclass of FileInputStream which overrides the mark/reset methods to do nothing. if there is a good reason, you can implement a version of FileInputStream which supports mark/reset using a FileChannel.
  • 23. Re: Need an example of read a file into ByteArrayInputStream
    baftos Expert
    Currently Being Moderated
    Although the FileChannel may be the best solution (honestly I don't know how this works), here is two other possibilities:
    1 - Ask them how far back they need to reset(). Do theye really go to the begining of the file, or just a few bytes here and there? Is this predictable or not?
    2 - If they don't answer, decompile their code and, if not too obfuscated, try to get the answers by yourself.
    What I propose is not great, as implementations may change, but we are stuck anyway, so you may give it a try.

    Edited by: baftos on Feb 17, 2013 1:17 PM
  • 24. Re: Need an example of read a file into ByteArrayInputStream
    Aacc Newbie
    Currently Being Moderated
    jtahlborn,
    Could you please give more detail about how to "implement a version of FileInputStream which supports mark/reset using a FileChannel"?
  • 25. Re: Need an example of read a file into ByteArrayInputStream
    EJP Guru
    Currently Being Moderated
    I don't normally do this, but:
    import java.io.File;
    import java.io.FileDescriptor;
    import java.io.FileInputStream;
    import java.io.IOException;
    
    public class MarkableFileInputStream extends FileInputStream
    {
         private long     mark = -1;
         
         public MarkableFileInputStream(File file) throws IOException
         {
              super(file);
         }
         
         public MarkableFileInputStream(FileDescriptor fd) throws IOException
         {
              super(fd);
         }
         
         public MarkableFileInputStream(String file) throws IOException
         {
              super(file);
         }
         
         @Override
         public void     mark(int readLimit)
         {
              try
              {
                   this.mark = getChannel().position();
              }
              catch (IOException exc)
              {
                   throw new RuntimeException(exc);
              }
         }
         
         @Override
         public boolean     markSupported()
         {
              return true;
         }
         
         @Override
         public void     reset() throws IOException
         {
              if (mark >= 0)
              {
                   super.getChannel().position(mark);
              }
              else
              {
                   throw new IOException("stream not marked");
              }
         }
    }
    Errors & omissions excepted.

    Edited by: EJP on 18/02/2013 11:32
  • 26. Re: Need an example of read a file into ByteArrayInputStream
    jtahlborn Expert
    Currently Being Moderated
    yep, that's basically it, but it's even simpler than that. the FileChannel already tracks the position, so you don't need to override the read methods, and the mark method is simply:
         @Override
         public void     mark(int readLimit)
         {
              this.mark = getChannel().position();
         }
  • 27. Re: Need an example of read a file into ByteArrayInputStream
    EJP Guru
    Currently Being Moderated
    Almost. You have to catch IOException. I'll edit the post above.
1 2 Previous Next

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points