This discussion is archived
6 Replies Latest reply: Mar 10, 2011 4:24 PM by EJP RSS

FileChannel.write(ByteBuffer[]) only writes out the first 16 ByteBuffer

846113 Newbie
Currently Being Moderated
Hi,
This does NOT happen on Windows. I've only had this problem on Solaris (10 u8).
I've tried this in jdk1.4, 1.5 and 1.6. All showed the same behavior on Solaris.
I tried to write bytes to a file using FileChannel.write(ByteBuffer[]) method.
I noticed that, if the ByteBuffer array contains more than 16 ByteBuffer, the FileChannel.write(ByteBuffer[] src) method only writes out contents of the first 16 ByteBuffer (src[0] thru src[15]).
It worked fine if I combined all bytes into a single (big) ByteBuffer. (FileChannel.write(ByteBuffer) works fine)
Has anyone experienced similar problem? Is it a bug in the JRE?

Thanks.
  • 1. Re: FileChannel.write(ByteBuffer[]) only writes out the first 16 ByteBuffer
    EJP Guru
    Currently Being Moderated
    That's probably the value of IOV_MAX in <limits.h> in Solaris, and if so you are stuck with it. See man writev(1), which is the system call underlying this Java method. As it's specified to fail if the count is too large I speculate that Java caps it for you. Whether it should really do that, or loop, or fail itself, is another question.

    Never seen anybody use 16 buffers to write to a file before!
  • 2. Re: FileChannel.write(ByteBuffer[]) only writes out the first 16 ByteBuffer
    846113 Newbie
    Currently Being Moderated
    Thank you very much for the response. That might explain the behavior. I was just trying to optimize some existing code and found that write(ByteBuffer[]) is available in the API.
    The code currently calls FileChannel.write(ByteBuffer) for each ByteBuffer. I was hoping I could batch it up by calling the write(ByteBuffer[]) method (to reduce the number of writes).
  • 3. Re: FileChannel.write(ByteBuffer[]) only writes out the first 16 ByteBuffer
    EJP Guru
    Currently Being Moderated
    It does indeed reduce the number of writes, but if they are above about 1k each there is no significant saving. You would get the same effect with much simpler coding via e.g. new BufferedOutputStream(new FileOutputStream(..));
  • 4. Re: FileChannel.write(ByteBuffer[]) only writes out the first 16 ByteBuffer
    800381 Explorer
    Currently Being Moderated
    shawnstc wrote:
    Hi,
    This does NOT happen on Windows. I've only had this problem on Solaris (10 u8).
    I've tried this in jdk1.4, 1.5 and 1.6. All showed the same behavior on Solaris.
    I tried to write bytes to a file using FileChannel.write(ByteBuffer[]) method.
    I noticed that, if the ByteBuffer array contains more than 16 ByteBuffer, the FileChannel.write(ByteBuffer[] src) method only writes out contents of the first 16 ByteBuffer (src[0] thru src[15]).
    It worked fine if I combined all bytes into a single (big) ByteBuffer. (FileChannel.write(ByteBuffer) works fine)
    Has anyone experienced similar problem? Is it a bug in the JRE?

    Thanks.
    You do check and properly handle partial success for every single write you do, correct?
  • 5. Re: FileChannel.write(ByteBuffer[]) only writes out the first 16 ByteBuffer
    846113 Newbie
    Currently Being Moderated
    Yes. Based on the returned value (i.e, the number of bytes actually written), "partial success" can be handled. Thanks.
  • 6. Re: FileChannel.write(ByteBuffer[]) only writes out the first 16 ByteBuffer
    EJP Guru
    Currently Being Moderated
    The correct way to do that is something like this:
    long count = 0;
    ByteBuffer[] buffers; // buffers to write, assumed to be unflipped at this point
    long max; // maximum data to transfer, i.e. sum of available() over buffers
    do
    {
      for (ByteBuffer buffer : buffers)
      {
        if (offset+1 < length && buffer.available() == 0)
          offset++;
        buffer.flip();
      }
      try
      {
        count += fch.write(buffers, offset, length);
      }
      finally
      {
        for (ByteBuffer buffer : buffers)
          buffer.compact();
      }
    } while (count < max);
    Warning: totally untested. But you get the idea. It caters for partial writes and also for the case where the number of iovecs is capped as appears to be happening here.

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points