This discussion is archived
7 Replies Latest reply: Jun 12, 2012 3:31 AM by Mohan RSS

NIO - Read and Write to a ByteBuffer

Mohan Newbie
Currently Being Moderated
Warning : This is a simple question from a beginner.

When I test this code it works but I get values like '271385604' when I read the integers back. So if I write 10 I get a long number back. Does it have something to do with any error in the code that does not update the position properly. I write some values and the program exist. Later I look at the file and it has 204b bytes

channelPosition: 0 bufferPosition: 0 limit: 2048 remaining: 2048 capacity: 16.

But the value read by readInt() is '271385604' which is not what I wrote. I am basically writing an off-heap file to gather throughput and latency at high speed without overwhelming the heap. The reason is to analyze later. The format I am writing is
       writeInt( readableBytes );
       writeInt( '-' );
       writeLong( latency );
       writeInt( '\n');
How do I get back these values as they are writen ? This is a beginner's error.
public class MetricCapture implements AutoCloseable{

    private long position;

    private RandomAccessFile memoryMappedFile;

    private MappedByteBuffer out;

    public MetricCapture(){

        try{
                getNativeByteBuffer();
        }catch(Exception e ){
            throw new IllegalArgumentException( "Unable to get native buffer");
        }
    }

    public void getNativeByteBuffer() throws IOException {

        memoryMappedFile = new RandomAccessFile( "R-LatencyVsThroughput.txt",
                                                 "rw");
        out = memoryMappedFile.getChannel().map( FileChannel.MapMode.READ_WRITE,
                                                 0,
                                                 2048 );
        out.order( ByteOrder.nativeOrder() );

    }

    public void writeLong( long l ){

        out.putLong((int) position++, l);

    }

    public void writeInt( int i ){

        System.out.format( "Writing %d" , i ) ;
        out.putInt((int) position++, i);

    }

    public int readInt() {
        int i = out.getInt((int) position);
        position += 4;
        return i;
    }


    public long readLong() {
        long l = out.getLong((int) position);
        position += 8;
        return l;
    }

    public void close() throws Exception {

        memoryMappedFile.close();
    }
Edited by: Mohan on Jun 7, 2012 3:51 AM
  • 1. Re: NIO - Read and Write to a ByteBuffer
    EJP Guru
    Currently Being Moderated
    Don't bother to maintain the position. You're doing it wrongly because you're not allowing for the different sizes of what you're reading and writing, and you're doing it differently between reading and writing, but as it's all sequential it's unnecessary. Just use the APIs without the position arguments.
  • 2. Re: NIO - Read and Write to a ByteBuffer
    KevinPas Explorer
    Currently Being Moderated
    EJP is correct. But just in case you don't see it, and really want this, you need to do the same in both your read and write:

    public void writeInt( int i ){

    System.out.format( "Writing %d" , i ) ;
    position += 4;
    out.putInt((int) position, i);

    }

    public int readInt() {
    int i = out.getInt((int) position);
    position += 4;
    return i;
    }
  • 3. Re: NIO - Read and Write to a ByteBuffer
    Mohan Newbie
    Currently Being Moderated
    Reply to both of you. Thanks.

    It does work if I use the API directly. Is there a recommended to way write back the actual characters written back to an output stream like this ?

    I get bytes back but not the actual characters.
                ReadableByteChannel r = new FileInputStream( "R-LatencyVsThroughput.txt" ).getChannel();
                ByteBuffer byteBuffer = ByteBuffer.allocate(2 * 1024);
                while (r.read(byteBuffer) != -1) {
    
                   byteBuffer.flip();
                   w.write(byteBuffer);
    
                   byteBuffer.compact();
                 }
                 byteBuffer.flip();
                 while (byteBuffer.hasRemaining()) {
                   w.write( byteBuffer );
                 }
    I even introduced a CharsetDecoder

    Example snippet,
                CharsetDecoder decoder = Charset.forName( "UTF-8" ).newDecoder();
    
                    byteBuffer.flip(); // Drain mode
    
                   decoder.decode( byteBuffer,
                                   charBuffer,
                                   false );
    I get the bytes back but don't see the actual integers and characters.
  • 4. Re: NIO - Read and Write to a ByteBuffer
    EJP Guru
    Currently Being Moderated
    I don't know what this means. 'See the actual integers and characters where? If you want to see integers and chars you have to read them. All you are doing here is copying bytes. Your code will do that, although it can be simplified:
    while ((r.read(byteBuffer) != -1 || byteBuffer.hasRemaining())
    {
      byteBuffer.flip();
      w.write(byteBuffer);
      byteBuffer.compact();
    }
  • 5. Re: NIO - Read and Write to a ByteBuffer
    Mohan Newbie
    Currently Being Moderated
    It works. But I was trying to find out if I can write by calling putInt, putChar etc. so that the target stream or file has readable values.

    Something like this.
                while ( r.read(byteBuffer) != -1  ) {
    
                   byteBuffer.flip();
                   p.write(byteBuffer.getInt());
                    p.write(byteBuffer.getInt());
                    p.println(byteBuffer.getLong());
    
                   byteBuffer.compact();
                 }
                 byteBuffer.flip();
                 while (byteBuffer.hasRemaining()) {
                     p.write(byteBuffer.getInt());
                      p.write(byteBuffer.getInt());
                      p.println(byteBuffer.getLong());
                 }
    }
     
  • 6. Re: NIO - Read and Write to a ByteBuffer
    EJP Guru
    Currently Being Moderated
    You would need to check that the buffer actually contained that much data (16 bytes) before doing all those getXXX's. You could probably combine that with my version, something like this (using your first version of the desired output):
    while ((r.read(byteBuffer) != -1 || byteBuffer.hasRemaining())
    {
      byteBuffer.flip();
      if (buffer.position() >= 12)
      {
        w.println(byteBuffer.getInt()+"-"+byteBuffer.getLong());
      }
      byteBuffer.compact();
    }
  • 7. Re: NIO - Read and Write to a ByteBuffer
    Mohan Newbie
    Currently Being Moderated
    Embarrasingly once I commented
    out.order( ByteOrder.nativeOrder() );
    all my code works. It seems that I have written code correctly but was tripped by this.

    Apology.

    I did check for the 'remaining' bytes ( 12 ) but now for testing purposes allocated only what I need instead of 2 * 1024. I think that later depending on paging I needed to fix this value. I am assuming paging is a serious factor in this.

    Edited by: Mohan on Jun 12, 2012 3:29 AM

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points