7 Replies Latest reply: Jun 12, 2012 5:31 AM by Mohan RSS

    NIO - Read and Write to a ByteBuffer

    Mohan
      Warning : This is a simple question from a beginner.

      When I test this code it works but I get values like '271385604' when I read the integers back. So if I write 10 I get a long number back. Does it have something to do with any error in the code that does not update the position properly. I write some values and the program exist. Later I look at the file and it has 204b bytes

      channelPosition: 0 bufferPosition: 0 limit: 2048 remaining: 2048 capacity: 16.

      But the value read by readInt() is '271385604' which is not what I wrote. I am basically writing an off-heap file to gather throughput and latency at high speed without overwhelming the heap. The reason is to analyze later. The format I am writing is
             writeInt( readableBytes );
             writeInt( '-' );
             writeLong( latency );
             writeInt( '\n');
      How do I get back these values as they are writen ? This is a beginner's error.
      public class MetricCapture implements AutoCloseable{
      
          private long position;
      
          private RandomAccessFile memoryMappedFile;
      
          private MappedByteBuffer out;
      
          public MetricCapture(){
      
              try{
                      getNativeByteBuffer();
              }catch(Exception e ){
                  throw new IllegalArgumentException( "Unable to get native buffer");
              }
          }
      
          public void getNativeByteBuffer() throws IOException {
      
              memoryMappedFile = new RandomAccessFile( "R-LatencyVsThroughput.txt",
                                                       "rw");
              out = memoryMappedFile.getChannel().map( FileChannel.MapMode.READ_WRITE,
                                                       0,
                                                       2048 );
              out.order( ByteOrder.nativeOrder() );
      
          }
      
          public void writeLong( long l ){
      
              out.putLong((int) position++, l);
      
          }
      
          public void writeInt( int i ){
      
              System.out.format( "Writing %d" , i ) ;
              out.putInt((int) position++, i);
      
          }
      
          public int readInt() {
              int i = out.getInt((int) position);
              position += 4;
              return i;
          }
      
      
          public long readLong() {
              long l = out.getLong((int) position);
              position += 8;
              return l;
          }
      
          public void close() throws Exception {
      
              memoryMappedFile.close();
          }
      Edited by: Mohan on Jun 7, 2012 3:51 AM
        • 1. Re: NIO - Read and Write to a ByteBuffer
          EJP
          Don't bother to maintain the position. You're doing it wrongly because you're not allowing for the different sizes of what you're reading and writing, and you're doing it differently between reading and writing, but as it's all sequential it's unnecessary. Just use the APIs without the position arguments.
          • 2. Re: NIO - Read and Write to a ByteBuffer
            KevinPas
            EJP is correct. But just in case you don't see it, and really want this, you need to do the same in both your read and write:

            public void writeInt( int i ){

            System.out.format( "Writing %d" , i ) ;
            position += 4;
            out.putInt((int) position, i);

            }

            public int readInt() {
            int i = out.getInt((int) position);
            position += 4;
            return i;
            }
            • 3. Re: NIO - Read and Write to a ByteBuffer
              Mohan
              Reply to both of you. Thanks.

              It does work if I use the API directly. Is there a recommended to way write back the actual characters written back to an output stream like this ?

              I get bytes back but not the actual characters.
                          ReadableByteChannel r = new FileInputStream( "R-LatencyVsThroughput.txt" ).getChannel();
                          ByteBuffer byteBuffer = ByteBuffer.allocate(2 * 1024);
                          while (r.read(byteBuffer) != -1) {
              
                             byteBuffer.flip();
                             w.write(byteBuffer);
              
                             byteBuffer.compact();
                           }
                           byteBuffer.flip();
                           while (byteBuffer.hasRemaining()) {
                             w.write( byteBuffer );
                           }
              I even introduced a CharsetDecoder

              Example snippet,
                          CharsetDecoder decoder = Charset.forName( "UTF-8" ).newDecoder();
              
                              byteBuffer.flip(); // Drain mode
              
                             decoder.decode( byteBuffer,
                                             charBuffer,
                                             false );
              I get the bytes back but don't see the actual integers and characters.
              • 4. Re: NIO - Read and Write to a ByteBuffer
                EJP
                I don't know what this means. 'See the actual integers and characters where? If you want to see integers and chars you have to read them. All you are doing here is copying bytes. Your code will do that, although it can be simplified:
                while ((r.read(byteBuffer) != -1 || byteBuffer.hasRemaining())
                {
                  byteBuffer.flip();
                  w.write(byteBuffer);
                  byteBuffer.compact();
                }
                • 5. Re: NIO - Read and Write to a ByteBuffer
                  Mohan
                  It works. But I was trying to find out if I can write by calling putInt, putChar etc. so that the target stream or file has readable values.

                  Something like this.
                              while ( r.read(byteBuffer) != -1  ) {
                  
                                 byteBuffer.flip();
                                 p.write(byteBuffer.getInt());
                                  p.write(byteBuffer.getInt());
                                  p.println(byteBuffer.getLong());
                  
                                 byteBuffer.compact();
                               }
                               byteBuffer.flip();
                               while (byteBuffer.hasRemaining()) {
                                   p.write(byteBuffer.getInt());
                                    p.write(byteBuffer.getInt());
                                    p.println(byteBuffer.getLong());
                               }
                  }
                   
                  • 6. Re: NIO - Read and Write to a ByteBuffer
                    EJP
                    You would need to check that the buffer actually contained that much data (16 bytes) before doing all those getXXX's. You could probably combine that with my version, something like this (using your first version of the desired output):
                    while ((r.read(byteBuffer) != -1 || byteBuffer.hasRemaining())
                    {
                      byteBuffer.flip();
                      if (buffer.position() >= 12)
                      {
                        w.println(byteBuffer.getInt()+"-"+byteBuffer.getLong());
                      }
                      byteBuffer.compact();
                    }
                    • 7. Re: NIO - Read and Write to a ByteBuffer
                      Mohan
                      Embarrasingly once I commented
                      out.order( ByteOrder.nativeOrder() );
                      all my code works. It seems that I have written code correctly but was tripped by this.

                      Apology.

                      I did check for the 'remaining' bytes ( 12 ) but now for testing purposes allocated only what I need instead of 2 * 1024. I think that later depending on paging I needed to fix this value. I am assuming paging is a serious factor in this.

                      Edited by: Mohan on Jun 12, 2012 3:29 AM