4 Replies Latest reply: Sep 20, 2012 10:06 AM by jtahlborn RSS

    MappedByteBuffer writes garbage values into the File

    963079
      Hi,
      I'm trying to build a logging framework using 'MappedByteBuffer', for the same I tested the following code,

      package test;

      import java.io.RandomAccessFile;
      import java.nio.MappedByteBuffer;
      import java.nio.channels.FileChannel;

      public class CanExecuteTest {

      private static int count = 1024;

      public static void main(String[] args) throws Exception {
      RandomAccessFile memoryMappedFile = new RandomAccessFile("test.txt", "rw");
      FileChannel fc = memoryMappedFile.getChannel();
      // Mapping a file into memory
      MappedByteBuffer out = fc.map(FileChannel.MapMode.READ_WRITE, 0, count);

      // Writing into Memory Mapped File
      for (int i = 0; i < count / 2; i++) {
      out.put((byte) 'A');
      }
      System.out.println("Writing to Memory Mapped File is completed");
      }
      }


      In the above example if I write bytes of size ( 'i' in example) less than the mapped size(count) into the 'MappedByteBuffer' then the file shows the rest of the size as some garbage value.
      I can remove it by truncating the FileChannel with the position, but in my case if some abnormal termination occurs then again the same problem will arise (It will write the rest of bytes with garbage values).
      Is there any way to avoid this problem ?. I think while in the mapping stage itself it is writing all the bytes into the file and then overwriting these bytes, is it?
        • 1. Re: MappedByteBuffer writes garbage values into the File
          Kayaman
          960076 wrote:
          the file shows the rest of the size as some garbage value.
          It's not garbage values, it's just the values what happened to be in the memory.
          Is there any way to avoid this problem?
          Yes. Don't use memory mapping if the file size might change or you can't truncate it.

          Also, unless this is a school project you really shouldn't be building a logging framework.
          • 2. Re: MappedByteBuffer writes garbage values into the File
            963079
            Hi,
            First of all I want to inform you that mine is not a school project, and that's the reason I'm looking for such a logging framework, which should be faster and reliable.
            We have some critical logging and those should not affect the application throughput. Currently we have that inefficiency with Logback. That's why I'm here with memory mapping.
            So you are saying that I can't avoid the unwanted bytes while using the existing memory mapping ?.
            • 3. Re: MappedByteBuffer writes garbage values into the File
              EJP
              MappedByteBuffers aren't much use on a file that you need to extend: you need to close and reopen, and there is no official way of unmapping them. You'd be far better off looking at a BufferedWriter with a large buffer size.
              • 4. Re: MappedByteBuffer writes garbage values into the File
                jtahlborn
                For logging, it's tough to get speed and reliability. one option to decouple the log writing from the application throughput, however, is to move the log writing to a separate thread (aka "write behind"). change the main "log" calls to just push the log record onto a queue and use a separate thread to monitor the queue and actually write the changes to disk. logback may have such an option already built in. obviously, you lose a bit on reliability here as you may have records in memory which get lost in the event of a crash.