Forum Stats

  • 3,768,730 Users
  • 2,252,841 Discussions
  • 7,874,701 Comments

Discussions

OutOfMemoryError when writing?

nitroamos
nitroamos Member Posts: 5
edited Oct 5, 2016 12:41PM in Berkeley DB Java Edition

I've incorporated je berkeleydb into my application and it seems to be working very well. Today a user reported the exception I pasted below. As far as I can tell from the log file, heap memory was available. I've pasted my config below, too.

Any ideas?

java.lang.OutOfMemoryError: null

  at java.io.RandomAccessFile.writeBytes0(Native Method) ~[na:1.7.0_45]

  at java.io.RandomAccessFile.writeBytes(Unknown Source) ~[na:1.7.0_45]

  at java.io.RandomAccessFile.write(Unknown Source) ~[na:1.7.0_45]

  at com.sleepycat.je.log.FileManager.writeToFile(FileManager.java:1749) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.je.log.FileManager.writeLogBuffer(FileManager.java:1663) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.je.log.LogBufferPool.writeBufferToFile(LogBufferPool.java:449) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.je.log.LogBufferPool.writeDirty(LogBufferPool.java:408) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.je.log.LogManager.log(LogManager.java:365) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.je.tree.LN.logInternal(LN.java:728) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.je.tree.LN.optionalLog(LN.java:447) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.je.dbi.CursorImpl.insertRecordInternal(CursorImpl.java:1478) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.je.dbi.CursorImpl.insertOrUpdateRecord(CursorImpl.java:1280) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.je.Cursor.putNoNotify(Cursor.java:2504) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.je.Cursor.putNotify(Cursor.java:2365) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.je.Cursor.putNoDups(Cursor.java:2223) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.je.Cursor.putInternal(Cursor.java:2060) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.je.Cursor.put(Cursor.java:730) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.util.keyrange.RangeCursor.put(RangeCursor.java:1066) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.collections.DataCursor.put(DataCursor.java:788) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.collections.StoredContainer.putKeyValue(StoredContainer.java:359) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.collections.StoredMap.put(StoredMap.java:315) ~[je-6.4.25.jar:6.4.25]

   private void initDb(File file) throws IOException {

      log.info("opening cache " + file);

      EnvironmentConfig envConfig = new EnvironmentConfig();

      envConfig.setTransactional(false);

      envConfig.setAllowCreate(true);

      DbInternal.disableParameterValidation(envConfig);

      envConfig.setConfigParam(EnvironmentConfig.CLEANER_MIN_UTILIZATION,"90");

      envConfig.setConfigParam(EnvironmentConfig.LOG_FILE_MAX,"10000000");

      envConfig.setDurability(Durability.COMMIT_NO_SYNC);

      envConfig.setCacheSize(128000000);

      env = new Environment(file,envConfig);

      DatabaseConfig dbConfig = new DatabaseConfig();

      dbConfig.setTransactional(false);

      dbConfig.setAllowCreate(true);

      Database catalogDb = env.openDatabase(null,CLASS_CATALOG,dbConfig);

      classCatalog = new StoredClassCatalog(catalogDb);

      db = env.openDatabase(null,"mydatastore",dbConfig);

      // setup binding

      EntryBinding<K> keyBinding = new SerialBinding<K>(classCatalog,keyType);

      EntryBinding<V> valueBinding = new SerialBinding<V>(classCatalog,

            valueType);

      keyValueMap = new StoredMap<K, V>(db,keyBinding,valueBinding,true);

   }

Answers

  • Greybird-Oracle
    Greybird-Oracle Member Posts: 2,690
    edited Oct 5, 2016 12:23PM

    Since the OOME is in RandomAccessFile.writeBytes0(Native Method), the system must overall must be low on memory. In other words, the OOME doesn't seem to be due to lack of Java heap space and therefore something else (another process?) is filling memory.

    However, just in case the error is misleading, you should ensure that your heap size is around 2X the size of the JE cache. You didn't say what heap size you're using.

    --mark

  • nitroamos
    nitroamos Member Posts: 5
    edited Oct 5, 2016 12:32PM

    Thanks, Greybird-Oracle.

    Here's the memory read out when the exception occurred:

    memory: 403 free, 870 max, 870 total

    for freeMemory, maxMemory, totalMemory respectively.

    So I think my heap is sufficiently large... I haven't found a way to track native memory usage/availability.

    How is it possible that my process would be competing for memory with another process?

    Are you implying that lowering my je cache size might help?

  • Greybird-Oracle
    Greybird-Oracle Member Posts: 2,690
    edited Oct 5, 2016 12:41PM

    It seems that RandomAccessFile.writeBytes0(Native Method) is allocating memory *outside* the Java heap, and there isn't memory available there. So something else on your machine is using this memory. In this case the JE cache size is not relevant, since it only impacts what is used inside the Java heap.

    You could try reducing your heap size or look at what else is running on the machine and how much memory is available. To start with, look at your heap size relative to the total RAM.

    --mark

This discussion has been closed.