Skip to Main Content
Forums

Berkeley DB Family

Announcement

For appeals, questions and feedback about Oracle Forums, please email oracle-forums-moderators_us@oracle.com. Technical questions should be asked in the appropriate category. Thank you!

OutOfMemoryError when writing?

nitroamosOct 5 2016 — edited Oct 5 2016

I've incorporated je berkeleydb into my application and it seems to be working very well. Today a user reported the exception I pasted below. As far as I can tell from the log file, heap memory was available. I've pasted my config below, too.

Any ideas?

java.lang.OutOfMemoryError: null

  at java.io.RandomAccessFile.writeBytes0(Native Method) ~[na:1.7.0_45]

  at java.io.RandomAccessFile.writeBytes(Unknown Source) ~[na:1.7.0_45]

  at java.io.RandomAccessFile.write(Unknown Source) ~[na:1.7.0_45]

  at com.sleepycat.je.log.FileManager.writeToFile(FileManager.java:1749) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.je.log.FileManager.writeLogBuffer(FileManager.java:1663) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.je.log.LogBufferPool.writeBufferToFile(LogBufferPool.java:449) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.je.log.LogBufferPool.writeDirty(LogBufferPool.java:408) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.je.log.LogManager.log(LogManager.java:365) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.je.tree.LN.logInternal(LN.java:728) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.je.tree.LN.optionalLog(LN.java:447) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.je.dbi.CursorImpl.insertRecordInternal(CursorImpl.java:1478) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.je.dbi.CursorImpl.insertOrUpdateRecord(CursorImpl.java:1280) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.je.Cursor.putNoNotify(Cursor.java:2504) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.je.Cursor.putNotify(Cursor.java:2365) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.je.Cursor.putNoDups(Cursor.java:2223) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.je.Cursor.putInternal(Cursor.java:2060) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.je.Cursor.put(Cursor.java:730) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.util.keyrange.RangeCursor.put(RangeCursor.java:1066) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.collections.DataCursor.put(DataCursor.java:788) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.collections.StoredContainer.putKeyValue(StoredContainer.java:359) ~[je-6.4.25.jar:6.4.25]

  at com.sleepycat.collections.StoredMap.put(StoredMap.java:315) ~[je-6.4.25.jar:6.4.25]

   private void initDb(File file) throws IOException {

      log.info("opening cache " + file);

      EnvironmentConfig envConfig = new EnvironmentConfig();

      envConfig.setTransactional(false);

      envConfig.setAllowCreate(true);

      DbInternal.disableParameterValidation(envConfig);

      envConfig.setConfigParam(EnvironmentConfig.CLEANER_MIN_UTILIZATION,"90");

      envConfig.setConfigParam(EnvironmentConfig.LOG_FILE_MAX,"10000000");

      envConfig.setDurability(Durability.COMMIT_NO_SYNC);

      envConfig.setCacheSize(128000000);

      env = new Environment(file,envConfig);

      DatabaseConfig dbConfig = new DatabaseConfig();

      dbConfig.setTransactional(false);

      dbConfig.setAllowCreate(true);

      Database catalogDb = env.openDatabase(null,CLASS_CATALOG,dbConfig);

      classCatalog = new StoredClassCatalog(catalogDb);

      db = env.openDatabase(null,"mydatastore",dbConfig);

      // setup binding

      EntryBinding<K> keyBinding = new SerialBinding<K>(classCatalog,keyType);

      EntryBinding<V> valueBinding = new SerialBinding<V>(classCatalog,

            valueType);

      keyValueMap = new StoredMap<K, V>(db,keyBinding,valueBinding,true);

   }

Comments

Greybird-Oracle

Since the OOME is in RandomAccessFile.writeBytes0(Native Method), the system must overall must be low on memory. In other words, the OOME doesn't seem to be due to lack of Java heap space and therefore something else (another process?) is filling memory.

However, just in case the error is misleading, you should ensure that your heap size is around 2X the size of the JE cache. You didn't say what heap size you're using.

--mark

nitroamos

Thanks, Greybird-Oracle.

Here's the memory read out when the exception occurred:

memory: 403 free, 870 max, 870 total

for freeMemory, maxMemory, totalMemory respectively.

So I think my heap is sufficiently large... I haven't found a way to track native memory usage/availability.

How is it possible that my process would be competing for memory with another process?

Are you implying that lowering my je cache size might help?

Greybird-Oracle

It seems that RandomAccessFile.writeBytes0(Native Method) is allocating memory *outside* the Java heap, and there isn't memory available there. So something else on your machine is using this memory. In this case the JE cache size is not relevant, since it only impacts what is used inside the Java heap.

You could try reducing your heap size or look at what else is running on the machine and how much memory is available. To start with, look at your heap size relative to the total RAM.

--mark

1 - 3
Locked Post
New comments cannot be posted to this locked post.

Post Details

Locked on Nov 2 2016
Added on Oct 5 2016
#berkeley-db-java-edition
3 comments
482 views