I am using JE 3.3.69 and am seeing a memory issue that I do not see with JE 3.2.76. The application loads the database (filling the DB cache in the process), fully closes the environment, re-opens the environment, and then preloads databases until the DB cache of the new environment is full. What seems to be happening is that the the DB cache from the closed environment is never garbage collected and the preload causes an out-of-memory error since the heap is not big enough for two full caches.
I saw that there is a static field in WriteLockInfo referencing a database (through the abortDb field) and is thereby holding on to the objects that make up the DB cache of the closed environment.
static final WriteLockInfo basicWriteLockInfo =
Should this static field belong to the environment instead?
Is there a work-around I can try?
The first load of the data is non-transactional. The subsequent preload is in a transactional environment.
I made some JE source changes to move the static basicWriteLockInfo out of WriteLockInfo and make it a non-static field of EnvironmentImpl. Initial testing is encouraging and suggests that this fixes the problem, i.e. the db cache from the first load of the data is garbage collected before the preload. I need to do some more testing though.
This bug, or something very similar, still exists for databases using the shared cache.
The whole shared cache ends up leaked.
I don't know what is causing the leak, and if it is the same or similar internal cause as the previous bug.
I can confirm that for databases using their own (non-shared) cache, it no longer occurs but did in the past.
Open Environment A (large) with locking disabled, using the shared cache, set the cache to a known percent value.
Open Environment B, C, D, etc in the same shared cache
Access Environment A until a large amount of RAM is used by the cache. Stop activity. Garbage Collect. Note remaining heap size (JConsole, Jstat, JMX, various other ways).
Open Environment A2 (same thing, different dir -- in our case we are swapping out updated data).
Close Environment A. Garbage collect -- no difference in size. Note, A2 has not been accessed at all.
Access Environment A2 again causing more memory to be used. It will grow well past its previous level above.
Open Environment A3.
Close Environment A2. Garbage collect -- it will free up all the stuff associated with A2, but all the stuff from A still remains.
The effect is as if the first shared lru cache is lost. Repeated opening and closing recycles the memory properly. But whatever is accessed in the "first" opening is lost forever.
When you use a cache that is 40% of the heap size (3500MB heap) and you lose 40% of the heap, its bad.
We are unable to use the shared cache feature due to this bug, though we are thinking about investigating a purposeful open/access/close upon startup to work around it.
I won't have time today to get to testing with EnvironmentConfig.setLocking(true); I will be unavailable for a few weeks and be able to test with locking enabled when I return.
The parameter has always been false for us; this process does not write at all but has multiple readers. We first started seeing memory problems after upgrading to the version with shared caches, we noticed the newer version that had a claimed fix for the leak but it did not help. After some experimentation, we found that it was the environments using the shared cache that leaked, and not the others. Switching all but the very small ones (which we don't care if they leak once) to not use the shared cache is the current work-around.
All we know is the following:
Older versions (prior to shared caches): locking = false, no leak but we were only using one environment in the process.
First version with shared caches, using shared caches and more than one environment: locking = false, leaks with shared caches
After the leak fix ( .75), using shared caches with locking = false, leaks. Using non-shared caches with locking = false, does not leak.