I'm trying to migrate to berkeley db je 5.0.34 and I'm the testing phase. I started with a valid db with 100M entries and minimum utilization factor = 50% and I tried to bump it up to 65% to see what the performance impact would be. In my first attempt, I tried with 4 cleaner threads, and after it finished cleaning (cleaner back log = 0), I run a simulation test that mainly reads random entries from the database and updates them several times and I got:
Caused by: voldemort.store.PersistenceFailureException: (JE 5.0.34) Environment must be closed, caused by: com.sleepycat.je.EnvironmentFailureException: Environ
ment invalid because of previous exception: (JE 5.0.34) ./config/data/bdb fetchTarget of 0x1ced/0x3d3561b parent IN=116781 IN class=com.sleepycat.je.tree.BIN la
stFullVersion=0xbc40/0x2ca3f3a lastLoggedVersion=0xbcc3/0x6a37712 parent.getDirty()=false state=0 LOG_FILE_NOT_FOUND: Log file missing, log is likely invalid. E
nvironment is invalid and must be closed.
at sun.reflect.GeneratedConstructorAccessor5.newInstance(Unknown Source)
I though this could be related to some kind of problem with multiple cleaner threads, so I started the whole process again, but with only 1 cleaner thread this time. Unfortunately I got the same issue (with a different log file, or course). The interesting part is that this happens after several entries were processed, the last time it happened at around 8M entries (out of 16M).
This sounds like a bug in the cleaner process, is there any other similar bug report or workaround that I can use?
More stack trace lines:
Caused by: com.sleepycat.je.EnvironmentFailureException: Environment invalid because of previous exception: (JE 5.0.34) ./config/data/bdb fetchTarget of 0x345c/0x3b7636f parent IN=492995 IN class=com.sleepycat.je.tree.BIN lastFullVersion=0x4a69/0x1e6d053 lastLoggedVersion=0x4b73/0x736179b parent.getDirty()=false state=0 LOG_FILE_NOT_FOUND: Log file missing, log is likely invalid. Environment is invalid and must be closed.
Caused by: java.io.FileNotFoundException: ./config/data/bdb/0000345c.jdb (No such file or directory)
at java.io.RandomAccessFile.open(Native Method)
Unless you accidentally deleted a .jdb file at some point, i.e., during a faulty backup/restore procedure, this error indicates a JE bug and is not caused by your configuration.
Please save the environment and send me email. The only way for me to debug this is to examine your .jdb files.