Discussions
Categories
- 197.1K All Categories
- 2.5K Data
- 546 Big Data Appliance
- 1.9K Data Science
- 450.7K Databases
- 221.9K General Database Discussions
- 3.8K Java and JavaScript in the Database
- 31 Multilingual Engine
- 552 MySQL Community Space
- 479 NoSQL Database
- 7.9K Oracle Database Express Edition (XE)
- 3.1K ORDS, SODA & JSON in the Database
- 555 SQLcl
- 4K SQL Developer Data Modeler
- 187.2K SQL & PL/SQL
- 21.4K SQL Developer
- 296.3K Development
- 17 Developer Projects
- 139 Programming Languages
- 293K Development Tools
- 110 DevOps
- 3.1K QA/Testing
- 646.1K Java
- 28 Java Learning Subscription
- 37K Database Connectivity
- 158 Java Community Process
- 105 Java 25
- 22.1K Java APIs
- 138.2K Java Development Tools
- 165.3K Java EE (Java Enterprise Edition)
- 19 Java Essentials
- 162 Java 8 Questions
- 86K Java Programming
- 81 Java Puzzle Ball
- 65.1K New To Java
- 1.7K Training / Learning / Certification
- 13.8K Java HotSpot Virtual Machine
- 94.3K Java SE
- 13.8K Java Security
- 205 Java User Groups
- 24 JavaScript - Nashorn
- Programs
- 468 LiveLabs
- 39 Workshops
- 10.2K Software
- 6.7K Berkeley DB Family
- 3.5K JHeadstart
- 5.7K Other Languages
- 2.3K Chinese
- 175 Deutsche Oracle Community
- 1.1K Español
- 1.9K Japanese
- 233 Portuguese
BerkeleyDB LOG_INTEGRITY errors

Greetings!
We run BerkeleyDB embedded in our java application. Everything runs fine normally. Trouble happens if we run two instances of the same application. The folders where database files are created are separate for both. But it seems that when one stops something deletes the database files of the running one and it crashes.
A little more detail on how we run things:
init.d script (1) --> Starts a python module as root (2) ---> Starts a Java program as a regular user (3) ---> Forks off java programs that use individual instances of BerkeleyDB (4)
So if we start (3) manually things seem to run fine even if we have two instances of 4. It is only when the full thing runs that the error happens - the first process runs fine, the second process starts and completes and then the first process crashes because the entire content of its database folder is gone.
One more observation - if we set up the HOME environment variable to "/root" before starting (3) above - things do not fail. We cannot do that for obvious reasons, but this is something we found by chance.
The stack trace is attached below. If anyone can throw some light on what may be causing this that will be very very helpful.
Regards,
Arnab.
Caused by: com.sleepycat.je.EnvironmentFailureException: (JE 5.0.103) Environment must be closed, caused by: com.sleepycat.je.EnvironmentFailureException: Environment invalid because of previous exception: (JE 5.0.103) db com.sleepycat.je.EnvironmentFailureException: (JE 5.0.103) db java.io.FileNotFoundException: db/00000001.jdb (No such file or directory) LOG_INTEGRITY: Log information is incorrect, problem is likely persistent. UNEXPECTED_EXCEPTION_FATAL: Unexpected internal Exception, unable to continue. Environment is invalid and must be closed.
at com.sleepycat.je.EnvironmentFailureException.wrapSelf(EnvironmentFailureException.java:246)
at com.sleepycat.je.dbi.EnvironmentImpl.checkIfInvalid(EnvironmentImpl.java:1618)
at com.sleepycat.je.dbi.CursorImpl.checkEnv(CursorImpl.java:2859)
at com.sleepycat.je.Cursor.checkEnv(Cursor.java:4181)
at com.sleepycat.je.Cursor.close(Cursor.java:517)
at com.sleepycat.je.Database.putInternal(Database.java:1584)
at com.sleepycat.je.Database.put(Database.java:1448)
at com.veracode.webscan.scanner.BerkeleyDbStorage$Db.put(BerkeleyDbStorage.java:585)
at com.veracode.webscan.scanner.BerkeleyDbStorage$Db.put(BerkeleyDbStorage.java:615)
at com.veracode.webscan.scanner.BerkeleyDbStorage$Db.access$3400(BerkeleyDbStorage.java:568)
at com.veracode.webscan.scanner.BerkeleyDbStorage$NodeProviderImpl.setProp(BerkeleyDbStorage.java:850)
at com.veracode.webscan.scanner.BerkeleyDbStorage$Node.setProp(BerkeleyDbStorage.java:148)
at com.veracode.dscancommon.util.BaseMutablePropsAsMutableProps.setProp(BaseMutablePropsAsMutableProps.java:62)
... 34 more
Caused by: com.sleepycat.je.EnvironmentFailureException: Environment invalid because of previous exception: (JE 5.0.103) db com.sleepycat.je.EnvironmentFailureException: (JE 5.0.103) db java.io.FileNotFoundException: db/00000001.jdb (No such file or directory) LOG_INTEGRITY: Log information is incorrect, problem is likely persistent. UNEXPECTED_EXCEPTION_FATAL: Unexpected internal Exception, unable to continue. Environment is invalid and must be closed.
at com.sleepycat.je.EnvironmentFailureException.unexpectedException(EnvironmentFailureException.java:351)
at com.sleepycat.je.log.LogBufferPool.writeBufferToFile(LogBufferPool.java:443)
at com.sleepycat.je.log.LogBufferPool.writeDirty(LogBufferPool.java:370)
at com.sleepycat.je.log.LogManager.multiLog(LogManager.java:426)
at com.sleepycat.je.log.LogManager.log(LogManager.java:350)
at com.sleepycat.je.tree.IN.logInternal(IN.java:3547)
at com.sleepycat.je.tree.IN.optionalLog(IN.java:3494)
at com.sleepycat.je.tree.IN.splitInternal(IN.java:2737)
at com.sleepycat.je.tree.IN.split(IN.java:2535)
at com.sleepycat.je.tree.Tree.forceSplit(Tree.java:1992)
at com.sleepycat.je.tree.Tree.searchSubTreeSplitsAllowed(Tree.java:1768)
at com.sleepycat.je.tree.Tree.searchSplitsAllowed(Tree.java:1318)
at com.sleepycat.je.tree.Tree.findBinForInsert(Tree.java:2246)
at com.sleepycat.je.dbi.CursorImpl.putInternal(CursorImpl.java:875)
at com.sleepycat.je.dbi.CursorImpl.put(CursorImpl.java:816)
at com.sleepycat.je.Cursor.putAllowPhantoms(Cursor.java:2397)
at com.sleepycat.je.Cursor.putNoNotify(Cursor.java:2352)
at com.sleepycat.je.Cursor.putNotify(Cursor.java:2260)
at com.sleepycat.je.Cursor.putNoDups(Cursor.java:2149)
at com.sleepycat.je.Cursor.putInternal(Cursor.java:2117)
at com.sleepycat.je.Database.putInternal(Database.java:1580)
... 41 more
Caused by: com.sleepycat.je.EnvironmentFailureException: (JE 5.0.103) db java.io.FileNotFoundException: db/00000001.jdb (No such file or directory) LOG_INTEGRITY: Log information is incorrect, problem is likely persistent.
at com.sleepycat.je.log.FileManager$LogEndFileDescriptor.getWritableFile(FileManager.java:2981)
at com.sleepycat.je.log.FileManager$LogEndFileDescriptor.access$200(FileManager.java:2682)
at com.sleepycat.je.log.FileManager.writeLogBuffer(FileManager.java:1607)
at com.sleepycat.je.log.LogBufferPool.writeBufferToFile(LogBufferPool.java:411)
... 60 more
Caused by: java.io.FileNotFoundException: db/00000001.jdb (No such file or directory)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.<init>(Unknown Source)
at java.io.RandomAccessFile.<init>(Unknown Source)
at com.sleepycat.je.log.FileManager$DefaultRandomAccessFile.<init>(FileManager.java:3181)
at com.sleepycat.je.log.FileManager$6.createFile(FileManager.java:3209)
at com.sleepycat.je.log.FileManager.openFileHandle(FileManager.java:1278)
at com.sleepycat.je.log.FileManager.makeFileHandle(FileManager.java:1245)
at com.sleepycat.je.log.FileManager.access$1500(FileManager.java:76)
at com.sleepycat.je.log.FileManager$LogEndFileDescriptor.getWritableFile(FileManager.java:2959)
... 63 more
Best Answer
-
Based on your description I'm guessing that either your script/app or something external is deleting the files. It is very unlikely that JE is deleting all the files in the environment directory. We've never had a report of such a problem anyway, and it's difficult for me to imagine how it could happen. For example, JE never cleans/deletes the two most recent files.
I should also mention that we do not certify the product on Windows, so there is a possibility that you're running into a file locking issue that we don't know about. Normally JE uses a lock file to ensure that only one JE process (JVM instance) can write to a given directory. If you are accidentally allowing two JVMs to write to the same directory, and Windows file locking is misbehaving, that might cause the problem. However, I think this is unlikely.
--mark
Answers
-
Based on your description I'm guessing that either your script/app or something external is deleting the files. It is very unlikely that JE is deleting all the files in the environment directory. We've never had a report of such a problem anyway, and it's difficult for me to imagine how it could happen. For example, JE never cleans/deletes the two most recent files.
I should also mention that we do not certify the product on Windows, so there is a possibility that you're running into a file locking issue that we don't know about. Normally JE uses a lock file to ensure that only one JE process (JVM instance) can write to a given directory. If you are accidentally allowing two JVMs to write to the same directory, and Windows file locking is misbehaving, that might cause the problem. However, I think this is unlikely.
--mark
-
Thanks a lot for your mail. I will see if something else is doing this. BTW, we are running this in a linux environment.
Regards,
Arnab. -
Mark - you were right ... some old code (separate) was doing a cleaning job removing files which deleted these... thanks!
-
Good, glad you resolved it!
--mark