While starting a replica node, attempting to create a StoredClassCatalog causes a ReplicaWriteException:
com.sleepycat.je.rep.ReplicaWriteException: (JE 4.0.103) The current state is:REPLICA. The node transitioned to this state at:Tue Aug 03 14:28:28 PDT 2010
Code basically looks like the following:
ReplicatedEnvironment repEnv = new ReplicatedEnvironment(envHome, repConfig, envConfig);
Database catalogDb = repEnv.openDatabase(null, CLASS_CATALOG, dbConfig);
StoredClassCatalog javaCatalog = new StoredClassCatalog(catalogDb);
where the last line of code in the above snippet throws the exception. All example code I can find uses the DPL and I'm at a loss as to what I'm doing wrong.
Using bdbje version 4.0.103
[java] com.sleepycat.je.rep.ReplicaWriteException: (JE 4.0.103) The current state is:REPLICA. The node transitioned to this state at:Tue Aug 03 14:28:28 PDT 2010
[java] at grid.service.command.sleepycat.SleepycatHA.setupDatabases(SleepycatHA.java:239)
[java] at grid.service.command.sleepycat.SleepycatHA.startup(SleepycatHA.java:119)
[java] at grid.service.command.sleepycat.SleepycatDataAccessor.<init>(SleepycatDataAccessor.java:52)
[java] at grid.service.command.C2DataAccessorFactory.getC2DataAccessor(C2DataAccessorFactory.java:15)
[java] at grid.service.command.C2Controller.<init>(C2Controller.java:55)
[java] at grid.service.command.CommandService.init(CommandService.java:52)
[java] at grid.Main.main(Main.java:50)
[java] Caused by: com.sleepycat.je.rep.ReplicaWriteException: (JE 4.0.103) The current state is:REPLICA. The node transitioned to this state at:Tue Aug 03 14:28:28 PDT 2010
[java] at com.sleepycat.je.rep.txn.ReadonlyTxn.lockInternal(ReadonlyTxn.java:61)
[java] at com.sleepycat.je.txn.Locker.lock(Locker.java:453)
[java] at com.sleepycat.je.dbi.CursorImpl.put(CursorImpl.java:1204)
[java] at com.sleepycat.je.Cursor.putAllowPhantoms(Cursor.java:1799)
[java] at com.sleepycat.je.Cursor.putNoNotify(Cursor.java:1756)
[java] at com.sleepycat.je.Cursor.putNotify(Cursor.java:1689)
[java] at com.sleepycat.je.Cursor.putInternal(Cursor.java:1626)
[java] at com.sleepycat.je.Database.putInternal(Database.java:1186)
[java] at com.sleepycat.je.Database.putNoOverwrite(Database.java:1109)
[java] at com.sleepycat.bind.serial.StoredClassCatalog.<init>(StoredClassCatalog.java:125)
[java] at grid.service.command.sleepycat.SleepycatHA.setupDatabases(SleepycatHA.java:184)
Thanks for the stack trace. On first look, this is a bug in JE. The StoredClassCatalog should not attempt to write to the catalog unless this is necessary. We will confirm this with a test, and supply a fix in the next JE release.
I believe a workaround is to open the catalog DB for read-only access -- use DatabaseConfig.setReadOnly(true) -- on the replica. This means that when a replica is elected master, you'll have to close the catalog DB and then re-open it for read-write access, close and re-open the StoredClassCatalog, and also recreate any other objects dependent on the catalog, such as SerialBinding objects.
If this workaround is not acceptable to you, and you don't want to wait for the next JE release for the fix, please let me know and we'll work with you to get the fix to you as soon as it is available. Also, if you have Oracle support, be sure to let me know so we can prioritize accordingly. You can contact me over email: mark.hayes at the o.com.
Thanks for reporting this, and sorry for your trouble.
Updated my code as per your instructions and that seems to have fixed the problem.
When creating a DatabaseConfig instance, I check my current state. If I am a master, I invoke DatabaseConfig.setReadOnly(false) otherwise DatabaseConfig.setReadOnly(true);
Any time there is a state transition (ie MASTER -> REPLICA), all databases are closed via the Database.close() method and reopened with a new DatabaeConfig object initialized as described above. Reopening the databases also includes reinitializing all EntityBindings and StoredMaps.
Thanks for your help and quick response.