I try to evolve my model by adding a keyfield inside the composite primary key of an entity.
I have modified my class (i.e. added the new field as @KeyField) and now i want to migrate the database.
I use the RawStore class, but I have got a problem: when i try to get the primary index from the RawStore, an exception occurs:
Exception in thread "main" com.sleepycat.je.EnvironmentFailureException: (JE 5.0.58) D:\ db fetchTarget of 0x7/0x2afd08 parent IN=3 IN class=com.sleepycat.je.tree.BIN lastFullVersion=0x7/0x2b7b1a lastLoggedVersion=0x7/0x2b7b1a parent.getDirty()=false state=0 java.lang.IllegalArgumentException: All composite key instance fields must be key fields: dao.berkeleydb.PersistentPublicationKey.sourceLocation LOG_INTEGRITY: Log information is incorrect, problem is likely persistent.
Caused by: java.lang.IllegalArgumentException: All composite key instance fields must be key fields: dao.berkeleydb.PersistentPublicationKey.sourceLocation
... 12 more
It seems that the RawStore uses the class declaration…
You've run into an "interesting" problem in that it wasn't anticipated when RawStore was designed and implemented. The error occurs when the RawStore opens a JE Database. Apparently your key class implements Comparable and this causes it to be used in JE to provide custom key comparison at a low level. To open the database JE must load your key class and configure it as a comparator. When it does this, it validates the key class. The error occurs when it discovers that the key class is invalid according to the last known metadata stored for the database, i.e., it notices that there is an extra field.
In fact this error could occur much earlier, in the Environment constructor, and it just happened to occur during the RawStore creation because you happened to have a clean checkpoint when the environment was last closed. I point this out just to emphasize that JE needs to be able to access the key comparator, even before the Environment is fully open. So it is very important that the key comparator is valid, and has not changed.
We can think about how this might be handled more gracefully in JE, but in the meantime I would like to suggest a workaround. Instead of adding a field to your existing key class, can you try creating a new key class (with a different class name) to use with your new/evolved persistent classes? The old class would exist solely for opening and reading the old store, and performing key comparison correctly for that old-format database.
Please let us know whether this approach works for you.
In fact, the problem did occur while opening the Environment, but i have made my own workaround : in a "legacy" project I have copied all persistent classes (old version) and opened, cleaned (with cleanLog) then closed the environment.
Following your suggestion, here is my solution :
In the "legacy" project
1) make a PKTmp class with the same fields as the old PK class, but which doesn't implements Comparable
2) make a ETmp class with the same fields as the old E(ntity) class, but the primary key is replaced by a PKTmp instance
3) iterate over the E entities cursor, creating a new ETmp for each E (i have deleted E instances during the iteration, but i think this is not mandatory)
4) call cleanLog on the environment, and force a checkpoint (this time the cleanLog was not enough, i don't know why)
In the real project, with new versions of the classes (i.e. the PK as Comparable and with the new field)
1) open the RawStore to read from and the entity store to write to
2) open an index on ETmp classname (E.getClassname() + "Tmp"), this time the class is not used and is not necessary in the classpath.
3) iterate over the entites, copy into the new store.