This discussion is archived
8 Replies Latest reply: Aug 21, 2008 4:05 PM by greybird RSS

Deadlocks while inserting entity and its subclass.

652881 Newbie
Currently Being Moderated
We are getting deadlocks when inserting an entity and a subclass of the entity during the same transaction. This is only happens the first time we insert the objects. I think the deadlock happens during the creation of the secondary databases( the shared secondary keys between the entity class and the subclass ) in the same transaction. When the databases are created we dont have this problem.

Is there anyway to avoid this?
  • 1. Re: Deadlocks while inserting entity and its subclass.
    Linda Lee Journeyer
    Currently Being Moderated
    Can you help us understand the situation by giving us a sketch of the classes involved?

    Thanks,

    Linda
  • 2. Re: Deadlocks while inserting entity and its subclass.
    greybird Expert
    Currently Being Moderated
    In addition to what Linda asked, please post the stack trace for the deadlock.
    Thanks,
    --mark                                                                                                                                                                                               
  • 3. Re: Deadlocks while inserting entity and its subclass.
    652881 Newbie
    Currently Being Moderated
    Here is an example of one of our entity classes and a subclass of the entity.

    @Entity
    public class Statement implements Serializable {
    @PrimaryKey(sequence="Statement")
    protected long id;

    @SecondaryKey(relate=Relationship.MANY_TO_ONE,relatedEntity=MMTObject.class,onRelatedEntityDelete=DeleteAction.CASCADE)
    protected long object;

    @SecondaryKey(relate=Relationship.MANY_TO_ONE)
    protected final String name;

    protected long parent = -1;

    protected List<Long> children = new ArrayList<Long>();

    protected Map<String,String[]> properties = new HashMap<String,String[]>();

    protected List<Integer> lines = new ArrayList<Integer>();

    protected String start = "";
    protected String end = "";
    }


    @Persistent
    public class ExtendedStatement extends Statement implements java.io.Serializable {
    @SecondaryKey(relate=Relationship.MANY_TO_MANY)
    private List<String> propertiesNames = new ArrayList<String>();
    @SecondaryKey(relate=Relationship.MANY_TO_MANY)
    private List<PropertyKey> propertiesId = new ArrayList<PropertyKey>();

    private transient Map<String,Property> propertiesObject = new LinkedHashMap<String,Property>();

    }

    Here is a stacktrace of the error:

    ERROR OIParserSentencesThread.class com.hp.mmt.proc.OIParserSentencesThread[8] 2008.08.11 10:49:12.671(48875)ms - com.sleepycat.util.RuntimeExceptionWrapper: (JE 3.3.62) Lock expired. Locker 11965543 57_class com.hp.mmt.proc.OIParserSentencesThread[8]_Txn: waited for lock on database=persist#EntityStore#com.hp.mmt.data.Statement LockAddr:1222210 node=1204 type=READ grant=WAIT_NEW timeoutMillis=10000 startTime=1218444542625 endTime=1218444552625
    Owners: [<LockInfo locker="3920972 52_class com.hp.mmt.proc.OIParserSentencesThread[8]_Txn" type="WRITE"/>]
    Waiters: []

    com.sleepycat.util.RuntimeExceptionWrapper: (JE 3.3.62) Lock expired. Locker 11965543 57_class com.hp.mmt.proc.OIParserSentencesThread[8]_Txn: waited for lock on database=persist#EntityStore#com.hp.mmt.data.Statement LockAddr:1222210 node=1204 type=READ grant=WAIT_NEW timeoutMillis=10000 startTime=1218444542625 endTime=1218444552625
    Owners: [<LockInfo locker="3920972 52_class com.hp.mmt.proc.OIParserSentencesThread[8]_Txn" type="WRITE"/>]
    Waiters: []

    at com.sleepycat.persist.impl.PersistCatalog.getFormat(PersistCatalog.java:738)
    at com.sleepycat.persist.impl.PersistEntityBinding.getValidFormat(PersistEntityBinding.java:156)
    at com.sleepycat.persist.impl.PersistEntityBinding.objectToKey(PersistEntityBinding.java:125)
    at com.sleepycat.persist.PrimaryIndex.assignKey(PrimaryIndex.java:490)
    at com.sleepycat.persist.PrimaryIndex.putNoReturn(PrimaryIndex.java:425)
    at com.hp.mmt.persistent.Inventory.putNoReturnStatement(Inventory.java:158)
    at com.hp.mmt.proc.OIParserSentencesThread.save(OIParserSentencesThread.java:203)
    at com.hp.mmt.proc.OIParserSentencesThread.parse(OIParserSentencesThread.java:304)
    at com.hp.mmt.proc.OIParserSentencesThread.run0(OIParserSentencesThread.java:171)
    at com.hp.mmt.server.MMTThread.run(MMTThread.java:53)
    at java.lang.Thread.run(Thread.java:619)
    Caused by: com.sleepycat.je.DeadlockException: (JE 3.3.62) Lock expired. Locker 11965543 57_class com.hp.mmt.proc.OIParserSentencesThread[8]_Txn: waited for lock on database=persist#EntityStore#com.hp.mmt.data.Statement LockAddr:1222210 node=1204 type=READ grant=WAIT_NEW timeoutMillis=10000 startTime=1218444542625 endTime=1218444552625
    Owners: [<LockInfo locker="3920972 52_class com.hp.mmt.proc.OIParserSentencesThread[8]_Txn" type="WRITE"/>]
    Waiters: []

    at com.sleepycat.je.txn.LockManager.makeTimeoutMsgInternal(LockManager.java:463)
    at com.sleepycat.je.txn.SyncedLockManager.makeTimeoutMsg(SyncedLockManager.java:77)
    at com.sleepycat.je.txn.LockManager.lock(LockManager.java:272)
    at com.sleepycat.je.txn.Txn.lockInternal(Txn.java:425)
    at com.sleepycat.je.txn.Locker.lock(Locker.java:360)
    at com.sleepycat.je.dbi.CursorImpl.lockLNDeletedAllowed(CursorImpl.java:2516)
    at com.sleepycat.je.dbi.CursorImpl.lockLN(CursorImpl.java:2438)
    at com.sleepycat.je.dbi.CursorImpl.fetchCurrent(CursorImpl.java:2368)
    at com.sleepycat.je.dbi.CursorImpl.getCurrentAlreadyLatched(CursorImpl.java:1427)
    at com.sleepycat.je.Cursor.positionAllowPhantoms(Cursor.java:1586)
    at com.sleepycat.je.Cursor.position(Cursor.java:1515)
    at com.sleepycat.je.SecondaryDatabase.init(SecondaryDatabase.java:239)
    at com.sleepycat.je.SecondaryDatabase.initNew(SecondaryDatabase.java:171)
    at com.sleepycat.je.Environment.setupDatabase(Environment.java:632)
    at com.sleepycat.je.Environment.openSecondaryDatabase(Environment.java:505)
    at com.sleepycat.compat.DbCompat.openSecondaryDatabase(DbCompat.java:326)
    at com.sleepycat.persist.impl.Store.openSecondaryIndex(Store.java:702)
    at com.sleepycat.persist.impl.Store.openSecondaryIndexes(Store.java:634)
    at com.sleepycat.persist.impl.PersistCatalog.getFormat(PersistCatalog.java:735)
                        ... 10 more

    To reproduce this error the insertions into the database were done by a single thread and in a single transaction. After the tables are
    created( after the deadlock ), we do not have this problem even while inserting from various threads.

    To avoid this, we have created an AnnotationModel and registered all our classes with it
    Here is the code:
         AnnotationModel model = new AnnotationModel();
         model.registerClass( Statement.class );
         model.registerClass( ExtendedStatement.class );
         StoreConfig storeConfig = new StoreConfig();
         storeConfig.setAllowCreate( true );
         storeConfig.setTransactional( true );
         storeConfig.setModel( model );
         entityStore = new EntityStore( environment, "EntityStore", storeConfig );
         storeconfig.setModel( model ).

    Although this works, everytime we implement a new class, we have to register it with the model while creating the entityStore.
    Hope this information is helpful, if require more please let me know.
  • 4. Re: Deadlocks while inserting entity and its subclass.
    greybird Expert
    Currently Being Moderated
    Thanks for the code and stack trace. This should be enough information for me to find the problem. I'll take a look and get back to you in the next couple of days.
    --mark                                                                                                                                                                                                                                                                                                                                                           
  • 5. Re: Deadlocks while inserting entity and its subclass.
    greybird Expert
    Currently Being Moderated
    Part of the resolution to this problem may be to recommend that you call registerClass or getSubclassIndex up front -- before accessing data via the indices. We're still evaluating this. Of course, we also want to ensure that the deadlock is avoided.

    But rather than call registerClass, have you tried calling EntityStore.getSubclassIndex for the ExtendedStatement class? Since this subclass contains a secondary key, this is the correct way to open the index.

    In general, when opening your indices, if the primary index is non-empty and a secondary index is opened that has never been opened previously, the DPL will enumerate all records in the primary index and index them by the new secondary key. This is going to cause performance problems for you -- and possibly deadlocks -- if you're actively using the primary index at that time.

    To avoid this, you can call getSecondaryIndex or getSubclassIndex, as appropriate, for all secondary indices before starting to access the primary index. This is a more natural solution than calling registerClass.

    Could you please try this and let us know how it works for you?

    To be clear: We will be working on the deadlock problem that occurs with a single thread is used. But in general, the best thing is to open your indices up front.

    Thanks,
    --mark                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   
  • 6. Re: Deadlocks while inserting entity and its subclass.
    652881 Newbie
    Currently Being Moderated
    I have tried to open all secondary indices of the ExtendedStatement class to see if the deadlocks can be avoided but I keep getting the following error:

    java.lang.IllegalArgumentException: Not a key class: com.hp.mmt.data.PropertyKey
    Caused by: java.lang.IllegalArgumentException: Not a key class: com.hp.mmt.data.PropertyKey
    at com.sleepycat.persist.impl.PersistKeyCreator.<init>(PersistKeyCreator.java:60)
    at com.sleepycat.persist.impl.Store.getSecondaryConfig(Store.java:1047)
    at com.sleepycat.persist.impl.Store.openSecondaryIndex(Store.java:664)
    at com.sleepycat.persist.impl.Store.getSecondaryIndex(Store.java:597)
    at com.sleepycat.persist.EntityStore.getSubclassIndex(EntityStore.java:352)
    at com.hp.mmt.persistent.PersistentManager.test(PersistentManager.java:266)


    The seconday Indices are opened with the following code:
    entityStore.getSubclassIndex( entityStore.getPrimaryIndex(Long.class , Statement.class ), ExtendedStatement.class , String.class , "propertiesNames");
    entityStore.getSubclassIndex( entityStore.getPrimaryIndex(Long.class , Statement.class ), ExtendedStatement.class , PropertyKey.class , "propertiesId" );

    This PropertyKey class -
    public class PropertyKey implements java.io.Serializable {
    @KeyField(1)
    private long statement;

    @KeyField(2)
    private final String name;

    public PropertyKey() { statement = -1L; name = "none"; }
    protected PropertyKey( Statement statement, String name ) {
    statement = _statement.getId();
    name = _name;
    }
    }

    I tried opening only one of the two indices( "propertiesNames" ) and it gives me the same error wile saving :
    java.lang.IllegalArgumentException: Not a key class: com.hp.mmt.data.PropertyKey
    at com.sleepycat.persist.impl.PersistKeyCreator.<init>(PersistKeyCreator.java:60)
    at com.sleepycat.persist.impl.Store.getSecondaryConfig(Store.java:1047)
    at com.sleepycat.persist.impl.Store.openSecondaryIndex(Store.java:664)
    at com.sleepycat.persist.impl.Store.openSecondaryIndexes(Store.java:634)
    at com.sleepycat.persist.impl.PersistCatalog.getFormat(PersistCatalog.java:735)
    at com.sleepycat.persist.impl.PersistEntityBinding.getValidFormat(PersistEntityBinding.java:156)
    at com.sleepycat.persist.impl.PersistEntityBinding.objectToKey(PersistEntityBinding.java:125)
    at com.sleepycat.persist.PrimaryIndex.assignKey(PrimaryIndex.java:490)
    at com.sleepycat.persist.PrimaryIndex.putNoReturn(PrimaryIndex.java:425)
    at com.hp.mmt.persistent.Inventory.putNoReturnStatement(Inventory.java:158)
    at com.hp.mmt.proc.OIParserSentencesThread.save(OIParserSentencesThread.java:203)

    Is there anything there anything wrong with the PropertyKey Class? When we call registerClass, we call it for the Statement class and the ExtendedStatement class and we dont have any problems with the PropertyKey class.

    Thanks
  • 7. Re: Deadlocks while inserting entity and its subclass.
    greybird Expert
    Currently Being Moderated
    PropertyKey is missing @Persistent. Was that line accidentally left out of the post?
    --mark                                                                                                                                                                                           
  • 8. Re: Deadlocks while inserting entity and its subclass.
    greybird Expert
    Currently Being Moderated
    I went ahead and created a test program using the code you posted. I was able to reproduce the problem, exactly as you reported it. PropertyKey is missing @Persistent in your post, but that was not the problem.

    We will be working on this problem, although I don't have an estimate for when a fix will be available. It may not be until the next major release. In the meantime, please use the registerClass workaround. You don't need to register all classes, just the entity subclass (ExtendedStatement).

    Thanks for reporting this.

    --mark