Forum Stats

  • 3,851,525 Users
  • 2,263,993 Discussions


locktimeout on secondary database inserts

Hello all,

My current setup uses BDBJE 4.1.10. I have 10 threads trying to write a Keyword object to the database.

The nature of the data is that the secondary keys - stores and items have a lot of duplicates. Meaning, many Keyword objects map to the same store and items entries.



public class Keyword implements Serializable {


    private String key;

    @SecondaryKey(relate = Relationship.MANY_TO_MANY)

    private List<Long> stores = new ArrayList<Long>();

    @SecondaryKey(relate = Relationship.MANY_TO_MANY)

    private List<Long> items = new ArrayList<Long>();


About 10% (out of 100,000) of the secondary database inserts fail with a LockTimeoutException.

Looking at the stacktrace, I understand that the thread 6 waited to obtain the lock for 2000ms before giving up and there are 3 more threads waiting.

Essentially, the operation that the Owner thread (thread 10) is performing is taking more than 2000ms. Thread 6 is trying to insert a record in the secondary database items.

Stacktrace: (JE 4.1.10) Lock expired. Locker 786527582 -1_fileChannelTaskExecutor-6_ThreadLocker: waited for lock on database=persist#faceted_store#com.xyzq.seo.bdb.entity.Keyword#items LockAddr:43409842 node=281964 type=WRITE grant=WAIT_NEW timeoutMillis=2000 startTime=1538759059393 endTime=1538759061393 Owners: [<LockInfo locker="786774600 -1_fileChannelTaskExecutor-10_ThreadLocker" type="WRITE"/>] Waiters: [<LockInfo locker="1270863098 -1_fileChannelTaskExecutor-9_ThreadLocker" type="WRITE"/>, <LockInfo locker="1244207474 -1_fileChannelTaskExecutor-4_ThreadLocker" type="WRITE"/>, <LockInfo locker="1391625295 -1_fileChannelTaskExecutor-1_ThreadLocker" type="WRITE"/>] at at at at at at at at at at at at at at at at at at at at at at at at com.sleepycat.persist.PrimaryIndex.putNoReturn( at com.sleepycat.persist.PrimaryIndex.putNoReturn( at com.xyzq.bdb.cache.da.impl.BDBDataAccessor.create(

I tried the following

- Reduced the lock time. Hoping that the winner would release the lock as soon as its done insering records into

- Catch LockTimeOutException and retry inserting the keyword later. This works but it's manual and takes time. 10% of keywords fail because of a LockTimeOutException. ~ 10k out of 100k.

I have a few questions:

Is there a limit on the number of threads that can write to the DB or does it depend on the data?

Would it help if I model the entities as pure key value pairs instead of having secondary databases?




    - key




    -key [reference to Keyword Obj]



    - id

    - key [reference to Keyword obj.]


- K