This discussion is archived
5 Replies Latest reply: Aug 2, 2013 8:12 AM by Winter Zhang RSS

Berkley Database stalls adding 65,536 entry

user634811 Newbie
Currently Being Moderated

I've been playing around using BDB on a couple of different platforms.  I noticed a similar situation.  When I sit in a loop doing adds, at the 65,536th add BDB stalls for minute or two.  After a minute or two, the add succeeds.

 

This situation seems to happen when I have around 43 10MB log files.  During the stall, I notice many log files are being written (another 25 or so), which is a much quicker rate than was being written prior to the stall. 

 

The stall only happens once. I added another 350,000 entries and no more stalls.

 

Trying to understand what BDB is doing during the stall. 

 

Thanks for any help,

 

Mark

  • 1. Re: Berkley Database stalls adding 65,536 entry
    Winter Zhang Explorer
    Currently Being Moderated

    Hi Mark,

     

    Can you provide more information about you application ? Like the following:

     

    1 What is the access method you are using ? Btree, Hash, Recno, Queue or Heap ? And what's the page size you are using ?

    2 What is the entry like ? The average length of the key and data.

    3 How do you configure your environment ? I see you are using logs, so are you opening the environment using DB_INIT_TXN, and DB_INIT_LOCK ? What's the cache size of your environment and are you doing checkpoint periodically ?  Are you using replication ?

     

    It is best if you can provide a sample program or at least provide the code about environment configuration.

     

    Regards,

    -Winter, Oracle Berkeley DB

  • 2. Re: Berkley Database stalls adding 65,536 entry
    user634811 Newbie
    Currently Being Moderated

    I'm using BDB as a backend from OpenLDAP.  So, I'm not sure I know all your answers, but I hope your familiar with OpenLDAP's use of BDB.

     

    1) Btree - at least I've certainly seen btree routines being called.

    2) There are 7 data items in an entry (at least in OpenLDAP terms).

    3) Some values from my DB_CONFIG file:

    set_cachesize 0  20971520 1

    set_lg_regionmax  1048576

    set_lg_max        10485760  

    set_lg_bsize      2097152

    set_lk_max_locks   2000

    set_lk_max_objects 2000

    set_open_flags db_private

     

    I was hoping the question was really a general question in nature and the details were not important, but maybe not.

     

    I guess it's possible the OpenLDAP is causing the delay, but it seemed like BDB was doing all the activity in this period.

  • 3. Re: Berkley Database stalls adding 65,536 entry
    Winter Zhang Explorer
    Currently Being Moderated

    Hi Mark,

     

    From the DB_CONFIG, you are using a private environment which prevents db_stat utility from getting statistics out of the environment.

     

    I have no idea of how OpenLDAP uses BerkeleyDB. I will take a look, but that will take some time, it will take me a little longer to get to you again.

     

    Regards,

    -Winter

  • 4. Re: Berkley Database stalls adding 65,536 entry
    user634811 Newbie
    Currently Being Moderated

    I asked this question to OpenLDAP and they believe it is an OpenLDAP condition that happens relating to indexes.  If my understanding is correct, after 65535, the max number of index slots in the index table, the table is collapsed into ranges.  This reorganizing of the table is causing my delay.

     

    Thanks for any time you spent looking at this issue.

  • 5. Re: Berkley Database stalls adding 65,536 entry
    Winter Zhang Explorer
    Currently Being Moderated

    Hi Mark,

     

    It is good to know that you get the answer from OpenLDAP.  Sorry for not being able to get your further.

     

    Regards,

    --Winter

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points