I've been playing around using BDB on a couple of different platforms. I noticed a similar situation. When I sit in a loop doing adds, at the 65,536th add BDB stalls for minute or two. After a minute or two, the add succeeds.
This situation seems to happen when I have around 43 10MB log files. During the stall, I notice many log files are being written (another 25 or so), which is a much quicker rate than was being written prior to the stall.
The stall only happens once. I added another 350,000 entries and no more stalls.
Trying to understand what BDB is doing during the stall.
Thanks for any help,
Can you provide more information about you application ? Like the following:
1 What is the access method you are using ? Btree, Hash, Recno, Queue or Heap ? And what's the page size you are using ?
2 What is the entry like ? The average length of the key and data.
3 How do you configure your environment ? I see you are using logs, so are you opening the environment using DB_INIT_TXN, and DB_INIT_LOCK ? What's the cache size of your environment and are you doing checkpoint periodically ? Are you using replication ?
It is best if you can provide a sample program or at least provide the code about environment configuration.
-Winter, Oracle Berkeley DB
I'm using BDB as a backend from OpenLDAP. So, I'm not sure I know all your answers, but I hope your familiar with OpenLDAP's use of BDB.
1) Btree - at least I've certainly seen btree routines being called.
2) There are 7 data items in an entry (at least in OpenLDAP terms).
3) Some values from my DB_CONFIG file:
set_cachesize 0 20971520 1
I was hoping the question was really a general question in nature and the details were not important, but maybe not.
I guess it's possible the OpenLDAP is causing the delay, but it seemed like BDB was doing all the activity in this period.
From the DB_CONFIG, you are using a private environment which prevents db_stat utility from getting statistics out of the environment.
I have no idea of how OpenLDAP uses BerkeleyDB. I will take a look, but that will take some time, it will take me a little longer to get to you again.
I asked this question to OpenLDAP and they believe it is an OpenLDAP condition that happens relating to indexes. If my understanding is correct, after 65535, the max number of index slots in the index table, the table is collapsed into ranges. This reorganizing of the table is causing my delay.
Thanks for any time you spent looking at this issue.