Thanks for you answer but actually it doesn't help much...
Let me please explain once again.
I have some code that compacts 32 env sequentially and to do so smoothly, it dynamically increases the cache size of the env being compacted and decreases when done. This code works normally with BDB 4.7.25 but leaks so I tried to upgrade to 5.1 and then to 5.3 but with those newer versions this code crashes when decreasing the cache size to its original value (see previous stack trace).
Let me summarize here what I believe is wrong with BDB code (or at least not following documentation):
Before open, I do:
DBEnvironment->set_cachesize((u_int32_t)0, (u_int32_t)512*1024*1024/32, 1);
DBEnvironment->get_cache_max(&gbytes, &obytes); // gives gbytes=1 and obytes=0
Here everything is good but after open, I have the following:
which is strange, right? But please have a look at the following:
1. DBEnvironment->set_cache_size(0, 9502720, 0)
Here I increase cache to cache_max and get:
memp_stat: outlinks cache_size=27328512 cache_ncache=54 max_ncache=58
This doesn't make sense, why cache_size ((ui64)sp->st_gbytes * GIGA + sp->st_bytes)) was increased? why my cache is now more than 1GB when I specify 9MB...
2. DBEnvironment->set_cache_size(0, 18644992, 0)
Then I try to decrease cache to its original value and get: cannot resize to 142 cache regions: maximum is 58.
This doesn't make sense either.
3. DBEnvironment->set_cache_size(0, 9502720/2, 0)
If I try to decrease cache to cache_max / 2, I get the stack trace I posted before:
Program terminated with signal 11, Segmentation fault.
#0 __memp_fput (dbmfp=0x11a62b780, ip=0x0, pgaddr=0x214155058,
Is there some magic formula out there to apply on cache sizes for these methods to behave correctly?
Hope that helps. I really would like to make this work but really don't understand what I do wrong or what workaround I can use here.
Thanks a lot.
Could you please help confirm the details so that I can reproduce the issue better?
Here is what I think the case is.
I confirm that your description is correct, that's exactly how it goes for me.
I'm using FreeBSD 8.2, Berkeley DB 5.3.15: (December 19, 2011) and DB_BTREE tables.
Here are my env (open) flags: DB_CREATE | DB_INIT_LOCK | DB_INIT_LOG | DB_INIT_MPOOL | DB_INIT_TXN | DB_RECOVER | DB_PRIVATE | DB_THREAD;
Here are my env flags: 0 | DB_AUTO_COMMIT | DB_TXN_NOSYNC;
Sorry for the long delay. We have reproduced some issues with the cache resizing and are working to fix them. I can't give you a timeline for the fix. I will let you know when we have more information. Thanks for your patience.
I regret and apologize that we've been moving so slowly on this issue, but we do have some progress to report.
At this point, we have reproduced several errors, and applied fixes so that increasing the cache size works reliably. We are working on fixes so decreasing the cache size will work as well. Mostly what I can tell you is that we have not forgotten about you or about this issue.
Thanks for reminding me about this. An (at least partial) fix has been checked into the current code line, which means it will be part of the next Berkeley DB release. I have asked an engineer to look into whether that fix can be applied against the 5.3 release. I'll let you know what I find out.