I'm trying to limit the running of the background threads in BerkeleyDB JE 6.0.10.
When I open a database/environment for writing, apart from my application thread, I see one compressor thread, one cleaner thread and one checkpointer thread. So for each DB, that is 4 threads.
For 1000+ databases, this is rapidly spiraling up to 4000+ threads and although it is possible to configure the OS to cater for this, I would like to have finer grain control over these housekeeping threads and run them at a time that I am in control of to even out the thread-load.
Reading through the documentation, I understand that the checkpointer thread must always run for db consistency. Fair enough.
My understanding is that the cleaner thread compacts the je logfiles and that the compressor thread is used for trimming the B-Tree during deletions.
What I don't understand, is that I run many insertions (well, to be honest, they are updates), I get all three threads running. I don't understand why I see compressor threads unless the update is being treated as a delete and then insert.
What I would really like to do is stop the cleaner and compressor threads running automatically and live with the fact that the database is not optimal.
Then what I would really like to do is ask the database "do you need cleaning/compressing?" and if so, then open->housekeep->close.
I find in the GSG documentation this piece of information:
"Note that you can prevent a background thread from running by using the appropriate je.properties parameter, but this is not recommended for production use and those parameters are not described here. "
The GSG tells me this about the compressor thread:
"There is no need for you to manage the compressor and so it is not described further in this manual. "
This implies it may be possible but we won't tell you how :-) Is that the case - I realize this is all at my own risk here!! ?
When it comes to the cleaner thread, the GSG says there are a couple of properties I can control, one being
It may be possible to set this to zero to prevent cleaning and then at a later time increase the value ?
So I guess I'm saying, although its not recommended, I would like to stop one or more of the background threads from running automatically on a db write. I understand this may lead to disk bloat and may slow down query but I'm in the game of reducing instantaneous threadcount and then come back and tidyup at a more convenient time. Is this something that is considered feasible with BerkeleyDB JE and if so, what are these 'properties that are not recommended' ? I'd like to try it out as this is going to be my bottleneck, rather than RAM or diskspace.
Thanks for taking the time to read my rambling if you get this far, I look forward to any response !!