4 Replies Latest reply: Jun 12, 2014 12:00 PM by kiwiclive RSS

    Life Cycle of the background threads

    kiwiclive

      Hi Guys,

       

      I was wondering if anyone knows the lifecycle of the background threads (compressor, cleaner, checkpointer etc) ?

       

      The reason I ask is that when I see a insert application thread run, the cleaner and compressor threads run. So far so good. It does appear that these threads do not terminate. Looking through the available methods in EnvironmentConfig, I see things like COMPRESSOR_WAKEUP_INTERVAL which implies the compressor is a daemon thread that once started will keep running in the host JVM.


      Would someone be able to outline the background thread lifecycle, as I was expecting something like this:


      1. application: openEnvironment()

      2. application: open databases()

      3. application: modifyDatabases() + commit

      4. At this stage I would expect checkpointer/cleaner/compressor threads to run.

      5. application: closeDatabases()

      6. application: closeEnvironment()

      7. application: thread terminates.

      ...

      8. After a while, compressor/checkpointer/cleaner threads terminate.


      Is this final assumption correct? From observations, it appears that the background threads do not terminate but keep running on the host JVM (perhaps asleep) rather than terminate.


      And then the big question: If they do not terminate, is there a way to ask them do so when they have completed, thereby freeing up a pid?


      Sorry for the seemingly trivial question, but I would like to have finer grain control over the background threads.


      Thanks,

      Clive


        • 1. Re: Life Cycle of the background threads
          greybird

          The JE background threads are started when the Environment is opened (in the constructor), and are stopped by Environment.close.  We have not seen a case where the threads keep running after Environment.close, so if you believe this is happening you'll need to give us more information, preferably a standalone test case the demonstrates the problem.

           

          --mark

          • 2. Re: Life Cycle of the background threads
            kiwiclive

            Hi Mark,

             

            Thanks for the fast reply. This was more a case of an observation under load with hundreds of parallel environments and was more a case of clarifying my limited understanding rather than indicating a potential problem. Once we have some data in the store, I'll  run some more targeted tests to make sure this is not an issue as I realize we are using BerkeleyDB in a pretty non-standard setup.

             

            Thanks,
            Clive

            • 3. Re: Life Cycle of the background threads
              greybird

              Glad to hear there's not a known problem at this point.

               

              Were you able to use setSharedCache(true), so that at least the evictor threads are shared by all environments?  Other users with large numbers of environments have had success with this.

               

              --mark

              • 4. Re: Life Cycle of the background threads
                kiwiclive

                Hi Mark,

                 

                The problem was due to multiple threads running one one environment. The docs do state after the 'last' environment is closed, the background threads will terminate. Multiple versions of the environment meant the threads were being kept open. When analyzing in a profiler, I do see that they are labelled as Daemon threads which does imply they hang around for a while.

                 

                However, thanks for all the pointers to how to switch them off in the EnvironmentMultableConfig, we have a mechanism for keeping the proliferation under control now :-)


                I am sharing the cache so its nice to know the Evictor threads are shared too.

                 

                Many thanks,

                Clive