1 2 3 Previous Next 30 Replies Latest reply: Aug 23, 2012 5:15 AM by userBDBDMS-Oracle Go to original post RSS
      • 15. Re: Problems with increasing/decreasing cache size when live
        897965
        Hi,
        Thanks for you answer but actually it doesn't help much...
        Let me please explain once again.

        I have some code that compacts 32 env sequentially and to do so smoothly, it dynamically increases the cache size of the env being compacted and decreases when done. This code works normally with BDB 4.7.25 but leaks so I tried to upgrade to 5.1 and then to 5.3 but with those newer versions this code crashes when decreasing the cache size to its original value (see previous stack trace).

        Let me summarize here what I believe is wrong with BDB code (or at least not following documentation):

        Before open, I do:
        DBEnvironment->set_cachesize((u_int32_t)0, (u_int32_t)512*1024*1024/32, 1);
        DBEnvironment->set_cache_max(1*1024*1024*1024, 0);
        DBEnvironment->get_cache_max(&gbytes, &obytes); // gives gbytes=1 and obytes=0

        Here everything is good but after open, I have the following:

        DBEnvironment->get_cache_max(&gbytes, &obytes); // gives gbytes=0 and obytes=9502720
        memp_stat: cache_size=18644992 cache_ncache=1, max_ncache=58

        which is strange, right? But please have a look at the following:

        1. DBEnvironment->set_cache_size(0, 9502720, 0)

        Here I increase cache to cache_max and get:
        memp_stat: outlinks cache_size=27328512 cache_ncache=54 max_ncache=58

        This doesn't make sense, why cache_size ((ui64)sp->st_gbytes * GIGA + sp->st_bytes)) was increased? why my cache is now more than 1GB when I specify 9MB...

        2. DBEnvironment->set_cache_size(0, 18644992, 0)

        Then I try to decrease cache to its original value and get: cannot resize to 142 cache regions: maximum is 58.

        This doesn't make sense either.

        3. DBEnvironment->set_cache_size(0, 9502720/2, 0)

        If I try to decrease cache to cache_max / 2, I get the stack trace I posted before:

        Program terminated with signal 11, Segmentation fault.
        #0 __memp_fput (dbmfp=0x11a62b780, ip=0x0, pgaddr=0x214155058,
        priority=DB_PRIORITY_UNCHANGED)
        at /place/home/bdb5.0/mp/mp_fput.c:123
        in /place/home/bdb5.0/mp/mp_fput.c

        Is there some magic formula out there to apply on cache sizes for these methods to behave correctly?

        Hope that helps. I really would like to make this work but really don't understand what I do wrong or what workaround I can use here.
        Thanks a lot.
        • 16. Re: Problems with increasing/decreasing cache size when live
          Oracle,CindyZeng
          Hi,

          Thanks for providing the information.

          Could you please help confirm the details so that I can reproduce the issue better?

          Here is what I think the case is.

          //Before open
          DBEnvironment->set_cachesize((u_int32_t)0, (u_int32_t)512*1024*1024/32, 1);
          DBEnvironment->set_cache_max(1*1024*1024*1024, 0);
          DBEnvironment->get_cache_max(&gbytes, &obytes); // gives gbytes=1 and obytes=0

          //After open
          DBEnvironment->get_cache_max(&gbytes, &obytes); // gives gbytes=0 and obytes=9502720
          memp_stat: cache_size=18644992 cache_ncache=1, max_ncache=58

          //Resize cache to maximum size just got
          DBEnvironment->set_cache_size(0, 9502720, 0)
          memp_stat: outlinks cache_size=27328512 cache_ncache=54 max_ncache=58

          //Resize cache to original cache size after open
          DBEnvironment->set_cache_size(0, 18644992, 0)
          get: cannot resize to 142 cache regions: maximum is 58

          //Resize the cache to half of the maximum size
          DBEnvironment->set_cache_size(0, 9502720/2, 0)
          get: segmentation falult

          Is that correct?
          • 17. Re: Problems with increasing/decreasing cache size when live
            897965
            Oracle, Cindy Zeng wrote:
            Hi,

            Thanks for providing the information.

            Could you please help confirm the details so that I can reproduce the issue better?

            Here is what I think the case is.
            Hi,

            I confirm that your description is correct, that's exactly how it goes for me.

            I'm using FreeBSD 8.2, Berkeley DB 5.3.15: (December 19, 2011) and DB_BTREE tables.
            Here are my env (open) flags: DB_CREATE | DB_INIT_LOCK | DB_INIT_LOG | DB_INIT_MPOOL | DB_INIT_TXN | DB_RECOVER | DB_PRIVATE | DB_THREAD;
            Here are my env flags: 0 | DB_AUTO_COMMIT | DB_TXN_NOSYNC;

            Thanks you again.
            • 18. Re: Problems with increasing/decreasing cache size when live
              897965
              Oracle, Cindy Zeng wrote:
              Hi,

              Thanks for providing the information.

              Could you please help confirm the details so that I can reproduce the issue better?
              Hi,

              Is there any news about what could be done here?

              Thank you.
              • 19. Re: Problems with increasing/decreasing cache size when live
                924602
                Hi

                it is quite interesting to know if the problem is reproduced and can we expect to get it fixed. We will appreciate any details about is status and possible resolution time.

                Thanks in advance.

                --
                p.s.
                • 20. Re: Problems with increasing/decreasing cache size when live
                  526060
                  Hi,

                  Sorry for the long delay. We have reproduced some issues with the cache resizing and are working to fix them. I can't give you a timeline for the fix. I will let you know when we have more information. Thanks for your patience.

                  Regards,
                  Alex Gorrod,
                  Oracle Berkeley DB
                  • 21. Re: Problems with increasing/decreasing cache size when live
                    924602
                    Thanks a lot for the update. Looking forward to get the patch and try it.

                    BTW we can try some preliminary patches if you wish.

                    p.s.
                    • 22. Re: Problems with increasing/decreasing cache size when live
                      924602
                      Hi

                      Do you have any news regarding the issue?

                      --
                      p.s.
                      • 23. Re: Problems with increasing/decreasing cache size when live
                        924602
                        Hi

                        We are still waiting for any update and for the fix of course. We need it for the functionality developed several months ago that can't work in production due to crash in BDB.

                        Could you please provide any information that might help us to adjust our expectation? Currently situation is completely unclear and it looks like no one is working on it.

                        Thanks in advance.

                        --
                        p.s
                        • 24. Re: Problems with increasing/decreasing cache size when live
                          carol.sandstrom
                          Hi,

                          I regret and apologize that we've been moving so slowly on this issue, but we do have some progress to report.

                          At this point, we have reproduced several errors, and applied fixes so that increasing the cache size works reliably. We are working on fixes so decreasing the cache size will work as well. Mostly what I can tell you is that we have not forgotten about you or about this issue.

                          Regards,
                          Carol
                          • 25. Re: Problems with increasing/decreasing cache size when live
                            924602
                            Thanks for the update, Carol

                            Do you have any estimation when this work can be completed?

                            --
                            p.s.
                            • 26. Re: Problems with increasing/decreasing cache size when live
                              carol.sandstrom
                              Hi Sergey,

                              My best guess is that it will be fixed this summer. It is hard to provide a good estimate since it is not the highest priority work for the engineers assigned to it.

                              Regards,
                              Carol
                              • 27. Re: Problems with increasing/decreasing cache size when live
                                924602
                                Hi Carol

                                I hope that my repeated question won't annoy you :) Do you have any good news for us?

                                p.s.
                                • 28. Re: Problems with increasing/decreasing cache size when live
                                  carol.sandstrom
                                  Hi Sergey,

                                  Thanks for reminding me about this. An (at least partial) fix has been checked into the current code line, which means it will be part of the next Berkeley DB release. I have asked an engineer to look into whether that fix can be applied against the 5.3 release. I'll let you know what I find out.

                                  Regards,
                                  Carol
                                  • 29. Re: Problems with increasing/decreasing cache size when live
                                    924602
                                    Thanks for these good news!

                                    What is BDB version where this change is going to be included and when we can expect its release?