3 Replies Latest reply: Feb 21, 2013 10:48 AM by Bogdan Coman RSS

    Closing berkeley db makes my Linux system halt

    988496
      I have a program using berkeley db as the data store, and when the program finishes its job and exits, it triggers berkeley db to close.

      Ok, here is the problem. Nine times out of ten, my Linux system goes down when berkeley db is closing. By `goes down`, I mean linux shell cannot respond to any linux command (ls,top,ps...) I typed in, just like the system is dead.

      According to documentation of berkeley db's api for `close()`, it will flash all the data in cache back to disk. But I wonder how could the db-closing make my system halt? Does berkeley db spawn lots of threads or takes up too much cpu when closing?
        • 1. Re: Closing berkeley db makes my Linux system halt
          Bogdan Coman
          985493 wrote:
          I have a program using berkeley db as the data store, and when the program finishes its job and exits, it triggers berkeley db to close.
          What is the program doing more exactly? How many records are you writing, what size? How does you DB_CONFIG look like?
          Ok, here is the problem. Nine times out of ten, my Linux system goes down when berkeley db is closing. By `goes down`, I mean linux shell cannot respond to any linux command (ls,top,ps...) I typed in, just like the system is dead.

          According to documentation of berkeley db's api for `close()`, it will flash all the data in cache back to disk. But I wonder how could the db-closing make my system halt? Does berkeley db spawn lots of threads or takes up too much cpu when closing?
          Do you get a core dump? Would you be able to keep track what the process utilization of the CPU is?
          It may be an issue of misallocation of resources. What's the cache size?
          • 2. Re: Closing berkeley db makes my Linux system halt
            988496
            Bogdan Coman wrote:
            985493 wrote:
            I have a program using berkeley db as the data store, and when the program finishes its job and exits, it triggers berkeley db to close.
            What is the program doing more exactly? How many records are you writing, what size? How does you DB_CONFIG look like?
            Ok, here is the problem. Nine times out of ten, my Linux system goes down when berkeley db is closing. By `goes down`, I mean linux shell cannot respond to any linux command (ls,top,ps...) I typed in, just like the system is dead.

            According to documentation of berkeley db's api for `close()`, it will flash all the data in cache back to disk. But I wonder how could the db-closing make my system halt? Does berkeley db spawn lots of threads or takes up too much cpu when closing?
            Do you get a core dump? Would you be able to keep track what the process utilization of the CPU is?
            It may be an issue of misallocation of resources. What's the cache size?
            1. My program just does the normal things with bdb, like updating, deleting or inserting records into bdb. Those records vary in size, but generally not larger than 4k, most of them are around 100b in size.
            2. No core dump. I cannot track CPU utilization, cuz system doesn't respond to me when bdb is closing. CACHE size is 80G, quite large.

            When the program finishes, the data-store object (has bdb as a member of class) will be destructed, which triggers the bdb to close, and from now on, the system acts slowly and does not respond to any keyboard input any longer, just like dead. I just wait, and it may come back again, and by that time, bdb closing should be done.
            • 3. Re: Closing berkeley db makes my Linux system halt
              Bogdan Coman
              Cache size is 80GB? That's a lot of cache. What system are you running on? How much ram do you have?

              Do you see the same issue when running with a more humble cache size, like 1GB?

              If you intend to keep your data in memory, you may be interested in reading the following page: http://docs.oracle.com/cd/E17076_02/html/programmer_reference/program_ram.html