2 Replies Latest reply: Mar 9, 2012 10:41 AM by 906284 RSS

    Make BDB blindingly fast

    906284
      Hi,

      Currently I'm using BDB in my project (gitorious.org/libbitcoin), and there's an area where BDB is really slow but I don't need ACID really.

      My environment flags are:
      DB_CREATE
      DB_RECOVER
      DB_INIT_LOCK
      DB_INIT_LOG
      DB_INIT_TXN
      DB_INIT_MPOOL
      DB_THREAD
      DB_TXN_NOSYNC
      DB_CXX_NO_EXCEPTIONS

      env = new DbEnv(0);
      env->set_lk_max_locks(10000);
      env->set_lk_max_objects(10000);
      // Then open environment and create databases

      And my database flags:
      DB_CREATE
      DB_THREAD

      I realised I don't really need transactions, or ACID since the data is non-critical and can easily be replicated in the case of a major error. So how can I disable ACID completely? Speed is the most critical thing needed here above all else.

      Thanks

      Edited by: 903281 on 04-Mar-2012 11:16

      Edited by: 903281 on 04-Mar-2012 11:17
        • 1. Re: Make BDB blindingly fast
          Ashok_Ora
          Hello there,

          Have you looked at the "Data Store" and "Concurrent data store" feature sets? They don't support ACID transactions, but are significantly faster.

          Just out of curiosity, when you say "blindingly fast" - how fast do you mean? can you let us know what throughput you're looking for?

          Warm regards.
          ashok
          • 2. Re: Make BDB blindingly fast
            906284
            It's for bitcoin and I need to store around 160k blocks that are 30 Kb each. A block consists of a header which is put in 1 database, a block hash which is put in a secondary database and then a list of transactions together with a secondary database for transaction hashes.

            Right now, it takes hours to save all the blocks. I'm not sure how to tune this, but most blocks are fast but sometimes it can get a bit slower to save (taking a few seconds per block).