1 Reply Latest reply: Nov 8, 2012 6:42 AM by Ashok.Ora-Oracle RSS

    Insert data to Berkeley DB very poor performance in Big Data

      I have already held the 80G Berkeley DB file. I measure the average insert speed is 8ms for one record(32 byte key/100 byte value) without transaction.

      Compare to insert to empty database with same interface, the average speed is 3~6 us.
        • 1. Re: Insert data to Berkeley DB very poor performance in Big Data

          When the database is empty, the inserts are being written to Berkeley DB's buffer cache, so you're able to insert at memory speeds. Once the amount of data that you store exceeds the available memory on the machine, Berkeley DB will need to write data to disk, which will increase latency.

          How much memory are you using for Berkeley DB? How much data do you need to load into Berkeley DB? There are many options in Berkeley DB that can help. For example, Berkeley DB has a feature to write database pages to disk in the background (trickle). You can run Berkeley DB as a pure in-memory database.

          It would be great if you could post some additional information about your application and also the Berkeley DB configuration that you're using for your test.

          Thanks and warm regards.