When the database is empty, the inserts are being written to Berkeley DB's buffer cache, so you're able to insert at memory speeds. Once the amount of data that you store exceeds the available memory on the machine, Berkeley DB will need to write data to disk, which will increase latency.
How much memory are you using for Berkeley DB? How much data do you need to load into Berkeley DB? There are many options in Berkeley DB that can help. For example, Berkeley DB has a feature to write database pages to disk in the background (trickle). You can run Berkeley DB as a pure in-memory database.
It would be great if you could post some additional information about your application and also the Berkeley DB configuration that you're using for your test.