Hi,
I'm doing some performance testing of the Berkeley DB (C/C++ version) on a Windows 7 x64 platform and wanted to check whether the results I'm getting can be improved.
I've simply been testing the time taken to write a million (key,value) pairs where the key is a 64 bit integer and the mapped value is a buffer of some fixed size. I'm using my laptop which has a pair of SSDs in RAID0 so bandwidth is excellent.
I'm using an environment with a 512MB cache (see DB_CONFIG below), and in a loop performing 1000 transactions (not flushed), and in each transaction I write 1000 (key,value) pairs. The key is just an incrementing integer.
Taking over 2 minutes to write 4GB of data isn't very good.
Are these results expected?
Cheers,
David
Mapped Time Database Log __db Effective Disk space
value (sec) size files files Rate wastage
size size size (MB/sec) factor
-------------------------------------------------------------------------------------------------
4 3.78 30416896 178257920 550969344 3.03 63.30
8 3.80 34021376 188743680 550969344 4.02 48.36
16 3.83 45162496 199229440 550969344 5.98 33.14
32 4.21 72228864 251658240 550969344 9.06 21.87
64 4.44 102146048 314572800 550969344 15.46 13.44
128 5.49 230932480 503316480 550969344 23.62 9.45
256 7.36 421306368 828375040 550969344 34.21 6.82
512 11.9 678215680 1342177280 550969344 41.67 4.94
1024 30.9 1756733440 2967470080 550969344 31.85 5.11
2048 66.4 8226021376 2380267520 550969344 29.53 5.43
4096 140 8226021376 4424990720 550969344 27.96 3.22
8192 212 16418021376 8671723520 550969344 36.89 3.13
// Test code minus error handling and timing:
void BerkeleyTest(int objectSize)
{
const char* environPath = "env";
const char* dbPath = "my_db.db";
DbEnv env(0);
env.open(environPath, DB_CREATE | DB_INIT_LOCK | DB_INIT_LOG | DB_INIT_MPOOL | DB_INIT_TXN, 0);
Db database(&env, 0);
database.open(NULL,dbPath,NULL,DB_BTREE,DB_CREATE | DB_AUTO_COMMIT,0);
__int64 keyid = 0;
std::vector<char> buffer(objectSize);
// note: only this for loop is being timed
for (int i=0 ; i < 1000 ; ++i)
{
DbTxn* txn = NULL;
env.txn_begin(NULL, &txn, 0);
for (int j=0 ; j < 1000 ; ++j)
{
Dbt key(&keyid, sizeof(keyid));
Dbt data(buffer.data(),objectSize);
database.put(txn, &key, &data, DB_NOOVERWRITE);
++keyid;
}
txn->commit(0);
}
database.close(0);
env.close(0);
}
# DB_CONFIG
set_cachesize 0 536870912 0
set_flags DB_TXN_NOSYNC
set_lg_regionmax 1048576
set_lg_max 10485760
set_lg_bsize 2097152