Why are there so many errors. [Use DbEnv for C++ in windows]

John Qiang

The version: Berkeley DB 11g Release 2, library version 11.2.5.3.15: (December 19, 2011)

16:30 2014/2/18 [Problem]

[1. Hard disk no enough space.]

db_err_handle - BDB3015 sound.bpkg: write failed for page 19360

db_err_handle - BDB0018 write: 0x80ed6e4, 4096: 磁盘空间不足。\r

db_err_handle - BDB3027 sound.bpkg: unable to flush page: 19360

BDB:\'res/map/fbdx/mini_map/m_1_2.dds\'. Put fail.28

db_err_handle - BDB4519 txn_checkpoint: failed to flush the buffer cache: No space left on device

BDB: Checkpoint err exception !DbEnv::txn_checkpoint: No space left on device

BDB:\'res/sound/playerword/err_skill_outofrange01_m.xwb\'. Put fail.12 [not enough space]

[2. Transaction Db->cur_locker != NULL]

db_err_handle - BDB0101 Transaction that opened the DB handle is still active

BDB:Could not get data \'res/character/ani/npc/qt803_attack02.ani\'. Giving up.22

sqr::CBerkeleyDB::ExistsDb::exists: Invalid argument

[3. DB_PAGE_NOTFOUND]

BDB:\'res/tile/model/閲庡鍦板浘/閲庡05/ywsd_fangzi_06.mod\'. Put fail.-30986

BDB:Could not get data size \'lang/chinese_simple/fonts/fzzzh.ttf\'. Giving up.-30986

[4. Recovery, no idea.]

sqr::CBerkeleyDB::PutDb::put: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery

db_err_handle - BDB0058 page 167715090: illegal page type or format [error code :2416045436]

db_err_handle - BDB0061 PANIC: Invalid argument

db_err_handle - BDB0060 PANIC: fatal region error detected; run recovery

[5. else]

BDB:\'res/character/ani/npc/qt172_cast01.ani\'. Put fail.22

db_err_handle - BDB4519 txn_checkpoint: failed to flush the buffer cache: Bad address

BDB: Checkpoint err exception !DbEnv::txn_checkpoint: Bad address

BDB:\'res/tile/model/閲庡鍦板浘/閲庡03/yw03_ydzp_01.mod\'. Put fail.14

db_err_handle - BDB0018 write: 0xc87f08b, 1: 由于 I/O 设备错误，无法运行此项请求。\r [4031010145]

db_err_handle - BDB3015 terrain.bpkg: write failed for page 81601

  剩余物理内存: 60 MB;

  剩余页交换文件: 4095 MB;

  占用的物理内存: 797 MB;

  占用的页交换文件: 691 MB

db_err_handle - BDB3018 role.bpkg: unwritable page 11329 remaining in the cache after error 14

  剩余物理内存: 238 MB;

  剩余页交换文件: 4095 MB;

  占用的物理内存: 757 MB;

  占用的页交换文件: 601 MB

db_err_handle - BDB4524 operation not permitted during recovery

This is my key code.

DbEnv * m_pDbEnv;


/******** create db_env *******/
m_uEnvFlags = DB_CREATE | DB_RECOVER | DB_INIT_LOCK | DB_INIT_LOG | DB_INIT_TXN | DB_INIT_MPOOL | DB_THREAD
m_pDbEnv = new DbEnv(0);


m_pDbEnv->set_errfile(m_pFile);
m_pDbEnv->log_set_config(DB_LOG_AUTO_REMOVE, 1);   // remove all no use log after check_point or recovered
m_pDbEnv->set_flags(DB_TXN_NOSYNC, 1);   // no sync db data, need sync or check_point by programmer in the code.


m_pDbEnv->set_lg_bsize(1 * 1024 * 1024); // 1 M for log buf.
m_pDbEnv->set_tx_max(16 * 1024);
// m_pDbEnv->mutex_set_max(100000);
m_pDbEnv->set_cachesize((u_int32_t)0, (u_int32_t)(64 * 1024 * 1024), 1);


m_pDbEnv->open(m_sEnvHome.c_str(), m_uEnvFlags, 0);


m_pDbEnv->set_lk_detect(DB_LOCK_DEFAULT);




/******** create db *******/
for (uint32 i = 0; i < pkgSize; ++i)
{
  CBerkeleyDB* fileDB = new CBerkeleyDB(m_pDbEnv, pkgName[i], false, &db_err_handle);
  m_dbContainer.insert(make_pair(dbHome, fileDB));
}




/******** put db *******/
Dbt key((void*)pathkey.c_str(), pathkey.size() + 1);
Dbt data((void*)buf, size);


CBerkeleyDB* db = GetFileDB(dbHome); // from m_dbContainer
int32 ret = db->Put(NULL, &key, &data, 0);
db->checkpoint(30 * 1024, 1, 0);


/******** get file size *******/
Dbt key(buff, (u_int32_t)size);
Dbt data(NULL, 0);
data.set_flags(DB_DBT_MALLOC);
CBerkeleyDB* db = GetFileDB(dbHome); // from m_dbContainer
int ret = db->Get(NULL, &key, &data, DB_READ_COMMITTED);
if (ret == 0)
  lenght = data.get_size();
  ......
free(data.get_data());


/******** get file buf *******/
Dbt key(buff, (u_int32_t)size);
Dbt data(NULL, 0);
data.set_flags(DB_DBT_MALLOC);
CBerkeleyDB* db = GetFileDB(dbHome); // from m_dbContainer
int ret = db->Get(NULL, &key, &data, DB_READ_COMMITTED);
if (ret == 0)
  memcpy(pBuf, data.get_data(), data.get_size()); // space for pBuf opt being outside(new and delete)
  ......
free(data.get_data());


/******** get file buf *******/
Dbt key(buff, (u_int32_t)size);
CBerkeleyDB* db = GetFileDB(dbHome); // from m_dbContainer
int ret = db->Exists(NULL, &key, DB_READ_COMMITTED);


/******** close *******/
CloseAllDb();
CloseEnv(); // m_pDbEnv->close(0);
TearDownEnv();




class CBerkeleyDB{
  ....
  cFlags_(DB_CREATE | DB_THREAD | DB_AUTO_COMMIT)
  ....
  Db m_DB;
  /***** init db ****/
  m_DB.set_error_stream(&std::cerr);
  callback == NULL ? m_DB.set_errcall(&my_error_handler) : m_DB.set_errcall(callback);
  m_DB.set_errpfx(dbFileName_.c_str());


  SYSTEM_INFO info;
  GetSystemInfo(&info);
  if(info.dwPageSize > 0)
  m_DB.set_pagesize(info.dwPageSize); 
  else
  m_DB.set_pagesize(4 * 1024); // set page size 4K for db by rong.


  // Open the database
  int ret = m_DB.open(NULL, dbFileName_.c_str(), NULL, DB_BTREE, cFlags_, 0);
  ....


  /**** Put ****/
  ret = m_DB.put(txn, key, data, flag);

  /**** Get ****/
  ret = m_DB.get(txn, key, data, flag);


  /**** Exist ****/
  ret = m_DB.exists(txn, key, flag);

  /**** close ****/
  m_DB.close(0);


};

John Qiang

i use multi thread in one process, and run on many machine. The windows system may be difference.

Any help at all would be great, thanks.

userBDBDMS-Oracle

Looking at the errors, it seems that you either have a corrupt disk or the disk you are using does not have enough space.   The errors you have seem to be compounding on one another.   The recommendation is to start with the first one and look at why your application thinks there is no disk space available.

thanks

mike

John Qiang

Hi mike

thanks for your response.

However, all of the error is not present on the same machine.

These different errors are from different machines. These errors have been classified by type. It means that many errors are not because of lack of disk space.

I just want my application to continue running. How should i do?

thanks

john

userBDBDMS-Oracle

Hi John,

   First, have you dealt with the errors dealing with disk space.   For disk space errors we get an error message from the OS and pass it through our code.    We will need to take the errors one at a time.   Which one do you want to look at next?   The code that you provided -- is that enough to reproduce the problem?     What type of machine are you using?  How much memory?  how much disk space?

thanks

mike

John Qiang

Hi mike

     Thank you for reminding me. Yes, most errors are from OS, when i check the first error.

     We have developed a game, and BDB is our file engine.

     Our machines are usually home machine. Memory is about 4G, Hard drive takes about 3G, system Win7 x64/Win7 x86/XP in assembly machine.

Thank you very much

John