3 Replies Latest reply: May 1, 2012 10:13 AM by Laurenfoutz-Oracle RSS

    Read locks created after committing a transaction?

    929429
      I'm running into an issue where when I open (possibly creating) a database under a transaction, it seems as though when I invoke the commit on the transaction, it acquires a read lock on the database, and I have no way of releasing these locks. I assume I'm just doing something wrong, but don't know what -- the documentation hasn't been helpful for determining what I'm doing wrong.

      This is problematic, because it's not possible to acquire a WRITE lock on these databases later, when trying to invoke a dbremove on both.

      The sequence of events, along with what locks are acquired, is this:

      1. Begin a transaction.
      2. Create the main and secondary database get created (the second will be a secondary index on the first).
      3. Open the main database; a WRITE lock is acquired on the main database handle with the transaction's lock ID.
      4. Set a key comparison function for the secondary database.
      5. Open the secondary database; a WRITE lock is acquired on the secondary database handle with the transaction's lock ID.
      6. Associate the main and secondary databases.
      7. Create and initialize a sequence.
      8. Open the sequence in the main database; a WRITE lock is acquired on a page in the main database, using the transaction's lock ID.
      9. Commit the transaction; a READ lock is acquired on the main and secondary DB handles with plain locker ID's (not the transaction's lock ID).

      It also acquires similar READ locks if I, say, open a transaction, do a stat on both databases, and commit the transaction.

      Here is the code that is opening the databases:

      // Open the env, in one function...
      if (env->env->open(env->env, name, DB_CREATE | DB_INIT_LOG | DB_INIT_LOCK | DB_INIT_MPOOL | DB_THREAD | DB_INIT_TXN, 0) != 0)
      {
      free(env);
      return NULL;
      }

      ...

      // Open a database, in another function...
      if (txn_begin(env->env, NULL, &txn, DB_TXN_WRITE_NOSYNC) != 0)
      {
      free(store);
      return LA_STORAGE_OPEN_ERROR;
      }
      if (db_create(&store->db, env->env, 0) != 0)
      {
      txn_abort(txn);
      free(store);
      return LA_STORAGE_OPEN_ERROR;
      }
      if (db_create(&store->seq_db, env->env, 0) != 0)
      {
      txn_abort(txn);
      free(store);
      return LA_STORAGE_OPEN_ERROR;
      }
      dbflags = 0;
      if (flags & LA_STORAGE_OPEN_FLAG_CREATE)
      dbflags = DB_CREATE;
      if (flags & LA_STORAGE_OPEN_FLAG_EXCL)
      dbflags |= DB_EXCL;
      if ((ret = store->db->open(store->db, txn, path, NULL, DB_BTREE, dbflags | DB_MULTIVERSION | DB_THREAD, 0)) != 0)
      {
      txn_abort(txn);
      free(store);
      if (ret == EEXIST)
      return LA_STORAGE_OPEN_EXISTS;
      if (ret == ENOENT)
      return LA_STORAGE_OPEN_NOT_FOUND;
      return LA_STORAGE_OPEN_ERROR;
      }
      seqpath = string_append(path, ".seq");
      if (seqpath == NULL)
      {
      store->db->close(store->db, 0);
      txn_abort(txn);
      free(store);
      return LA_STORAGE_OPEN_ERROR;
      }
      if (store->seq_db->set_bt_compare(store->seq_db, compare_seq) != 0)
      {
      store->db->close(store->db, 0);
      txn_abort(txn);
      free(store);
      return LA_STORAGE_OPEN_ERROR;
      }
      if (store->seq_db->open(store->seq_db, txn, seqpath, NULL, DB_BTREE, DB_CREATE | DB_THREAD, 0) != 0)
      {
      free(seqpath);
      store->db->close(store->db, 0);
      txn_abort(txn);
      free(store);
      return LA_STORAGE_OPEN_ERROR;
      }
      free(seqpath);
      store->db->associate(store->db, txn, store->seq_db, seqindex, 0);
      if (db_sequence_create(&store->seq, store->db, 0) != 0)
      {
      store->seq_db->close(store->seq_db, 0);
      store->db->close(store->db, 0);
      txn_abort(txn);
      free(store);
      return LA_STORAGE_OPEN_ERROR;
      }
      store->seq->initial_value(store->seq, 1);
      seq_name[0] = '\0';
      seq_name[1] = 'S';
      seq_name[2] = 'E';
      seq_name[3] = 'Q';
      seq_key.data = seq_name;
      seq_key.size = 4;
      seq_key.ulen = 4;
      seq_key.flags = DB_DBT_USERMEM;
      if (store->seq->open(store->seq, txn, &seq_key, DB_CREATE | DB_THREAD) != 0)
      {
      store->seq_db->close(store->seq_db, 0);
      store->db->close(store->db, 0);
      txn_abort(txn);
      free(store);
      return LA_STORAGE_OPEN_ERROR;
      }
      txn_commit(txn, DB_TXN_NOSYNC);
        • 1. Re: Read locks created after committing a transaction?
          Laurenfoutz-Oracle
          The read lock on the database is expected. BDB holds a read lock on a database as long as it has an open handle in order to prevent the database from being deleted out from under the handle, which would cause crashes and other errors. If you want to use dbremove on a database you have to close all handles on that database, then call dbremove.

          Lauren Foutz
          • 2. Re: Read locks created after committing a transaction?
            929429
            Makes sense, but:

            1. Calling close on the database handles does not appear to release these locks. I do call close on the database handles before attempting the dbremove, and the close calls are succeeding.
            2. Multiple locks appear to be created; for instance, each time I perform a stat on the databases, another lock is acquired. Why is this?

            Am I using transactions wrong? Generally each operation does this:

            begin transaction
            perform the operation(s), abort if any errors occur.
            commit the transaction

            The "stat" I'm doing does these two operations:


            DB_SEQUENCE_STAT *stat;
            db_seq_t seq;
            if (store->seq->stat(store->seq, &stat, 0) != 0)
            return 0;
            seq = stat->st_current;
            free(stat);
            return seq;

            And:

            DB_TXN *txn;
            DB_BTREE_STAT mainStat, seqStat;
            if (txn_begin(store->env->env, NULL, &txn, DB_READ_COMMITTED | DB_TXN_NOSYNC) != 0)
            return -1;
            if (store->db->stat(store->db, txn, &mainStat, DB_FAST_STAT | DB_READ_COMMITTED) != 0)
            {
            txn_abort(txn);
            return -1;
            }
            if (store->db->stat(store->seq_db, txn, &seqStat, DB_FAST_STAT | DB_READ_COMMITTED) != 0)
            {
            txn_abort(txn);
            return -1;
            }
            stat->numkeys = mainStat->bt_nkeys;
            stat->size = ((uint64_t) mainStat->bt_pagecnt * (uint64_t) mainStat->bt_pagesize)
            + ((uint64_t) seqStat->bt_pagecnt * (uint64_t) seqStat->bt_pagesize);
            free(mainStat);
            free(seqStat);
            txn_commit(txn, DB_TXN_NOSYNC);
            return 0;

            (txn_commit et al, BTW, are macros/functions I have that also log the event in my debug builds)
            • 3. Re: Read locks created after committing a transaction?
              Laurenfoutz-Oracle
              The db_sequence holds open a handle on the DB database you used to open it with, because it uses that database to backup its values. So you will have to close the db_sequence before you can remove the database. As for the extra read you are seeing when you do a db_stat, db_stat does have to open the database to collect its data, so you will see that read lock in its output. The read lock will be released after db_stat finishes.

              Lauren Foutz