This discussion is archived
7 Replies Latest reply: Dec 15, 2011 11:19 PM by 857786 RSS

TXN_SYNC understanding

857786 Newbie
Currently Being Moderated
Hi,

The documentation states the following regarding the documentation of function txn_begin() and the flag DB_TXN_SYNC:

DB_TXN_SYNC

Synchronously flush the log when this transaction commits or prepares. This means the transaction will exhibit all of the ACID (atomicity, consistency, isolation, and durability) properties.

This behavior is the default for Berkeley DB environments unless the DB_TXN_NOSYNC flag was specified to the DbEnv::set_flags() method. Any value specified to this method overrides that setting.

I have an ambiguity regarding the last sentence:
"Any value specified to this method overrides that setting."

What method? The just aforementioned set_flags of DbEnv or txn_begin() itself.

In other terms if I specify this in the environment (DB_INIT_REP | DB_TXN_NOSYNC); as the flags(equivalent to set_flags) of it but then when creating a transaction with this environment using txn_begin() i specify DB_TXN_SYNC, which one will be taken into account? Will the transaction be SYNC or NOSYNC

I am using BDB 5.1.19.

Thx in advance.
  • 1. Re: TXN_SYNC understanding
    Oracle, Sandra Whitman Journeyer
    Currently Being Moderated
    Hello,

    If I understand your question correctly you want to know if
    you invoke DB_ENV->txn_begin() with the DB_TXN_SYNC flag
    and open the environment DB_ENV->open() with the DB_TXN_NOSYNC
    flag will the transaction synchronously flush the log when
    the transaction commits or prepares.

    In this case the transaction will flush the log synchronously
    because DB_TXN_NOSYNC was not specified to DB_ENV->set_flags().
    It is actually the case that DB_ENV->open() does not make use
    of the DB_TXN_NOSYNC flag. If you look at the documentation
    on DB_ENV->open() you will notice DB_TXN_NOSYNC is not mentioned:
    http://docs.oracle.com/cd/E17076_02/html/api_reference/C/envopen.html
    The value of DB_TXN_NOSYNC from <platform>/db.h is:
    #define DB_TXN_NOSYNC 0x00000001
    This is the same as DB_CREATE:
    #define DB_CREATE 0x00000001

    So setting DB_TXN_NOSYNC is the same as using DB_CREATE in
    DB_ENV->open(). It will not influence the way the transaction flushes
    the log on commit or prepare.


    Thank you,
    Sandra
  • 2. Re: TXN_SYNC understanding
    857786 Newbie
    Currently Being Moderated
    Thx Sandra, that's helpful.

    If i call explicitly to set_flags on the environment with TXN_NOSYNC, and create the transaction tn_begin() with TXN_SYNC, which one will be taken into account in this case when committing the transaction? Will it be flushed to log synchronously(like the transaction) or asynchronoulsy(like the environment)?

    Also I am using BDB 5.1.19, and i actually use the replication manager. Do i understand correctly that the difference between TXN_SYNC and TXN_NOSYNC as following?

    TXN_SYNC: this means that when the thread calls the function commit() on a given transaction, the call will be blocked(and hence the thread) until the data of the transaction is written to disk and sent to the replica and ack is received. Is this understanding accurate?

    TXN_NOSYNC: this means that when the thread calls the function commit() on a given transaction, the writin to disk + sending and receiving ack on the transaction will be done asynchronously which i believe is using another thread to get the async behavior. Is that understanding correct?

    Our application is very sensitive to latency and throughput and it's critical for us to get an accurate understanding of things work. Th in advance for your explanations.
  • 3. Re: TXN_SYNC understanding
    857786 Newbie
    Currently Being Moderated
    If i open a ticket in the support will the question be answered faster?
    Thx again for cooperation.
  • 4. Re: TXN_SYNC understanding
    Paula B Explorer
    Currently Being Moderated
    If i call explicitly to set_flags on the environment with TXN_NOSYNC, and create the transaction tn_begin() with TXN_SYNC, which one will be taken into account in this case when committing the transaction? Will it be flushed to log synchronously(like the transaction) or asynchronoulsy(like the environment)?
    The setting on the transaction is the one that is used and it overrides the setting on the environment.
    Also I am using BDB 5.1.19, and i actually use the replication manager. Do i understand correctly that the difference between TXN_SYNC and TXN_NOSYNC as following?

    TXN_SYNC: this means that when the thread calls the function commit() on a given transaction, the call will be blocked(and hence the thread) until the data of the transaction is written to disk and sent to the replica and ack is received. Is this understanding accurate?
    Basically you are correct with a few additional points.

    The commit thread waits up to the DB_REP_ACK_TIMEOUT for its acks rather than waiting forever.

    Also, there are some cases where we can determine in advance that we cannot possibly get enough acks to satisfy the ack policy. For example, if you have specified the ACKS_ALL policy but one site is not currently connected, we don't have to wait for the acks to know that we won't get enough. In these cases, we do not wait, but instead return immediately.
    TXN_NOSYNC: this means that when the thread calls the function commit() on a given transaction, the writin to disk + sending and receiving ack on the transaction will be done asynchronously which i believe is using another thread to get the async behavior. Is that understanding correct?
    No, this is not correct. Notice that our terminology is SYNC/ NOSYNC rather than SYNC /ASYNC.

    For TXN_NOSYNC, the commit of a transaction will not write its changes to the log and will not flush the log. Writing the changes to the log and flushing the log are the "SYNC" that we avoid doing if TXN_NOSYNC is set.

    Setting TXN_NOSYNC has no effect on whether Replication Manager waits for acks. The commit thread still waits for acks, subject to the exceptions I mentioned above.

    Paula Bingham
    Oracle
  • 5. Re: TXN_SYNC understanding
    857786 Newbie
    Currently Being Moderated
    Thx a lot that is very helpful.

    A few other quesitons:
    You say:
    "For TXN_NOSYNC, the commit of a transaction will not write its changes to the log and will not flush the log. Writing the changes to the log and flushing the log are the "SYNC" that we avoid doing if TXN_NOSYNC is set."

    I am confused about the TXN_NOSYNC, are you saying that when using TXN_NOSYNC, all the db-put of this transactions will not be written to the disk at all?
    If this is true does that mean if the slave is not there for 1 hour, and the master box reboots, does that mean that the all the transactions of the last hour will be lost?

    This makes me re-think my understanding of how BDB works.
    Is the following correct?
    A typical application like the one we develop is basically writing to database(db_put() for ex) and then committing transactions.
    My understanding is:
    - Every db->put will never write to disk but to the shared mem buffer of BDB. Is it correct or db->put could trigger write to disk too?
    - At commit time, all of the changes in the transaction will be put to log file as well as sent to the slave. Is it correct?

    Are you saying that at commit time with the transaction is TX_NOSYNC it will be just sent to the slave but not written to disk? Or is the NOSYNC saying just that the data is not written to the log immediately when performing the commit() call but later using some kind of thread?

    Thx in advance.

    Edited by: 854783 on Dec 15, 2011 12:27 AM
  • 6. Re: TXN_SYNC understanding
    Paula B Explorer
    Currently Being Moderated
    You say:
    "For TXN_NOSYNC, the commit of a transaction will not write its changes to the log and will not flush the log. Writing the changes to the log and flushing the log are the "SYNC" that we avoid doing if TXN_NOSYNC is set."

    I am confused about the TXN_NOSYNC, are you saying that when using TXN_NOSYNC, all the db-put of this transactions will not be written to the disk at all?
    No, the write of the changes to the log file and the log flush will happen at the earlier of the following two events: 1. your next checkpoint or 2. when the in-memory log buffer fills up and needs more space.

    You get a performance benefit by not doing separate disk I/Os for each transaction commit. These disk I/O's are instead batched up and done together at one of the above two times.
    If this is true does that mean if the slave is not there for 1 hour, and the master box reboots, does that mean that the all the transactions of the last hour will be lost?
    If you haven't done a checkpoint or if the in-memory log buffer hasn't happened to fill up in the last hour, then yes, those transactions are lost.

    The checkpoints are under your control. Use of TXN_NOSYNC needs to be paired with a checkpoint strategy that reflects the maximum amount of time that you want to risk transaction durability.


    >
    This makes me re-think my understanding of how BDB works.
    Is the following correct?
    A typical application like the one we develop is basically writing to database(db_put() for ex) and then committing transactions.
    My understanding is:
    - Every db->put will never write to disk but to the shared mem buffer of BDB. Is it correct or db->put could trigger write to disk too?
    No, db_put does not cause the log to be written to disk.
    - At commit time, all of the changes in the transaction will be put to log file as well as sent to the slave. Is it correct?
    If you do not use TXN_NOSYNC, the changes are written to the log file at commit time.

    The changes are sent to the slave as they occur. At commit time, a commit log record is sent to the slave. The replication master waits for acks only on a commit record (not all the other changes) if required by your ack policy.

    >
    Are you saying that at commit time with the transaction is TX_NOSYNC it will be just sent to the slave but not written to disk? Or is the NOSYNC saying just that the data is not written to the log immediately when performing the commit() call but later using some kind of thread?
    Regardless of TXN_NOSYNC, changes are sent to the slave as they occur.

    When using TXN_NOSYNC, changes are written to the in-memory log buffer as they occur. The log buffer is written to the log disk file and the log disk file is flushed either at your next checkpoint or when the log buffer fills up. The write to the log disk file is not done in another thread, but rather in the same thread at one of these two times.

    If you anticipate failures at replicated sites, you can also add more replicated sites to improve the chances that a committed transaction is durably stored somewhere.

    Paula Bingham
    Oracle
  • 7. Re: TXN_SYNC understanding
    857786 Newbie
    Currently Being Moderated
    Thx a lot. That is helpful.

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points