8 Replies Latest reply on Mar 21, 2013 9:07 PM by vinothchandar

    BIN delta logging

    vinothchandar
      Hi,

      I would like to better understand BIN delta logging and have some questions around it.

      -- From the code, I see that the BINDeltas only contain a reference to the full version, while I expected it to point to the last delta for the same BIN. I assume when a delta is logged, the parent IN is dirtied to point to the delta instead of the BIN? then how does a tree lookup reconstruct the entire BIN with all the deltas applied, without pointers to other deltas?


      -- Does the cleaner reconstruct the entire BIN when it encounters the a delta in the log? or simply relog the delta and dirty the parent IN.. (I guess the answer to the question above would auto clarify this as well)..

      Is there some design document somewhere that discusses this.

      Thanks
      Vinoth
        • 1. Re: BIN delta logging
          vinothchandar
          Hmmm.. I went over the 2006 architecture paper again and I kind of understand now I think.

          If I am right, at any given point there is one valid BIN and one valid delta log entry for a BIN? For example, if at time 1, we logged a full BIN and at time 2, key k2 was dirty, we would log a delta for the changed key k2 alone and point it to the full BIN. then if at time 3, a different key k3 was dirty, we would read the k2 delta and log a new delta with k2 and k3 combined? So, the size of the delta log would keep increasing until the configurable thresholds are hit?
          • 2. Re: BIN delta logging
            vinothchandar
            after going through the 2006 architecture paper again, I think I understand this. But would like a confirmation..

            At time 1, we log a new BIN. at time 2, key k2 in the BIN gets updated and a delta is logged for k2 pointing to full BIN. At time 3, when one more key k3 is dirtied, the delta entry for k2 is read and a new delta with both k2 and k3 is logged pointing again to the full BIN? So, the size of the delta records for the same BIN keep increasing until the configurable thresholds are hit and then another full BIN is logged? The cleaner would simply clean up the dead BIN delta entries..

            Thanks
            Vinoth
            • 3. Re: BIN delta logging
              Linda Lee-Oracle
              Vinoth,

              Yes, you have it right. A BINDelta encompasses all changes to a BIN, and so therefore at any one time, there is only a single BINDelta that needs to be applied. BINDeltas accumulate changes, so if a BINDelta is active, all precursors of that BINDelta are known to be obsolete.

              As of JE 5.0, on disk representations of higher level btree nodes (INs) may point to a child BIN, or to a BINDelta that represents that BIN. That BINDelta would refer to a single, full BIN parent.

              Linda
              • 4. Re: BIN delta logging
                vinothchandar
                Thanks for confirming.. One more question, given we are on SSDs and have no problems with random reads whatsoever, is there a downside to bumping up the TREE_MAX_DELTA parameter to its maximum? Intuitively, it seems like this can only help.

                Thanks
                Vinoth
                • 5. Re: BIN delta logging
                  Greybird-Oracle
                  Hi Vinoth,
                  Thanks for confirming.. One more question, given we are on SSDs and have no problems with random reads whatsoever, is there a downside to bumping up the TREE_MAX_DELTA parameter to its maximum? Intuitively, it seems like this can only help.
                  On JE 4.1, this may not be a good idea. The advantage of logging a full BIN in JE 4.1 is that the BIN will be marked non-dirty and won't be logged the next checkpoint. If a BIN is dirty but is no longer changing (no records are being written in the BIN), in JE 4.1 a delta will be logged at every checkpoint. Eventually, the size of these repeated deltas will be greater than if a full BIN were logged. This is because in JE 4.1, logging a delta does not mark the BIN non-dirty.

                  On JE 5, this is definitely a good idea and in fact I think we should disable and deprecate this setting in JE 5. In JE 5, logging a delta does mark the BIN non-dirty. There is a description of this change in the JE 5 change log -- search for [#19671].

                  --mark                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       
                  • 6. Re: BIN delta logging
                    vinothchandar
                    Hi Mark,

                    Thanks for the information. We are beginning to slowly rollout 5.0.73 in production. So, understanding this really helps.
                    If a BIN is dirty but is no longer changing (no records are being written in the BIN), in JE 4.1 a delta will be logged at every checkpoint.
                    I wonder what the 4.1 behavior is if the cache cannot fit all BINs and a steady state random read/write workload keeps dirtying BINs in random order. In most cases, then, only one key entry would be dirtied before the BIN is evicted. Would eviction log a delta and mark the BIN non-dirty then? since the BIN is no longer in cache, the checkpointer would not log anything for that BIN (of course the other BINs in the cache at the time of checkpoint, would undergo the behavior you described) ?

                    Thanks
                    Vinoth
                    • 7. Re: BIN delta logging
                      Greybird-Oracle
                      Good question.

                      In JE 4.1, BIN eviction always logs the full BIN, never a delta. This is because, in order to remove it from memory, we have to have a reference to it from the Btree. In 4.1, there was no support for a persistent reference from the Btree to a delta.

                      So in 4.1, if your workload is causing most BINs to be evicted, rather than checkpointed, you won't see very many deltas compared to full BINs. Deltas are only logged by checkpoints.

                      In JE 5, eviction does log a delta exactly as you described, and the Btree refers to the delta. So with the same workload in JE 5, you'll see a larger percentage of deltas compared to full BINs, which translates to less overall logging.

                      --mark                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       
                      • 8. Re: BIN delta logging
                        vinothchandar
                        Thanks mark.. That was helpful..

                        Thanks
                        Vinoth