2 Replies Latest reply: Dec 12, 2012 2:45 AM by 714794 RSS

    Trail record size & check frecuency for end of uncommitted transactions

    714794
      Hi, everyone,

      Does anyone know if the trail record size can be changed from its default value of 4K?

      What about how long the data pump process delays before searching for more data to proccess in its source trail while it waits for the end of a uncommitted transaction (which is 1 second)?
        • 1. Re: Trail record size & check frecuency for end of uncommitted transactions
          MikeN
          beatrizia wrote:
          Does anyone know if the trail record size can be changed from its default value of 4K?
          In short, no. But I'm not sure what the goal is... GoldenGate more or less handles this automatically already; the record size can be larger than 4K in some cases (which creates certain limitations, e.g,. w/ RmtTask -- in which case, just don't use RmtTask and instead use a normal extract with RmtFile + MaxFiles 99999).

          But I think you may be talking about how GG sends data across the network vs. how GG stores the data in the trail file; in this case, see the TcpBufSize option of RmtHost to tune either your network or GG (see ref guide, "Determining the optimum buffer size").

          What about how long the data pump process delays before searching for more data to proccess in its source trail while it waits for the end of a uncommitted transaction (which is 1 second)?
          See EofDelay or EofDelayCSecs option (or EofDelayMS for RAC). In low volume replication, this setting can be used to either increase the delay between checks for more data in the redo logs to reduce I/O on the DB, or it can be used to decrease latency by checking more frequently (at the expense of hitting the disk more frequently). For data pumps (i.e., extract reading trails, vs. extract reading DB redo logs), I don't think you can set it less than a second, though. Check the docs in your specific release of GG, as this default has changed across versions to be optimal for most people. The default for the DB redo as the source is different than the default for a trail file source. (I think the default is 250ms in GG 11.2 for RAC.)

          Note that in high-volume processing, when there is no "waiting" for more data, this setting has no effect. So to achieve the lowest latency (delay between source & target), it's best when there is actually high and consistent volume (i.e, don't run benchmarking tests that insert 10k rows, sleep 2 seconds, insert another 10k rows, sleep, ....; plus, that's not "real world" behavior).
          • 2. Re: Trail record size & check frecuency for end of uncommitted transactions
            714794
            Thank you for your answer, MikeN

            The delay I'm referring to cannot be set with EofDelay or EofDelayCSecs: these parameters establish how much time the Data Pump process sleeps when it has nothing new in its source trail. The delay that bothers me seems to happen when the Data Pump has nothing new in its source trail but it is in the middle of an open transaction processing.

            I think is better explained with an example (which I think explains the goal of changing the record size too):

            This is an excerpt for the Extract process trace:

            *09:51:59.653622 write(20, "G\1\0\307H\0\0<E\4\0A\0F\5\377\2\361\361\317\21\232\265\300\0\0\0\0\0)h\20"..., 4096) = 4096* <----- Extract writes down the first 4K record of the transaction
            09:51:59.653690 time(NULL) = 1349769119
            09:51:59.653726 time(NULL) = 1349769119
            09:51:59.653763 time(NULL) = 1349769119
            09:51:59.653803 time(NULL) = 1349769119
            09:51:59.653838 time(NULL) = 1349769119
            09:51:59.653877 time(NULL) = 1349769119
            09:51:59.653913 time(NULL) = 1349769119
            09:51:59.653948 time(NULL) = 1349769119
            09:51:59.653987 time(NULL) = 1349769119
            09:51:59.654024 time(NULL) = 1349769119
            09:51:59.654058 time(NULL) = 1349769119
            09:51:59.654097 time(NULL) = 1349769119
            09:51:59.654140 time(NULL) = 1349769119
            09:51:59.654174 gettimeofday({1349769119, 654182}, NULL) = 0
            09:51:59.654207 clock_gettime(CLOCK_REALTIME, {1349769119, 654216293}) = 0
            09:51:59.654234 futex(0x9b62584, FUTEX_WAIT_PRIVATE, 957, {0, 999965707}) = 0
            09:51:59.751502 futex(0x9b62568, FUTEX_WAKE_PRIVATE, 1) = 0
            09:51:59.751554 llseek(19, 2722304, [2722304], SEEKSET) = 0
            09:51:59.751608 futex(0x9b62534, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x9b62530, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
            09:51:59.751682 nanosleep({0, 0}, NULL) = 0
            (….)
            *09:52:00.162689 write(20, "\0D\0\0O\0\0\0\30\0\0\0\0240000100050134977631"..., 2374) = 2374* <----- Extract writes down the remaining data for the transaction

            And this is an excerpt of the corresponding Data Pump process trace:

            09:51:59.653398 read(11, "F\0\4/0\0\1\3210\0\0\10GG\r\nTL\n\r1\0\0\2\0\0032\0\0\4 \0"..., 1048576) = 7604
            09:51:59.653472 stat64("/stella_dat/ggate/tlstella/tl000195", 0xbfca2a0c) = -1 ENOENT (No such file or directory)
            09:51:59.653543 nanosleep({0, 0}, NULL) = 0
            09:51:59.653651 llseek(11, 0, [0], SEEKSET) = 0
            *09:51:59.653543 nanosleep({0, 0}, NULL) = 0* <---- This is EOFDELAY: it's set to 0
            09:51:59.653651 llseek(11, 0, [0], SEEKSET) = 0
            *09:51:59.653709 read(11, "F\0\4/0\0\1\3210\0\0\10GG\r\nTL\n\r1\0\0\2\0\0032\0\0\4 \0"..., 1048576) = 11700* <----- Data Pump detects a new record in the source trail
            09:51:59.653767 read(11, "", 1048576) = 0
            09:51:59.653840 time(NULL) = 1349769119
            09:51:59.653910 time(NULL) = 1349769119
            09:51:59.653959 time(NULL) = 1349769119
            09:51:59.654014 time(NULL) = 1349769119
            09:51:59.654067 time(NULL) = 1349769119
            09:51:59.654123 time(NULL) = 1349769119
            09:51:59.654181 time(NULL) = 1349769119
            09:51:59.654232 time(NULL) = 1349769119
            09:51:59.654274 time(NULL) = 1349769119
            09:51:59.654312 time(NULL) = 1349769119
            09:51:59.654351 time(NULL) = 1349769119
            09:51:59.654389 time(NULL) = 1349769119
            09:51:59.654428 time(NULL) = 1349769119
            09:51:59.654467 time(NULL) = 1349769119
            09:51:59.654505 time(NULL) = 1349769119
            09:51:59.654543 time(NULL) = 1349769119
            09:51:59.654582 time(NULL) = 1349769119
            09:51:59.654620 time(NULL) = 1349769119
            09:51:59.654657 time(NULL) = 1349769119
            09:51:59.654695 time(NULL) = 1349769119
            09:51:59.654733 time(NULL) = 1349769119
            09:51:59.654771 time(NULL) = 1349769119
            09:51:59.654809 time(NULL) = 1349769119
            09:51:59.654844 read(11, "", 1048576) = 0
            *09:51:59.654881 nanosleep({1, 0}, NULL) = 0* <----- This is the 1 second delay that I want to get rid of
            *09:52:00.655079 read(11, "\0D\0\0O\0\0\0\30\0\0\0\0240000100050134977631"..., 1048576) = 2374* <----- Data Pump reads the second record of the transaction