One of the common causes of this is that the system clocks are not adequately in sync between the two machines. TimesTen A/S pair replication requires the system clocks to be synchronised to an accuracy of < 250 ms difference. Check the message and error logs (<tt_install_dir>/info/ttmesg.log and <tt_install_dir>/info/tterrors.log>) for mesages complaining about clock skew.
We'd recommend that you use NTP or similar to keep the clocks in sync.
Wow you nailed it. That was the issue. I completely forgot to look at the tterrors.log the problem was in clear sight...ugh. Thank you so much.
I have my TimesTen servers setup on Oracle Linux and I sync'd the clocks on both servers by doing
yum install ntpdate
ntpdate 0.rhel.pool.ntp.org 1.rhel.pool.ntp.org
chkconfig ntpdate on
Message was edited by: Victor updated the ntpdate command I used as well as added the "chkconfig ntpdate on" command