2 Replies Latest reply on Aug 29, 2013 3:49 PM by rmoff

    Keeping two very large datastores in sync


      I'm looking at options for keeping a very large (potentially 400GB) TimesTen ( datastore in sync between a Production server and a [warm] Standby.


      Replication has been discounted because it doesn't support compressed tables, nor the types of table our closed-code application is creating (without non-null PKs)


      1. I've done some testing with smaller datastores to get indicative numbers, and a 7.4GB datastore (according to dssize) resulted in a 35GB backup set (using ttBackup -type fileIncrOrFull). Is that large increase in volume expected, and would it extrapolate up for a 400GB data store (2TB backup set??)?
      2. I've seen that there are Incremental backups, but to maintain our standby as warm, we'll be restoring these backups and from what I'd read & tested only a ttDestroy/ttRestore is possible, i.e. complete restore of the complete DSN each time, which is time consuming. Am I missing a smarter way of doing this?
      3. Other than building our application to keep the two datastores in sync, are there any other tricks we can use to efficiently keep the two datastores in sync?
      4. Random last question - I see "datastore" and "database" (and to an extent, "DSN") used apparently interchangeably - are they the same thing in TimesTen?


      Update: the 35GB compresses down with 7za to just over 2.2GB, but takes 5.5 hours to do so. If I take a standalone fileFull backup it is just 7.4GB on disk, and completes faster too.






      Message was edited by: rmoff - add additional detail

        • 1. Re: Keeping two very large datastores in sync

          This must be an Exalytics system, right? I ask this because compressed tables are not licensed for use outside of an Exalytics system...


          As you note, currently replication is not possible in an Exalytics environment, but that is likely to change in the future and then it will definitely be the preferred mechanism for this. There is not really any other viable way to do this other than through the application.


          With regard to your specific questions:


          1.   A backup consists primarily of the most recent checkpoint file plus all log files/records that are newer than that file. So, to minimise the size of a full backup ensure

               that a checkpoint occurs (for example 'call ttCkpt' from a ttIsql session) immediately prior to starting the backup.


          2.   No, only complete restore is possible from an incremental backup set. Also note that due to the large amount of rollforward needed, restoring a large incremental backup set may take quite a long time. Backup and restore are not really intended for this purpose.


          3.   If you cannot use replication then some kind of application level sync is your only option.


          4.   Datastore and database mean the same thing - a physical TimesTen database. We prefer the term database nowadays; datastore is a legacy term. A DSN is a different thing (Data Source Name) and should not be used interchangeably with datastore/database. A DSN is a logical entity that defines the attributes for a database and how to connect to it. It is not the same as a database.



          • 2. Re: Keeping two very large datastores in sync

            Awesome, thanks Chris. Yes you're right, Exalytics.