Forum Stats

  • 3,855,236 Users
  • 2,264,483 Discussions
  • 7,905,940 Comments

Discussions

Federated Cache Pre-loading

James99991
James99991 Member Posts: 3
edited Oct 30, 2017 12:02PM in Coherence Support

Hi,

I'm planning on setting up a Federated Cache over two sites and am wondering about how the loading of the caches works. I have a few  questions on this topic really.

For the purposes of my questions, lets say my sites are set up with two clusters, Cluster A and Cluster B.

1, Have I understood the documentation right that (aside from HotCache) the suggested way of preloading a cache is to do a putAll from an external application ( https://docs.oracle.com/middleware/12213/coherence/develop-applications/performing-basic-cache-operations.htm#COHDG4859 )?

2, When each cluster starts, is it possible to get them to automatically load all the required data before they are usable/queryable?

3, If both clusters are fully populated. Cluster B goes down but A remains and is still successfully processing updates. When site B comes back up and the preloading needs to be done, how is this managed with respect to the federated changes that have been building up and are going to feed through when Cluster A sees that Cluster B is now available? Do the federated updates come through first before preloading or vice-versa?

Many thanks for your time,

James

Answers

  • Shyam Radhakrishnan-Oracle
    Shyam Radhakrishnan-Oracle Member Posts: 80
    edited Oct 26, 2017 9:06AM

    putAll can be one way, but should be batched. Another way of pre-loading is explained in the blog http://www.ateam-oracle.com/bulk-loading-a-coherence-cache-from-an-oracle-database/

    No, it is not possible to pre-load when the cluster is not usable, as the pre-loading will use the cluster itself. You can choose to use a CacheStore if pre-loading is not essential/

    When cluster B comes up, Cluster A will push all the data which could not federated into Cluster B. You could also choose to use the FederationManagerMBean operations https://docs.oracle.com/middleware/1221/coherence/manage/appendix_mbean.htm#COHMG5551

  • James99991
    James99991 Member Posts: 3
    edited Oct 27, 2017 3:54AM

    Hi Shyam, thanks for the reply. These look like good ways of loading data on a per cluster basis, and I've worked out that I can separate proxy nodes/members from data storage nodes thereby preventing clients from connecting to an individual cluster before it's loaded - I just start up the proxy nodes after I've preloaded. The thing I still need to resolve is (in relation to my question 3) whether or not I can preload the data in cluster B before cluster A realizes it's back online and attempts to federate any changed data across as this obviously doesn't use a proxy. If the federation happens before my preload and I then preload I run the risk of writing over the latest data with old. Any ideas on this would be much appreciated.

  • Viktar C
    Viktar C Member Posts: 31
    edited Oct 27, 2017 4:18PM

    If one of federated cluster for any reason has been restarted, you should manually trigger data replication for this cluster.

    Otherwise, only new cached data will be in sync.

  • James99991
    James99991 Member Posts: 3
    edited Oct 30, 2017 4:56AM

    Hi Viktar, thanks for the response. So are you saying the following should take place?

    1, Bring Cluster B back online

    2, Cluster A detects Cluster B is back and starts to send across the changes that occurred whilst cluster B was down.

    3, Manually tell Cluster A to replicateAll to ClusterB which then sends all the remaining data?

    The thing that concerns me about this is the potential for a large about of data being sent across the network to repopulate cluster B. If I had the data stored locally (to B) ready to preload into the cache then would it not be quicker to repopulate locally using a preload and then to allow federated changes through? If this is not possible then I understand.

    Thanks again.

  • Viktar C
    Viktar C Member Posts: 31
    edited Oct 30, 2017 12:02PM
    So are you saying the following should take place?1, Bring Cluster B back online2, Cluster A detects Cluster B is back and starts to send across the changes that occurred whilst cluster B was down.3, Manually tell Cluster A to replicateAll to ClusterB which then sends all the remaining data?

    That's correct.

    If I had the data stored locally (to B) ready to preload into the cache then would it not be quicker to repopulate locally using a preload and then to allow federated changes through?

    Pre-population could lead to clusters' data desync since a pre-populated entry could become outdated.

This discussion has been closed.