This discussion is archived
10 Replies Latest reply: Jun 21, 2010 8:05 AM by 701681 RSS

refresh a restarted spoke with a full set of data from the hub

701681 Explorer
Currently Being Moderated
I have asked this question sometime back and was told it would be provided in a future implementation, is this available yet:

Using push replication hub/spoke, how do I refresh a restarted spoke with a full set of data from the hub (i.e. not just the delta's that have been queued up on the hub since the restarting the spoke).

Cheers,
Neville.
  • 1. Re: refresh a restarted spoke with a full set of data from the hub
    Bob Hanckel Explorer
    Currently Being Moderated
    Hi Neville,

    This has not been implemented yet but is a very high priority since we are hearing similar
    requests from various customers. For the moment, the workaround is to have an application
    force updates on cache elements in the hub, effectively updating all entries and scheduling
    then for replication to the spokes.

    Regards,

    Bob
  • 2. Re: refresh a restarted spoke with a full set of data from the hub
    701681 Explorer
    Currently Being Moderated
    Thanks Bob, do you have any rough timeframes of when this will be implemented?

    WRT manually updating all cache items in the hub, I am guessing an EntryProcessor is the safest way to do this, as we can take a fine grain lock and perhaps just write back the same binary value (i.e. avoid serialization cost).

    Something like this:

    public class TouchEntryPropcessor extends AbstractProcessor {
    public Object process(InvocableMap.Entry e) {
    BinaryEntry entry = (BinaryEntry) e;
    entry.updateBinaryValue(entry.getBinaryValue());
    return null;
    }
    }

    namedCache.invokeAll(new AlwaysFilter(), new TouchEntryPropcessor());

    Would the above work and force items to be replicated, i.e. even though the binary object has not actually changed? If not, do you have any other suggestions, as we do not really want to change the object.

    Cheers,
    Neville.
  • 3. Re: refresh a restarted spoke with a full set of data from the hub
    Bob Hanckel Explorer
    Currently Being Moderated
    Hi Neville,

    We cannot publicly announce dates for future functionality, but synchronization
    of a cache between two clusters is very high priority. We are hearing this
    from a lot of customers.

    Getting to your workaround, this looks correct and the most efficient way
    of touching entries to force publicataion.

    Regards,

    Bob
  • 4. Re: refresh a restarted spoke with a full set of data from the hub
    701681 Explorer
    Currently Being Moderated
    Hi Bob,

    I have just tried to use the touch EP:

    public class TouchEntryPropcessor extends AbstractProcessor {
    public Object process(InvocableMap.Entry e) {
    BinaryEntry entry = (BinaryEntry) e;
    entry.updateBinaryValue(entry.getBinaryValue());
    return null;
    }
    }

    but unfotunetely it does not seem to generate a replication event (but it does fire an attached trigger).

    Is PRP doing some internal optimization so that it will not push duplicate binary values?

    Cheers,
    Neville.
  • 5. Re: refresh a restarted spoke with a full set of data from the hub
    648232 Explorer
    Currently Being Moderated
    Hey Nevill,

    How do you know that the entry you've touched is not being replicated? If a trigger fires, the cache store should fire which should cause replication to happen. We do have a coalescing batch publisher, but you'd have to configure that, by default we replicate every mutating operation on a key in the cache.

    -Noah
  • 6. Re: refresh a restarted spoke with a full set of data from the hub
    701681 Explorer
    Currently Being Moderated
    I am 100% sure this is not firing a replication event as when i do a "normal" update i see:

    Site A:
    PRP logging saying its publsihing to remote site
    My trigger fires and I see loggng for this.

    Site B:
    I see logging in my Conflict Resolution code, + some PRP logging.
    My trigger fires and I see loggng for this.

    When I try the TouchEP:

    Site A:
    My trigger fires and I see loggng for this. No logging at all for PRP.

    Site B:
    Nothing at all.

    Cheers,
    Neville.
  • 7. Re: refresh a restarted spoke with a full set of data from the hub
    648232 Explorer
    Currently Being Moderated
    Gotcha Neville,

    I just went and looked at how Coherence works for EntryProcessors and I found that Coherence proper will optimize out an update where the values are the same for storing in the backing map, but not for calling triggers. As such the cache store isn't fired for an equivalent update. What you could do is drop an additional decoration on the data. If you look at our PublishingCacheStore you'll see that we decorate the binaries with a map of values. I would recommend adding a value to that map, and if it doesn't exist, add the map and a new value. It can be small, just something to mutate the value.

    -Noah
  • 8. Re: refresh a restarted spoke with a full set of data from the hub
    701681 Explorer
    Currently Being Moderated
    Perfect, that worked. (i.e. custom decoration forces Binary to be different, which in turn forces push rep to site B, nice!)

    Final EP is:

    private static final int TOUCH_KEY = 1;

    public Object process(InvocableMap.Entry entry) {
    BinaryEntry binaryEntry = (BinaryEntry) entry;

    //decorate
    BackingMapManagerContext bmCtx = binaryEntry.getContext();
    Map oldDecorations = (Map) bmCtx.getInternalValueDecoration(binaryEntry.getBinaryValue(), BackingMapManagerContext.DECO_CUSTOM);

    Boolean flag;
    if (oldDecorations == null || ((flag = (Boolean) oldDecorations.get(TOUCH_KEY)) == null)) {
    oldDecorations = new HashMap();
    oldDecorations.put(TOUCH_KEY, true);
    } else {
    oldDecorations.put(TOUCH_KEY, !flag); //make sure flag is different so EP can be run multiple times
    }

    Binary decoratedValue = (Binary) bmCtx.addInternalValueDecoration(binaryEntry.getBinaryValue(), BackingMapManagerContext.DECO_CUSTOM, oldDecorations);

    binaryEntry.updateBinaryValue(decoratedValue);
    return null;
    }

    Cheers,
    Neville.
  • 9. Re: refresh a restarted spoke with a full set of data from the hub
    648232 Explorer
    Currently Being Moderated
    Hey Neville,

    That looks good, but I would recommend that you keep in mind that after the first time you run this entryprocessor it won't work again because you always set the value you to one.

    -Noah
  • 10. Re: refresh a restarted spoke with a full set of data from the hub
    701681 Explorer
    Currently Being Moderated
    EP changes flag from true to false (and vice versa) on every run right?

    oldDecorations.put(TOUCH_KEY, !flag); //make sure flag is different so EP can be run multiple times

    hence binary will be different.

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points