10 Replies Latest reply: Jun 21, 2010 10:05 AM by 701681 RSS

    refresh a restarted spoke with a full set of data from the hub

    701681
      I have asked this question sometime back and was told it would be provided in a future implementation, is this available yet:

      Using push replication hub/spoke, how do I refresh a restarted spoke with a full set of data from the hub (i.e. not just the delta's that have been queued up on the hub since the restarting the spoke).

      Cheers,
      Neville.
        • 1. Re: refresh a restarted spoke with a full set of data from the hub
          Bob Hanckel
          Hi Neville,

          This has not been implemented yet but is a very high priority since we are hearing similar
          requests from various customers. For the moment, the workaround is to have an application
          force updates on cache elements in the hub, effectively updating all entries and scheduling
          then for replication to the spokes.

          Regards,

          Bob
          • 2. Re: refresh a restarted spoke with a full set of data from the hub
            701681
            Thanks Bob, do you have any rough timeframes of when this will be implemented?

            WRT manually updating all cache items in the hub, I am guessing an EntryProcessor is the safest way to do this, as we can take a fine grain lock and perhaps just write back the same binary value (i.e. avoid serialization cost).

            Something like this:

            public class TouchEntryPropcessor extends AbstractProcessor {
            public Object process(InvocableMap.Entry e) {
            BinaryEntry entry = (BinaryEntry) e;
            entry.updateBinaryValue(entry.getBinaryValue());
            return null;
            }
            }

            namedCache.invokeAll(new AlwaysFilter(), new TouchEntryPropcessor());

            Would the above work and force items to be replicated, i.e. even though the binary object has not actually changed? If not, do you have any other suggestions, as we do not really want to change the object.

            Cheers,
            Neville.
            • 3. Re: refresh a restarted spoke with a full set of data from the hub
              Bob Hanckel
              Hi Neville,

              We cannot publicly announce dates for future functionality, but synchronization
              of a cache between two clusters is very high priority. We are hearing this
              from a lot of customers.

              Getting to your workaround, this looks correct and the most efficient way
              of touching entries to force publicataion.

              Regards,

              Bob
              • 4. Re: refresh a restarted spoke with a full set of data from the hub
                701681
                Hi Bob,

                I have just tried to use the touch EP:

                public class TouchEntryPropcessor extends AbstractProcessor {
                public Object process(InvocableMap.Entry e) {
                BinaryEntry entry = (BinaryEntry) e;
                entry.updateBinaryValue(entry.getBinaryValue());
                return null;
                }
                }

                but unfotunetely it does not seem to generate a replication event (but it does fire an attached trigger).

                Is PRP doing some internal optimization so that it will not push duplicate binary values?

                Cheers,
                Neville.
                • 5. Re: refresh a restarted spoke with a full set of data from the hub
                  648232
                  Hey Nevill,

                  How do you know that the entry you've touched is not being replicated? If a trigger fires, the cache store should fire which should cause replication to happen. We do have a coalescing batch publisher, but you'd have to configure that, by default we replicate every mutating operation on a key in the cache.

                  -Noah
                  • 6. Re: refresh a restarted spoke with a full set of data from the hub
                    701681
                    I am 100% sure this is not firing a replication event as when i do a "normal" update i see:

                    Site A:
                    PRP logging saying its publsihing to remote site
                    My trigger fires and I see loggng for this.

                    Site B:
                    I see logging in my Conflict Resolution code, + some PRP logging.
                    My trigger fires and I see loggng for this.

                    When I try the TouchEP:

                    Site A:
                    My trigger fires and I see loggng for this. No logging at all for PRP.

                    Site B:
                    Nothing at all.

                    Cheers,
                    Neville.
                    • 7. Re: refresh a restarted spoke with a full set of data from the hub
                      648232
                      Gotcha Neville,

                      I just went and looked at how Coherence works for EntryProcessors and I found that Coherence proper will optimize out an update where the values are the same for storing in the backing map, but not for calling triggers. As such the cache store isn't fired for an equivalent update. What you could do is drop an additional decoration on the data. If you look at our PublishingCacheStore you'll see that we decorate the binaries with a map of values. I would recommend adding a value to that map, and if it doesn't exist, add the map and a new value. It can be small, just something to mutate the value.

                      -Noah
                      • 8. Re: refresh a restarted spoke with a full set of data from the hub
                        701681
                        Perfect, that worked. (i.e. custom decoration forces Binary to be different, which in turn forces push rep to site B, nice!)

                        Final EP is:

                        private static final int TOUCH_KEY = 1;

                        public Object process(InvocableMap.Entry entry) {
                        BinaryEntry binaryEntry = (BinaryEntry) entry;

                        //decorate
                        BackingMapManagerContext bmCtx = binaryEntry.getContext();
                        Map oldDecorations = (Map) bmCtx.getInternalValueDecoration(binaryEntry.getBinaryValue(), BackingMapManagerContext.DECO_CUSTOM);

                        Boolean flag;
                        if (oldDecorations == null || ((flag = (Boolean) oldDecorations.get(TOUCH_KEY)) == null)) {
                        oldDecorations = new HashMap();
                        oldDecorations.put(TOUCH_KEY, true);
                        } else {
                        oldDecorations.put(TOUCH_KEY, !flag); //make sure flag is different so EP can be run multiple times
                        }

                        Binary decoratedValue = (Binary) bmCtx.addInternalValueDecoration(binaryEntry.getBinaryValue(), BackingMapManagerContext.DECO_CUSTOM, oldDecorations);

                        binaryEntry.updateBinaryValue(decoratedValue);
                        return null;
                        }

                        Cheers,
                        Neville.
                        • 9. Re: refresh a restarted spoke with a full set of data from the hub
                          648232
                          Hey Neville,

                          That looks good, but I would recommend that you keep in mind that after the first time you run this entryprocessor it won't work again because you always set the value you to one.

                          -Noah
                          • 10. Re: refresh a restarted spoke with a full set of data from the hub
                            701681
                            EP changes flag from true to false (and vice versa) on every run right?

                            oldDecorations.put(TOUCH_KEY, !flag); //make sure flag is different so EP can be run multiple times

                            hence binary will be different.