1 Reply Latest reply: Oct 27, 2013 4:56 PM by Leo_TA RSS

    Issue to setup local Coherence cluster with WKA (well-known-address)

    Sajeev Pynadath

      Hello - I have started local coherence cluster using WKA with single node,but when I start CacheFactory (coherence.cmd) with same configuration it throws following error message.

       

      Any help is appricicated.

       

      JVM startup Arrgument

      -Dtangosol.coherence.override=cluster.xml

       

      cluster.xml

      <?xml version="1.0" encoding="UTF-8"?>

      <coherence xmlns="http://xmlns.oracle.com/coherence/coherence-operational-config" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-operational-config http://xmlns.oracle.com/coherence/coherence-operational-config/1.1/coherence-operational-config.xsd">

      <cluster-config>

        <unicast-listener>

         <well-known-addresses>

          <socket-address id="1">

           <address>171.193.103.25</address>

           <port>8088</port>

          </socket-address>

         </well-known-addresses>

            </unicast-listener>

      </cluster-config>

      <logging-config>

        <destination>stdout</destination>

        <severity-level>9</severity-level>

      </logging-config>

      </coherence>

       

       

      Cluster startup Message

       

      WellKnownAddressList(Size=1,
        WKA{Address=171.193.103.25, Port=8088}
        )

      MasterMemberSet(
        ThisMember=Member(Id=1, Timestamp=2013-10-24 11:07:18.603, Address=171.193.103.25:8088, MachineId=9041, Location=site:,machine:FD4C9EF534D5D,process:16704, Role=CoherenceServer)
        OldestMember=Member(Id=1, Timestamp=2013-10-24 11:07:18.603, Address=171.193.103.25:8088, MachineId=9041, Location=site:,machine:FD4C9EF534D5D,process:16704, Role=CoherenceServer)
        ActualMemberSet=MemberSet(Size=1
          Member(Id=1, Timestamp=2013-10-24 11:07:18.603, Address=171.193.103.25:8088, MachineId=9041, Location=site:,machine:FD4C9EF534D5D,process:16704, Role=CoherenceServer)
          )
        MemberId|ServiceVersion|ServiceJoined|MemberState
          1|3.7.1|2013-10-24 11:07:48.843|JOINED
        RecycleMillis=1200000
        RecycleSet=MemberSet(Size=0
          )
        )

      TcpRing{Connections=[]}
      IpMonitor{AddressListSize=0}

      2013-10-24 11:07:48.869/31.794 Oracle Coherence GE 3.7.1.0 <D5> (thread=Invocation:Management, member=1): Service Management joined the cluster with senior service member 1
      2013-10-24 11:07:49.058/31.983 Oracle Coherence GE 3.7.1.0 <D5> (thread=DistributedCache, member=1): Service DistributedCache joined the cluster with senior service member 1
      2013-10-24 11:07:49.077/32.002 Oracle Coherence GE 3.7.1.0 <D6> (thread=DistributedCache, member=1): Service DistributedCache: sending PartitionConfig ConfigSync to all
      2013-10-24 11:07:49.121/32.046 Oracle Coherence GE 3.7.1.0 <D5> (thread=ReplicatedCache, member=1): Service ReplicatedCache joined the cluster with senior service member 1
      2013-10-24 11:07:49.128/32.053 Oracle Coherence GE 3.7.1.0 <D5> (thread=OptimisticCache, member=1): Service OptimisticCache joined the cluster with senior service member 1
      2013-10-24 11:07:49.131/32.056 Oracle Coherence GE 3.7.1.0 <D5> (thread=Invocation:InvocationService, member=1): Service InvocationService joined the cluster with senior service member 1
      2013-10-24 11:07:49.132/32.057 Oracle Coherence GE 3.7.1.0 <Info> (thread=main, member=1):
      Services
        (
        ClusterService{Name=Cluster, State=(SERVICE_STARTED, STATE_JOINED), Id=0, Version=3.7.1, OldestMemberId=1}
        InvocationService{Name=Management, State=(SERVICE_STARTED), Id=1, Version=3.1, OldestMemberId=1}
        PartitionedCache{Name=DistributedCache, State=(SERVICE_STARTED), LocalStorage=enabled, PartitionCount=257, BackupCount=1, AssignedPartitions=257, BackupPartitions=0}
        ReplicatedCache{Name=ReplicatedCache, State=(SERVICE_STARTED), Id=3, Version=3.0, OldestMemberId=1}
        Optimistic{Name=OptimisticCache, State=(SERVICE_STARTED), Id=4, Version=3.0, OldestMemberId=1}
        InvocationService{Name=InvocationService, State=(SERVICE_STARTED), Id=5, Version=3.1, OldestMemberId=1}
        )

      Started DefaultCacheServer...

       

      Error Message from CacheFactory

       

      C:\Users\Zk5rjg8>C:\coherence37\bin\coherence.cmd
      ** Starting storage disabled console **
      java version "1.6.0_51"
      Java(TM) SE Runtime Environment (build 1.6.0_51-b11)
      Java HotSpot(TM) 64-Bit Server VM (build 20.51-b01, mixed mode)

      2013-10-24 11:13:22.851/0.392 Oracle Coherence 3.7.1.0 <Info> (thread=main, member=n/a): Loaded operational configuration from "jar:file:/C:/coherence37/lib/coherence.jar!/tangosol-coherence.xml"
      2013-10-24 11:13:22.920/0.462 Oracle Coherence 3.7.1.0 <Info> (thread=main, member=n/a): Loaded operational overrides from "file:/C:/coherence37/cluster.xml"
      2013-10-24 11:13:22.924/0.465 Oracle Coherence 3.7.1.0 <D5> (thread=main, member=n/a): Optional configuration override "/custom-mbeans.xml" is not specified
      2013-10-24 11:13:22.924/0.465 Oracle Coherence 3.7.1.0 <D6> (thread=main, member=n/a): Loaded edition data from "jar:file:/C:/coherence37/lib/coherence.jar!/coherence-grid.xml"

      Oracle Coherence Version 3.7.1.0 Build 27797
      Grid Edition: Development mode
      Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.

      2013-10-24 11:13:23.722/1.263 Oracle Coherence GE 3.7.1.0 <D4> (thread=main, member=n/a): TCMP bound to /171.193.103.25:8090 using SystemSocketProvider
      2013-10-24 11:13:54.001/31.542 Oracle Coherence GE 3.7.1.0 <Warning> (thread=Cluster, member=n/a): This Member(Id=0, Timestamp=2013-10-24 11:13:23.762, Address=171.193.103.25:8090, MachineId=9041, Location=site:,machine:FD4C9EF534D5D,process:17192, Role=CoherenceConsole) has been attempting to joi
      2013-10-24 11:13:54.001/31.542 Oracle Coherence GE 3.7.1.0 <Warning> (thread=Cluster, member=n/a): Delaying formation of a new cluster; waiting for well-known nodes to respond
      2013-10-24 11:14:24.402/61.943 Oracle Coherence GE 3.7.1.0 <Warning> (thread=Cluster, member=n/a): Delaying formation of a new cluster; waiting for well-known nodes to respond
      2013-10-24 11:14:54.805/92.346 Oracle Coherence GE 3.7.1.0 <Warning> (thread=Cluster, member=n/a): Delaying formation of a new cluster; waiting for well-known nodes to respond
      2013-10-24 11:15:25.207/122.748 Oracle Coherence GE 3.7.1.0 <Warning> (thread=Cluster, member=n/a): Delaying formation of a new cluster; waiting for well-known nodes to respond
      2013-10-24 11:15:55.610/153.151 Oracle Coherence GE 3.7.1.0 <Warning> (thread=Cluster, member=n/a): Delaying formation of a new cluster; waiting for well-known nodes to respond
      2013-10-24 11:16:26.012/183.553 Oracle Coherence GE 3.7.1.0 <Warning> (thread=Cluster, member=n/a): Delaying formation of a new cluster; waiting for well-known nodes to respond
      2013-10-24 11:16:56.414/213.955 Oracle Coherence GE 3.7.1.0 <Warning> (thread=Cluster, member=n/a): Delaying formation of a new cluster; waiting for well-known nodes to respond
      2013-10-24 11:17:26.817/244.358 Oracle Coherence GE 3.7.1.0 <Warning> (thread=Cluster, member=n/a): Delaying formation of a new cluster; waiting for well-known nodes to respond
      2013-10-24 11:17:57.219/274.760 Oracle Coherence GE 3.7.1.0 <Warning> (thread=Cluster, member=n/a): Delaying formation of a new cluster; waiting for well-known nodes to respond
      2013-10-24 11:17:58.271/275.812 Oracle Coherence GE 3.7.1.0 <Error> (thread=Cluster, member=n/a): Detected soft timeout) of {WrapperGuardable Guard{Daemon=IpMonitor} Service=ClusterService{Name=Cluster, State=(SERVICE_STARTED, STATE_ANNOUNCE), Id=0, Version=3.7.1}}
      2013-10-24 11:17:58.273/275.814 Oracle Coherence GE 3.7.1.0 <Error> (thread=Recovery Thread, member=n/a): Full Thread Dump

      Thread[PacketListener1,8,Cluster]
              java.net.PlainDatagramSocketImpl.receive0(Native Method)
              java.net.PlainDatagramSocketImpl.receive(Unknown Source)
              java.net.DatagramSocket.receive(Unknown Source)
              com.tangosol.coherence.component.net.socket.UdpSocket.receive(UdpSocket.CDB:22)
              com.tangosol.coherence.component.net.UdpPacket.receive(UdpPacket.CDB:1)
              com.tangosol.coherence.component.util.daemon.queueProcessor.packetProcessor.PacketListener.onNotify(PacketListener.CDB:20)
              com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:42)
              java.lang.Thread.run(Unknown Source)

      Thread[PacketReceiver,7,Cluster]
              java.lang.Object.wait(Native Method)
              com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:18)
              com.tangosol.coherence.component.util.daemon.queueProcessor.packetProcessor.PacketReceiver.onWait(PacketReceiver.CDB:2)
              com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:39)
              java.lang.Thread.run(Unknown Source)

      Thread[Attach Listener,5,system]

      Thread[PacketPublisher,6,Cluster]
              java.lang.Object.wait(Native Method)
              com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:18)
              com.tangosol.coherence.component.util.daemon.queueProcessor.packetProcessor.PacketPublisher.onWait(PacketPublisher.CDB:2)
              com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:39)
              java.lang.Thread.run(Unknown Source)

      Thread[Cluster|STATE_ANNOUNCE|Member(Id=0, Timestamp=2013-10-24 11:13:23.762, Address=171.193.103.25:8090, MachineId=9041, Location=site:,machine:FD4C9EF534D5D,process:17192, Role=CoherenceConsole),5,Cluster]
              sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
              sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(Unknown Source)
              sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(Unknown Source)
              sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source)
              sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source)
              sun.nio.ch.SelectorImpl.select(Unknown Source)
              com.tangosol.coherence.component.net.TcpRing.select(TcpRing.CDB:11)
              com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.ClusterService.onWait(ClusterService.CDB:6)
              com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:39)
              java.lang.Thread.run(Unknown Source)

      Thread[Reference Handler,10,system]
              java.lang.Object.wait(Native Method)
              java.lang.Object.wait(Object.java:485)
              java.lang.ref.Reference$ReferenceHandler.run(Unknown Source)

      Thread[Finalizer,8,system]
              java.lang.Object.wait(Native Method)
              java.lang.ref.ReferenceQueue.remove(Unknown Source)
              java.lang.ref.ReferenceQueue.remove(Unknown Source)
              java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)

      Thread[Signal Dispatcher,9,system]

      Thread[PacketSpeaker,8,Cluster]
              java.lang.Object.wait(Native Method)
              com.tangosol.coherence.component.util.queue.ConcurrentQueue.waitForEntry(ConcurrentQueue.CDB:16)
              com.tangosol.coherence.component.util.queue.ConcurrentQueue.remove(ConcurrentQueue.CDB:7)
              com.tangosol.coherence.component.util.Queue.remove(Queue.CDB:1)
              com.tangosol.coherence.component.util.daemon.queueProcessor.packetProcessor.PacketSpeaker.onNotify(PacketSpeaker.CDB:21)
              com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:42)
              java.lang.Thread.run(Unknown Source)

      Thread[Logger@1457155060 3.7.1.0,3,main]
              java.lang.Object.wait(Native Method)
              com.tangosol.coherence.component.util.Daemon.onWait(Daemon.CDB:18)
              com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:39)
              java.lang.Thread.run(Unknown Source)

      Thread[PacketListener1P,8,Cluster]
              java.net.PlainDatagramSocketImpl.receive0(Native Method)
              java.net.PlainDatagramSocketImpl.receive(Unknown Source)
              java.net.DatagramSocket.receive(Unknown Source)
              com.tangosol.coherence.component.net.socket.UdpSocket.receive(UdpSocket.CDB:22)
              com.tangosol.coherence.component.net.UdpPacket.receive(UdpPacket.CDB:1)
              com.tangosol.coherence.component.util.daemon.queueProcessor.packetProcessor.PacketListener.onNotify(PacketListener.CDB:20)
              com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:42)
              java.lang.Thread.run(Unknown Source)

      Thread[main,5,main]
              java.lang.Object.wait(Native Method)
              com.tangosol.coherence.component.util.daemon.queueProcessor.Service.start(Service.CDB:18)
              com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.start(Grid.CDB:6)
              com.tangosol.coherence.component.net.Cluster.onStart(Cluster.CDB:56)
              com.tangosol.coherence.component.net.Cluster.start(Cluster.CDB:11)
              com.tangosol.coherence.component.util.SafeCluster.startCluster(SafeCluster.CDB:3)
              com.tangosol.coherence.component.util.SafeCluster.restartCluster(SafeCluster.CDB:10)
              com.tangosol.coherence.component.util.SafeCluster.ensureRunningCluster(SafeCluster.CDB:26)
              com.tangosol.coherence.component.util.SafeCluster.start(SafeCluster.CDB:2)
              com.tangosol.net.CacheFactory.ensureCluster(CacheFactory.java:427)
              com.tangosol.coherence.component.application.console.Coherence.run(Coherence.CDB:25)
              com.tangosol.coherence.component.application.console.Coherence.main(Coherence.CDB:3)
              sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
              sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
              java.lang.reflect.Method.invoke(Unknown Source)
              com.tangosol.net.CacheFactory.main(CacheFactory.java:827)

      Thread[Recovery Thread,5,Cluster]
              java.lang.Thread.dumpThreads(Native Method)
              java.lang.Thread.getAllStackTraces(Unknown Source)
              com.tangosol.net.GuardSupport.logStackTraces(GuardSupport.java:810)
              com.tangosol.internal.net.cluster.DefaultServiceFailurePolicy.onGuardableRecovery(DefaultServiceFailurePolicy.java:44)
              com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid$WrapperGuardable.recover(Grid.CDB:1)
              com.tangosol.net.GuardSupport$Context$1.run(GuardSupport.java:653)
              java.lang.Thread.run(Unknown Source)

      2013-10-24 11:17:58.273/275.814 Oracle Coherence GE 3.7.1.0 <Warning> (thread=Recovery Thread, member=n/a): Attempting recovery of Guard{Daemon=IpMonitor}
      Exception in thread "main" 2013-10-24 11:18:24.025/301.566 Oracle Coherence GE 3.7.1.0 <Error> (thread=main, member=n/a): Error while starting cluster: com.tangosol.net.RequestTimeoutException: Timeout during service start: ServiceInfo(Id=0, Name=Cluster, Type=Cluster
        MemberSet=MasterMemberSet(
          ThisMember=null
          OldestMember=null
          ActualMemberSet=MemberSet(Size=0
            )
          MemberId|ServiceVersion|ServiceJoined|MemberState
          RecycleMillis=1200000
          RecycleSet=MemberSet(Size=0
            )
          )
      )
              at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.onStartupTimeout(Grid.CDB:3)
              at com.tangosol.coherence.component.util.daemon.queueProcessor.Service.start(Service.CDB:28)
              at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.start(Grid.CDB:6)
              at com.tangosol.coherence.component.net.Cluster.onStart(Cluster.CDB:56)
              at com.tangosol.coherence.component.net.Cluster.start(Cluster.CDB:11)
              at com.tangosol.coherence.component.util.SafeCluster.startCluster(SafeCluster.CDB:3)
              at com.tangosol.coherence.component.util.SafeCluster.restartCluster(SafeCluster.CDB:10)
              at com.tangosol.coherence.component.util.SafeCluster.ensureRunningCluster(SafeCluster.CDB:26)
              at com.tangosol.coherence.component.util.SafeCluster.start(SafeCluster.CDB:2)
              at com.tangosol.net.CacheFactory.ensureCluster(CacheFactory.java:427)
              at com.tangosol.coherence.component.application.console.Coherence.run(Coherence.CDB:25)
              at com.tangosol.coherence.component.application.console.Coherence.main(Coherence.CDB:3)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
              at java.lang.reflect.Method.invoke(Unknown Source)
              at com.tangosol.net.CacheFactory.main(CacheFactory.java:827)
      java.lang.reflect.InvocationTargetException

      2013-10-24 11:18:24.025/301.566 Oracle Coherence GE 3.7.1.0 <D5> (thread=Cluster, member=n/a): Service Cluster left the cluster at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

              at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
              at java.lang.reflect.Method.invoke(Unknown Source)
              at com.tangosol.net.CacheFactory.main(CacheFactory.java:827)
      Caused by: com.tangosol.net.RequestTimeoutException: Timeout during service start: ServiceInfo(Id=0, Name=Cluster, Type=Cluster
        MemberSet=MasterMemberSet(
          ThisMember=null
          OldestMember=null
          ActualMemberSet=MemberSet(Size=0
            )
          MemberId|ServiceVersion|ServiceJoined|MemberState
          RecycleMillis=1200000
          RecycleSet=MemberSet(Size=0
            )
          )
      )
              at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.onStartupTimeout(Grid.CDB:3)
              at com.tangosol.coherence.component.util.daemon.queueProcessor.Service.start(Service.CDB:28)
              at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.start(Grid.CDB:6)
              at com.tangosol.coherence.component.net.Cluster.onStart(Cluster.CDB:56)
              at com.tangosol.coherence.component.net.Cluster.start(Cluster.CDB:11)
              at com.tangosol.coherence.component.util.SafeCluster.startCluster(SafeCluster.CDB:3)
              at com.tangosol.coherence.component.util.SafeCluster.restartCluster(SafeCluster.CDB:10)
              at com.tangosol.coherence.component.util.SafeCluster.ensureRunningCluster(SafeCluster.CDB:26)
              at com.tangosol.coherence.component.util.SafeCluster.start(SafeCluster.CDB:2)
              at com.tangosol.net.CacheFactory.ensureCluster(CacheFactory.java:427)
              at com.tangosol.coherence.component.application.console.Coherence.run(Coherence.CDB:25)
              at com.tangosol.coherence.component.application.console.Coherence.main(Coherence.CDB:3)
              ... 5 more

      C:\Users\Zk5rjg8>

        • 1. Re: Issue to setup local Coherence cluster with WKA (well-known-address)
          Leo_TA

          Hi SajeevPynadath

           

          1

          First start the server process  with "cache-server.cmd"

           

          2

          After that you can start another server or client process,  the "coherence.cmd" script is to start a client process to join the cluster .

           

          3

          Then now you have 2 processes , and your cluster.xml will look like this :

           

          <socket-address id="serverprocess">

               <address>171.193.103.25</address>

               <port>8088</port>

              </socket-address>

          <socket-address id="clienprocess">

               <address>171.193.103.25</address>

               <port>8089</port>

              </socket-address>


          4

          Before start each process remember put in java command line :

          for server

          -Dtangosol.coherence.localhost=171.193.103.25 -Dtangosol.coherence.localport=8088

          for client

          -Dtangosol.coherence.localhost=171.193.103.25 -Dtangosol.coherence.localport=8089

           

           

          regards,

          Leo_TA