Forum Stats

  • 3,815,185 Users
  • 2,258,973 Discussions
  • 7,892,982 Comments

Discussions

Configuring Remote Application to Connect to Coherence Cache through HAProxy

3245865
3245865 Member Posts: 151
edited Dec 19, 2018 2:27PM in Coherence Support

Good morning,

I'm pretty new to Coherence and am looking for some guidance.

I have a functioning Coherence cluster deployed to Kubernetes (bare metal) as a POC. We also have a single HA Proxy instance (outside of Kubernetes) that is configured to talk TCP to each Coherence Pod that is part of the cluster. I am having issues using this instance to proxy 'get' and 'put' requests from a remote application using Coherence Extend. Perhaps I am misunderstanding what Coherence Extend is capable of.

Here is my cluster configuration:

<?xml version="1.0"?>

<cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

   xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"

   xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config

   coherence-cache-config.xsd">

   <caching-scheme-mapping>

      <cache-mapping>

         <cache-name>hello-example</cache-name>

         <scheme-name>extend</scheme-name>

      </cache-mapping>

   </caching-scheme-mapping>

   <caching-schemes>

      <distributed-scheme>

         <scheme-name>extend</scheme-name>

         <lease-granularity>member</lease-granularity>

         <backing-map-scheme>

            <local-scheme/>

         </backing-map-scheme>

         <autostart>true</autostart>

      </distributed-scheme>

      <proxy-scheme>

         <service-name>ExtendTcpProxyService</service-name>

         <thread-count>5</thread-count>

         <acceptor-config>

            <tcp-acceptor>

               <local-address>

                  <address>localhost</address>

                  <port>7574</port>

               </local-address>

            </tcp-acceptor>

         </acceptor-config>

         <autostart>true</autostart>

      </proxy-scheme>

   </caching-schemes>

</cache-config>

Here is my client configuration, deployed with my remote application and referenced using -Dtangosol.coherence.cacheconfig=classpath:mycache.xml

<?xml version='1.0'?>

<cache-config xmlns:processing="class://com.oracle.coherence.patterns.processing.config.xml.ProcessingPatternNamespaceHandler"

                          xmlns:element="class://com.oracle.coherence.common.namespace.preprocessing.XmlPreprocessingNamespaceHandler"

                          element:introduce-cache-config="coherence-cache-config.xml">

        <caching-scheme-mapping>

                <cache-mapping>

                        <cache-name>hello-example</cache-name>

                        <scheme-name>remote</scheme-name>

                </cache-mapping>

        </caching-scheme-mapping>

        <caching-scheme>

                <remote-cache-scheme>

                        <scheme-name>remote</scheme-name>

                        <service-name>ExtendTcpCacheService</service-name>

                        <initiator-config>

                                <tcp-initiator>

                                        <remote-addresses>

                                                <socket-address>

                                                        <address>[haproxy ip]</address>

                                                        <port>[haproxy port]</port>

                                                </socket-address>

                                        </remote-addresses>

                                        <connect-timeout>10s</connect-timeout>

                                </tcp-initiator>

                                <outgoing-message-handler>

                                        <request-timeout>5s</request-timeout>

                                </outgoing-message-handler>

                        </initiator-config>

                </remote-cache-scheme>

                <remote-invocation-scheme>

                        <scheme-name>extend-invocation</scheme-name>

                        <service-name>ExtendTcpInvocationService</service-name>

                        <initiator-config>

                                <tcp-initiator>

                                        <remote-addresses>

                                                <socket-address>

                                                        <address>[ha-proxy ip]</address>

                                                        <port>[ha-proxy port]</port>

                                                </socket-address>

                                        </remote-addresses>

                                        <connect-timeout>10s</connect-timeout>

                                </tcp-initiator>

                                <outgoing-message-handler>

                                        <request-timeout>5s</request-timeout>

                                </outgoing-message-handler>

                        </initiator-config>

                </remote-invocation-scheme>

        </caching-scheme>

</cache-config>

The remote Java application is coded to perform 'get' and 'put' operations. In short:

public void put(...) {

  CacheFactory.getCache("hello-example").put(key, value);

}

public String get(...) {

   return (String) CacheFactory.getCache("hello-example").get(key);

}

Any advice would be greatly appreciated - thank you!

Best Answer

Answers

  • 3245865
    3245865 Member Posts: 151
    edited Dec 18, 2018 9:13AM

    Upon further inspection of the documentation, I changed the 'get' method to use an InvocationService defined in the mycache.xml file.

    InvocationService invocationService = (InvocationService) CacheFactory.getConfigurableCacheFactory().ensureService("ExtendTcpInvocationService");

    Map map = invocationService.query(new AbstractInvocable()

    {

       public void run()

      {

      setResult(CacheFactory.getCache("dist-extend").get(key));

      }

    }, null);

    return (String) map.get(invocationService.getCluster().getLocalMember());

    However, now I am receiving the following error when I try and issue a request.

    java.lang.IllegalArgumentException: No scheme found for service ExtendTcpInvocationService

            at com.tangosol.net.ExtensibleConfigurableCacheFactory.ensureService(ExtensibleConfigurableCacheFactory.java:302) ~[coherence-12.1.3-0-0.jar:12.1.3.0.0]

            ...

  • Tmiddlet-Oracle
    Tmiddlet-Oracle Member Posts: 125
    edited Dec 18, 2018 8:03PM Answer ✓

    Hi

    You definitely shouldn't be using invocation service for get/put. The invocation service is for running code on individual members.

    As you are using coherence 12.1.3, Take a look at the following documentation on setting up extend.

    https://docs.oracle.com/middleware/1213/coherence/develop-remote-clients/gs_configextend.htm#COHCG4877

    Your server side cache config proxy-scheme acceptor is running on 7574 which is the default cluster address for later coherence version, so probably best to change it to another value and expose that on your docker image.

    You are using HA proxy, so you must also ensure you set client based load balancing (https://docs.oracle.com/middleware/1213/coherence/develop-remote-clients/gs_configextend.htm#COHCG5168  ) whic means the clietn will do the load balancing and not the service.

    Also see JK's blog which talks about Running coherence in docker.

    https://thegridman.com/coherence/oracle-coherence-on-docker/#extend

    Tim

  • 3245865
    3245865 Member Posts: 151
    edited Dec 19, 2018 2:27PM

    Tim, thank you for your assistance!

    I was able to get it working using the documents that you provided. I ended up changing the port I was using, ditching the InvocationService in my code, and removing the remote-invocation-scheme in my client configuration.