4 Replies Latest reply on Jan 2, 2013 10:44 AM by 981080

    Problem with JMS over SSL in clustered environment


      We are running Weblogic 11g Cluster (domain) which consists of admin server AS and two managed servers MS1, MS2.
      AS and MS1 run on machine 1, MS2 runs on machine 2. Both machines have two network interfaces, one public used for client connections and one internal for cluster communication, monitoring etc. The default channel of each Weblogic server is listening on the internal network interface and besides we have two channels (for http and t3 protocol) configured for the public interface.
      Both managed servers serve as JMS Providers and there is a JMS Module myModule in the domain with the following JMS resources: custom connection factory myConnFactory (Load Balancing Enabled=true, Server affinity=false, targets: whole JMS cluster) and myQueue, which is a Uniform Distributed Queue (targets: MS1, MS2). The queue is accessible via its logical JNDI name, but physically it is located on each managed server.

      JMS communication normally flows through the dedicated t3 channel listening on the public interface. However, a new external client is going to send messages to myQueue and the communication must be encrypted for security reasons. For this reason, we have set up SSL. Instead of enabling a DefaultSecure channel, we left "SSL Listen Port Enabled"=false (as the default channel would be bound to the internal network interface) and created a new channel T3SChannel for t3s protocol on the public interface for incoming client connections.

      The client creates one t3s connection to the cluster (via T3SChannel) and gets the connection factory and the queue using JNDI lookup ( source). The JMS connection is in real made with MS1. If we want to create two consumers for that queue, the fist consumer is created on MS1 and the second is going to be created on MS2 (thanks to enabled load-balancing). However, the creation of second consumer fails with an exception (it is thrown on the client):

      java.rmi.ConnectException: No known valid port for: 'DefaultSecure[t3s]:t3s(t3s):mserver1-internal.company.com:56213:null:-1'; No available router to destination
           at weblogic.rjvm.ConnectionManager.bootstrap(ConnectionManager.java:464)
           at weblogic.rjvm.ConnectionManager.bootstrap(ConnectionManager.java:396)
           at weblogic.rjvm.RJVMImpl.ensureConnectionEstablished(RJVMImpl.java:303)
           at weblogic.rjvm.RJVMImpl.getOutputStream(RJVMImpl.java:347)
           at weblogic.rjvm.RJVMImpl.getRequestStreamInternal(RJVMImpl.java:610)
           ... 18 more

      It was told us that the exception can be avoided with <default-protocol>t3s</default-protocol> element (default value is t3) added to config.xml of Weblogic domain. If we configure t3s as the default protocol, we must also enable the DefaultSecure channel on each server and then everything works and the client is able to create consumers correctly.

      However, as a side-effect, the whole cluster communication on weblogic.rjvm layer is then realized via t3s. We do not want that because the internal cluster communication is secured enough with other methods and it would have noticeable performance impacts in the production environment. In principle, it should be possible to enable the external client to connect to the JMS provider via new, secure channel without affecting the existing internal communication of the cluster (which should be a black box for the client).

      My question: is it possible to run described example without setting the default-protocol to t3s?

      Thank you for the answer.
        • 1. Re: Problem with JMS over SSL in clustered environment
          René van Wijk
          Not sure here, but you could try to get it to work with network channels - http://docs.oracle.com/cd/E12839_01/web.1111/e13701/network.htm

          "A network channel is a conceptual combination of a Listen Address, Listen Port, and Protocol
          that must be unique within a server. Network channels can share the same address and port number
          provided that their protocols are different. When this happens, WebLogic Server combines these
          channels and creates a single listen thread and port that accepts all of the protocols with that address
          and port number combination. The choice of protocols includes t3, IIOP, HTTP, COM, LDAP, SNMP,
          t3s, IIOPS, HTTPS, LDAPS, cluster-broadcast, cluster-broadcast-secure, or admin. WebLogic
          Server’s cluster-broadcast protocol supports routing unicast-based cluster messages between servers;
          cluster-broadcast-secure does the same over SSL. The admin protocol is simply a network channel
          that accepts only administrative requests and requires the use of SSL.

          Network channels also support network address translation (NAT) firewalls directly by providing the
          ability to specify the External Listen Address and External Listen Port attributes that WebLogic
          Server should use when communicating with clients through this channel. In addition, each network
          channel has its own TCP-related configuration parameters that you will find under the Advanced area of
          the channel’s General Configuration tab in the WebLogic Console."

          In your case you would configure something like

          Name: JustANameForTheChannel
          Protocol: T3S
          Listen Address: hostname1 for MS1, hostname2 for MS2
          Listen Port: 8888 (can be any port number, of course)
          External Listen Address: ...
          External Listen Port: ...
          HTTP Enabled for This Protocol: No (unchecked)

          which cna be configured under the protocols, channels tab of the managed server in question.
          1 person found this helpful
          • 2. Re: Problem with JMS over SSL in clustered environment
            Tom B-Oracle
            My question: is it possible to run described example without setting the default-protocol to t3s?
            Thanks for the very clear problem description. I double-checked with our top customer support guru and I'm pretty sure the short answer to your question is no. I think you've run into a known issue and have already hit upon the recommended work-around .

            That said, you may be able to at least partially avoid the problem by setting "server-affinity=true" on your CF. As you probably already know very well, affinity=false encourages consumer and producer traffic to route from client, over it's server connection host, and then potentially on a "second hop" to another server in the cluster. It looks like the attempt at an implicit downgrade of an originally SSL secured request in the first hop over an insecure channel in this second hop is what's throwing the exception.


            • 3. Re: Problem with JMS over SSL in clustered environment
              René van Wijk: thank you for your suggestion, but we already are using the network channels (the channels I mentioned in the description were meant as network channels).

              • 4. Re: Problem with JMS over SSL in clustered environment
                Tom: thank you for the qualified answer. I thought this is an issue but I needed to check by someone else if there is another solution/workaround known.

                BTW: Happy new year 2013!