4 Replies Latest reply: Jul 5, 2010 2:16 AM by PhHein RSS

    jconsole connection trouble. Port redirection?

    843798
      Hello!

      I have a strange issue with jconsole and my JMX enabled daemon. It used to work fine until sysadmin in datacenter placed out machine behind a kind of firewall (cisco accesslist or something like that).

      After that remote connection between jconsole and server no longer can be established.

      Let's say ip of machine is: host_ip_address

      Server side code to start the app:
      /usr/java/bin/java -server -Xms64m -Xmx512m -Djava.util.logging.config.file=bin/../conf/logging.properties -Duser.timezone=Europe/Amsterdam -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.host=host_ip_address -Dcom.sun.management.jmxremote.port=8184 -Dcom.sun.management.jmxremote.password.file=bin/../conf/jmx.password -Dcom.sun.management.jmxremote.access.file=bin/../conf/jmx.access -Djava.naming.factory.initial=org.jnp.interfaces.NamingContextFactory -Djava.naming.factory.url.pkgs=org.jboss.naming:org.jnp.interfaces -Djava.naming.provider.url=jnp://localhost:1099 -Dsecurity.policy=client.policy -Djava.security.auth.login.config=/usr/local/jboss/client/auth.conf -cp bin/../lib/Daemon.jar:bin/../lib/Cashier-client.jar:bin/../lib/CoreEJB-client.jar:/usr/local/jboss/client/jbossall-client.jar daemon.Main
      Sorry for long string. Just wanted to show it asis.

      The daemon used to be controllable by jcosole using remote connection to host_ip_address:8184
      And one day control was lost, jconsole tells that "connecction could ont be established"
      After some investigation I found out that jconsole starts connecting to 8184. At this step it works. After that jconsole decides to connect to host_ip_address:49706. Why? I'm amazed.

      I will appriciate any ideas how to get out of the problem.
      Or probably explanation about that port 49706 redirection.

      Here is log of jconsole. Sorry it is quite long, but contains everything.


      19.06.2007 18:47:18 sun.rmi.transport.tcp.TCPEndpoint <clinit>
      FINE: JConsole.addHost: localHostKnown = true, localHost = 192.168.1.5
      19.06.2007 18:47:18 sun.rmi.server.UnicastRef newCall
      FINE: JConsole.addHost: get connection
      19.06.2007 18:47:18 sun.rmi.transport.tcp.TCPTransport <init>
      FINE: JConsole.addHost: Version = 2, ep = [192.168.1.5:0]
      19.06.2007 18:47:18 sun.rmi.transport.tcp.TCPEndpoint getLocalEndpoint
      FINE: JConsole.addHost: created local endpoint for socket factory null on port 0
      19.06.2007 18:47:18 sun.rmi.transport.tcp.TCPChannel createConnection
      FINE: JConsole.addHost: create connection
      19.06.2007 18:47:18 sun.rmi.transport.tcp.TCPEndpoint newSocket
      FINER: JConsole.addHost: opening socket to [host_ip_address:8184,javax.rmi.ssl.SslRMIClientSocketFactory@170bea5]
      19.06.2007 18:47:19 sun.rmi.server.UnicastRef newCall
      FINE: JConsole.addHost: get connection
      19.06.2007 18:47:19 sun.rmi.transport.tcp.TCPChannel createConnection
      FINE: JConsole.addHost: create connection
      19.06.2007 18:47:19 sun.rmi.transport.tcp.TCPEndpoint newSocket
      FINER: JConsole.addHost: opening socket to [host_ip_address:8184]
      19.06.2007 18:47:19 sun.rmi.transport.proxy.RMIMasterSocketFactory createSocket
      FINE: JConsole.addHost: host: host_ip_address, port: 8184
      19.06.2007 18:47:19 sun.rmi.transport.tcp.TCPChannel createConnection
      FINER: JConsole.addHost: server suggested 91.76.3.103:60229
      19.06.2007 18:47:19 sun.rmi.transport.tcp.TCPChannel createConnection
      FINER: JConsole.addHost: using 192.168.1.5:0
      19.06.2007 18:47:19 sun.rmi.server.UnicastRef newCall
      FINER: JConsole.addHost: create call context
      19.06.2007 18:47:19 sun.rmi.server.UnicastRef logClientCall
      FINER: JConsole.addHost: outbound call: [endpoint:[host_ip_address:8184](remote),objID:[0:0:0, 0]] : sun.rmi.registry.RegistryImpl_Stub[0:0:0, 0]: java.rmi.Remote lookup(java.lang.String)
      19.06.2007 18:47:19 sun.rmi.transport.StreamRemoteCall <init>
      FINER: JConsole.addHost: write remote call header...
      19.06.2007 18:47:19 sun.rmi.transport.StreamRemoteCall getOutputStream
      FINER: JConsole.addHost: getting output stream
      19.06.2007 18:47:19 sun.rmi.server.UnicastRef invoke
      FINER: JConsole.addHost: execute call
      19.06.2007 18:47:19 sun.rmi.transport.StreamRemoteCall getInputStream
      FINER: JConsole.addHost: getting input stream
      19.06.2007 18:47:20 sun.rmi.server.LoaderHandler loadClass
      FINE: JConsole.addHost: name = "javax.management.remote.rmi.RMIServerImpl_Stub", codebase = "", defaultLoader = sun.misc.Launcher$AppClassLoader@d9f9c3
      19.06.2007 18:47:20 sun.rmi.server.LoaderHandler loadClass
      FINER: JConsole.addHost: class "javax.management.remote.rmi.RMIServerImpl_Stub" found via defaultLoader, defined by null
      19.06.2007 18:47:20 sun.rmi.server.LoaderHandler loadClass
      FINE: JConsole.addHost: name = "java.rmi.server.RemoteStub", codebase = "", defaultLoader = sun.misc.Launcher$AppClassLoader@d9f9c3
      19.06.2007 18:47:20 sun.rmi.server.LoaderHandler loadClass
      FINER: JConsole.addHost: class "java.rmi.server.RemoteStub" found via defaultLoader, defined by null
      19.06.2007 18:47:20 sun.rmi.server.LoaderHandler loadClass
      FINE: JConsole.addHost: name = "java.rmi.server.RemoteObject", codebase = "", defaultLoader = sun.misc.Launcher$AppClassLoader@d9f9c3
      19.06.2007 18:47:20 sun.rmi.server.LoaderHandler loadClass
      FINER: JConsole.addHost: class "java.rmi.server.RemoteObject" found via defaultLoader, defined by null
      19.06.2007 18:47:20 sun.rmi.server.UnicastRef done
      FINE: JConsole.addHost: free connection (reuse = true)
      19.06.2007 18:47:20 sun.rmi.transport.tcp.TCPChannel free
      FINE: JConsole.addHost: reuse connection
      19.06.2007 18:47:20 sun.rmi.transport.tcp.TCPChannel free
      FINE: JConsole.addHost: create reaper
      19.06.2007 18:47:20 sun.rmi.server.UnicastRef newCall
      FINE: JConsole.addHost: get connection
      19.06.2007 18:47:20 sun.rmi.transport.tcp.TCPChannel createConnection
      FINE: JConsole.addHost: create connection
      19.06.2007 18:47:20 sun.rmi.transport.tcp.TCPEndpoint newSocket
      FINER: JConsole.addHost: opening socket to [host_ip_address:49706]
      19.06.2007 18:47:20 sun.rmi.transport.proxy.RMIMasterSocketFactory createSocket
      FINE: JConsole.addHost: host: host_ip_address, port: 49706
      19.06.2007 18:47:35 sun.rmi.transport.tcp.TCPChannel$1 run
      FINER: RMI Scheduler(0): wake up
      19.06.2007 18:47:35 sun.rmi.transport.tcp.TCPChannel freeCachedConnections
      FINER: RMI Scheduler(0): connection timeout expired
      19.06.2007 18:47:35 sun.rmi.transport.tcp.TCPConnection close
      FINE: RMI Scheduler(0): close connection
      19.06.2007 18:47:41 sun.rmi.server.UnicastRef newCall
      FINE: RMI RenewClean-[host_ip_address:49706]: get connection
      19.06.2007 18:47:41 sun.rmi.transport.tcp.TCPChannel createConnection
      FINE: RMI RenewClean-[host_ip_address:49706]: create connection
      19.06.2007 18:47:41 sun.rmi.transport.tcp.TCPEndpoint newSocket
      FINER: RMI RenewClean-[host_ip_address:49706]: opening socket to [host_ip_address:49706]
      19.06.2007 18:47:41 sun.rmi.transport.WeakRef pin
      FINER: JConsole.addHost: strongRef = sun.rmi.transport.DGCImpl@11c2b67
      19.06.2007 18:47:41 sun.rmi.transport.ObjectTable putTarget
      FINER: JConsole.addHost: add object [0:0:0, 2]
      19.06.2007 18:47:41 sun.rmi.transport.ConnectionInputStream done
      FINER: JConsole.addHost: send ack
      19.06.2007 18:47:41 sun.rmi.transport.proxy.RMIMasterSocketFactory createSocket
      FINE: RMI RenewClean-[host_ip_address:49706]: host: host_ip_address, port: 49706
      19.06.2007 18:47:41 sun.rmi.transport.tcp.TCPChannel createConnection
      FINE: JConsole.addHost: create connection
      19.06.2007 18:47:41 sun.rmi.transport.tcp.TCPEndpoint newSocket
      FINER: JConsole.addHost: opening socket to [host_ip_address:8184]
      19.06.2007 18:47:41 sun.rmi.transport.proxy.RMIMasterSocketFactory createSocket
      FINE: JConsole.addHost: host: host_ip_address, port: 8184
      19.06.2007 18:47:41 sun.rmi.transport.tcp.TCPChannel createConnection
      FINER: JConsole.addHost: server suggested 91.76.3.103:60247
      19.06.2007 18:47:41 sun.rmi.transport.tcp.TCPChannel createConnection
      FINER: JConsole.addHost: using 192.168.1.5:0
      19.06.2007 18:47:41 sun.rmi.transport.tcp.TCPChannel free
      FINE: JConsole.addHost: reuse connection
      19.06.2007 18:47:41 sun.rmi.transport.tcp.TCPChannel free
      FINE: JConsole.addHost: create reaper
      19.06.2007 18:47:41 RMIConnector connect
      FINER: [javax.management.remote.rmi.RMIConnector: rmiServer=RMIServerImpl_Stub[UnicastRef [liveRef: [endpoint:[host_ip_address:49706](remote),objID:[-57f44ee5:111746985d0:-8000, 2408504052969290368]]]]] connecting...
      19.06.2007 18:47:41 RMIConnector connect
      FINER: [javax.management.remote.rmi.RMIConnector: rmiServer=RMIServerImpl_Stub[UnicastRef [liveRef: [endpoint:[host_ip_address:49706](remote),objID:[-57f44ee5:111746985d0:-8000, 2408504052969290368]]]]] finding stub...
      19.06.2007 18:47:41 RMIConnector connect
      FINER: [javax.management.remote.rmi.RMIConnector: rmiServer=RMIServerImpl_Stub[UnicastRef [liveRef: [endpoint:[host_ip_address:49706](remote),objID:[-57f44ee5:111746985d0:-8000, 2408504052969290368]]]]] connecting stub...
      19.06.2007 18:47:41 RMIConnector connect
      FINER: [javax.management.remote.rmi.RMIConnector: rmiServer=RMIServerImpl_Stub[UnicastRef [liveRef: [endpoint:[host_ip_address:49706](remote),objID:[-57f44ee5:111746985d0:-8000, 2408504052969290368]]]]] getting connection...
      19.06.2007 18:47:41 sun.rmi.server.UnicastRef invoke
      FINER: VMPanel.connect: method: public abstract javax.management.remote.rmi.RMIConnection javax.management.remote.rmi.RMIServer.newClient(java.lang.Object) throws java.io.IOException
      19.06.2007 18:47:41 sun.rmi.server.UnicastRef logClientCall
      FINER: VMPanel.connect: outbound call: [endpoint:[host_ip_address:49706](remote),objID:[-57f44ee5:111746985d0:-8000, 2408504052969290368]] : javax.management.remote.rmi.RMIServerImpl_Stub[-57f44ee5:111746985d0:-8000, 2408504052969290368]: public abstract javax.management.remote.rmi.RMIConnection javax.management.remote.rmi.RMIServer.newClient(java.lang.Object) throws java.io.IOException
      19.06.2007 18:47:41 sun.rmi.transport.tcp.TCPChannel createConnection
      FINE: VMPanel.connect: create connection
      19.06.2007 18:47:41 sun.rmi.transport.tcp.TCPEndpoint newSocket
      FINER: VMPanel.connect: opening socket to [host_ip_address:49706]
      19.06.2007 18:47:41 sun.rmi.transport.proxy.RMIMasterSocketFactory createSocket
      FINE: VMPanel.connect: host: host_ip_address, port: 49706
      19.06.2007 18:47:56 sun.rmi.transport.tcp.TCPChannel$1 run
      FINER: RMI Scheduler(0): wake up
      19.06.2007 18:47:56 sun.rmi.transport.tcp.TCPChannel freeCachedConnections
      FINER: RMI Scheduler(0): connection timeout expired
      19.06.2007 18:47:56 sun.rmi.transport.tcp.TCPConnection close
      FINE: RMI Scheduler(0): close connection
      19.06.2007 18:48:02 RMIConnector connect
      FINER: [javax.management.remote.rmi.RMIConnector: rmiServer=RMIServerImpl_Stub[UnicastRef [liveRef: [endpoint:[host_ip_address:49706](remote),objID:[-57f44ee5:111746985d0:-8000, 2408504052969290368]]]]] failed to connect: java.rmi.ConnectException: Connection refused to host: host_ip_address; nested exception is: 
           java.net.ConnectException: Connection timed out: connect
      19.06.2007 18:48:02 RMIConnector close
      FINER: [javax.management.remote.rmi.RMIConnector: rmiServer=RMIServerImpl_Stub[UnicastRef [liveRef: [endpoint:[host_ip_address:49706](remote),objID:[-57f44ee5:111746985d0:-8000, 2408504052969290368]]]]] closing.
      19.06.2007 18:48:23 sun.rmi.server.UnicastRef newCall
      FINE: RMI RenewClean-[host_ip_address:49706]: get connection
      19.06.2007 18:48:23 sun.rmi.transport.tcp.TCPChannel createConnection
      FINE: RMI RenewClean-[host_ip_address:49706]: create connection
      19.06.2007 18:48:23 sun.rmi.transport.tcp.TCPEndpoint newSocket
      FINER: RMI RenewClean-[host_ip_address:49706]: opening socket to [host_ip_address:49706]
      19.06.2007 18:48:23 sun.rmi.transport.proxy.RMIMasterSocketFactory createSocket
      FINE: RMI RenewClean-[host_ip_address:49706]: host: host_ip_address, port: 49706
      19.06.2007 18:49:26 sun.rmi.server.UnicastRef newCall
      FINE: RMI RenewClean-[host_ip_address:49706]: get connection
      19.06.2007 18:49:26 sun.rmi.transport.tcp.TCPChannel createConnection
      FINE: RMI RenewClean-[host_ip_address:49706]: create connection
      19.06.2007 18:49:26 sun.rmi.transport.tcp.TCPEndpoint newSocket
      FINER: RMI RenewClean-[host_ip_address:49706]: opening socket to [host_ip_address:49706]
      19.06.2007 18:49:26 sun.rmi.transport.proxy.RMIMasterSocketFactory createSocket
      FINE: RMI RenewClean-[host_ip_address:49706]: host: host_ip_address, port: 49706