3 Replies Latest reply on Jul 6, 2011 8:26 PM by Paula B-Oracle

    JVM crashes after MASTER: 'accepted a new connection'

      Hi all,

      We run into a strange problem, and we hope you may give us some light.

      We have a web-app in Jetty that handles our Berkeley environment.
      We have enabled replication between 2 servers, one Master and one Replica.

      The problem is, lately, that if we start the Master server it crashes after a period of 15-30 minutes.

      I can see the following messages in the log:
      [6541:609000][3144/3736] MASTER: accepted a new connection
      although that the Replica server is down!!!

      WARNING: Database problem, it is required to shut down an run recovery
      from our custom replication EventHandlerAdapter.handlePanicEvent()

      And then JVM crashes and Jetty disappears.
      I enabled Verbose for all replication flags, in EnvironmentConfig, but I didn't get any more logs for this problem.

      I wan wondering if you can think of any ways to get into the problem and track it down.
      Replication runs on port 8080, is it possible that a request from somewhere that uses port 8080 to confuse the replication manager?
      Is there a way to get more information about this 'accepted' new connection and maybe debug it??

      Thanks in advance!

      We're using Berkeley DB 11gR2 5.1.25
        • 1. Re: JVM crashes after MASTER: 'accepted a new connection'
          Paula B-Oracle
          We've run into similar problems during some of our Replication Manager testing.

          One way to find out more about what port 8080 is doing if you are on Linux or a similar OS is to do the following:

          lsof -i TCP:8080

          The output will contain PIDs. You can see what a PID (e.g. 3892) is running with the following:

          ps -p 3892

          One issue we ran into was that our tests were sometimes using ephemeral ports, which can be arbitrarily assigned by the OS for other purposes. We resolved this issue by making sure we avoid using an ephemeral port on the OS where we are running a test. You may want to check if 8080 is in the ephemeral port range for your OS. Here's a link that explains more about ephemeral ports and how to determine the range on various operating systems:


          Paula Bingham
          • 2. Re: JVM crashes after MASTER: 'accepted a new connection'
            Hi Paula,

            Thanks you your reply.
            I managed to replicate the problem, just before seeing your post.
            In a few words, I can panic my database and eventually kill JVM by sending a simple http request to the replication port.

            Based on your suggestions, we need to make some firewall and port configuration on the server, to allow only valid requests reaching the replication port.

            My understanding, is that the replication manager does not validate incoming requests. I think it makes sense for the manager to check (a) if a request comes from one of the (defined in configuration) remote hosts and (b) if the request satisfies some information criteria. Of course, I'm not aware of implementation restrictions and problems, but I think this will be a nice future improvement, especially if the database needs shutdown and recovery.

            Again, thanks for your reply and thanks for sharing the ephemeral ports link.


            Edited by: Stelios Limnaios on Jul 1, 2011 1:21 AM
            • 3. Re: JVM crashes after MASTER: 'accepted a new connection'
              Paula B-Oracle
              Thanks for your enhancement request. Something similar is already on our list for future consideration, and I have forwarded your specific request to it so that your ideas are also considered.

              Paula Bingham