This discussion is archived
8 Replies Latest reply: Jul 20, 2012 12:37 PM by 810618 RSS

JDK 1.6.025 file descriptor leak

919462 Newbie
Currently Being Moderated
Hi,

We are using java.net.Socket to open a socket (snmp based communication) and we close the socket properly. But we are seeing "Too many open files" issue and this is happening in 1.6.0.25.

We dont see any problem (with the same code base) in JDK1.6.0_U3. The application is running without any issues..Even with the latest Java (1.7U4) we are seeing this problem.

I am seeing lot of file descriptor files opened under the /proc/<pid>/fd directory. But i couldnt read the file and all of them are UDP sockets.

And also our CPU usage goes upto 100% and we are not sure how to debug this problem. Clearly this pbm is not there in 1.6.0.U3, only with the later versions.. Can anyone facing this issue or help me to debug this issue? for me it looks like a java pbm...

Sam.
  • 1. Re: JDK 1.6.025 file descriptor leak
    919462 Newbie
    Currently Being Moderated
    As a additional info, the code works properly in JDK1.6U3 and the problem starts from JDK1.6U4..
  • 2. Re: JDK 1.6.025 file descriptor leak
    gimbal2 Guru
    Currently Being Moderated
    You say "We are properly closing the socket" - I would make triple sure that you're doing that everywhere. I would also make triple sure that the exception is actually coming from socket handles and not something else like a file handle.

    If you are sure then the only way forward is to create a small example program and see if you can reproduce it. If you can then you have the base for a bug report you should be making. If you can you should also try running on a different environment (for example if you're running on Windows, try running on a Linux JDK) to see if that makes a difference.

    Just to be sure: this is not Oracle tech support. You're telling other Java programmers just like yourself.
  • 3. Re: JDK 1.6.025 file descriptor leak
    BIJ001 Explorer
    Currently Being Moderated
    "Too many open files"
    What OS is it?

    Anyway, your OS might help checking the resources held by the Java process.

    On Unix sockets, pipes, open files all use (and therefore compete for) file descriptors.

    oracle@izsak:/tmp> ls -l /proc/24544/fd
    total 48
    lrwx------ 1 oracle oinstall 64 Jul 17 15:49 0 -> /dev/pts/2
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 1 -> /tmp/ivan_R3.log
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 10 -> /home/oracle/oc4j_R3/oc4j.ivan/j2ee/home/log/oc4j/log.xml
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 11 -> /home/oracle/oc4j_R3/oc4j.ivan/j2ee/home/log/wsmgmt/auditing/log.xml
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 12 -> /home/oracle/oc4j_R3/oc4j.ivan/j2ee/home/log/wsmgmt/logging/log.xml
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 13 -> /home/oracle/oc4j_R3/oc4j.ivan/j2ee/home/log/system-application.log
    lrwx------ 1 oracle oinstall 64 Jul 17 15:49 14 -> socket:[22197049]
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 15 -> /home/oracle/oc4j_R3/oc4j.ivan/j2ee/home/log/global-application.log
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 16 -> /home/oracle/oc4j_R3/oc4j.ivan/j2ee/home/application-deployments/javasso/application.log
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 17 -> /home/oracle/oc4j_R3/oc4j.ivan/j2ee/home/log/ascontrol-application.log
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 18 -> /home/oracle/oc4j_R3/oc4j.ivan/j2ee/home/application-deployments/TokenStatic-dev/application.log
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 19 -> /home/oracle/oc4j_R3/oc4j.ivan/j2ee/home/application-deployments/client-token-dev-user/application.log
    l-wx------ 1 oracle oinstall 64 Jul 17 15:07 2 -> /tmp/ivan_R3.log
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 20 -> /home/oracle/oc4j_R3/oc4j.ivan/j2ee/home/application-deployments/client-token-dev-lra/application.log
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 21 -> /home/oracle/oc4j_R3/oc4j.ivan/j2ee/home/application-deployments/pgadmin-dev/application.log
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 22 -> /home/oracle/oc4j_R3/oc4j.ivan/j2ee/home/application-deployments/szep3-dev/application.log
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 23 -> /home/oracle/oc4j_R3/oc4j.ivan/j2ee/home/application-deployments/szep1-dev/application.log
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 24 -> /home/oracle/oc4j_R3/oc4j.ivan/j2ee/home/application-deployments/smea-dev/application.log
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 25 -> /home/oracle/oc4j_R3/oc4j.ivan/j2ee/home/application-deployments/psm-dev/application.log
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 26 -> /home/oracle/oc4j_R3/oc4j.ivan/j2ee/home/application-deployments/szep4-dev/application.log
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 27 -> /home/oracle/oc4j_R3/oc4j.ivan/j2ee/home/application-deployments/PaymentGateway-dev/application.log
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 28 -> /home/oracle/oc4j_R3/oc4j.ivan/j2ee/home/application-deployments/szep2-dev/application.log
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 29 -> /home/oracle/oc4j_R3/oc4j.ivan/j2ee/home/application-deployments/szep5-dev/application.log
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 3 -> /home/oracle/oc4j_R3/nohup.out
    lrwx------ 1 oracle oinstall 64 Jul 17 15:49 30 -> socket:[22210040]
    lr-x------ 1 oracle oinstall 64 Jul 17 15:49 31 -> /dev/random
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 32 -> /home/oracle/oc4j_R3/oc4j.ivan/j2ee/home/log/jms.log
    lrwx------ 1 oracle oinstall 64 Jul 17 15:49 33 -> /home/oracle/oc4j_R3/oc4j.ivan/j2ee/home/persistence/jms.state
    lrwx------ 1 oracle oinstall 64 Jul 17 15:49 34 -> socket:[22197058]
    lrwx------ 1 oracle oinstall 64 Jul 17 15:49 35 -> socket:[22197056]
    lrwx------ 1 oracle oinstall 64 Jul 17 15:49 36 -> /home/oracle/oc4j_R3/oc4j.ivan/j2ee/home/persistence/Oc4jJmsExceptionQueue
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 37 -> /home/oracle/oc4j_R3/oc4j.ivan/j2ee/home/log/rmi.log
    lrwx------ 1 oracle oinstall 64 Jul 17 15:49 38 -> socket:[22197107]
    lrwx------ 1 oracle oinstall 64 Jul 17 15:49 39 -> socket:[22197109]
    lr-x------ 1 oracle oinstall 64 Jul 17 15:49 4 -> pipe:[22197042]
    lr-x------ 1 oracle oinstall 64 Jul 17 15:49 40 -> pipe:[22197473]
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 41 -> pipe:[22197473]
    lrwx------ 1 oracle oinstall 64 Jul 17 15:49 42 -> socket:[22197474]
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 43 -> /home/oracle/oc4j_R3/oc4j.ivan/j2ee/home/log/ascontrol.log
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 44 -> /home/oracle/oc4j_R3/oc4j.ivan/j2ee/home/szep0.log
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 45 -> /home/oracle/oc4j_R3/oc4j.ivan/j2ee/home/szep2.log
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 49 -> /home/oracle/oc4j_R3/oc4j.ivan/j2ee/home/log/global-application.log
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 5 -> pipe:[22197042]
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 50 -> /home/oracle/oc4j_R3/oc4j.ivan/j2ee/home/application-deployments/ebank2-dev/application.log
    lr-x------ 1 oracle oinstall 64 Jul 17 15:49 6 -> /dev/random
    lr-x------ 1 oracle oinstall 64 Jul 17 15:49 7 -> /dev/urandom
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 8 -> /home/oracle/oc4j_R3/oc4j.ivan/j2ee/home/application-deployments/dev-sme/application.log
    l-wx------ 1 oracle oinstall 64 Jul 17 15:49 9 -> /home/oracle/oc4j_R3/oc4j.ivan/j2ee/home/log/server.log
  • 4. Re: JDK 1.6.025 file descriptor leak
    919462 Newbie
    Currently Being Moderated
    Yes, i did. Why i am saying this because the same code is working properly with JDK.16U3 and not with JDK1.6U4.
  • 5. Re: JDK 1.6.025 file descriptor leak
    919462 Newbie
    Currently Being Moderated
    Its Redhat. And as you said its purely UDP sockets opened by our application and though we are calling socket.close() , the reference is not completely cleaned up. And this pbm occurs only in JDK1.6U4 and not in JDK1.6U3.

    I couldnt find anything in the JDK1.6U4 changelist, but i suspect some enhancements related to the GC ..
  • 6. Re: JDK 1.6.025 file descriptor leak
    810618 Newbie
    Currently Being Moderated
    Are you absolutely sure you are executing the close() statement? Is your close() statement in a finally block?
  • 7. Re: JDK 1.6.025 file descriptor leak
    919462 Newbie
    Currently Being Moderated
    Hi Kevin,

    I am very sure. As i mentioned already, i have two different java installation ..When i run my product with JDK1.6U4 i am getting this problem , and not with JDK1.6U3. No code change, no environment change (apart from JAVA)

    For me, when i switch to JDK1.6U4, getting this "Too many open files" , CPU goes to 98 %..
  • 8. Re: JDK 1.6.025 file descriptor leak
    810618 Newbie
    Currently Being Moderated
    I guess I should've explained the thinking behind my question. If, for whatever reason, an unchecked Exception or Throwable is being thrown in the later JVMs that wasn't in the previous ones, and if you don't catch it, your Thread will just 'go away' possibly before the close() is executed. This happened to me on a server that was supposed to be monitoring a smoke alarm system. It was discovered after months of running that it stopped monitoring. I tracked it down to an uncaught NullPointerException causing the Thread to just go away. So now, after my known 'catches', I add a catch Throwable.

    I know this is a long shot but if you've eliminated everything else, you might want to check to make sure there is no way this could be happening to you. It just seems to me that if this were a bug in the JVM, it would've been noticed long ago by others.

    Kev

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points