This discussion is archived
5 Replies Latest reply: Dec 11, 2012 4:35 AM by EJP RSS

Error when calling getLocalSocketAddress() on for a newly created socket

978944 Newbie
Currently Being Moderated
Hello,

I have an issue when using JVM 1.5.0.12 on HP-UX (IA64N):
java.lang.Error: java.net.SocketException: Bad file number (errno:9)
     at sun.nio.ch.Net.localAddress(Net.java:123)
     at sun.nio.ch.SocketChannelImpl.localAddress(SocketChannelImpl.java:394)
     at sun.nio.ch.SocketAdaptor.getLocalAddress(SocketAdaptor.java:147)
     at java.net.Socket.getLocalSocketAddress(Socket.java:708)
I get this error when I accept a connection from a remote peer and I try to call getLocalSocketAddress() method on the new accepted socket connection object.

Does anyone have an idea about the cause for this error?

I also get 2 other errors but in different places in the code:

1/ java.io.IOException: No buffer space available (errno:233)
     at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
     at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:145)
This one I get when my server socket accepts a connection. But even if the connection is accepted succesfully, I might get the "java.lang.Error: java.net.SocketException: Bad file number (errno:9)" when trying to call getLocalSocketAddress()

2/ Once I accept the new connection I call "socketChannel.socket().setTcpNoDelay(true)" but sometimes this fails with a SocketException: Invalid argument (errno:22).


Al the 3 errors are manifesting sporadically and I cannot find the cause.
  • 2. Re: Error when calling getLocalSocketAddress() on for a newly created socket
    978944 Newbie
    Currently Being Moderated
    Please find a fragment of our code bellow:

    SocketChannel socketChannel = serverSocketChannel.accept();
    if (socketChannel == null) {
    throw new NIOCoreException("No incoming connection to accept");
    }

    // Socket options
    if (params.getSocketBufferSize() > 0) {
    socketChannel.socket().setSendBufferSize(params.getSocketBufferSize());
    }

    try {
    socketChannel.socket().setTcpNoDelay(true);
    } catch (SocketException ex) {
    if (log.isDebugEnabled() || log.isTraceEnabled())
    log.warn(logPrefix + "could not set TCP_NODELAY socket option.You may experience performance loss", ex);
    else
    log.warn(logPrefix + "could not set TCP_NODELAY socket option. You may experience performance loss. Cause : " + ex.getMessage());
    }

    ......
    // later if the connection is accepted succesfully this line is executed
    SocketAddress localAddress = socketChannel.socket().getLocalSocketAddress();


    The three instances when my application fails (but not in every instance when the remote peers connects, though):

    1/ When I accept a connection from the remote peers using java NIO's ServerSocketChannel.accept() method, the call fails from time to time with
    12-11-27 06:17:09,302 ERROR incall : [Listen:8] Error while accepting connection
    java.io.IOException: No buffer space available (errno:233)
         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:145)
         at com.axway.niocore.acceptor.Acceptor.acceptEvent(Acceptor.java:151)
    We catch this error and log the error message. This error is not fatal for us in the sense that the server socket channel will continue to listen for further connections (by the way we do SFTP).
    But the current connection fails...


    2/ If the connection is accepted (as I said, it fails only from time to time), trying to enable TCP_NODELAY option wil sporadically fail with:
    could not set TCP_NODELAY socket option. You may experience performance loss. Cause : Invalid argument (errno:22)
    I do not have the entire stack trace for this because we only log the exception message

    3/ Also, a little later in the code we try to get the local socket address, but this also sometimes fails
    12-11-27 08:17:27,856 ERROR NIOCore : Error on ACCEPT event for listener Acceptor(bound:/192.49.223.165:22, bindAddr:/192.49.223.165:22)
    java.lang.Error: java.net.SocketException: Bad file number (errno:9)
         at sun.nio.ch.Net.localAddress(Net.java:123)
         at sun.nio.ch.SocketChannelImpl.localAddress(SocketChannelImpl.java:394)
         at sun.nio.ch.SocketAdaptor.getLocalAddress(SocketAdaptor.java:147)
         at java.net.Socket.getLocalSocketAddress(Socket.java:708)
         at com.axway.niocore.communicator.Communicator.<init>(Communicator.java:54)
         at com.axway.niocore.communicator.ClearCommunicator.<init>(ClearCommunicator.java:33)
         at com.axway.niocore.acceptor.Acceptor.acceptEvent(Acceptor.java:180)

    This error is actually fatal! We catch this error and we close the server socket channel deliberately.
    Our customers are complaining about this.
    We could implement something that will call getLocalAddress in a loop and provide retries if the call fails but I know this is not a Java good practice.


    Anyway, three weeks ago, we had a similar issue, but this time on IBM AIX system:
    https://www.ibm.com/developerworks/forums/thread.jspa?threadID=460428

    I have opened a service request to IBM but the investigation is still in progress. Because of customer pressure, when getLocalAddress failed, we catch this specific error but we changed the normal logic and we do not close the server socket channel anymore. This helped because our application still accepts new connections but the issue still persists (as expected) and some of them are still failing because of this java.lang.Error.

    But now we have a similar issue on HP-UX (similar in the sense that java.net.Socket.getLocalSocketAddress() routine also fails with java.lang.Error).


    Best regards,
    Alex
  • 3. Re: Error when calling getLocalSocketAddress() on for a newly created socket
    EJP Guru
    Currently Being Moderated
    Well unless you're closing those channels these all sound like JVM bugs which you are already addressing correctly.

    I can't imagine why you would close a ServerSocketChannel when you get an exception on a SocketChannel. Not a solution in any sense.
  • 4. Re: Error when calling getLocalSocketAddress() on for a newly created socket
    978944 Newbie
    Currently Being Moderated
    Hello,

    Thanks for your quick response!

    We actually do something like this:

    // NIO engine build around a Selector for asynchronous IO developpement.
    public class NIOCore implements Runnable {

    private final Selector selector;
    private final Set<SelectableChannelListener> channels = new HashSet<SelectableChannelListener>();
    ...

    public void run() {
    ...
    for (final Iterator<SelectionKey> i = selector.selectedKeys().iterator(); i.hasNext();) {
    final SelectionKey selectionKey = i.next();
    i.remove();

    final SelectableChannelListener selectableChannelListener = (SelectableChannelListener) selectionKey.attachment();

    if (!selectionKey.isValid()) {
    continue;
    }

    // Check what event is available and deal with it
    if (selectionKey.isAcceptable()) {
    if (log.isTraceEnabled()) {
    log.debug("Processing ACCEPT event for listener: " + selectableChannelListener);
    }
    try {
    selectableChannelListener.acceptEvent();
    } catch (final Throwable t) {
    log.error("Error on ACCEPT event for listener " + selectableChannelListener, t);
    selectableChannelListener.close();
    }
    }
    .....
    }

    }

    }

    As you ca see, we use to do a callback on our SelectableChannelListener objects. In our case the SelectableChannelListener object that actually propagates the JVM Error is this one:

    public final class Acceptor extends SelectableChannelListener {
    protected NIOCore core;
    ...
    public void acceptEvent() {
    ...
    SocketChannel socketChannel = serverSocketChannel.accept();
    if (socketChannel == null) {
    throw new NIOCoreException("No incoming connection to accept");
    }

    // Socket options
    if (params.getSocketBufferSize() > 0) {
    socketChannel.socket().setSendBufferSize(params.getSocketBufferSize());
    }

    try {
    socketChannel.socket().setTcpNoDelay(true);
    } catch (SocketException ex) {
    if (log.isDebugEnabled() || log.isTraceEnabled())
    log.warn(logPrefix + "could not set TCP_NODELAY socket option.You may experience performance loss", ex);
    else
    log.warn(logPrefix + "could not set TCP_NODELAY socket option. You may experience performance loss. Cause : " + ex.getMessage());
    }

    ......
    // later if the connection is accepted succesfully this line is executed
    SocketAddress localAddress = socketChannel.socket().getLocalSocketAddress();
    }

    @Override
    public void close() {
    core.unregister(this);
    try {
    selectableChannel.close();
    } catch (final IOException e) {
    // Ignore
    }
    NIOPortManager.getInstance().setListenBindSapState(getBoundSAP(), true);
    }

    }


    So bassically when we call selectableChannelListener.close() we indeed close the ServerSocketChannel channel, but we also unregister the Acceptor observer object from NIOCore in order to free memory and set the SAP as available.
    The problem is that we cannot reproduce this issue (it reproduces only on our customers environment) and I consider rather risky to remove the selectableChannelListener.close() call.


    Actually I would catch the Error and package it in an internal runtime exception:
    SocketAddress localAddress = null;
    try {
    localAddress = socketChannel.socket().getLocalSocketAddress();
    }
    catch (Throwable t) {
    throw new MyRunntimeException(t);
    }

    And then to catch this new runtime exception in NIOCore class when this call is made:
    try {
    selectableChannelListener.acceptEvent();
    } catch (final MyRunntimeException e) {
    log.debug("JVM error occured!");
    } catch (final Throwable t) {
    log.error("Error on ACCEPT event for listener " + selectableChannelListener, t);
    selectableChannelListener.close();
    }

    I know this is silly but at least we can use it as a temporary workaround until these JVM issues are fixed.

    Also since socketChannel.socket().getLocalSocketAddress() API call only throws that specific error from time to time, we could implement a retry mechanism. What I mean is to call getLocalSocketAddress() in a while loop and if the call fails with Error just continue to retry...we could provide 3 retries for example. If on the 3 ^rd^ retry the call also fails, just let it be failed and treat it in normal way.


    Best regards,
    Alex
  • 5. Re: Error when calling getLocalSocketAddress() on for a newly created socket
    EJP Guru
    Currently Being Moderated
    Well I won't repeat myself but I would comment that it is not a major error for a non-blocking accept() to return null. Instead of throwing an exception here, I would just return.

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points