Skip to Main Content

Infrastructure Software

Announcement

For appeals, questions and feedback about Oracle Forums, please email oracle-forums-moderators_us@oracle.com. Technical questions should be asked in the appropriate category. Thank you!

Solaris DHCP server blocks DHCP client after client reboots

932891Apr 24 2012
Sorry if this is a bit long, but it's a strange issue that needs some detail to explain and to be clear.

I have a single Solaris 11 x86 server doing several things (DHCP, Dovecot, Apache, MySQL, Samba, OpenVPN, sendmail). DHCP is doing something strange. Here are the specifics...

I have configured the Solaris DHCP server as follows:
a) Configured to give out 8 client IPs
b) 6 IPs are reserved (a Client ID is specified and the Reserved check box is ticked)
c) Remaining 2 IPs are not reserved (Client ID is 00 and the Reserved check box is unticked)
d) All IPs are dynamic assignment for 1 week

(Unrelated, but just as an FYI, I chose to use reserved IPs as I don't allow DHCP to update DNS, so I wanted my clients to always be assigned the same IP via DHCP.)

The very first time a new client powers on it is assigned its reserved IP address by the DHCP server. The client is able to communicate with the server (for http, email, DB, smb, etc) without any problems. The client stays powered on 24/7, and when the 1 week DHCP lease expires, the client renews the lease (and is given the same reserved IP), no problems. Everything is fine, and can continue this way indefinitely.

The problem occurs when the client is powered off for a few minutes then powered on again.

Upon powering back on, the client requests an IP address, the DHCP server assigns the client its reserved IP address, and from that point onward, the server blocks all incoming network traffic from that client - absolutely everything is blocked. I have snooped whilst doing all sorts of direct and broadcast network tests (TPC, UDP and ICMP) from the client and the server sees none of it.

To get the client's traffic through to the server again I can do either of the following:
a) Reboot the server (a bit drastic)
b) From the client, release the IP address, then renew the IP address (effectively restarting the 1 week IP lease)

This problem only occurs when a client requests an IP address whilst a previous 1 week lease exists and is still active for the same client. If a client has never had a lease, or a previous 1 week lease has expired, then the client is assigned its reserved IP address and communicates with the server without a problem.

In the tests I've done, I've been able to replicate this problem with Windows, MacOS and Solaris clients.

Does the DHCP server think that the client is an imposter because it is requesting an IP address when it's already been assigned one?
Is there some sort of security switched on by the DHCP server that's causing this?
As you can probably tell, I'm totally stumped by this behaviour.

Any ideas or assistance would be appreciated.

Cheers
Craig

Comments

What kind of application are you running? I'm thinking possibly the issue is MBean server related somehow.

Do you get the same problem if you just run something really simple, like HelloWorld with a for(;;) loop.

(Or start JDK/bin/jdb with the same jvm flags, just prepend them all with -J)

1044925

My application is a daemon that accepts TCP Connections and processing data.

When i created a simple Main class that loops and sleeps on every iteration, it worked.

Any idea what could be wrong with MBean server ?

Is it possible to run it normally, ie. not in daemon mode, and see if that makes any difference?

1044925

I mean i am not doing anything special to run it in Daemon mode. My app runs infinitely loops on a blocking call ( accept() ).

1044925

I also get problem when doing HeapDump. It gives error

Problems when doing Heap Dump for JOverflow analysis

Problems when doing Heap Dump for JOverflow analysis

And when i try to use DTrace from within my JavaMissionControl. I get this error

com.jrockit.mc.dtrace.recorder.DTraceRecorderException: Remote machine not setup correctly for running DTrace.

  at com.jrockit.mc.dtrace.recorder.Factory.createRemoteDTraceRecorder(Factory.java:70)

  at com.jrockit.mc.dtrace.ui.DTraceStartJob.createRemoteRecorder(DTraceStartJob.java:112)

  at com.jrockit.mc.dtrace.ui.DTraceStartJob.startRemote(DTraceStartJob.java:79)

  at com.jrockit.mc.dtrace.ui.DTraceStartJob.run(DTraceStartJob.java:60)

  at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)

Caused by: java.lang.RuntimeException: Could not invoke operation SetupState on DTraceMBean

  at com.jrockit.mc.dtrace.recorder.internal.remote.MBeanToolkit.rethrowBadInvoke(MBeanToolkit.java:85)

  at com.jrockit.mc.dtrace.recorder.internal.remote.MBeanToolkit.invokeOperation(MBeanToolkit.java:62)

  at com.jrockit.mc.dtrace.recorder.internal.remote.RemoteRecorder.getSetupState(RemoteRecorder.java:95)

  at com.jrockit.mc.dtrace.recorder.Factory.createRemoteDTraceRecorder(Factory.java:63)

  ... 4 more

Caused by: javax.management.InstanceNotFoundException: com.oracle.management:type=DTrace

  at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1095)

  at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:643)

  at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)

  at com.sun.jmx.remote.security.MBeanServerAccessController.getAttribute(MBeanServerAccessController.java:320)

  at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1464)

  at javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97)

  at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1328)

  at java.security.AccessController.doPrivileged(Native Method)

  at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1427)

  at javax.management.remote.rmi.RMIConnectionImpl.getAttribute(RMIConnectionImpl.java:657)

  at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source)

  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

  at java.lang.reflect.Method.invoke(Method.java:606)

  at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)

  at sun.rmi.transport.Transport$1.run(Transport.java:177)

  at sun.rmi.transport.Transport$1.run(Transport.java:174)

  at java.security.AccessController.doPrivileged(Native Method)

  at sun.rmi.transport.Transport.serviceCall(Transport.java:173)

  at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556)

  at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811)

  at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670)

  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

  at java.lang.Thread.run(Thread.java:724)

  at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)

  at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)

  at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)

  at com.sun.jmx.remote.internal.PRef.invoke(Unknown Source)

  at javax.management.remote.rmi.RMIConnectionImpl_Stub.getAttribute(Unknown Source)

  at javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.getAttribute(RMIConnector.java:902)

  at com.jrockit.mc.rjmx.subscription.internal.AttributeValueToolkit.getAttribute(AttributeValueToolkit.java:51)

  at com.jrockit.mc.rjmx.internal.MCMBeanServerConnection.getAttribute(MCMBeanServerConnection.java:153)

  at com.jrockit.mc.dtrace.recorder.internal.remote.MBeanToolkit.invokeOperation(MBeanToolkit.java:53)

  ... 6 more

1044925

So i experimented with different things and i realized that if i TURN OFF jmx authentication then Flight Recorder works.

Thanks for your help.

I was just about to suggest that it might be an issue with authentication. Does it work if you give readWrite permissions (if you had it set to readOnly before)?

1044925

Permission was readWrite and it didnt work

Hirt-Oracle
Answer

Hi there!

You need to ensure that the user you want to access the flight recorder has permission to create the relevant MBeans. You have probably defined the roles in the jmxremote.access file. Try adding:

<your-role-name-goes-here> readwrite \
  create com.sun.management.*,com.oracle.jrockit.* \
  unregister

Kind regards,

Marcus

Marked as Answer by 1044925 · Sep 27 2020

Ah, so that's how you do it. Must be many people having the same problem. We should add it to the FAQ.

Hirt-Oracle

Yep. I've been thinking about writing up a blog on how to safely navigate JMX configuration hell, as there are quite a few more thinks that can go wrong. It will have to wait a bit though. Too busy.

1 - 11
Locked Post
New comments cannot be posted to this locked post.

Post Details

Locked on May 22 2012
Added on Apr 24 2012
0 comments
847 views