This is x-posted from Driver Development. That forum does not seem to get much traffic.
I have written a driver that will accept only 1 open. A application can successfully call an open to access the driver. If a 2nd application tries another open, the driver will return EBUSY. This 2nd application receives the EBUSY error correctly.
However on Solaris 11 the OS seems to call a close() on our driver in response to the failed open attempt. This close causes the first application's file descripter to become invalid.
Solaris 10 does not call close on a failed open attempt.
Why is Solaris 11 making this close() call on the driver when:
a) the open call was not successful.
b) the first application should still have the open count at 1. The OS should only call close when the last reference is closed.
Is there anything required by a driver to prevent this from happening.
I have also written the application that will access the driver and when the application is closed, the system panics.