This discussion is archived
7 Replies Latest reply: Feb 16, 2012 7:56 PM by 796440 RSS

Noobish "efficiency" question on DataOutputStream

796440 Guru
Currently Being Moderated
So, I currently have this:
byte[] buf = some_bytes;
DataOutputStream out = new DataOutputStream(socket.getOutputStream());
out.writeInt(buf.length);
out.write(buf);
I've always thought that calling write(byte[]) and any given OutputStream will be "efficient", in that things will be chunked under the hood to avoid incurring whatever I/O overhead is present for each and every byte. So, as I understand it, we only need to explicitly use BufferedOutputStream if we're writing one or a few bytes at a time.

Now I'm not so sure.

Digging through the core API source code a bit, it seems that this ultimately calls OutputStream.write(byte) in a loop, which, presumably, is bad unless the OutputStream in question is a Buffered one.

Am I shooting myself in the foot here? Should I be wrapping that DataOuputStream around a BufferedOutputStream? Or vice versa?

Or should I just scrap the DataOuputStream altogether and write the int's bytes out myself?

I'm going to test it both ways, but in the meantime, and just in case the tests are not definitive, I'm wondering if anybody here can tell me what I should expect to see.

I think I've been staring at this stuff for a bit too long and am second-guessing myself to death here no matter which way I look at it. So thanks in advance for any nudge you can give me back toward sanity.

Edited by: jverd on Feb 16, 2012 3:59 PM
  • 1. Re: Noobish "efficiency" question on DataOutputStream
    796440 Guru
    Currently Being Moderated
    Actually, I may have answered my own question, though I'd still be happy for some confirmation.

    I'm assuming that the underlying implementation will ultimately be a SocketOuptuStream, and if so, then this is what gets called:
        private void socketWrite(byte b[], int off, int len) throws IOException {
    ...
         FileDescriptor fd = impl.acquireFD();
         try {
             socketWrite0(fd, b, off, len);
    ...
    where socketWrite0 is a native method, which, presumably, has some intelligence in handling the byte array.

    I hope. :-)
  • 2. Re: Noobish "efficiency" question on DataOutputStream
    EJP Guru
    Currently Being Moderated
    Should I be wrapping that DataOuputStream around a BufferedOutputStream?
    Always.
  • 3. Re: Noobish "efficiency" question on DataOutputStream
    796440 Guru
    Currently Being Moderated
    EJP wrote:
    Should I be wrapping that DataOuputStream around a BufferedOutputStream?
    Always.
    Okay, fair enough.

    So, what's the point of the basic OutputStream not being buffered then? As I said, I thought that it was only very small, separate writes that benefited from buffering, but that it wouldn't matter for write(large_byte_buf). Am I just totally wrong there? Or is it something more subtle?

    (I should be receiving a certain book any day now, if it didn't already come today without me noticing. Maybe the answer will be in there?)
  • 4. Re: Noobish "efficiency" question on DataOutputStream
    EJP Guru
    Currently Being Moderated
    So, what's the point of the basic OutputStream not being buffered then?
    I guess the idea was that if you want buffering you say so, but that falls down in the case of ObjectOutputStream which does it anyway. I think you'll find that every significant class that extends OutputStream (specifically FileOutputStream and SocketOutputStream) overrides write(byte[], int, int) to do an atomic write to the OS, so it isn't really such an issue except in the case of DataOutputStream (and not ObjectOutputStream, see above).
    (I should be receiving a certain book any day now, if it didn't already come today without me noticing. Maybe the answer will be in there?)
    Nope. I took the reader's knowledge of Java I/O streams for granted. I had written a biggish section about it in the RMI book, but I cut it in the quest for page count (and probably relevance).
  • 5. Re: Noobish "efficiency" question on DataOutputStream
    796440 Guru
    Currently Being Moderated
    EJP wrote:
    So, what's the point of the basic OutputStream not being buffered then?
    I guess the idea was that if you want buffering you say so,
    Ok.
    I think you'll find that every significant class that extends OutputStream (specifically FileOutputStream and SocketOutputStream) overrides write(byte[], int, int) to do an atomic write to the OS, so it isn't really such an issue except in the case of DataOutputStream (and not ObjectOutputStream, see above).
    Okay, so, in this case, I've got a DataOutputStream wrapped around a SocketOutputStream. It's not an issue for SOS, but it is for the wrapping DOS. Yes?

    So to overcome the DOS doing a bunch of piddly 1-byte writes to the SOS, which in turn could result in a bunch of piddly 1-byte writes to the network, which I don't want, I inject a BOS between them. Yes?

    Thanks for the help. I can't believe after all these years I never got these details sorted out. I guess it never came up quite this way before.
  • 6. Re: Noobish "efficiency" question on DataOutputStream
    EJP Guru
    Currently Being Moderated
    Okay, so, in this case, I've got a DataOutputStream wrapped around a SocketOutputStream. It's not an issue for SOS, but it is for the wrapping DOS. Yes?
    Yes. I always wrap a socket output stream in a BufferedOutputStream/Writer before I wrap anything else on top. (And if there's a zipping output stream I put another BufferedOutputStream around that, so the zipper has as much as possible to zip at a time.)
    So to overcome the DOS doing a bunch of piddly 1-byte writes to the SOS, which in turn could result in a bunch of piddly 1-byte writes to the network, which I don't want, I inject a BOS between them. Yes?
    Yes.

    It's even more important when the socket is an SSL socket, as a 1-byte write becomes a 40+-byte SSL record.
  • 7. Re: Noobish "efficiency" question on DataOutputStream
    796440 Guru
    Currently Being Moderated
    Got it. Thanks much.

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points