This discussion is archived
1 2 Previous Next 28 Replies Latest reply: Sep 9, 2012 10:43 PM by EJP Go to original post RSS
  • 15. Re: Jre7 Socket problem
    960486 Newbie
    Currently Being Moderated
    Yes I do control all the code, I had a look at this earlier and discovered that the send and receive code was actually different from the other code. I have corrected this and the system works now.

    This was due to the earlier mentioned problems with mobile connections it had always worked and I assumed its code was shared with the others. So... never assume anything,

    sabre150 actually made me have doubts about the code on the receiving side because there were actual 512 bytes received at the receivers end... but... not by the relay and that's exactly where the bug was. That was an old piece of code which i never had touched after solving the mobile problems, it had always been working correctly but, you can guess it, it did not copy the actual bytes but the number of bytes of the buffer. I have consolidated all the send and receive code to prevent this from happening again.

    Apparently there is a difference with socket handling between Java 1.6 and Java 1.7 but its indeed not a bug.

    I guess this is not the only thread which ends like this.

    Thanks everyone!

    Edited by: 957483 on Sep 7, 2012 10:56 PM
  • 16. Re: Jre7 Socket problem
    EJP Guru
    Currently Being Moderated
    Apparently there is a difference with socket handling between Java 1.6 and Java 1.7 but its indeed not a bug.
    You've provided no evidence for either claim. All you've provided and told us about is some very poor quality code that has finally failed due to a major error that had always been there and that could have failed any time TCP decided to deliver the data differently. You should be counting your blessings that it ever worked at all.
  • 17. Re: Jre7 Socket problem
    sabre150 Expert
    Currently Being Moderated
    957483 wrote:
    Yes I do control all the code, I had a look at this earlier and discovered that the send and receive code was actually different from the other code. I have corrected this and the system works now.
    Maybe you should now take another look at the problem that required you to break the data into 512 byte chunks. Maybe you would now find that it was not necessary!

    P.S. I'm having trouble reconciling your explanation of the problem with the fact that the bytes were the character '0' and not the value 0x00.
  • 18. Re: Jre7 Socket problem
    960486 Newbie
    Currently Being Moderated
    EJP:
    Code has a reason for its existence, I've tried Data(Input/Output)Streams, Buffered(Reader/Writer) in the past and they could not provide the stability needed. It was until I broke up the stream into smaller chunks when it became stable, something which it has been for three years afterwards. This solution worked perfect and I had to meet my deadline for this particular project so I moved on to other pressing matters. The examples provided might not have been of good quality but that is what you get after trying to pinpoint the problem.

    Let me be clear, this code/system has been working fine for three years straight, there has never been reported any problem on this subject. It is in use daily by multiple users and it was only, and only, until users upgraded to Java 1.7 when this problem began. It is even that when I downgrade to Java 1.6 on my system, without reboot, the same program worked and when I upgrade to Java 1.7 the program failed. Exactly the same for other users, so yes I believe that the observation that Java 1.7 handles sockets different internally is correct, that is not a horrific problem or a fault of the developers of Java, its an observation and not more than that.

    sabre150:
    It might be indeed an idea to try but I believe there was a reason for being like that and it had something to do with remote locations and bad gprs cover. Anyways, I'm going to try when I have found a remote spot with bad cover here. I think the 0s were 0x00s which I mis-interpreted after staring myself blind on this.


    To all, thanks for helping!

    Edited by: 957483 on Sep 8, 2012 9:39 AM
  • 19. Re: Jre7 Socket problem
    sabre150 Expert
    Currently Being Moderated
    957483 wrote:
    sabre150:
    It might be indeed an idea to try but I believe there was a reason for being like that and it had something to do with remote locations and bad gprs cover. Anyways, I'm going to try when I have found a remote spot with bad cover here. I think the 0s were 0x00s which I mis-interpreted after staring myself blind on this.
    My last words on this - to quote your original post
    >
    This arrives at the server as follows:

    T00000000000000000000000000000000000000000000000000000000000000000000000000
    0000000000000000000000000000000000000000000000000000000000000000000000G9yZW
    0gaXBzdW0gZG9sb3Igc2l0IGFtZXQsIGNvbnNlY3RldHVyIGFkaXBpc2NpbmcgZWxpdC4gSW50ZWd
    lciB

    which shows that the zeros were the character '0' . I deduce two things from this. First, if the corruption had been 0x00 then this would not have displayed as the character '0' since in no character encoding I am aware of is 0x00 a representation of the character '0' . Second, the line lengths are 72 characters which is the standard for Base64. This means that the corruption had to occur prior to the Base64 encoding and not after as you are saying. Of course, as hinted at in one of your earlier posts, that might not have been actual data in which case your posting it was very silly since any deductions made from analizing it are worthless!

    You still have not convinced me that there was ever a need to break the data into 512 byte blocks because TCP is by design a reliable protocol even when sent over an unreliable channel. The implication of your assertion that the 512 byte blocks are needed and that this is not as result of your code is that along the channel some system component is badly broken. Though there may be a bug in the system code my experience over many many years is that user code is more likely to have bugs than system code. I know nothing at all about GPRS so I could be very very wrong but I would still bet a lot of money on the problem being in your code.

    Bye
  • 20. Re: Jre7 Socket problem
    EJP Guru
    Currently Being Moderated
    Code has a reason for its existence
    Broken code has no reason to exist. You've already posted one sample, and you've alluded to the existence of another.
    I've tried Data(Input/Output)Streams, Buffered(Reader/Writer) in the past and they could not provide the stability needed.
    So your code was at fault, see above (passim),a nd also your investigative procedures. I've been using all of those for over fifteen years, and I have never noticed a 'stability' problem.
    It was until I broke up the stream into smaller chunks when it became stable, something which it has been for three years afterwards.
    This only goes to prove my assertion. I have been using chunk sizes of 8192 or more over TCP for over twenty years without problems, whether 'stabililty' or otherwise.
    This solution worked perfect
    No it didn't. Otherwise you wouldn't be posting here. You've already admitted one major bug; I have already shown you a way to reduce 26 lines of spaghetti to 4 lines of correct code; and DrClap has done the same for 30 other lines of 'perfect' code. Your evidence for any of these propositions is badly lacking.
    and I had to meet my deadline for this particular project so I moved on to other pressing matters.
    In other words you didn't test properly, and you allowed flawed code that you didn't completely understand to go into production. If there was nothing wrong with the code you wouldn't be making these excuses and indeed you wouldn't be posting here at all. QED.
    The examples provided might not have been of good quality but that is what you get after trying to pinpoint the problem.
    Unless you are actually asserting that for some reason you made the code worse while trying to fix it, this is meaningless.
    Let me be clear, this code/system has been working fine for three years straight, there has never been reported any problem on this subject.
    Let me be clear in turn. It has been wrong for three years straight. Any change from a new NIC, to an equipment change in an ISP rack, to an operating system upgrade, to a C library change, to a Java version change, could have broken it, and finally one of those did. The miracle is that it ever worked at all. You were lucky, simple as that.
  • 21. Re: Jre7 Socket problem
    960486 Newbie
    Currently Being Moderated
    Code exists and therefore has (or had) its reason of existence. This is a fact, whether we want it to exist is a whole different question.
    In this case no, it should have not existed, but the problem is that it seemed to work fine and therefore did not raise any suspicion. I can not imagine you never had a similar issue.

    To add a little more information, this system has been used on many different types of hardware, uses different types of modems (2g and now 3g different types/brands), different operating systems and this solution has worked on all of these. An example it worked on my laptop with any of the following connections: wifi/lan/tethered android phone, it worked on other computers, in the field as well as workstations. More, it is at times in use in Australia and the server is in Europe and it worked.

    Please watch the word "worked" this means in the past, that means until Java 1.7. and yes... that is when I got here because that is when the problems started to manifest itself. It first seemed to be Java 1.7 because only and only Java 1.7 clients gave these problems. Of course it could have also been triggered by a hardware change but it did not, apparently 512 bytes was a very safe choice back then.

    This forum is about solving problems and answering questions (including issues like this), at this moment you are not doing any of them, you are now just trying to prove your point which does not help anyone. Remember, in the end it was simply a bug and people make mistakes.
  • 22. Re: Jre7 Socket problem
    960486 Newbie
    Currently Being Moderated
    sabre150: The zeroes were apparently copied due to the relay in between happily copied over buffers with the wrong number of bytes. I got to this when I realized that this could happen if there was a problem when I receive e.g. 10 bytes but the system copy the full 512 bytes in the buffer. It made it possible to send 0x00 bytes and that is what happened.

    Like I said earlier, I made the wrong assumption which made me look in the wrong direction. The fact that it worked flawless in the past 3 years just made it worse. I've solved it by consolidating all send and receive code to one class.

    There was a very good reason to break it into 512 bytes and this might have been the cause but it does not really matter now, it works and in a new major version we'll try and test it while writing the whole buffer.

    Anyways, many thanks for sharing your thoughts on this issue.
  • 23. Re: Jre7 Socket problem
    sabre150 Expert
    Currently Being Moderated
    957483 wrote:
    sabre150: The zeroes were apparently copied due to the relay in between happily copied over buffers with the wrong number of bytes.
    I thought I had nothing more to say in this thread but I can't let this pass. Once again, one deduces from the sample output you gave that the corruption occurred BEFORE the Base64 encoding and not after you sent the encoded data to the relay or even when sending the encoded data to the relay !!!!!!!!!!!!!!!!!.
  • 24. Re: Jre7 Socket problem
    960486 Newbie
    Currently Being Moderated
    I'm glad i'm keeping your minds occupied.

    When I wrote the question I wanted to save you guys from having to read lots and lots of bytes on the forum:

    AFSDFSFADASFFCSDS <-- packet to send

    A000000000000000 <-- packet fragment 1

    FSDFSFADASFFCSDS <-- packet fragment 2

    I also did not expect that it would generate so much comments.

    I'll be more specific next time, it makes the picture more clear.

    Again, many thanks
  • 25. Re: Jre7 Socket problem
    sabre150 Expert
    Currently Being Moderated
    957483 wrote:
    I'm glad i'm keeping your minds occupied.

    When I wrote the question I wanted to save you guys from having to read lots and lots of bytes on the forum:

    AFSDFSFADASFFCSDS <-- packet to send

    A000000000000000 <-- packet fragment 1

    FSDFSFADASFFCSDS <-- packet fragment 2

    I also did not expect that it would generate so much comments.

    I'll be more specific next time, it makes the picture more clear.
    Which of us is missing the point! IF, AS YOU HAVE SHOWN TWICE NOW, YOUR CORRUPTION WAS ASCII '0' BYTES THEN THE CORRUPTION TOOK PLACE PRIOR TO THE BASE64 ENCODING AND not not not , AS YOU KEEP ASSERTING, AFTER BASE64 ENCODING.
  • 26. Re: Jre7 Socket problem
    EJP Guru
    Currently Being Moderated
    Also worth noting that 512-byte buffering cannot possibly have had the magical 'stabilizing' properties claimed for it, as all the data gets buffered in the socket send buffer of at least 8k in the kernel anyway. There is also the Nagle algorithm.

    Clearly there is far more to this than meets the eye. Anybody's eye.
  • 27. Re: Jre7 Socket problem
    960486 Newbie
    Currently Being Moderated
    no... a print directly after public void send( byte[] data ) { showed me that the data array was indeed intact, correct base64 encoded data. I've already spent time verifying that. way before coming here.

    It turned out to be somewhere else, and it was certainly not magical as i described but an assumption which i should not have made.
  • 28. Re: Jre7 Socket problem
    EJP Guru
    Currently Being Moderated
    See here for a parallel story.
1 2 Previous Next

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points