13 Replies Latest reply on Feb 10, 2012 3:25 AM by EJP

    Socket - very slow HTTP 1.1 request

    916217
      Hello,

      I am connection to http server via socket. I have very simple HTTP request:

      String out="GET / HTTP/1.0 +CRLF
      Host: www.example.com +CRLF
      +CRLF";

      Socket sock2 = new Socket(ADDRESS, PORT);
      BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(sock2.getOutputStream()));
      writer.write(out);
      writer.flush();

      BufferedReader rd = new BufferedReader(new InputStreamReader(sock2.getInputStream()));
      String line;

      while ((line = rd.readLine()) != null) {
      //...
      }

      When I use HTTP/1.0 everything works perfectly, the response is very quick. When I use HTTP/1.1 response is 5-10x slower but I have no idea why.

      Does anyone know where the problem might be? Thank you.
        • 1. Re: Socket - very slow HTTP 1.1 request
          sabre150
          There seems to be some quotes missing in the 'out' string so that can't be your code.

          In any case, you should use a URLConnection obtained from connecting through a URL and caste the result to an HttpURLConnection.

          One possible problem with your code is that HTTP 1.1 does not close the socket to indicate an end of response so I suspect you will be waiting for a timeout to close the connection and indicate the end of data.
          • 2. Re: Socket - very slow HTTP 1.1 request
            916217
            Could you be more specific? I can not use HttpURLConnection, because I need some kind of low-level information.

            Previously mentioned HTTP ver. 1.0 takes on average 200ms and the same HTTP 1.1 request takes on average 5200ms each time. It realy seems to happen there some kind of timeout.

            P.S. Of course this is my code - structure of HTTP request is for illustration only ;)
            • 3. Re: Socket - very slow HTTP 1.1 request
              DrClap
              Then that would imply that the timeout which sabre150 referred to is 5 seconds. If you don't want that to happen, then you would have to write code which could find out when it had read the last line of the response, and disconnect immediately. Your code waits for the socket's input stream to realize that there is no more data to be read; as sabre150 said, it can't determine that until the timeout period is finished.
              • 4. Re: Socket - very slow HTTP 1.1 request
                EJP
                because I need some kind of low-level information.
                Such as? You can get every response header and the entire body: what else is there?
                • 5. Re: Socket - very slow HTTP 1.1 request
                  916217
                  "...you would have to write code which could find out when it had read the last line of the response, and disconnect immediately..."

                  I am not sure what / how do you mean to disconnect immdiately - now I close the socket after the all response is read. I cannot to close the socket earlier.

                  Could you provide me a piece of code or try specify what method use or send me some URL with an example - I'll find out the rest.
                  • 6. Re: Socket - very slow HTTP 1.1 request
                    EJP
                    HttpURLConnection conn = (HttpURLConnection)new URL("http://www.example.com/").openConnection();
                    InputStream in = conn.getInputStream();
                    // etc
                    • 7. Re: Socket - very slow HTTP 1.1 request
                      916217
                      Thank you.

                      As I mentioned previously, for now I do not use HttpURLConnection but sockets. Do you know some similar solution for sockets?

                      Edited by: 913214 on 8.2.2012 23:58
                      • 8. Re: Socket - very slow HTTP 1.1 request
                        gimbal2
                        913214 wrote:
                        Thank you.

                        As I mentioned previously, for now I do not use HttpURLConnection but sockets. Do you know some similar solution for sockets?
                        You start by reading the RFC on HTTP 1.0 and HTTP 1.1. If you want to use low level sockets, you need to accept the consequences: you'll need to do everything yourself, including dealing with the protocol. So study it, learn it, apply it.

                        OR listen to what people have been telling you and stop trying to reinvent the wheel.
                        • 9. Re: Socket - very slow HTTP 1.1 request
                          916217
                          I tried it HttpURLConnection - it works perfectly but is not suitable for me, I need to get more information about the connection to the server (such as connect time, read time and a receiving time). I am afraid this is not possible to get these values separated. If so, try be something more specific. If not, tell me and I'll try to find another solution...

                          Edited by: 913214 on 9.2.2012 5:52
                          • 10. Re: Socket - very slow HTTP 1.1 request
                            sabre150
                            913214 wrote:
                            , I need to get more information about the connection to the server (such as connect time, read time and a receiving time).
                            We may be able to suggest a better approach if we understood what you are trying to do. Why do you need these parameters? Can you provide definitions of these parameters? How do you get these values using your original approach ?
                            • 11. Re: Socket - very slow HTTP 1.1 request
                              916217
                              I am trying to create simple program to get these values for my URLs:

                              - Connect time - time needed to establish connection with server
                              - Read time - time between sending the request to the server and obtaining the response
                              - Receiving time - time needed to download the response from the server back to the client (this time will be always shorter than Read time)
                              • 12. Re: Socket - very slow HTTP 1.1 request
                                sabre150
                                913214 wrote:
                                I am trying to create simple program to get these values for my URLs:

                                - Connect time - time needed to establish connection with server
                                - Read time - time between sending the request to the server and obtaining the response
                                - Receiving time - time needed to download the response from the server back to the client (this time will be always shorter than Read time)
                                I would be interested in how you obtain these values with your current approach.

                                In order to complete this project it looks to me like you are going to have to implement the equivalent of HttpURLConnection for the HTTP 1.1 protocol; non-trivial so I doubt if you are going to get away with a "simple program". You are going to have to spend some time studying rfc2616 (Google is your friend). Off the top of my head I would expect some form of callback approach to offer the most flexibility with callback events being generated at each state transition. This would allow you to decouple your home grown HttpURLConnection from the time calculations.
                                • 13. Re: Socket - very slow HTTP 1.1 request
                                  EJP
                                  - Connect time - time needed to establish connection with server
                                  A second or two at the outside, more probably a fraction of a second.
                                  - Read time - time between sending the request to the server and obtaining the response
                                  That's not 'read' time. That's (a) request transmission time + (b) time to service the request at the server + (c) network latency from the server to the client.
                                  - Receiving time - time needed to download the response from the server back to the client
                                  That's what I would call 'read time'.
                                  (this time will be always shorter than Read time)
                                  No it won't. It depends on (a), (b), and (c) above, and on the payload size. For a payload of 1 byte that will be true. For a payload consisting of a 1G pre-existing file it will certainly not be true. It depends.

                                  My next question is why measure all this with HTTP 1.1 at all? There is no reason to expect it to be any different from the HTTP 1.0 figures. Just use those.